Hectic times

October 8, 2007

Exegesis and audiovisual work due in a matter of weeks. Terrifying for two reasons:

    1. My HDD died. Partition stuffed. Some pictures, documents, video and music (expensively) recovered, but all my code is gone.

    2. I’ve just started full-time work at a wonderful place called Icelab.

Although coding again from scratch is frustrating, it’s most likely a good thing. The cleanliness and quality of my code will surely be better for it. Thought my computer would be happier with a fresh, clean hard drive and a re-install of windows, but no, its surely dying.

So rather than wrapping up my honours work I’ve been at work learning Motion 3 and how to build websites properly. Expect a decently designed blog at some stage.

After getting my head around Motion’s 3D navigation I discovered a really impressive piece of software. Purely as an animation or graphics tool its daunting, but its video features and connectivity with Final Cut make for a piece of software with potential that I am yet to even grasp. Playing with nice macs and flash new software is fun.

My Reas/Fry processing book came in the mail and it’s fantastic. If only I had it at the start of the year. Countless small programming concepts I’ve spent hours frustrating over are explained concisely and effortlessly. Solid hard-cover, well laid out, and cleverly structured. I shipped to Australia from Barnes and Noble. Took about two weeks, but it was far cheaper than buying here.

Finally did a little bit of programming tonight. Some results:




September 4, 2007

Currently writing exegesis rather than blog posts.

Another clue as to how my animation might look. This time with paint instead of pencil:

In ‘Analysing Musical Multimedia’ Nicholas Cook provides an extremely clear general theory of how media engage with one another in multimedia works in general (from opera through to commercials). Judging the book solely by its cover and thinking I had found a first year textbook, I almost placed it back on the library shelf. Thankfully, I checked the contents-

Chapter 1 – Synaesthesia and Similarity
Chapter 2 – Multimedia as Metaphor
Chapter 3 – Models of Multimedia
and so on, and decided it might be worth a shot.

Cook’s thoughts on synaesthesia were extremely refreshing, ‘Synaesthesia provides some hints as to what multimedia is: but, perhaps more importantly, it supplies an illuminating model of what multimedia is not.’ Cook believes the most useful way to study multimedia is in its element’s differences and interactions, rather than through the eyes of synaesthesia, a phenomenon based on similarity, duplication and translation.

Speaking on Skriabin’s (an apparent synaesthete) fifth symphony, Prometheus, which includes a part for a Tastiera per luce, or colour keyboard:

    The luce part literally does add little; for while the slower part has no discernible relationship to what is heard, the faster part simply duplicates information that is already present in the music. In neither case is there a substantial degree of perceptual interaction between what is seen and what is heard—which means that, in a significant sense, Prometheus does not belong to the history of multimedia at all. And to say this is to suggest there is a definite limit to what the phenomenon of synaesthesia can tell us about multimedia, because synaesthesia consists precisely of the duplication of information across different sensory modes. To demand something other than duplication is to go beyond the bounds of synaesthetic correspondence.

The paragraph alone makes Cook’s attack seem more severe than in correct context but there remains an interesting point. Synaesthesia is an amazing phenomenon, and has metaphorical relevance but it also has its limits; its subjective nature among synaesthetes and its complete adherence to similarity. We need to be armed with a wider approach to cross-media works. Cook sees synaesthesia as potentially ‘enabling condition for multimedia, but not a sufficient one.’

    To analyse music is also to be committed to the idea that we perceive notes in terms of the relationships between them: we perceive each note as influencing, and being influenced by other notes—or at any rate, if we do not, it is hard to see what we could be analysing. In a nutshell, we analyse the interaction between the elements of the music: that is what analysing music means. And exactly the same applies to multimedia. To analyse something as multimedia is to be committed to the idea that there is some kind of perceptual interaction between its various individual components, such as music, speech, moving images, and so on: for without such interaction there is nothing to analyse.

The rest of the book is based on this idea of interaction and the idea of multimedia as metaphor. Cooks sees in media interaction the potential for emergence (the result of putting medias together as more than the sum of its parts) and sees the metaphor model as invoking ‘similarity not as an end, but as a means.’ Cook talks about semiotics, motion and gesture (something I might elaborate later on with my reading of The Sonic Self) and arrives at a basic model for the analysis of multimedia.

For a full understanding, read the book, but I’ll explain simply and inadequately. We start at the top and decide whether two media’s relationships are consistent or coherent. A colour organ or keyboard would be a simple and perfect example of a consistent relationship (a relationship he sees as quite rare in current multimedia) where as coherence allows for differential elaboration. The next distinction is a fuzzy one, which Cook acknowledges; he also acknowledges the potential for media forms to shift between classifications. These two quotes might help:

    Conformance begins with originary meaning, whether located within one medium or diffused between all; contest, on the other hand, ends in meaning. And the association of conformant models with synaesthestic and metaphysical speculation demonstrates, conformance tends towards the static and the essentialized, whereas contest is intrinsically dynamic and contextual.
    The term ‘contest’ is intended to emphasize the sense in which different media are, so to speak, vying for the same terrain, each attempting to impose its own characteristics upon the other. One might develop the analogy by saying that each medium strives to deconstruct the other, and so create space for itself.
    The mid-point between these two extremes is represented by the third model of multimeda, complementation, which Figure 3.1 represents in negative terms as that which exhibits neither consistency nor contradiction… complmentation is readily associated with the succesive phases of multimedia production. The classical Hollywood film for example, for instance, was in general virtually complete before it was passed onto the composer for scoring: the composers job was understood as one of complementing….

The main point I wanted to get to was Cook’s ideas on contestation. The possible repercussion being that that works such as mine; generative AV work (dynamic and cross imposing characteristics), form an AV relationship of contestation, rather than harmony. Cook would be useful in supporting the argument that such works are linked through metaphor, motion, gesture (and illusion) rather than similarity or some sort of natural harmony.


August 10, 2007

“So long as the expression ‘Gesamtkunstwerk’ (or ‘integrated work of art’) means that the integration is a muddle, so long as the arts are supposed to be ‘fused’ together, the various elements will all be equally degraded, and each will act as a mere ‘feed’ to the rest. The process of fusion extends to the spectator, who gets thrown into the melting pot too and becomes a passive (suffering) part of the total work of art. Witchcraft of this sort must of course be fought against. Whatever is intended to produce hypnosis, is likely to induce sordid intoxication, or creates fog, must be given up.”

Brecht on Theatre: The Development of an Aesthetic. 1964

In Audio-Vision, Chion boldy suggests that our senses are separate and disparate, describing synchresis as falsely ‘natural’ and ‘immediate’ and ultimately an illusionary experience. Rather than balancing the argument with talk of synesthesia and it’s associated art works I want to focus on this idea of illusion.

An earlier post on David Lynch provides a wonderful example of Lynch exposing this illusion. In ‘Club Silencio’ Lynch creates a moment of almost reverse synchresis providing an AV experience that feels unnatural, stressing that we cannot help but be affected and fooled by this AV trickery. Lynch provides a similar effect through eerie drones, the lack of recorded sound, or audio and visual effects/distortions, (think the Red Room in Twin Peaks) expossing an inescapable illusion.

This blog post really got me thinking. Lynch is essentially and brilliantly pointing out, that illusion (or fantasy) is inescapable and essential to reality. Enter Lacanian thought/Slavoj Zizek.

In Cyberspace, Zizek writes about the interface and the ‘fantasy screens’ of computing:

    What is a fantasy screen, an “interface”? Sometimes we find it even in nature, as in the case of the Cerknica lake in Slovenia: this intermittent lake (during its seasonal eruption, the water throws fishes into the air) was experienced as a kind of magic screen, a miracle of something emerging out of a void. As early as the seventeenth century, this phenomenon intrigued natural scientists—a Slovene author, Janez Valvasor, became a member of the British Royal Academy for providing an explanation of this mystery (an intricate network of underground channels with different pressures). Perhaps, this is the most elementary definition of a mechanism: a machine which produces an effect in the precise sense of a ‘magical’ effect of sense, of an event which involves a gap between itself as the raw bodily materiality—a mechanism is that which accounts for the emergence of an “illusion.” The crucial point here is that the insight into the mechanism does not destroy the illusion, the “effect,” it even strengthens it insofar as it renders palpable the gap between the bodily causes and their surface-effect.

The illusion of synchresis could be seen in the same light. We may know the mechanics of its illusion, but the illusion is just as, if not more powerful. Lynch’s audiovisual disjunctions reveal a gap between surface effect and reality – could fused AV works through the creation of an illusion mechanism, through simulating an unreal relationship, reveal the same gap?

Zizek’s quote would suggest that the pursuit of audio visual fusion is not the pursuit of the pure and interrupted functioning of fantasy. A constructed AV relationship while an effective illusion could also be seen to render palpable the gap between audio and vision. Perhaps this is it’s lure?

The hard part now. Investigating this creatively…

The Real as the irreducible gap between vision and audio?
Could AV illusion offer us a peak into the impossible Real?
How to investigate the fantasmic aspects of the audio-visual relation?

In reverse?

July 31, 2007

In this post I believed I was working to the reverse process to Chion’s analysis; that I was adding visuals to audio rather than the other way round. The more I think about it, the more I realise that I may be wrong. The graphics system I am building is being built independently of my chosen soundtrack and to be honest, adding the audio is the final step. Certain properties of the visuals are permanent, while the soundtrack is completely interchangeable.

This process only differs to film sound in that the visuals are affected not only by the phenomena of ‘added value’, due to synchresis, but also directly in form and shape by audio analysis. Possibly an extension of Chion’s ideas on film sound?

    Considering arts techniques from the broad perspective of the present, I observed that the best “computer art” did not compare
    well with lacework from Belgium made a century ago. But the computer possessed a unique capability of making very complex pattern flow. One could plan exacting and explicit patterns of action and distinctive motions as intricate as lace, but in a way no Belgian lace maker would ever imagine. – John Whitney, 1980.

This 1975 film is reportedly John Whitney’s first foray into computer graphics. Until ‘Arabesque’, Whitney used a converted mechanism of a World War II M-5 Antiaircraft Gun. Essentially a twelve-foot-high analog computer of amazing complexity; where design templates were placed on three different layers of rotating tables and photographed by multiple-axis rotating cameras.

In Digital Harmony (1980), the book that describes his life’s work, his hypothesis –

    …assumes the existence of a new foundation for a new art. It assumes a broader context in which Pythagorean laws of harmony operate. These laws operate in a graphic context parallel to the established context of music. In other words, the hypothesis assumes that the attractive and repulsive forces of harmony’s consonant/dissonant patterns function outside the dominion of music.

Whitney acknowledges that, ‘Music does not need images any more than paintings need sound’ but saw in computing, ‘a visual medium which is more malleable and swifter than musical airwaves. That medium is light itself.’

The book often communicates personal opinion rather than rigorous argument but Whitney makes some original and interesting points. It seems Whitney is not really pursuing visualisation or a tightly fused AV form. Whitney’s search is instead for abstract graphics with the fluidity, expressiveness and structural qualities of music. Whitney begins the book by highlighting the inherent spatial and visual qualities of music and damning early ‘visual music’ inventions:

    Most people visualize music as two-dimensional, with time represented by the horizontal lines and pitch by vertically arrayed symbols, as is the convention on paper. But the perception of music is not two dimensional. The ears reside at the center of a spherical domain. We hear from all around. We hear music as patterns of ups and downs, to and fro in a distinctly three-dimensional space – a space within.

    The eye, more outwardly oriented, perceives objects and events outside at the point where our eyes focus. Yet the eye enjoys design equally as well as the ear. The mind’s eye shares with the ear any inward experience of architectonic spatial constructions and would perceive them with the same pleasure, were they to exist.

    The fact is, however, that these interior fluid visual edifices hardly exist. Anyone can visualize an architectural fantasy of music dancing in the head, but manifesting in reality is another matter! Each century since Leonardo, a vision, grand and obscure as its myth, compelled one or two inventors to struggle with the pathetic inadequacies of the color organ. Twentieth-century abstract art has been a training ground for visual response to musical experience, but in the mind’s eye, architecture in motion lies at the root of our enjoyment of music. Many people, with closed eyes at a concert, are “watching” the music, but after all these centuries, there still exists no universally acceptable visual equivalent to music! It should exist and it will soon.

Whitney also documents his and others failed attempts at experimental film based endeavours:

    Pointing their cameras at the world, all those “symphonists” inadvertently recorded the stasis of the world, even as they filmed its busiest moments – its winds and storms and birds and water and city traffic. Those films are not symphonies, I thought, poetry perhaps, but not liquid architecture, not music.

    …wherever I pointed my camera, I failed to discover that special quality of any material possessing the controllable visual fluidity that I desired … pointing my camera anywhere resulted in recording images of somewhere. If the camera’s record is unclear, blurred by the smear of too fast panning or being out of focus, the sense of somewhere as place is simply flattened. The spatial content of an image is flattened. The eye resists the attempt to domesticate abstraction. This sort of deception hardly satisfies the eye, because the sense of being (or seeing) somewhere is so strong. The eye is the natural master of pattern recognition. The eye demands satisfaction by invoking in us strong feelings of puzzlement.

And makes the important point that, “No abstraction in my camera had the generative potential, the capability to propagate fluid patterns or especially, the liquid variability of the intervallic families of music tones.”

This is where the computer comes into play and Whitney’s argument gets interesting. Whitney sees a parallel between musical tones and generative animation. Whitney sees music as an abstract and generative form in itself:

    There is no such thing as the harmonic organization of musical tone in nature. Occasionally a stone may ring like a bell, birds pattern “song,” but there are few natural bells, fewer natural flues where the winds sound organ tones. Even the whistle of the wind is eerie and non-musical. Patterning of musical tones is a man-made reality of the aural world, universally accepted as such, but nowhere looked upon as an abstraction that has been extracted (or abstracted) out of the natural environment, nowhere regarded as a manifestation of the environment.

Whitney in deciding that music is not an abstracted picture of anything, allows for his second level of pure abstraction and generation. He focuses on three qualities applicable to both forms:

    A benchmark was reached when I began to apprehend the relationship of the three terms: differential, resonance and harmony. First, motion becomes pattern if objects move differentially. Second, a resolution to order in patterns of motion occurs at points of resonance. And third, this resolution at resonant events, especially at the whole number ratios, characterizes the differential resonant phenomena of visual harmony.

    What I knew about music confirmed for me that emotion derives from the force-fields of musical structuring in tension and motion. Structured motion begets emotion. This, now, is true in a visual world, as it is a truism of music.

Digital Harmony, the documentation of a life’s work is the most comprehensive study of generative animation and its musical potential that I have found yet. It provides some useful counterpoints when compared to Chion’s deconstruction of audio visual relations. A simple reading of Chion would state that audio is predominately temporal while vision is predominately spatial but Whitney’s musical ‘liquid architecture’ metaphor is a wonderful one. Regardless, I’m starting to side with Chion’s idea of ‘audiovisual illusion’ and perhaps through a lifetime of work and focus, Whitney has merely become a better magician.

This is not to say Whitney is wasting his time. Magic is an art form. This also doesn’t devalue his ideas of visual consonance, dissonance, harmony and disharmony. A work where consonance and dissonance is linked between audio and visual, temporally and structurally without doubt creates moments of audio-visual resonance. These ideas are particularly interesting in regards to my choice of song and visual aesthetic.

Evil and spikey.

July 24, 2007

Hand drawn element sketch no. 3 (click to enlarge)

Hand-drawn elements

July 23, 2007

First attempt at using hand-drawn elements. Took a little more care with the etching and transparency settings. (Click to enlarge)

In motion.

July 11, 2007

Sorry about the quality. I just couldn’t be bothered with setting up for rendering, rendering, loading into after effects, and rendering again….