Considering arts techniques from the broad perspective of the present, I observed that the best “computer art” did not compare
    well with lacework from Belgium made a century ago. But the computer possessed a unique capability of making very complex pattern flow. One could plan exacting and explicit patterns of action and distinctive motions as intricate as lace, but in a way no Belgian lace maker would ever imagine. – John Whitney, 1980.

This 1975 film is reportedly John Whitney’s first foray into computer graphics. Until ‘Arabesque’, Whitney used a converted mechanism of a World War II M-5 Antiaircraft Gun. Essentially a twelve-foot-high analog computer of amazing complexity; where design templates were placed on three different layers of rotating tables and photographed by multiple-axis rotating cameras.

In Digital Harmony (1980), the book that describes his life’s work, his hypothesis –

    …assumes the existence of a new foundation for a new art. It assumes a broader context in which Pythagorean laws of harmony operate. These laws operate in a graphic context parallel to the established context of music. In other words, the hypothesis assumes that the attractive and repulsive forces of harmony’s consonant/dissonant patterns function outside the dominion of music.

Whitney acknowledges that, ‘Music does not need images any more than paintings need sound’ but saw in computing, ‘a visual medium which is more malleable and swifter than musical airwaves. That medium is light itself.’

The book often communicates personal opinion rather than rigorous argument but Whitney makes some original and interesting points. It seems Whitney is not really pursuing visualisation or a tightly fused AV form. Whitney’s search is instead for abstract graphics with the fluidity, expressiveness and structural qualities of music. Whitney begins the book by highlighting the inherent spatial and visual qualities of music and damning early ‘visual music’ inventions:

    Most people visualize music as two-dimensional, with time represented by the horizontal lines and pitch by vertically arrayed symbols, as is the convention on paper. But the perception of music is not two dimensional. The ears reside at the center of a spherical domain. We hear from all around. We hear music as patterns of ups and downs, to and fro in a distinctly three-dimensional space – a space within.

    The eye, more outwardly oriented, perceives objects and events outside at the point where our eyes focus. Yet the eye enjoys design equally as well as the ear. The mind’s eye shares with the ear any inward experience of architectonic spatial constructions and would perceive them with the same pleasure, were they to exist.

    The fact is, however, that these interior fluid visual edifices hardly exist. Anyone can visualize an architectural fantasy of music dancing in the head, but manifesting in reality is another matter! Each century since Leonardo, a vision, grand and obscure as its myth, compelled one or two inventors to struggle with the pathetic inadequacies of the color organ. Twentieth-century abstract art has been a training ground for visual response to musical experience, but in the mind’s eye, architecture in motion lies at the root of our enjoyment of music. Many people, with closed eyes at a concert, are “watching” the music, but after all these centuries, there still exists no universally acceptable visual equivalent to music! It should exist and it will soon.

Whitney also documents his and others failed attempts at experimental film based endeavours:

    Pointing their cameras at the world, all those “symphonists” inadvertently recorded the stasis of the world, even as they filmed its busiest moments – its winds and storms and birds and water and city traffic. Those films are not symphonies, I thought, poetry perhaps, but not liquid architecture, not music.

    …wherever I pointed my camera, I failed to discover that special quality of any material possessing the controllable visual fluidity that I desired … pointing my camera anywhere resulted in recording images of somewhere. If the camera’s record is unclear, blurred by the smear of too fast panning or being out of focus, the sense of somewhere as place is simply flattened. The spatial content of an image is flattened. The eye resists the attempt to domesticate abstraction. This sort of deception hardly satisfies the eye, because the sense of being (or seeing) somewhere is so strong. The eye is the natural master of pattern recognition. The eye demands satisfaction by invoking in us strong feelings of puzzlement.

And makes the important point that, “No abstraction in my camera had the generative potential, the capability to propagate fluid patterns or especially, the liquid variability of the intervallic families of music tones.”

This is where the computer comes into play and Whitney’s argument gets interesting. Whitney sees a parallel between musical tones and generative animation. Whitney sees music as an abstract and generative form in itself:

    There is no such thing as the harmonic organization of musical tone in nature. Occasionally a stone may ring like a bell, birds pattern “song,” but there are few natural bells, fewer natural flues where the winds sound organ tones. Even the whistle of the wind is eerie and non-musical. Patterning of musical tones is a man-made reality of the aural world, universally accepted as such, but nowhere looked upon as an abstraction that has been extracted (or abstracted) out of the natural environment, nowhere regarded as a manifestation of the environment.

Whitney in deciding that music is not an abstracted picture of anything, allows for his second level of pure abstraction and generation. He focuses on three qualities applicable to both forms:

    A benchmark was reached when I began to apprehend the relationship of the three terms: differential, resonance and harmony. First, motion becomes pattern if objects move differentially. Second, a resolution to order in patterns of motion occurs at points of resonance. And third, this resolution at resonant events, especially at the whole number ratios, characterizes the differential resonant phenomena of visual harmony.

    What I knew about music confirmed for me that emotion derives from the force-fields of musical structuring in tension and motion. Structured motion begets emotion. This, now, is true in a visual world, as it is a truism of music.

Digital Harmony, the documentation of a life’s work is the most comprehensive study of generative animation and its musical potential that I have found yet. It provides some useful counterpoints when compared to Chion’s deconstruction of audio visual relations. A simple reading of Chion would state that audio is predominately temporal while vision is predominately spatial but Whitney’s musical ‘liquid architecture’ metaphor is a wonderful one. Regardless, I’m starting to side with Chion’s idea of ‘audiovisual illusion’ and perhaps through a lifetime of work and focus, Whitney has merely become a better magician.

This is not to say Whitney is wasting his time. Magic is an art form. This also doesn’t devalue his ideas of visual consonance, dissonance, harmony and disharmony. A work where consonance and dissonance is linked between audio and visual, temporally and structurally without doubt creates moments of audio-visual resonance. These ideas are particularly interesting in regards to my choice of song and visual aesthetic.

In ‘Untitled Iterations’, Ben Bogart writes on generative art, consciousness and perception in a straightforward and useful way.

    Consciousness is the process of turning noise into pattern. When we look around the world and listen we do not see or hear the cascade of unimaginable amounts of interacting particles, from waves of light scattering off objects to the molecules that carry the sound we hear; rather, we see a defined, understandable structure of our world. We do not see the world as science defines it. We instead collectively participate in the process of creating the structure that comprises our world.

It’s easy to forget the infinite amount of raw data the human brain filters and processes every day. Without getting into perception or philosophies of consciousness, (something I know little about) it’s clear that the human brain has an amazing ability to find pattern and structure in chaos. A perfect example – noise rock.

    Noise and pattern are not two concepts that are mutually exclusive. In fact, noise and pattern are markers at two ends of a single continuum that classify all types of structure.

For example, we could place Sunn o))) with their sub-bass drones of distortion fairly concretely and deeply into the noise side of the spectrum.

We could place a minimal, repetitive electronic number (something I again, know little about) closer to the pattern end.

Brisk, the band I’ve chosen to ‘visualise’ won’t settle on its place on the spectrum. ‘Hell or High Water’ dances up and down it at will. Six instruments all screaming at once, generally in dissonance, turns to two guitars playing a pleasing melody. Sometimes with, and sometimes without a typical song (verse/chorus) structure. Generally adhering to the laws of rhythm, sometimes barely, sometimes with overlapping time signatures and complex polyrhythms. It’s possible, for the few who are entertained by this toying with chaos that the kick is in decoding the chaos. Or to use a cheesy analogy; possessing the correct, obscure codec.

From Australian academic Paul Carter’s ‘Material Thinking’:

    We pointed out that, in ancient Greek thought, chaos did not have its present-day meaning. It signified ‘the yawning, or gaping open of time and space to permit creation … Greek chaos imagines the interpenetration of lines, a crossing that does not cancel out but mutually transforms.

Bogart considers, ‘the vast area between noise and pattern as chaos.’ So, this is what I’m getting at:


Interesting is the computer’s struggle to successfully interpret and create pattern from raw audio data. Picking up even the simplest of patterns; a beat, even in wild, heavy music is a straightforward task for any person but beat detection is still, to this day, an unreliable technology.

Compared the rich abilities of the ear and brain, asking a computer to visualise music seems a strange concept. The question is – can applying structure, systems and process to music full of confusion, noise and chaos reveal new qualities? Enhance this process of creation, creating a new space of interpretation? Or at least create a worthwhile viewing experience? I’m hoping so in the context of the video clip.

I’ve just installed a bigger hard drive in my Xbox and put most of my music on it. XBMC comes with a variety of visualisers and when faced with the choice of which one, I chose none. It’s not the way I generally want to experience music. My media centre screensaver is now a black screen…