Designing
an Instrument to Perform
Abstract Animation in Real-Time
Fred Collopy, PhD
July 21, 2003
The designer of a visual instrument that is oriented to improvisation
must balance the demand for expressive utility with the need to produce
an instrument with a playable interface. My design goal has been to
create instruments that will allow about the same level of expressiveness
as a piano, guitar, or trap set and that make similar learning and interface
demands.
The problem was clearly articulated at least as early as 1930, by Adrian
Klein who wrote:
Within the prescribed limits of space, it is required that everything,
colour, form, and direction of movement shall be subject to any conceivable
modification….Somehow or other, we have got to treat light, form,
and movement, as sound has already been treated [Klein 1930, p.
195].
Around the same time, Thomas Wilfred argued that the same basic dimensions—color
form and motion—should serve as the basis of the new art he referred
to as lumia. While most of the experimental work until that time had
been largely related to color, Wilfred felt that form and motion were
the more important two [Wilfred 1947, p. 252].
Imager uses Max, a MIDI control environment that runs on Macintoshes,
to provide players with interfaces to control color, form, and motion.
My own performance environment uses such interfaces as the ZenDrum,
the Tactex controller and the Kurzweil ExpressionMate, along with foot
pedals, to play with colors, shapes, and movements rhythmically.
My initial design goals included creating an instrument that would allow
me to produce lumia that move like music, that approach the depth of
musical compositions, and whose individual frame’s would stand
alone as images. In addition, Imager had to be controllable by a single
player and permit improvisation in real time so that the player could
respond to music while listening to it.
So, what is required in order for an instrument to be expressive? The
cybernitician Gordon Pask posited that aesthetically potent environments
share four characteristics. First, they provide sufficient variety to
provide novelty. Next, they provide forms that can be interpreted at
various level of abstraction. They provide cues to guide learning. And
finally, they are responsive, engaging the player in discourse.
Getting sufficient variety is pretty easy. Each of the three factors
identified by Wilfred is itself made up of many dimensions. Color, for
example, is made up minimally of hue, saturation, and value. By combining
a number of colors so defined, one quickly has thousands or millions
of possibilities for color combinations at any moment. Indeed, the problem
once one begins implementing a graphic model based around color, form,
and motion, is not coming up with enough variety, but rather managing
the rapidly expanding sources of variety. As Brian Eno observed about
music synthesizer design, the temptation is to head "towards
giving you more and more options. It’s not more options that you
want. It’s more useful options [Imaginary Landscapes]."
Historically, we live in a propitious time for providing forms that
can be interpreted at various levels of abstraction. The modern abstract
artists have created an impressive body, both of artworks and of written
works, that address this very point. From Kandinsky’s Point and
Line to Plane, through Klee’s Notebooks and Gyorgy Kepes’
Language of Vision, they have provided quite detailed descriptions of
the use of form at various levels to achieve a variety of expressive
goals.
An issue that interested Kepes, for example, relates to achieving
"rhythmical patterning of the picture surface." He proposed
that rhythmic patterns can be achieved at several levels. On one level
are the relationships of objects subdividing the picture plane. A higher
level of rhythm is reached as the optical units are related to their
virtual movement to and from the picture plane. "Finally we
might have orderly changes or repetition of more complex configurations
of visual experience; rhythmic order of tension and repose, concentration
and rarefaction, harmony and discord". That these are similar
to ideas explored by composers was not lost on Kepes:
Music suggests an excellent analogy. A musical unit played by an instrument
is repeated contrapuntally on other instruments, on the strings, on
the brasses, on the woodwinds, even on percussion instruments. Each
plastic unit with its specific sensory quality echoes the previous one;
light, dark, color, shapes, forms, all mutually help one another, one
taking over the movement where another stopped, leading toward complete
unity [Kepes 1944, p. 54] .
The problem of providing cues to guide learning is at the very center
of modern instrument design. In a recent review of Pinch and Trocco’s
book on the history of the synthesizer, I wrote:
For the attentive designer, there are lots of other lessons in this
book. They found that one of the things that worked best about the control
panel for the Minimoog was the way in which knobs were staggered. "It
turned out that having things not all in military formation made it
a lot easier for someone to find a control." Further, tactile
controls like rocker switches permitted players to find their way around
using feel [p.225]. And it is important in a musical instrument for
the musician to be able to dynamically alter the sound in very small
ways. "For many musicians, it is the pitch wheel on the Minimoog
that enables them to make the instrument come alive. By bending a pitch
or adding vibrato, a note can be given that special personal touch that
violinists and guitarists find so important [p. 228]." These
and similar lessons will, I think, have wide application in the design
of instruments, even instruments that are used for purposes other than
making music [Collopy, 2003].
Choosing the dimensions that the player controls, and determining such
things as how presets are recalled and sequences are defined are problems
the designer must solve both at an instrument (general strategies) and
at an individual performance level (e.g., what exactly will the peddle
control during this particular piece?). Fortunately, modern synthesizer
design environments like Max make it possible to leave a lot of this
to the performer. In the design of Imager, I have provided a general
constructivist model of form and movement, along with a painterly color
model [Collopy 2000]. These are presented in an environment that allows
the user to configure and interact with them using whatever controllers
she wishes.
Pask’s last issue, engaging the user in discourse, is the most
complex. What makes an instrument engaging is an empirical question.
Given my experience to date, the instrument must be physical. I like
the Zendrum because it is possible to locate controllers in the dark,
remember their particular meaning by physical association and move easily
while playing it. It responds to speed and force equally well, and it
is easy to map rhythmic events in much the way that a drummer uses a
kit to do that same task (indeed it was designed for drummers). I like
the ExpressionMate because it permits me to bend or adjust a color or
movement after the initial attack. This attack and adjust strategy is
commonly employed by musicians and allows one to benefit from feedback.
No single instrument will capture all of the aspects of visual performance
that might interest audiences, any more than a single sonic instrument
does. A model based around color, form, and motion does, though, provide
a rich collection of visual elements that can be combined in ways that
reflect much of the expressiveness of music. Learning to control the
many dimensions of image is the challenge before us. The excitement
implicit in that is not unlike that which must have greeted the inventors
of the earliest rhythm instruments. So many possibilities; so much yet
to learn and develop.
References:
Collopy, Fred, "Exploring the Bourndaries of Sound: A Review
of Trevor Pinch and Frank Trocco’s Analog Days," forthcoming
in Metascience, 2003.
Collopy, F., "Color Form and Motion: Dimensions of a Musical
Art of Light," Leonardo, 33 (2000), 355-360.
Kandinsky, Wassily, Point and Line to Plane (1926), in Kenneth
C. Lindsay and Peter Vergo, eds., Kandinsky: Complete Writings on
Art, New York:
Da Capo Press, 1994, pp. 527-699.
Kepes, Gyorgy, Language of Vision, Chicago, IL: Paul Theobald,
1944)\,.
Klee, Paul, The Thinking Eye, New York: George Wittenborn, 1961.
Klein, Adrian Bernard, Colour-Music: The Art of Light, London:
Crosby Lockwood & Son, 1930,.
Pask, Gordon, "A Comment, a Case History, and a Plan,"
in Jasia Reichardt [ed.], Cybernetics, Art and Ideas, Studio
Vista London, 1971, pp. 76-99.
Pinch, Trevor and Frank Trocco, Analog Days: Tee Invention and Impact
of the Moog Synthesizer, Cambridge, MA: Harvard University Press,
2002.
Wilfred, Thomas, "Light and the Artist," Journal of
Aesthetics and Art Criticism 5 (June 1947), pp. 247-255.