Dreams of Computer Music: Then and Now
Author(s): F. Richard Moore
Source: Computer Music Journal, Vol. 20, No. 1 (Spring, 1996), pp. 25-41
Published by: The MIT Press
Stable URL: http://www.jstor.org/stable/3681267
Accessed: 28/01/2010 22:31
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=mitpress.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

The MIT Press is collaborating with JSTOR to digitize, preserve and extend access to Computer Music
Journal.

http://www.jstor.org

MooreF.Richard
Department of Music, and Center for Researchin Computing and the Arts
University of California, San Diego
La Jolla, California 92093-0037, USA
frm@ucsd.edu

of Computer
DreamsNow
Music-Then andobservations, first published in 1514, that led him
Computer music was first created nearly 40 years
to suggest that the earth revolves around the sunlaboratories
research
inanduniversities
byago
than vice versa, an observation that directly
ratherformusic.
instrumentsofpeople dreaming magicalcontradicted the teachings of the Roman Catholic
These instruments were like genies in that they
church. In 1633, just a few years after Bacon's
could grant wishes-musical wishes-to anyone
death, the church condemned Galileo Galilei towho possessed the required "bottle" of knowledge
about computers and music. These dreams also had life in prison on a charge of "vehement suspicionof heresy" for teaching Copernicanism. Francis
a correspondingtechnology in the real world, but
Bacon'sphilosophical works were held in high esmore than anyone would have daredto wish at theteem by many important scientists, including Robbeginning, the technology of computer music wasert Boyle, Robert Hooke, Isaac Newton, andtheareitselfto provepossiblyunique; computers
Thomashave
Hobbes, so much so that in the 18th cenactually gotten
only thing in existence that
Voltaireand Denis Diderot considered Francis
cheaper over the years while simultaneously becom- tury,
bethefather of modern science.
Bacon
toing more powerful. Now nearly everyone whoIn
two years before his death, Francis
about
wishes it has the means to create computer mu1626,
The
Bacon
wrote
New Atlantis. This is, ostensibly,
sic-but do they have the dreams?
the first work of what we today call science fiction.
Bacon was both a practical man and a dreamer.In
The New Atlantis, he described a society stumbled
andMusic
ScienceFiction,
Precursors,
upon by shipwrecked sailors, a kind of island utoDream:PowerOverSoundand Music
pia somewhere in the Atlantic Ocean. As the sailors were shown around the wonders of this place,
Reality:PurelyImaginary
Bacon describes many aspects of nature being studied. One of these places of study was described
As strange as it may sound, the idea that observawith the following words.
tion should be the basis of knowledge was not alandThe
great philosopher
ways commonplace.
We have also our sound-houses, where we pracstatesman Francis Bacon gave much impetus to thetice and demonstrate all sounds, and their gendevelopment of modern science by suggesting thateration. We have harmonies which you do not,observation of nature should be the basis for knowlof quarter-sounds,and lesser slides of sounds.
edge. At the time of Francis Bacon, the church, theDiverse instruments of music, likewise to you
state, and the nobility all had another theory; they
unknown, some sweeter than any you have; tobelieved that authoritative statements by highgether with bells and rings that are dainty andranking officials should be the basis for knowledge
sweet. We represent small sounds as great
and truth, rather than observations that could be
and deep; likewise great sounds extenuate andmade by just about anybody.The real revolution of
sharp;we make diverse tremblings and warscience was the replacement of authority with obof sounds, which in their original areblingsservation as the basis for truth. In fact, the very
entire. We represent and imitate all articulateterm revolution comes from Nicolaus Copernicus's
sounds and letters, and the voices and notes ofbeasts and birds. We have certain helps which
set to the ear do further the hearing greatly.We
Computer Music Journal,20:1, pp. 25-41 Spring 1996
? 1996 Massachusetts Institute of Technology.
have also diverse strange and artificial echoes,
Moore

25

I

reflecting the voice many times, and as it were
tossing it: and some that give back the voice
louder than it came; some shriller, and somedeeper;yea, some renderingthe voice differingin the letters or articulate sound from thatthey receive. We have also means to convey
sounds in trunks and pipes, in strange linesand distances.
In this small paragraphfrom The New Atlantis,
Francis Bacon described much of the modern fieldof computer music almost perfectly. In 1624, Bacon
was already dreaming of fundamental power oversound and music. What is remarkableis not somuch the dream itself, for Bacon was simply trying
to list everything that he could imagine doing withsound, but that it would take hundreds of years toachieve most of it. Alexander Graham Bell invented
the "trunks and pipes" and "strangelines" to convey sounds over distances-otherwise known as
the telephone-only in the 19th century. Hearing
aids, which Bacon described as "certain helps
which set to the ear do further the hearing greatly"
are still a matter of intensive research, thoughmuch has been achieved recently through the useof neural implants and microelectronics. Modern recording studios would be lost without their "diverse strange and artificial echoes," and computer
music techniques like the phase vocoder allow us
to "representsmall sounds as great and deep; likewise great sounds extenuate and sharp,"among
other things. The important thing is that Bacon
imagined-he dreamed-of being able to "practiceand demonstrate all sounds, and their generation."
As with Leonardoda Vinci's visions of flightthrough the air, and Jules Verne'svisions of travel
through space, without dreams, reality could not
possibly follow.
Dreams in this sense are a pure combination ofdesire and imagination without necessity of themeans to achieve something in reality. Dreaming isimagining what is desirable without having toworry about how to achieve it. The purpose of
dreaming is therefore to discover what we wish to
do. Only when we awaken do dreams become
attached to the typical clauses and conditions starting with "if only...."
26

I wish now to consider the development of the
field of computer music in terms of the dreams ofseveral of those who were active in its establishment. While Francis Bacon was dreaming in 1624
of doing what many now take for granted,I consider the real history of a separate field called computer music to have started in the mid-1950s withthe pioneering work of LejarenHiller and Max Mathews. Obviously such a treatment as this must be
somewhat speculative, for most of what remains is
the work of such people rather than records of their
dreams. But there is ample evidence that dreamthey did, and I suggest that it is these dreamsevery bit as much as the work-that form the basis
for what is now an established and increasingly
important field of inquiry.
Musique Concrete and Elektronische Musik
There were many other precursorsto computer music. Thaddeus Cahill's 200-ton Dynamophone (alsocalled the Telharmonium),invented around 1906,
toured the United States in several railroadboxcars,
astonishing and delighting audiences with its amazing ability to create musical sounds through theuse of huge electromagnetic inductors. Leon Theremin invented his famous mood-altering instrument
about 1924, around the time of the Ondes Martenot used by Olivier Messian and the Trautonium
used by Paul Hindemith. Sometime in the 1930s
the great conductor Leopold Stokowski wrote, "I
can see coming ahead a time when the musician
who is creator can create directly in tone not on
paper."In the late 1940s, with the advance of magnetic wire and tape, it became possible to manipulate sounds on recordingsin a manner similar to
the manipulation of images on photographicpaper.
Such manipulations gave rise to the school of musique concrete by the Groupe de Recherches Musicales at the national radio station of Francein theearly 1950s, with the work of Pierre ShaefferandPierre Henry and others. Musique concrete was tobe based on the "concrete" sounds of nature rather
than the "abstract"sounds of traditional musicalinstruments.
Also in the early 1950s, at the Cologne Radio StaComputer Music JournalFigure 1. John Cage andLejarenHiller at the console of the ILLIACcomputer they used in theircollaborative piece
HPSCHD, realized at the
University of Illinois in Urbana in 1968. (This photo-graph originally appearedin Source:Music of the
Avant-Garde2(2), 1968,
and is reprinted with permission of the Source Archives, Music Library,University of North Texas,
Denton, Texas, USA.)tion in Germany, Herbert Eimert and Robert Beyer
began to generate sound waves from their elemental building blocks-sine
waves-according to thefamous mathematical theory of Jean-Baptiste Joseph Fourier. Because the sine waves were createdelectronically, the result, as practiced by composer
Karlheinz Stockhausen and many others, was collectively known as elektronische Musik. In the
United States, Vladimir Ussachevsky and Otto
Leuning produced the first concert of what theycalled "tape music" at the Museum of Modern Art
in New York City on 28 October 1952.

Music:TheFirstDecade,1955-1965
ComputerDream:Mindof Science/Heart of Art
Reality:Existence Proofs
Hiller and the ILLIAC
True computer music did not begin, however, untila young research chemist and composer named
Lejaren A. Hiller began to investigate ways to apply
a process known as stochastic modeling to understanding the process of musical composition. Stochastic methods had been successfully used for molecular modeling in chemistry for some time, andcomputers were a perfect vehicle to exploit suchtechniques. Computers of the mid-1950s were notyet capable of sensing and producing arbitrarysounds. They were, however, capable of the highspeed calculations that made otherwise laborious
stochastic modeling a practical reality. Because heworked at the University of Illinois, Lejaren Hiller
had access to one of the earliest examples of what
we would now call a supercomputer: the ILLIAC.
He is shown in Figure 1 standing at the console ofthe ILLIAC with his collaborator John Cage.
Lejaren Hiller's basic idea was simple: have thecomputer generate a list of random numbers representing anything we like about music, such as
pitches, rhythms, dynamics, etc., then program aset of "rules" by which one choice would be allowed to follow another. For example, if we wish tocompose a melody, we might number the pitches
on a piano keyboard from 1 to 88, then (arbitrarily)

set the first note to number 40 (middle C). Given
that first choice, the rules might disallow any skip
greater than an octave, so if the computer chooses
a second note at random less than 28 or greaterthan 52, the choice would be rejected and the computer must try another, and another, until a choice
is found that doesn't break any of the "rules." Sucha technique is called a "Monte Carlo method." The
hard part of this procedure is coming up with rulesthat are both possible to obey and that give resultsthat sound musical. This is much more difficult
than it may appear at first glance; so difficult, infact, that Hiller tried several alternatives to thewhole procedure, most notably those based on themathematics of Andrei Andreevich Markov. Using
Markov's techniques, one examines any set of existing music, perhaps that of a composer generally
regarded to be reasonably good, such as Mozart.
The basic idea of Markov modeling is as follows.
Looking through one or more pieces by Mozart,
one can compute the probability that the note
middle C will be followed by D, or E, or E-sharp, or
F-sharp, etc. Once these probabilities are determined by analyzing existing music, the computercan be programmed to produce music that has exactly the same statistics, yet is different from theanalyzed music. If the statistics are based on Mozart, the computer would be more likely to follow
middle C with D than, say, F-sharp. Such techniques can be extended to pairs of notes, or triplets
Moore

27

of notes, and so on, yielding ever more restrictive
probabilities, while still allowing some freedom of
choice. Eventually one can imagine doing suchthings as feeding in Beethoven'snine symphonies
and asking the computer to compose the Tenth
Symphony that Beethoven might have written if he
had had the time.
LejarenHiller's dream was not only to compose
music with a computer, but to understandboth
how a composer thinks and even what makes one
piece of music good and another poor. Because suchwork had never been attempted before, the dreams
clearly (and typically) overshot any possibility ofrealization. This problem turned out to be muchharderthan anyone realized, before anyone had attempted to solve it, as problems often do. Nevertheless, the dream was no less than grandiose-deep
insight into the intelligence of the human mind, in
this case, the musical intelligence. The computer
allowed an experiment that most would say failed(if we listen to the resulting music), but I contend
that the experiment succeeded in producing one of
the most brilliant dreams ever had about music.
The dream exceeded even those of Francis Bacon,
whose brilliant imagination stopped short of including scientific investigation of the process of humanmusical thought. Sound example 1 (CD index 2) isthe second movement of LejarenHiller and Leonard Isaacson's1957 piece Illiac Suite for Stringare embodiments of objective research results.
They are laboratorynotebooks. Sometimes theresults have surprisedus because a compositional routine seems less effective than expected, sometimes more so. If I were to delete
everything that disturbs me aesthetically, I
would falsify the research record. So, for thepresent, my objective in composing music bymeans of computer programmingis not the immediate realization of an aesthetic unity, but
the providing and evaluation of techniques
whereby this goal can eventually be realized.
For this reason, in the long run I have no personal interest in using a computer to generate
known styles either as an end in itself or in order to provide an illusion of having achieved avalid musical form by a tricky new way of stating well-known musical truths.

Mathews: MUSIC I (1957)

Even in 1876, Alexander Graham Bell knew thatsound pressure waves could be converted into analogous electrical signals, and vice-versa, throughthe use of microphones and loudspeakers-that's
why we call them analog signals. The advantageof
electrical signals is that they travel much faster
Quartet.
through wires than sound does through air,
Hiller's
music
sound
Granted, Lejarenmay
pretty allowing electrical signals to travel much further inthatmake it good science? Werea small amount of time; hence the name tele (farbad, but does
Hiller and his associates actually using the scienoff) plus phone (sound).By continually reconnecttific method suggested by Bacon and Galileo anding a network of wires efficiently, the telephone system has probably done more to shrink the size ofothers, or were they simply playing with the new
the world than the airplane.When electrical signals
technology of computers? The following words,
written by LejarenHiller in the record liner notes
travel through wires, however, they pick up theelectrical equivalent of dust and dirt, called noise.
about the Illiac Suite, addressthis exact question.
If the wire is long enough, the signal becomes so
contaminated that it is no longer possible to underShould a person listen to these two pieces asstand the original sound. Practically the entire
he might "ordinarymusic?" Yes, I think, butwith this important qualification: They are
story of telecommunications is about ways to premuch more didactic than expressive compared vent, or at least to slow, this inevitable noise degrato most music. These pieces are truly experidation of signals as they are transmitted from one
mental because they are concerned with reveal- place to another.
Digital technology was developed in so manying process as well as being final product. They

28

Computer Music Journal

ways and places that there is no particularperson
identified with its origin. By the 1950s it was already known that digital signals could be transmitted from one place to another far more reliably
than analog electrical signals such as those in Bell's
original telephone. In a digital signal, information
about, say, a sound, is encoded as a series of binary
digits, or "bits,"each of which can have one of only
two possible values ("on/off,""yes/no," "1/0," etc.).
When a bit is transmitted electrically from one
place to another, it picks up noise just like any signal. The receiver,however, only needs to be able todistinguish whether a "yes" or a "no" was originally transmitted to have essentially perfect knowledge of the original signal. As long as it is not too
powerful compared to the "on/off" part of the signal, the noise can be eliminated completely, making digital transmission and recordingfar superior
to analog for many purposes. This fact was not loston the telephone company,and in the mid-1950s a
young telecommunications engineer and amateur
violinist named Max Mathews was hard at work at
AT&T Bell Telephone Laboratoriesinvestigating its
potential. At that time the telephone company wasinterested only in speech. From a technical standpoint, however, music is just like speech with a few
more frequencies-both are structured sound.
Using a special device called a digital-to-analog
converter,a computer can control the movementsof loudspeakers with a precision never before available. Max Mathews and others reasoned that because computers could be used to investigate
improvements in telephone speech analysis, processing, and synthesis, they could also be used to
improve the analysis, processing, and synthesis ofmusical sounds. Mathews therefore wrote the firstmusic-synthesizing programsin about 1957, turning the computer into a new kind of musical instrument that was capable, in principle at least, of producing any sound that could come from aloudspeaker.Along with LejarenHiller's efforts touse computers to investigate musical intelligenceand composition, the field of computer music wasnow born. The liner notes of Music from Mathematics, the first published recordingof computer
music, explained it as follows.

On this recording,we illustrate another advancement in the realm of tools available tothe music-maker: the computer and thedigital-to-sound transducer.This new "instrument" combination is not merely a gadget or acomplicated bit of machinery capable of producing new sounds. It opens the door to the
exploration and discovery of many new andunique sounds. However, its musical usefulness and validity go far beyond this. With
the development of this equipment carriedout
at the Bell Telephone Laboratories,the composer will have the benefits of a notational system so precise that future generations willknow exactly how the composer intended hismusic to sound. He will have at his command
an "instrument" which is itself directly involved in the creative process. In the words ofthree of the composers whose works are heard
on this recording,man'smusic has always been
acoustically limited by the instruments on
which he plays. These are mechanisms thathave physical restrictions. We have madesound and music directly from numbers, surmounting conventional limitations of instruments. Thus, the musical universe is now circumscribed only by man'sperceptions andcreativity.
Especially given this vision of the computer as acompletely general musical instrument, perhaps
the most obvious question one can ask about his
original computer music sound-synthesis programis, How did it sound? The answer is simple: inmusical terms, it sounded rather terrible. For one
thing, the audio quality of 1950s digital audio technology was poor by today's standards(as today's
technology will seem soon enough). Also, the musical quality was extremely stilted and unnatural dueto the simplistic means for specifying the musicalsound, which left out the performeraltogether. Finally, expectations placed on the output of earlycomputers were often unrealistic. Who would haveexpected that the first musical sounds producedby
a multimillion-dollar example of our most advanced technology would sound more like a child's

Moore

29

first violin lesson than the pinnacle of musical evolution? Despite its potential to act as a completely
general musical instrument, the computer's firstmusic lesson sounded no better than yours or
mine-and (not knowing how yours sounded), perhaps a lot worse.
For the remainder of computer music's first decade, most work occurred in a few laboratoriesand
universities where expensive and hard-to-use computers were available for this purpose. Programs
were typically written in assembly language using
punched cards or paper tape. To run a single computer programoften took a full day or more, andthe results were usually disappointing, assuming
the computer itself worked properly (which wasoften not the case). Printed output was usually onlarge fan-folded sheets of computer paper.If thecomputer also produced a sound, it was in the formof a large computer tape containing millions of
numbers representing the sound in digitized form.
If the sound-synthesis programsucceeded in producing the tape (which one often determined bywatching the tape move while the programran), itthen had to be carriedto a separate machine specially designed to feed these numbers at high speed
through a digital-to-analogconverter,to convert the
numbers into voltages. This signal could then beprocessed and fed into a loudspeaker (in the mid1960s, computer music researchersfrom Princeton
University had to drive to Bell Labs-a 150-km
round trip-about once a week with one or two
such tapes). One always had to remember to turnon the reel-to-reel tape recorderbefore starting theconversion into sound, or the whole effort might belost if the computer tape broke. The results typically fell into these categories: (1) silence (the mostcommon result), (2) a long, loud noise (also a common result), (3) a patterned noise sounding nothing
like expected (usually the result of a programming
error),or (4) a thin, weak, pale, sickly musicalsound (which happened in perhaps one of every fiveattempts). The sounds may have been sickly andpale, and the work may have been laborious andpainstakingly slow, but these results proved thatcomputer music existed. Now all it needed was
improvement!

ComputerMusic:TheSecondDecade,1965-1975

30

Computer Music JournalSoundsDream:Musical
Truly
Reality:Research

Risset and the Trumpet
It is difficult to make music on a musical instrument that does not sound like a musical instrument. Therefore, one of the first problems to be
tackled seriously in the field of computer music
was that of synthesizing sounds with truly musicalcharacteristics. Two basic approachesto this problem are either to imitate the sounds of traditionalmusical instruments or to synthesize completely
new sounds that are musically satisfying. Both ofthese problems are difficult, but for different reasons. Imitations are always tricky, because even theslightest discrepancywill alert the listener to try tofind others. Elsewhere I have called this the "oleomargarineeffect" by analogy with another famous
example of how substitutes rarely work perfectly.
The other approach,that of synthesizing new
sounds with musical properties, is also difficult
because listeners have such ingrained habits ofthought regardingwhat sounds are "musical."Despite years of the "liberation of noise" and othercampaigns by modern musicians, most people simply have different aesthetic expectations when they
hear traditional musical instruments, such as violins and guitars, as opposed to sounds never before
encountered. A kind of "categoricalperception" occurs in which some sounds are placed into the "musical" category while others (such as speech andnoise) are not. Some interesting sounds have been
discovered,however, most notably an entire category of "acoustical illusions," named for their ability to "fool the ear" in ways analogous to the manner in which optical illusions "fool the eye."By the mid-1960s, computer sound synthesis had
advancedto the point where musical researchers
could begin to address the problem of musical tone
color, or timbre. Based on the assumption that it
would be necessary to understand what makes thesounds of traditional instruments "musical" to be-Figure2. Jean-ClaudeRisset demonstrating a trumpet tone synthesized bycomputer at Bell Telephone Laboratoriesin
1965. (Photographcourtesy of Bell Telephone
Laboratories.)

ally any musical sound, but musicians are almost
always after a particular set of musical characteristics, usually a matter of perception. One of thethings early computer music researchers quickly
learned was that our knowledge of the relationshipbetween the physical and psychological characteristics of sound-psychoacoustics-is
about as limited as our knowledge of every other aspect of theIhuman mind. Looking at waveform plots, for ex91
ample, we soon learned that fairly large-looking differences between two sound waves don't necessarily
correspond to large-sounding differences, and the
opposite is also true (apparentlyhuman beings canboth see and hear because we get truly different
information about the environment from each ofthese senses).
Jean-ClaudeRisset's investigation into trumpet
sound led to a key insight into the sounds of virtually all brass musical instruments. The physical operation of brass instruments causes the amplitude,
or strength, of the sound to build up, reach a moregin with before new ones could be invented, JeanClaude Risset, a physicist and composer working
or-less steady state, then die away at the end, all inwith Max Mathews at Bell Labs, investigated themore-or-less exponential fashion (this is a common
sound of the trumpet. What he learned confirmed
characteristic of many physical systems). Risset
what had been suspected for some time-that
determined that the bandwidth of the sound-esthere is not a single recipe for the entirety of any
sentially the number of harmonics-grows inmusical sound. Rather, the recipe-the relative pro- direct proportion to the amplitude of the sound.
While that simple fact is not all there is to brassportions of sine waves that are added together toform a complete sound-must change in characinstrument sounds, it is so important that virtually
teristic ways from moment to moment within aany sound with this characteristic will sound brasseven
from
the
to
the
end
of
a
sound,
beginning
like, or at least more like brass than any other typeIn
note.
technical
the
of
the
of instrument.
singleterms,
spectrumsound waveform must vary over time in the correct
Despite its limitations, Jean-ClaudeRisset's
The
for
be
"brasstone hypothesis" was a real breakthrough,way.
equivalent problem
cooking might
to try to invent a food that tastes and feels like a
and an encouragement that at least some imgreen apple when you bite into it, then graduallybe- portant characteristics of musical timbre could becomes more like roast beef as you chew it, thenunderstood. Risset celebrated his discovery by featastes and feels like port wine as you swallow it.
turing his synthetic trumpet in computer-generated
That is essentially what all musical sounds do, exportions of the incidental music he wrote for Little
cept that taste and feel are replaced by timbre, theBoy, a play that tells the true story of the pilot who
mixtureofflew the plane that droppedthe atomic bomb on
time-varying
frequency componentsthat fuse to form a single musical sound. Figure 2
Hiroshima. The flight crew had named the bombshows Risset at Bell Labs in 1965 presenting his"Little Boy."Risset used his computer-synthesizedwork on trumpet tones.
trumpet to accompany sequences in the play in
Max Mathews's synthesis programsallowed
which the pilot dreamed about happier times in ajazz club he had frequented. The pilot-who later
people to specify the physical properties of virtu-

-t

Moore

31

I

went insane-was tormented by nightmares inwhich he "became" the bomb as it fell endlessly
toward its target. Here Risset found an eloquent
application for a computer-synthesized acoustic
illusion. Based on recently understood principles ofpsychoacoustics, he created a long tone whose
pitch gradually descends foreverwithout getting
any lower (like the barberpole stripes that appear
to go down foreverwithout getting anywhere) to accompany the pilot's dream of endless falling. Sound
example 2 (CD index 3) is an excerpt from JeanClaude Risset's 1968 Suite from Little Boy.

of which is that it requires intimate familiarity
with mathematical physics. Another problem withphysical modeling is that it is difficult even for thecomputer, because an enormous amount of computation must usually be done to obtain even modest
results. This fact relegates most physical modelingto the exotic realm of supercomputers.But the advantages of physical modeling are also very real,especially if one is interested in synthesizing thesound of traditional musical instruments. Pierre
Ruiz's early work in this field was so demanding
that, despite obtaining the best synthetic violin
tones made at the time, he left the field of computer music altogether, as have many others when
Ruiz and Physical Models
they discovered the true cost of some dreams. One
might have expected someone with Ruiz's mathebasedonundermatical abilities to investigate other issues in sciRisset's
successwasJean-Claude
the
of
brass
instruence or technology, but such was not the case. He
characteristics
standing
physicalwent
into experimental theater.
Max
Mathews's
ments and human perception.
programsallowed users to specify the physical characteristics of virtually any musical sound. Lejaren
Hiller borrowedstochastic methods used in model- Music V and GROOVE
ing molecular physics to model the music composiWhen I first arrivedat Bell Labs in 1967, aftertion process. A young graduatestudent of Hiller's,
Pierre Ruiz, working at the University of Illinois
studying music composition and performance atand later at Bell Labs with Risset, took computer
Carnegie-Mellon University and the University ofmusic synthesis a big step further. Physicists have
Illinois, one of my first jobs was to write the comknown for some time how to write differential
puter programthat processed computer tapes containing sounds. I also became deeply involved inequations that model the behavior of physical systems, including those of musical instruments. The Mathews's last sound synthesis project, called Music V because it was the fifth version of that protrouble is that such equations quickly grow more
in
solve
a
than
can
any person
general gram he had designed over about ten years. While
complicatedMusic V embodied many ideas and technical admathematical way. The computer, however, can bethevances of interminable interest to programmers,tosimulate
moment-to-moment
programmed
results of such equations even when they cannot be perhaps its most important feature was that it was
the first such programwritten in a portable prosolved generally.If the equations correctly model
the behavior of the musical instrument, then thegramming language-in this case FORTRANIVbe
able
to
the
rather than machine-specific assembly language.
shouldresultingcomputercomputeThis allowed Music V to be carriedfrom one comsound. Ruiz applied this approach,called physicalputer to another and still stand a fairly good chancemodeling, to the violin.
of working. As a public service to the new field ofWhile this may be said of all computer musicissound-synthesis techniques, physical modelingcomputer music, Bell Labs gave the Music V proin
whichmathematicalatruly techniqueequations gram away for free, primarily to universities that
become musical instruments. The computer, in ef- owned suitable computer systems such as IBM
fect, "plays"the formulae in ways that no human7094s, Honeywell/GE 645s, and Control Data
6600s. Keep in mind that a typical computer sysever could, thereby obtaining sounds. There are
tem was the size of a small house, and cost many
many problems with this technique, not the least
32

Computer Music JournalFigure3. The IBM 650 computer (first delivered in1954). (From S. Augarten,
1984. Bit by Bit-An Illustrated History of Comput-

ers. New York:Ticknor
and Fields, p. 210. Reusedby permission.)

Figure4. The highperformance CDC 6600
computer of 1963. Other
examples of this generation of machines were the
IBM 7090 or 7094 and the

GE 635/645, on which
Music V was developed.
(FromAugarten 1984,
p. 219. Reused by permission.)

Figure3
millions of dollars. The Music V program was distributed in several boxes, each having a capacity ofroughly 2,000 punched cards (don't worry-theywere numbered in case someone dropped them!).
Music V was a flexible and powerful tool for bothmaking computer-synthesized music and for studying the relationship between the physics of soundand its perception. It was also clear that the modus
operandi of Music V, which consisted of typed descriptions of one or more virtual musical "instruments" in a special language designed for that purpose, followed by a typed list of notes to be playedon these instruments, mimicked well the concepts
embodied in traditional written music. The program treated its input as the specification for asound, which it then proceeded to synthesize.
What this paradigm leaves out is anything in particular having to do with a performer's contribution
to music. When a performer plays a piece of music,
a great deal happens, most of which is poorly indicated-if at all-in a written score. There are usually noteworthy differences between reading thescript for a play and seeing it performed on stage orscreen. In both theater and music, these differences
are called performance. The long list of names atthe end of a movie testifies to all the people whohave added something to the book, which usually

Figure4

exists before the movie-making process even begins. A performance is filled with nuances and inflections associated with such things as the general
historical period and style of a piece, the performer's personal "interpretations,"and the physicalcharacteristics of the listening space. In the case ofmusic, the physical characteristics of an instrument also contribute a great deal to the resulting
sound. Without these refinements, a piece of music
usually tends to sound flat and lifeless, as when a
Moore

33

primitive computer speech synthesizer reads a love
poem.
There are two general approachesto solving theperformanceproblem in computer music. One is towrite more complicated scores that are designed to
specify the precise final sound of the finished music-a much more complex task than writing a traditional musical score. The second approachis toallow computer music to be played in the traditional manner with keyboardsand other controllers, thereby altogether avoiding the necessity of understandingperformance.In other words, we caneither try to understand performance or simply"capture"it. While some composers have been up
to the first task, most are not, partly because many
composers are not themselves performers,but
mostly because we do not necessarily understandwhat happens when performersperform. Because
it involves additional understanding, the first approach is a matter of further research, even today.
The second approachavoids the necessity for understanding by simply capturing aspects of performance and using them directly. As we will see
later, this "capturing"approachhas also been applied successfully in the area of sound synthesis.
As a pianist and percussionist as well as a composer, I was equally interested in the computer music problems of sound synthesis and performance.
The only trouble was that by the late 1960s there
was no performance capability in computer music,
which suggested to Max Mathews and me that weshould work on one. The result was a system wecalled GROOVE,for Generated Real-time Operations On Voltage-controlledEquipment.
At that time, there were no digital computers capable of synthesizing sound in real time. Music V
was an extremely powerful tool, but it worked independently of real time. This means that it synthesized the specified sounds as quickly as possible,
but not as the sound was heard. Non-real-timesound synthesis is similar to creating an animatedfilm; all the individual drawings must be completed before the film can be viewed and the motion implicit in those frames can be experienced.
While virtually anything can appearon thoseframes, the synthesis process generally takes much
longer than viewing the final result in real time.

Everything must be planned in such a situationevery gesture, every scene-no spontaneous "improvisation" is possible. If computer music wasever to sound as lively and expressive as other music, we needed a way to account for what performers contribute to music. Since we didn'treally understandperformancewell enough to specify itwith written instructions (and we still don't),wehad to resort to the capturingapproach-and to capture what performersdo, a computer music system
must operate in real time.
For the GROOVEsystem, Mathews and I used a
relatively small computer (one that occupied only
the space of an apartment rather than a house) tocontrol another roomful of voltage-controlled analog synthesis equipment that had become the basis
for such popular (at that time) electronic music devices as the Moog synthesizer (of Switched-On
Bach fame). It is shown in Figure 5. The voltages
that controlled the pitches and amplitudes of thesynthesis equipment were obtained through a bankof 14 digital-to-analogconverters attached to a Honeywell DDP-224 computer. These voltages were updated by the computer at a rate of between 100 and
200 times per sec instead of the 10,000 to 50,000
times per sec needed for direct digital sound synthesis. Though the sound potential of the analog synthesis equipment was far more limited than direct
digital synthesis, by using digital-to-analogconverters to control only such things as pitch and amplitude rather than the actual sound waveform itself,
we gained a factor of about 100 in available computation time, and 5 or 10 msec computers-even
1969 computers as big as apartments-can do a fairamount of computing! We then built and connected piano-like keyboards,pedals, joysticks,
knobs, and switches to the computer, all of which
could affect the sound being synthesized in realtime according to the user's program.Like a player
piano, the GROOVEsystem recordedthe actions
we made pushing keys and twisting knobs, rather
than the sounds produced. For the first time, we
could truly play the computer like a musical instrument-even though it still didn'treally soundlike one.
The result was that for the first time computers
could be involved in improvisation and every other

34

Computer Music JournalFigure5. The analog sideof the early-1970s
GROOVEsystem, developed at Bell Telephone
Laboratoriesby Max Ma-

thews and the author. The
left-most equipment rackcontains an Ampex twochannel tape recorderanda collection of Moog ana-kind of spontaneous musical decision-making.
These are things that trained musicians are extremely good at, and, like walking, are far easier todo than to explain. Much research on real-time systems for computer music has been done for the purpose of circumventing deeper questions regarding
how performers perform. Fortunately, the very existence of real-time computer music systems has
made it feasible to study performance in ways
never before possible. Robert Willey, a bright young
graduate student of mine at University of California, San Diego, recently earned a doctorate in music by studying the effect of instrument response delay on performers. This is just one of the manyquestions that can now be asked and answeredusing real-time computer music systems.
The CD that accompanies this issue includes

log synthesizer modules
above it. The other racks
include a variety of analog
voltage-controlled oscillators (with the large fre-quency tuning knobs) andfilters, and patch bays forconfiguring the interconnection of these elements.

three examples from the GROOVE system. The
piece at CD index 4 is an excerpt from EmmanuelGhent's work Phosphones, and indices 5 and 7 are
two short GROOVE pieces by Laurie Spiegel.
The example at index 6 on the CD was producedon the real-time interactive synthesizer developed
by Hal Alles and co-workers at Bell Telephone Laboratories; it is an excerpt from one of Laurie Spiegel's
improvisations on the machine.
Frequency Modulation
In the late 1960s, another young graduate student
of music named John Chowning was using Music
V at Stanford University to investigate the musicaleffects of extreme vibrato. Due to a programming
Moore

35

Figure6. A flow chart of aMusic V-style instrument
for frequency modulation(FM).The two oscillators
have icons that resemble

half-circles; the trapezoidal figures are timevarying envelope generators.

accident (as he himself describes it) he specified asound with a vibrato rate much largerthan a normal vibrato, which is usually about 5 or 6 vibrations per sec. In fact, the vibrato he specified was
as high or even higher than the fundamental frequency of the sound itself-something no singer ortraditional instrument could possibly produce.
Much to Chowning's surprise, what came out was
something that didn't sound like it had a vibrato atall. Instead, the result was a completely different
tone quality. The simple process of vibrato, when
carriedto such an extreme, can be understood byborrowing a theory from radio transmission, calledfrequency modulation (FM).Even though FM radiotransmission had been well understood since the1930s, no one ever thought of listening directly tothe modulated waveform itself before, and that's
not how John Chowning thought of it either. What
he did was accidentally transferthe FM theory
from radio to musical psychoacoustics, and voila! afascinating new music synthesis technique wasborn.
Of course, FM is not musically interesting because its theory comes from radio. It is interesting
because it is very simple to compute (one need
only modulate the frequency of one sine wave withanother-a simple and efficient process). Furthermore, two things that are easy to control in FM are
the modulating frequency,which correspondsto
the vibrato rate, and something called the modulation index, which is simply related to the vibrato
depth (the extent to which the frequency varies
above and below the main, or carrierfrequency). It
turns out that the ratio of the modulating and carrier frequencies determines whether the resulting
musical timbre has a harmonic or inharmonic spectrum, which is an extremely important musicalcharacteristic in perceptual terms. Furthermore,
the modulation index is directly related to anotherperceptually important quality of sound-its bandwidth, or "brightness."Figure 6 shows the block
diagramof an FM instrument.
Before John Chowning's wonderful discovery,noone had any idea that such a simple and efficientsynthesis process could provide such straightforward control over many perceptually importantcharacteristics of musical sound. Furthermore,noIn 1960 the first integrated circuits were fabricated,marking the real beginning of the digital revolution

36

Computer Music Journal

carrierfrequency} modulator
ij

ratio

carrierfrequency

one had any idea that such a discovery in the computer music field could be so profitable. Stanford
University patented the discovery,which was later
licensed to the YamahaCorporation,giving Yamaha a distinct early advantagein the computer
music instrument marketplace. For several years,
this patent produced royalties in excess of any everissued to StanfordUniversity. The discovery of FM
synthesis-more than any other single development-was responsible for the proliferation of computer music from universities and laboratoriesinto
studios, onto stages, into concerts, onto recordings,
and ultimately, into homes all over the world.
Sound example 7 (CD index 8) is John Chowning's
1972 piece Turenas.

ComputerMusic:TheThirdDecade,1975-1985
Dream:Proliferation
Reality:Development
The Digital Revolution

Figure 7. A 1961-vintage
integrated circuit (a dual
flip-flop) constructed by
the Fairchild Camera andInstrument Corp. The
white lines are aluminum
interconnects; the four(see Figures 7 and 8). About 10 years later, in 1971,
Marcian Hoff of Intel Corporation invented the microprocessor, placing an entire computer central
processor unit (CPU) on a single chip of silicon
about the size of a fingernail.
Such facts gave rise to a well-known maxim in
the semiconductor industry known as Moore's Law(after Gordon Moore of the Intel Corporation).
Moore's Law states that the complexity of integrated circuits is proportional to 2Y-1960,where Y isthe current year. In other words, every year (startingin 1960) the complexity of integrated circuits doubles, meaning that roughly twice as many gates,
bits of memory, or whatever, can be put on a single
chip as compared with the year before. A corollary
to Moore's Law states that cost is approximately
proportional to the square root of complexity, meaning that the cost of doing any particular thing withintegrated circuits will be cut in half about once every two years.
Moore's Law and its corollary have continued to
be valid (more or less) ever since they were introduced. They are important because they codify oneof the most important aspects of computer technology, namely, that it is getting cheaper as time goesby. Not only that, it is getting cheaper at a remarkable rate. Computer systems aren't getting cheaper
quite so fast, of course, because not everything in acomputer is an integrated circuit. The cost of anequivalent computer is only falling at the rate ofabout one half every three years, still an incredible
price reduction compared with virtually everythingelse in the known universe.
There is a dark side to Moore's Law, however. It
says that for the same price, complexity will increase by a factor of four every two years or so. In
other words, computers keep getting more andmore capable, so much so that after a computer isjust a few years old, it can no longer compete withnewer models in either performance or cost. At
some point it actually becomes cheaper to replace
an old computer than to upgrade or even repair it.
Moore's Law is therefore an important basis for thedreaded phenomenon known as technological obsolescence. Computer designs progress by "generations" that are only about three or four years long.
A "last-generation" computer may still be usable

conical shapes around thecenter of the figure are thetransistors. The entire device is 0.06-inch in diameter. (FromAugarten 1984,
p. 224. Reused by permission.)for many things, but by the time a computer is twogenerations old it becomes nearly useless, like lastmonth's newspapers.
Machines, Machines ...
The same principles hold true for any devices basedon integrated circuits, like digital music synthesizers. With the beginning of the real-time digitalmusic synthesis era, research in computer music
was booming. In 1975 I had temporarily left Bell
Labs to learn how to make digital synthesizers at
Stanford University, where I studied computer engineering and built the FRMbox, one of the world's
first all-digital music synthesizers shown in
Figure 9.
Computers were still the size of cars and small
trucks, and, much as with the GROOVE system,

Moore

37

Figure 8. Three historic
microprocessor chips fromIntel-a 4004 (1970; left-top), 8008 (1972; right-top)and 8080 (1974; leftbottom), and an Intel 1103

lk-bit memory chip (1970;
right-bottom). (FromAu-

garten 1984, p. 266.
Reused by permission.)

one entire computer had to be dedicated to control
my synthesizer. Its design was documented in mydissertation, and represented a kind of digitizationof the analog synthesis portion of the GROOVE system. Peter Samson of the Systems Concepts company also built a large digital synthesizer that became the basis for much music created at Stanford
University's Center for Computer Research in Music and Acoustics (CCRMA) which John Chowningand others founded around that time. Pierre Boulez
created the Institute de Recherche et Coordination,
Acoustique/Musique (IRCAM) in Paris, which
opened in 1978. At IRCAM, a brilliant young nuclear physicist named Giuseppe Di Giugno started

designing a digital synthesizer, working first with
Luciano Berio, and later with Pierre Boulez andmany others. Hal Alles built a research digital music synthesizer at Bell Labs around one of the new
LSI-11 microcomputer systems, shrinking the sizeof the control computer from pickup truck to TV
set. In 1979 I left Bell Labs to found the Computer
Audio Research Laboratory (CARL) project at the
University of California, San Diego, an effort dedicated to doing everything possible to sound and music with computers, and vice versa.
All of these efforts to build computer music instruments, among several others, were research
projects that consumed fairly large efforts and bud-

38

Computer Music JournalFigure 9. The 1975
FRMboxdigital synthesizer developed by the author at Stanford University. This was one of thefirst all-digital sound synthesis machines, devel-

oped around the sametime as the Systems Concepts Digital Synthesizer(or "Samson box") used
at CCRMA, and the "4"series of machines built
at IRCAMin Paris.

gets, especially if the cost of the computer was included. The Digital Equipment Corporation
VAX-11/780 (see figure 10) computer system thatwas the mainstay of the CARL project for many
years cost about half a million dollars (US$),
though it did serve many users simultaneously (albeit not too well, by today's standards). While it did
not seem unreasonable that a major research effort
in music might cost as much as a single grand opera production, tensions rose as with the costs ofthese efforts conflicted with the needs of individual
musicians who were used to working on more modest budgets. In those days we wanted real-time digital computer music synthesis, and thought interms of how many grand pianos it would cost. We

Figure 10. A Digital Equipment Corp. DEC VAX-11/
780 of the late 1970s.

(FromAugarten 1984,
p. 258. Reused by permission.)dreamed of the day when real-time computer music synthesis would cost no more than a single
grandpiano, making it possible for a dedicated individual musician to affordone. Then computer music might begin to be on the same social basis as
the rest of music, and we were sure that it would
proliferate widely after that.
Our dreams of proliferation were far too modest.
Within a few years, the YamahaCorporationhad developed inexpensive chips that could do much ofwhat the large, expensive research prototypes could
do. The first digital music synthesizer marketed in
the United States was the Synclavier, designed bySidney Alonso and others at New England Digital
Company,a small computer company. Such machines, which appearedin the late 1970s, carried
price tags similar to averageautomobiles, and werefairly limited in their capabilities.
But nothing could really compare on the priceperformance curve with the first good, cheap synthesizer, the YamahaDX-7, introduced in 1983. It
cost about US$ 2,000. Based on John Chowning's
FM synthesis algorithm, within a few years theDX-7 had sold more than 500,000 units to the international market. It was not only the world's mostpopular digital music synthesizer, but possibly theworld's most popular musical instrument.
In addition to the DX-7, another key development in computer music was the agreement among
several important manufacturers to a voluntarystandardfor connecting digital music synthesizers
to each other (regardlessof brand!)and to the new,small personal computers. The resulting Musical Instrument Digital Interface, or MIDI, allows people
Moore

39

display have already combined with interchange tocreate what is now called the World-WideWeb(WWW),but that is just the beginning. In effect,
the interconnected computers of the world are beginning to wake up, just as their users are beginning to realize how powerful an instrument ofmind (as Marvin Minsky called them) computersreally are. As we prepareto enter the 21st century,computers are empowering individuals in ways reminiscent of the scientific revolution-no longer isthe world's most powerful technology available
only to those in privileged positions.
Computer music has also jumped out of the laboratory and into the fire. Traditionalmusical instruments continue to get more expensive (like everything else) while computer music instruments justkeep getting better and cheaper.Most new pianos
sold are now digital, and this trend shows no signsof abating. Computer music is no longer an esoteric corner of music but has in fact become partof the mainstream, at least in terms of the marketplace.
The new face of computer music is as shallow
as it is broad, however. Rather than promoting research into basic musical questions, most com1985
toThe
FourthMusic:
Decade,Computerputer music now merely substitutes newer, cheaper
the Present (and Beyond)technology for strings and pipes. In that sense thefield of music has effectively "absorbed"computer
music with little or no effect on what music is proMindwithReality
Dream:Interfacing
duced or our understanding of it.
Reality:Virtual
Perhapsthis is as it should be. Technology is
In about 1985 the complexion of computer musicchanging much more rapidly than people.
After all is said and done, the dreams of today are
began to change; what was research oriented wasmuch different from those of our ancestors, exnot
thetremenoriented.
marketFollowing
becomingmusic
of
successful
cept, of course, that we have many more opportuniproliferation computer
douslyties to realize those dreams than they did.
instruments from the laboratoriesand into homes
and studios, shifts began to occur that are still proPerhapstoo, it is difficult to realize what a dreamabout while one is in the midst of having it.
isinCommusicfield
thegeneral.
foundly affecting
Since
at
and
to
shrinkcontinued
have
my own interest in computer music over the
get cheaper
putersbeen largely oriented toward creating newhas
rates predicted by Moore's Law to the point where
years
In
now
are
capabilities, perhaps the best way to access current
commonplace. fact, computers,theyfax
machines, television, radio, cinema, "dreams"is to examine what capabilities still dotelephones,
not exist but need to.
and print media all appearto be converging into aisthat
information
Clearly, the technology of computer music will
inherently
technology
singlecentered around real time, telecommunicating, and continue to improve and get cheaper. Generalaudio-visual computer-basedinformation exchange. purpose computers are becoming so fast that soon
we will have to slow them down-rather than
Explosions in information storage, processing, andto combine the features of different digital synthesizers and computers in various ways-a criticalmeans of combating technological obsolescence.
Whatever their shortcomings, MIDI and the DX-7
created a proliferation that was completely unprecedented in the field of music. Many other instruments and companies followed.
One device that deserves special mention in thiscontext was the Fairlight Computer Music Instrument (CMI).Just as it is possible to solve the performance problem without understandingit, it is possible to solve the sound synthesis problem in asimilar manner. The Fairlight was the first of a new
genre of digital music instruments that works byrecordingsounds in its digital memory (a process
called "sampling")in such a way that they can beplayed back at various pitches and amplitudes under control of a keyboard.The so-called sampling
synthesizer could put a recordingof virtually any
sound under any key, though there are some limits
to how much pitch modification can be done without degradingthe sound.

40

Computer Music Journal

I

speed them up with specialized-but-limited hardware-in order to achieve real-time performance.
This means that soon we will be able to retain thegenerality and flexibility of the Music V model andhave real-time interaction, too. Computers areor computing is-becoming an essentially free resource. This inevitably will put more and more of
our experience of the world into the "computermediated" category,meaning that our distinction
between reality and what we now call virtual reality will gradually disappear.Rather than being a"dial-a-dream"situation, the real world will actually be under the power and control of the mind.
As an example of this, consider the "fly-by-wire"
systems in modern aircraft.Instead of directly controlling the airplane, a pilot operates controls thatare sensed by the computer, which in turn flies the
airplane. In effect, the pilot's input only expresses
his wishes-the computer then responds to thesewishes in a manner that is screened for such things
as flight safety and passenger comfort. One result
is that far more complex airplanes can be built
than ever could be flown by an unassisted humanpilot. Another result is that airplanes sometimes
crash due to programmingerrors.
Imagine now a computer-basedmusic machinethat senses the musical desires of an individual listener. The listener might simply turn a knob one
way when the computer plays something the listener likes, the other way when the computer does
something less likable. Or, better yet, the computer
could sense the listener's responses directly using,say, body temperature,pulse rate, galvanic skin response, pupil size, blood pressure, etc. Imagine
what music would sound like if it continuallyadapteditself to your neurophysiological responses,
for as long as you wish. Such music might be more
addictive than any known drug, or it might cureany of several known medical disorders.
Of course, such a music machine cannot be built
today but that is precisely my point. Without such
dreams we have no idea how to direct our increasingly marvelous technology. Without dreams we
can only repeat what has alreadyhappened.
Dreams are the only true agents of change.

Moore

41