Combining granular synthesis with frequency modulation.
Kim ERVIK
Department of music
University of Science and Technology
Norwaykimer@stud.ntnu.noAbstract
Both granular synthesis and frequency
modulation are well-established synthesis
techniques that are very flexible. This paper
will investigate different ways of combining
the two techniques. It will describe the rulesof spectra that emerge when combining,
compare it to similar synthesis techniques andsuggest some aesthetic perspectives on thematter.
Keywords
Granular synthesis, frequency modulation,
partikkel, csound, sound synthesis

Fig 1:Grain Pitch modulation
1Introduction

1.1

MethodWorking on this subject, We´ve been using
csound for implementing and generating sound,and reviewing graphical analysis of the sounds
generated. We have been working systematic
trying to find similarities and differences between
regular FM, granular synthesis (GS), and FM in
GS. For the purpose of this project, we have beenusing sinusoidal grains.
1.2

Granular synthesis

The idea of granular synthesis can be traced
back to Gabor’s theory of acoustical quantum in1947 [7]. Thinking of sound as particles as aphilosophical and a musical point of view can bevery interesting, and has led to the development ofgranular synthesis, or particle synthesis. Granularsynthesis means synthesizing sound based on
adding thousands of sonically grains into largerØyvind BRANDSEGG
Department of music
University of Science and Technology
Norwayoyvind.brandsegg@ntnu.noacoustical events. Sound as particles has beenused in applications like independent time andpitch scaling, formant modification, analog synthmodeling, clouds of sound, granular delays andreverbs, etc [3]. Examples of granular synthesis
parameters are density, grain pitch, grain duration,grain envelope, the global arrangement of thegrains, and of course the content of the grains(which can be a synthetic waveform or sampledsound). Granular synthesis gives the musician vast
expressive possibilities [10].
1.3

FM synthesis

Frequency modulation has been a known
method of coding audio into radio signal since thebeginning of commercial radio. In 1964 that John
Chowning discovered the implication offrequency modulation of audio working on
synthesis of brass instrument at Stanford [2]. He
discovered that modulating the frequency resulted
in sideband emerging from the carrier frequency.
Yamaha later implemented the technique in thehugely successful DX7.
If we consider FM synthesis using sinusoidal
carrier and modulation oscillator, the spectral
components present in a FM sound can bemathematically stated as in figure 2.

f n= f c ±nf m
Fig 2:The spectral components present in a FM
sound where n is an integer, fc is the carrier
frequency and fm is the modulation frequency.
The intensity of the sidebands can be calculatedwith Bessel functions. When a sideband passes 0
Hz or the nyquist frequency, it is mirrored and 180
degrees phase shifted. The modulation frequency
is usually a product of a modulation ratio and thecarrier frequency, while the modulation amplitudeis the product of a modulation index and thecarrier frequency, which sets the sidebands in aconstant relation to the carrier frequency [9].
FM in GS is earlier mentioned in Jones andParks (1988) as a method to increase the range ofpossibilities available to the composer
researcher using granular synthesis [8].
2Combining the two techniques

2.1

FM of GS parameters

orIn granular synthesis the pitch interpretation ofthe sound is dependent on the grain rate. If thegrain rate is within the audible range, the pitch isgiven by the grain rate1. If the grain rate is subaudio, the pitch of the source waveform within thegrain defines the pitch of the generated sound. We
can do both FM of the grain rate, and FM of thepitch within the grain. We´ve looked at different
variants and investigated the differences inspectrum.
The grain pitch affects the spectrum of a GS
tone with high grain rate. When the grain rate is
low the spectrum might be affected by the grain
rate. So there are two parameters that oppose eachother. But seen as one, the two parameters
controlling pitch and timbre are grain rate andgrain pitch, and that’s why we have tried tomodulate the two.
If we look at regular FM synthesis, theperceived pitch is dependent on the modulation
ratio. If the FM ratio is 0.5 the pitch is interpret as
one octave below the carrier. The distance between
the frequency components of the spectrum is an
important part of determining the perceived pitch.
If a tone is generated with FM ratio at 7/8, thetimbre will be inharmonic resulting in a diffuse
fundamental pitch. As shown above in both FM
and GS we have many different factors
contributing to the perceived pitch.
To sum up we have two different ways ofcombining FM with GS:- Frequency modulation of the pitch within thegrain- Frequency modulation of the grain rate
3

Frequency modulation of the pitch within
the grain

Grain pitch modulation is easily done in csoundusing the partikkel opcode [1]. The particle opcode
has an a-rate input that alters the pitch of the grain,
implemented as phase modulation, directly
1With some exceptions. For instance if the phase or

the reading position of the grain source is modulated,
the pitch may be defined by the pitch of the content in
the grains. Another exception is if the grain stream isasynchronous. Then the result will be noisy dependent
on the random range of the grain distribution.

modifying the reading position of the source
waveform. This input can be fed with any audio
signal, for our experiments, we have used a sine
wave signal. A modulation oscillator can be set up
as in regular FM synthesis using the parameters
FM ratio and FM index, although the calculationof modulation index is done somewhat differently
than in regular FM [4].

Fig 3: Grain pitch FM
3.1

Grain pitch modulation with modulation
frequencies below 30 Hz

If we modulate the frequency at a rate below 30
Hz in a regular FM synthesis setup, the result isvibrato. The same thing happens in grain pitch
modulation with grain rates below 30 Hz. Withhigh grain rates the situation is somewhat
different, as the grain pitch will affect the timbre.
This means that a modulation frequency below 30
Hz applied to grain pitch modulation using highgrain rates will lead to periodic spectral sweeps
similar to filter sweeps.
There is an interesting transition area (approx.
20 – 50 Hz) both for grain rate and modulation
frequency. In this area we move from a pitch
perception defined by the grain pitch to the grainrate, similarly we move from vibrato to FM. We
also note significant interaction between theseparameters (grain rate and modulation frequency),
so the situation is somewhat complex. Further
exploration of this area might prove fruitful.
3.2

Grain pitch modulation with modulation
frequencies above 30 hz compared toregular FM synthesis.

Since the grain rate constitutes the perceived
pitch at high grain rates, it might be convenient tocalculate the modulation frequency as a relation
between the grain rate and the modulation ratio.
Calculating modulation frequency based on a
fixed pitch within the grain gives rather arbitrary
spectrum. But with modulation frequency

calculated based on the grain rate, a constantrelation between the fundamental pitch and thetimbre is obtained.
The spectral behavior in grain pitch modulation
is similar to regular FM. It might be relevant tocompare grain pitch modulation with regular FM
using a non-sinusoidal carrier wave. In thatexample all the harmonics get sidebands, resulting
in very dense spectra’s. The same thing happens in
grain pitch modulation. There are three majordifferences in the spectral behavior. First of all theFM index is behaving differently. In grain pitch
modulation the strength of the sidebands is weaker
than those in regular FM.
Secondly simple FM ratios behave differently.
The spectra of a regular FM tone will only containeven harmonics if the modulation ratio is two. Butin grain pitch modulation with integer FM ratios,for instance 2, the sidebands line up with every
other harmonics of the grain rate resulting in both
odd and even harmonics (se figure 4). This is not
the case though with complex FM ratios. FM ratio
at for instance 0,5 will cause the sidebands fromthe frequency modulation to line up between theharmonics of the grain rate resulting in a drop in
the perceived pitch (se figure 4). In some cases
complex FM ratios might give inharmonic spectra,
although the spectral behavior is different than inregular FM because of the dominant harmonics ofthe grain rate, which are still present.

variant of particle synthesis. FOF synthesis is aformant synthesis where the grain rate gives thefundamental pitch and the grain controls formant
frequency. The shape of the formant is given by
the grain shape and duration. The same thing
happens in grain pitch modulation, only with amuch wider peak in the spectrum. It’s so wide thatit’s perhaps not correct to call it a formant. Thisenables us to shift the spectral energy of the FM
synthesis upwards.
If one uses Gaussian grain envelope and thegrains don’t overlap, the result is similar toamplitude modulation. One very importantdifference is that the phase of the wave inside thegrains are resetting for each grain. That’s meansthat if the grain frequency is not in an integer
ration to the grain size, the wave cycles inside thegrains will be cut short, resulting in a spectrumwith a lot more components than in the case ofinteger ratios. So for cleaner sound one need thegrain frequency to be an integer of the grain size,
which limits the number of available grainfrequencies. The way to solve this is to use two
grain streams, each containing grain pitches at
integer ratio to the grain rate, and then crossfade
between them.
3.3

Similarities and differences between
grain pitch modulation and regular
granular synthesis using FM synthesizedsound as wave source.

It might be useful to compare grain pitch
modulation with regular granular synthesis using
an FM synthesized sound as source for the grains.
The major difference is the continuity of the phasein the FM synthesized source wave, whereas thephase will be reset for each new grain when doing
FM on the pitch within the grain. In otherrespects, grain pitch modulation is comparable tocombining FM with AM.
4Fig 4: Grain rate at 300 Hz, grain pitch at 300 Hz.
Shows the difference between grain rates at 3 andat 0,5.
The third difference is that in grain pitch
modulation sidebands will also emerge from a sub
harmonic tone at 0 hz, which is a sub harmonicfrom the grain rate in the GS. It is not audible, but
the sideband might be. This results in a much fattertone than in regular FM.
To understand the effect of the grain pitch thatare modulated, we must look at FOF synthesis, aFrequency modulation of the grain rate

Since the perceived pitch is given by the grain
rate with grain rates above 30 hz, its interesting toexperiment with modulation of the grain rate,using a regular sine modulator. The modulation
frequency can be calculated the same way as ingrain pitch modulation. When the grain rate ismodulated, it should be updated at audio rate.
Ideally the grain rate and the modulation wave
should be an a-rate variable. This is possible withthe partikkel opcode [5].

4.1

Grain rate modulation with modulation
frequencies below 30 hz

If we use modulation frequency below 30 hz wewill get vibrato, just like in grain pitch modulation
with low grain rates. But also here there are someinteresting transition areas to explore in both grain
rate and modulation frequency.

differences in the strength of the content result insounds that have different tonal quality. That goesfor grain pitch modulation versus regular FMsynthesis as well.
5

The two techniques described above can also becombined. This can be seen as two modulators inparallel. If one use grain pitch modulation ratio at½ and grain rate modulation ratio at 1/3 one get aperceived pitch equivalent to modulation ratio at
1/6 [6]. Of course there are some differences inthe strength of the sidebands leading to slightly
different sounding timbres.
6Fig 5: Grain rate modulation
4.2

Grain rate modulation with modulation
frequencies above 30 hz compared tograin pitch modulation

This implementation gives similar spectral
behavior as grain pitch modulation, except whenthe modulation index is above 1. The way wehave used modulation index here, we havecalculated it with a reference to the grain rate. In
this respect, the modulation index is a measure ofgrain displacement within 1/grain rate seconds. If
the modulation index exceeds one, the modulation
wave leads to a negative grain rate. This allows
extra grains to be generated, as the grain rate ismirrored around 0 Hz. This leads to an alterationof the perceived pitch.
Just as in grain pitch modulation, we alreadyhave harmonics from the granular synthesis before
we start modulating the grain rate. This results in aperceived pitch that never can be higher than thefundamental of the grain rate. But in grain rate
modulation the timbre don’t get brighter withinteger FM ratios above 1. This can be explainedby looking at the grain rate as a sampling
frequency. The modulation can only occur within
the sampling frequency. According to the nyquist
sampling theorem the highest frequencyrepresentable is ½ the sample rate. So the timbre ofa tone with FM ratio at 0.4 is the same as thetimbre of a tone with FM ratio at 0.6.
Of course even though the relation of theharmonics are similar in the two variants, theGrain rate and grain pitch modulation

Other considerations

Because of incomplete wave cycles inside thegrains, the spectra stretch further up in thefrequency range than in regular FM. This results
in a lot of aliasing. A suggested solution to thisdistortion is to use higher sample rates, or toimplement the suggested solution containing twograin generators described earlier in this paper.
6.1

What Granular synthesis brings to FM

FM synthesis is a very flexible synthesis
technique with a wide range of different sounding
spectra. It has defined an entire decade ofcontemporary music via its widespreadcommercial use in Yamaha synthesizers. When
combined with granular synthesis there are newunheard sounds and possibilities. Now you cancreate a Chowning brass sound, with overlapping
grains, or with steeper grain envelopes. It ispossible to make bells with grains panned in all
directions and much more. Thinking of the soundas particles instead of as waves results in new
ideas and alters the way we are creativ.
6.2

What FM brings to granular synthesis

To get a synth inside your granular engine
enables us even more control compared to using a
prerecorded sample as source inside the grains.
One very interesting thing is the shift of thespectra when the carrier pitch within the grain isaltered. For instance, for making new aggressive
bass sounds one could control the center pitch
inside the grain with an ADSR envelope. Onecould also make edgy sweep pads with a LFO on
the center frequency.
7Conclusion

We have investigated two different ways tocombine FM synthesis with granular synthesis,compared them, and presented our results. The

two are grain rate modulation and grain pitch
modulation. They behave similar to regular FM,
but there are some differences in the strength of
the sidebands, and in the spectra due to harmonics
from the grain rate. We have also suggested someaesthetic perspectives on the combination of thetwo techniques.
8

Acknowledgements

Thanks to Sigurd Saue and Victor Lazzarini forhelp resolving technical issues.
9Bibliography[1] Brandtsegg, Ø. Saue, S. and Johansen, T.
(2011) Particle synthesis, a unified model forgranular synthesis. Linux Audio Conference2011.
http://lac.linuxaudio.org/2011/papers/39.pdf[2] Chowning, John M. (1973) The synthesis ofcomplex audio spectra by means of frequencymodulation. Journal of the audio engineer
Society 21.
[3] Ervik, K. Brandsegg, Ø. (2011) Creating
reverb effects using granular synthesis. CsoundConference 2011.
[4] Ervik, Kim csound exsample nr1:
FM_Grain_Pitch.csd
http://folk.ntnu.no/kimer/LAC2013/FM_Grain_Pitch.csd
http://folk.ntnu.no/kimer/LAC2013/PartikkelArgs.inc[5] Ervik, Kim csound exsample nr2:
FM_Grain_Rate.csd
http://folk.ntnu.no/kimer/LAC2013/FM_Grain_Rate.csdhttp://folk.ntnu.no/kimer/LAC2013/PartikkelArgs.inc[6] Ervik, Kim csound exsample nr3:
FM_Grain_Rate_and_Pitch.csd
http://folk.ntnu.no/kimer/LAC2013/FM_Grain_Ra
te_and_Pitch.csd
http://folk.ntnu.no/kimer/LAC2013/PartikkelArgs.inc[7] Gabor, Dennis (1947) Acoustical quanta andthe theory of hearing. Nature 159[8] Jones, D. Parks, T (1988) Generation andCombination of grains for music synthesis.
Computer Music Journal, Vol 12, no 2.
[9] Manning, Peter (2004) Electronic andcomputer music. Oxford University Press, NewYork.
[10] Roads, Curtis (2001) Microsound. MIT Press,Cambridge, Massachusetts.