Functional Ambisonic Granulator

David Fierro, Alain Bonardi

► To cite this version:

David Fierro, Alain Bonardi. Functional Ambisonic Granulator. Proceedings of the 4th International Faust Conference, Nov 2024, Turin, Italy. hal-04849708

HAL Id: hal-04849708

https://hal.science/hal-04849708v1

Submitted on 6 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# FUNCTIONAL AMBISONIC GRANULATOR

David Fierro

CICM-MUSIDANSE

Université Paris 8

davidfierro@gmail.com

Alain Bonardi

CICM-MUSIDANSE

Université Paris 8

alain.bonardi@univ-paris8.fr

## ABSTRACT

This paper presents a functional ambisonic granulator developed using the Faust language, designed to generate diverse sound outputs from minimal inputs. Suitable for mixed music performances, it features configurable envelopes, distributed internal parameters, and feedback loops. The granulator's architecture includes input filtering, grain generation, modulation, transposition, and spatialization. Two spatialization approaches are implemented: "spatial sound transformation" and "point source to diffuse field".

## 1. INTRODUCTION

This paper introduces a functional ambisonic granulator, combining granular synthesis with ambisonic spatialization using Faust's functional programming paradigm. Developed for the BBDMI project,<fnref n="1" /> it aims to generate diverse sound outputs from minimal inputs, particularly suited for performances using electrophysiological signals. Its key features include:

* Configurable envelopes for grain shaping.

* Distributed internal parameters across channels.

* Strategic feedback loop positioning.

* Two distinct ambisonic spatialization techniques.

* Comprehensive user-controllable variables.

The granulator's architecture enables a wide spectrum of sound outputs. This paper details each component's implementation, explores the granulator's MaxMSP integration, and discusses future developments to enhance user accessibility and control. In general, the specificity of the proposed granulator lies in the highly configurable envelopes, the distribution of internal parameters for each instantiated channel, and the way feedback loops are positioned.

All the codes related to this paper can be found in our Gitlab repository.<fnref n="2" />

## 2. CONTEXT

The development of this granulator is proposed as a solution for generating a mixed music sound effect that operates with minimal input while producing a broad spectrum of diverse sound outputs. This type of sound effect is essential for our BBDMI project, which involves the performance of mixed music using EMG and EEG signals. Typically, the small number of extracted features from EEG signals and the limited number of channels in EMG devices result in a constrained set of variables for user interaction.

The concept of granular synthesis has a rich history that informs its current applications. Granular synthesis was initially conceptualized by Iannis Xenakis in the early 1950s, who explored the idea of sound as composed of small grains or particles, each contributing to a complex acoustic texture [1]. Xenakis's theoretical groundwork laid the foundation for later developments in the field.

In the 1970s and 1980s, Curtis Roads advanced granular synthesis by formalizing its methods and exploring its potential in digital sound processing. His work, particularly in Microsound, has been instrumental in defining the techniques used in granular synthesis today [2]. Roads highlighted the ability of granular synthesis to create new soundscapes by manipulating tiny fragments of sound, which is a core principle in the design of our ambisonic granulator.

Further contributions to the field were made by Horacio Vaggione, whose work on the articulation of microtime has significantly influenced the musical applications of granular synthesis. In his article "Articulating Microtime", Vaggione explores how the manipulation of microtemporal structures can be used to create complex sound forms that transcend traditional time scales [3]. His approach to granular synthesis emphasizes the importance of controlling the temporal evolution of grains, allowing composers to shape intricate sound textures at a microstructural level. This concept is reflected in the design of our granulator, which leverages feedback loops and delays, creating detailed and evolving soundscapes.

Barry Truax's work on real-time granular synthesis has also been influential, particularly in the context of live performance. Truax's approach to real-time processing aligns closely with the requirements of mixed music performances, where sound effects need to be generated dynamically based on live input [4]. This is directly applicable to the functionality of our granulator, which is designed for live use, incorporating feedback loops and real-time spatialization.

The design of this granulator was significantly influenced by the functional logic of the Faust programming language. The concept of a continuous signal flow, inherent to Faust, inspired the development of a flow of grains within the granulator. Through the incorporation of feedback loops, the granulator facilitates the creation of intricate sound textures. The development of this granulator was also inspired by the work conducted at the CICM lab [5]. By incorporating a final layer of spatialization, the granulator is capable of generating spatially diffuse sound fields and creating spatial sound transformations.

## 3. A LIVE MUSIC GRANULATOR

Following Truax's model of real-time granulation, this mixed music granulator requires an external sound input to generate audio.

<footnote n="1">¹BODY BRAIN DIGITAL MUSIC INSTRUMENTS. ANR-21-CE-38-00018. https://bbdmi.nakala.fr/</footnote>

<footnote n="2">²https://gitlab.huma-num.fr/bbdmi/bbdmi</footnote>
This operational approach makes it particularly suitable for live mixed music performances. The granulator can be configured in two distinct modes: as a multichannel sound effect with ambisonic output, or as an ambisonic sound effect when positioned between an ambisonic encoder and decoder.

## 4. MAXMSP WRAPPING

Throughout the development of our project, we have created numerous objects within the MaxMSP environment. To maintain workflow continuity, we developed a Max/MSP wrapper that facilitates interaction with the granulator within this development environment. The wrapper was initially designed to enable dynamic invocation of any Faust object by simply adjusting the number of channels within a bpatcher.<fnref n="3" /> The Figure 1 illustrates the variables within the wrapper, which include three additional options not originally present in the Faust code.

The first wrapper-specific option is the “channels” variable. By default, dynamically changing the number of channels for a Faust object within Max/MSP is not feasible. This variable, however, allows us to replace the Faust object by selecting the desired number of channels, thus offering greater flexibility in the development process.

The wrapper includes two additional variables, “randomize” and “dump”, each providing distinct functionalities. The “randomize” option assigns random values to all internal variables within their respective minimum and maximum ranges. This feature is crucial for exploring the full range of the granulator’s capabilities, especially given the complexity introduced by having 17 internal variables.

The “dump” option allows users to export the current configuration of the granulator for future retrieval with a single click. The interplay between the “randomize” and “dump” functions enables users to experiment with and save various configurations, facilitating both exploration and preservation of interesting settings.

## 5. ARCHITECTURE

The granulator’s behavior is determined by seven distinct components:

* Input filtering.

* Feedback loop of the input signal with delay.

* Grains generation.

* Modulation.

* Grains transposition.

* Spatial sound transformations.

* Feedback loop with transposition.

The Figure 2 represents these 7 modules. Subsequent sections will provide a detailed discussion of each module. It is crucial to note that two variables significantly influence the behavior of all other parameters in the granulator: the “variability” and “index distr” parameters (explained in Section 5.6.3). These variables introduce varying degrees of randomness into each of the other parameters, thereby impacting their behavior and interaction. The “variability” parameter modulates the extent of randomness applied, while the

<figure><img src="image_2.png" /><figcaption>Figure 1: *Max bpatcher*.</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Figure 2: *Simplified diagram*.</figcaption></figure>

“index distr” parameter governs how this randomness is distributed across the different channels or harmonics. Together, these variables play a central role in shaping the overall sonic output by dynamically altering the relationships between the granulator’s internal variables.

### 5.1. Input Filtering

The first effect applied to the input signal is a combination of low-pass and high-pass filtering. By default, the granulator does not apply any filtering to the input signal; however, it provides the flexibility to dynamically adjust the input filtering as needed.

### 5.2. Delayed Feedback Loop

The primary objective of this module is to blend the input signal with a delayed version of itself to create a more complex sound texture before segmenting it into grains. By adjusting the “maxdelay” variable, a delay is introduced to the input signal, which is

<footnote n="3">³A MaxMSP object that modularizes a patcher or subpatcher, displaying only specified visual elements in other patchers</footnote>
<figure><img src="image_4.png" /><figcaption>Figure 3: Input delayed feedback loop.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 4: Envelope design.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Figure 5: Morphing between envelopes and different grain sizes.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 6: Envelope amplitude modulation.</figcaption></figure>

then added to the input after being scaled by the “feedback” variable. Figure 3 illustrates the structure of this module.

These two variables allow the user to either process the clean input signal with no delay and zero feedback or to achieve a more intricate effect by mixing the input signal with its delayed version, with delays extending up to 10 seconds.

## 5.3. Grains Generation

This module is responsible for segmenting the processed input signal into grains. Two primary properties of the grain are addressed: the size of the grain and the grain envelope. The grain size determines the duration of each grain, while the grain envelope controls the temporal shape of the grain's amplitude profile.

### 5.3.1. Grain Size

Instead of employing a rarefaction logic, the granulator operates based on grain size and spacing. The grain size is calculated by adding the “grainoffset” variable to the product of the “grainsize” and the “variability” parameter. This configuration enables the generation of grains with a uniform size or with sizes varying within a specified range. The “spacing” variable multiplied by the “variability” parameter will determine the spacing between grains.

### 5.3.2. Grain Envelope

After evaluating various envelope shapes, we determined through experimental validation that a combination of Gaussian and logarithmic waveforms provides optimal results. The “grainenvmorph” variable enables dynamic adjustment of the envelope shape, ranging from zero to one. A value of zero corresponds to a Gaussian envelope, while a value of one corresponds to a logarithmic envelope. Figure 4 illustrates the envelope shapes for different values of the “grainenvmorph” variable. Figure 5 illustrates the creation of grains while the “grainenvmorph” and “grainsize” variable are changing.

## 5.4. Envelope Amplitude Modulation

Following the creation of each grain, this module enables the application of amplitude modulation to each grain through a variable modulation factor, “modfactor”. Figure 6 depicts the grain envelope under three conditions: without modulation, with 50% modulation, and with 100% modulation. The variable “modfreq” sets

the frequency of the amplitude modulation, ranging from 1 Hz to 15 kHz. Figure 7 demonstrates the effects of envelope modulation at three different frequencies: 10 Hz, 20 Hz, and 40 Hz.

Initially, the module utilized a single modulating signal for all grains. This approach led to the perception of a single, continuous tone after listening to a stream of grains, which did not align with our objectives. To address this issue, we implemented phase-controlled modulation for each grain. This modification ensures complete decorrelation between the shapes of the modulated envelopes of the grains. By employing an ad hoc modulating signal, the granulator is able to produce grains that share the same modulation characteristics without creating a cohesive tonal effect.

### 5.4.1. Modulating Signal Morphing

The ability to morph the waveform of the modulating signal in an amplitude modulation (AM) system plays a crucial role in shaping the sound of the resulting signal. For instance, using a square wave as the modulation signal tends to produce a bright, aggressive sound, whereas a sine wave results in a smoother, softer tone. The granulator’s “modmorph” feature acts as an interpolation controller between four waveform signals: sine, sawtooth, square, and sine again. Figure 8 illustrates the envelopes generated by varying values of the “modmorph” variable. It is important to note that integer values correspond to pure waveforms, while decimal values result in a morphed shape that blends two adjacent waveforms.

The design choice to position a sine wave at both the beginning and end of the modulation cycle allows the user to create any conceivable morph between the three basic waveforms. This capability enables the creation of interesting effects through waveform morphing. By gradually transitioning from one waveform shape to another, either rapidly or slowly, users can achieve smooth or abrupt changes in sound texture. Figure 9 illustrates the process of

<figure><img src="image_8.png" /><figcaption>Figure 7: Variable “modfreq”.</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Figure 8: Modulating signal morphing from sine to square.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Figure 9: Mix of different modulating signals.</figcaption></figure>

wave morphing applied to the grain envelope.

### 5.4.2. Modulating the Frequency of the Amplitude Modulating Signal

In addition to modulating the amplitude of each grain's envelope, the granulator also offers a variable called "modfreqmod". This variable controls the degree of frequency modulation applied to the amplitude modulation of the envelope. Figure 10 illustrates the impact of varying the "modfreqmod" variable from zero to one. In this context, a value of 0.5 represents the absence of frequency modulation, while values closer to zero or one introduce increasing levels of frequency modulation to the amplitude modulation envelope. This transformation enables the creation of envelopes where the amplitude modulation can dynamically shift from a high frequency to a low frequency, or vice versa, with each new grain. This modulation process is applied every time a grain is generated, allowing for varied and evolving sound textures.

## 5.5. Grains Transposition

At this stage of the process, the grains are transposed within a range of -24 to +24 semitones.

## 5.6. Spatialisation

Given that one of CICM laboratory's primary focus on sound spatialization and our prior advancements in the field of ambisonic

<figure><img src="image_11.png" /><figcaption>Figure 10: Frequency modulation of the amplitude modulating signal.</figcaption></figure>

spatialization, we decided to integrate an ambisonic spatialization layer into the granulator. There are various approaches to constructing this spatialization layer, and we chose to implement two distinct methods, each designed to produce different sonic outcomes. Currently, we have developed two versions of the granulator, each catering to different musical requirements. The version presented in this paper differs from the second version, particularly in its approach to spatialization. Here we discuss this two approaches:

### 5.6.1. Spatial Sound Transformation

The version of the granulator presented in this paper employs a method we refer to as "spatial sound transformation". Unlike traditional spatialization approaches that focus on placing sound at a specific point in space, this approach leverages spatialization algorithms to create new and dynamic sound effects [6].

In this configuration, the granulator could function as a multi-channel effect, with each channel acting independently, but in order to achieve a more intricate spatial result, the granulator is positioned between an ambisonic encoder and decoder. This approach contrasts with the traditional method of granulating multiple channels, placing them on the space and then projecting them through a sound system. By using the granulator as an ambisonic effect between the encoder and decoder, we modify the spatial representation sent from the encoder to the decoder, effectively altering how the spatial audio is perceived.

This "spatial sound transformation" enables a wide array of sound effects, one example is spatial decorrelation [7]. By applying different delays to each ambisonic harmonic channel, we can alter the reconstruction of the final "sound image", leading to unique spatial effects. In Section 5.6.3, we will explore how each variable of the granulator is configured to respond to different distributions among the ambisonic harmonics, further enhancing the complexity and richness of the spatial effects.

### 5.6.2. From Point Source to Diffuse Field

The second implementation of the granulator adheres to a more conventional approach to spatialization. In this version, each grain is positioned randomly within the 2D space before entering the final feedback loop, that will be discussed in Section 5.7.

Since each grain is spatialized using an ambisonic encoder, an ambisonic decoder of the same order is required for proper interpretation. What makes this implementation particularly interesting is that the initial "image" of the spatialized sound is precisely positioned in a specific location within the space. However, as the sound is reintroduced through the feedback loop, these subsequent "images" become increasingly diffuse, ultimately creating a more complex and enveloping diffuse sound field. This approach leverages the spatialization algorithms to transition from a clearly defined spatial sound to a more ambient and immersive auditory experience.

### 5.6.3. Ambisonic Distribution

When the granulator is used between an ambisonic encoder and decoder, every channel of the granulator is modifying a specific spherical (for 3D) or circular harmonics (for 2D). As every harmonic adds complementary information to the final reconstruction of the spatialised sound, the question of how to alter the parameters of each harmonic becomes relevant. As each channel of the
<figure><img src="image_12.png" /><figcaption>Figure 11: *Ambisonic distributions.*</figcaption></figure>

granulator is modifying a specific harmonic, the way each internal parameter of every channel of the granulator relates to one another determines the characteristics of the final ambisonic reconstruction. There are numerous possibilities for distributing the granulator's internal parameters across the spherical or circular harmonics, ranging from high-level control parameters to specific adjustments for each harmonic. Our proposition for distributing how each internal parameter of each channel changes is based on the transfer functions proposed by Alain Bonardi and Paul Goutmann in the Faust "HOA" library. With this method the variables of each channel will change taking in account the selected transfer function and the number of the harmonic. The "index distr" variable enables users to select from various distribution patterns. Figure 11 illustrates the potential distributions as mathematical functions. The first distribution is "x" which represents a linear distribution. Upon selecting a specific ambisonic distribution, the internal parameters of each channel in the granulator are adjusted according to the "variability" parameter and the corresponding value from the selected transfer function. For instance, consider the impact on the first low-pass and high-pass filters. For each harmonic, or granulator channel, the frequency of these filters is calculated by taking the base frequency value and adding an additional component. This component is the product of the base frequency and a scaling factor ranging from zero to one, which is distributed across all the harmonics. For a linear transfer function, or "x", the resulting sonic effect is that the higher the harmonic level, the more pronounced the effect of each parameter becomes. This configuration creates a gradient of effects, where parameters such as filter frequency progressively intensify with the level of the harmonic, contributing to a more dynamic and evolving sound texture.

## 5.7. Grains Feedback Loop and Transposition

At this stage, a grain with specific spatial properties and a modified spectrum, thanks to filtering and modulation, has already been created. The final step in the granulator's process involves the collection of these grains through a feedback loop. This final loop enables the granulator to produce a continuous stream of grains rather than discrete, isolated grains at the output. As detailed earlier in this paper, parameters such as grain size, spacing, and other characteristics are adjusted before each grain is generated. The outcome is a flux of grains with varying sizes and spectral contents, which are accumulated within a time window specified by the "maxdelay" variable.

Figure 12 illustrates the configuration of this final feedback loop. A critical aspect of this module is the transposition within the feedback loop. As shown in Figure 12, each grain is initially

<figure><img src="image_13.png" /><figcaption>Figure 12: *Feedback loop and transposition of grains.*</figcaption></figure>

outputted without any transposition. However, subsequent feedback signal of the same grain are increasingly transposed. This setup produces an interesting sonic effect, as the echoes of each grain begin to "escape" into either very low or very high frequency spectrums. The variable "transpout" controls the degree of transposition applied, allowing the user to shape the extent of this spectral shift.

## 5.8. Smoothing

A 1024-sample transition buffer is employed for all input variables to smooth the input before it is applied. This smoothing mechanism mitigates abrupt changes that could otherwise result in audible clicks.

# 6. MACRO CONTROLS

The granulator has 17 internal variables, which exceeds what a performer can feasibly adjust simultaneously, especially when working with EMG/EEG signals that offer limited channels for control. To maximize the granulator's potential, we have discovered that employing advanced mapping algorithms, such as regression models and neural networks, can yield highly effective results. These approaches allow for the complex and dynamic manipulation of the granulator's parameters, enabling performers to explore a broader range of sonic possibilities despite the constraints of limited input channels.

### 6.1. Regression

By employing the regression algorithm "RapidMax",<fnref n="4" /> we successfully recorded various configurations of the granulator, enabling them to evolve in response to inputs provided to the regression model. This approach offers users the flexibility not only to explore the extensive possibilities of the granulator but also to record these configurations and seamlessly morph between them. This dynamic capability enhances the creative potential of the granulator, allowing for the development of evolving soundscapes and complex transitions between different sonic states.

### 6.2. One Layer Perceptron

A comparable approach to the regression model was adopted by developing a single-layer perceptron neural network using Faust. This implementation relies on vector and matrix multiplications to evaluate the perceptron. However, as of now, we have not developed a training algorithm, so this method remains in the experimental stage. Currently, we can only test random weights and biases until an interesting outcome is achieved. The lack of a systematic training process limits the full potential of this method,

<footnote n="4">⁴https://github.com/samparkewolfe/RapidMax</footnote>
but it represents a promising direction for future development in
generating and exploring novel sound configurations.

7. USER CASES AND FUTURE DEVELOPMENTS

Within the context of the BBDMI project, we have had the oppor-
tunity to test the granulator in various settings. This experience
has provided valuable insights into both the possibilities and limi-
tations of the granulator. In concert settings, we received positive
feedback regarding the granulator's sound, which has garnered in-
terest from composers and musicians. However, despite this initial
enthusiasm, many users eventually discontinue using the granula-
tor due to its complexity.

This issue highlights the need for further development of the
internal mapping algorithms to facilitate more accessible and in-
tuitive macro controls directly within the Faust-compiled object.
Simplifying the user interface while maintaining the granulator's
powerful capabilities is crucial for broader adoption.

8. ACKNOWLEDGMENTS

The research presented in this paper has been supported by the
French national research agency (ANR-21-CE38-0018).

9. REFERENCES

[1] Iannis Xenakis, *Formalized music: Thought and mathemat-ics in composition*, Bloomington : Indiana University Press, 1971.

[2] Curtis Roads, *Microsound*, The MIT Press. MIT Press, Lon-don, England, Aug. 2004.

[3] Horacio Vaggione, “Articulating microtime,” *Computer Mu-sic Journal*, vol. 20, no. 2, pp. 33–38, 1996.

[4] Barry Truax, “Real-time granular synthesis with a digital sig-nal processor,” *Computer Music Journal*, vol. 12, no. 2, pp. 14–26, 1988.

[5] Alain Bonardi, “La librairie abclib : un ensemble de codes Faust rassemblant 20 ans de recherche, enseignement et création en musique mixte,” in *Proceedings of the Journées d’Informatique Musicale 2021*, videoconferencing due to Covid, July 2021, AFIM.

[6] Paul Goutmann and Alain Bonardi, “Approaching spatial au-dio processing by means of decorrelation and ring modulation in ambisonics,” in *Proceedings of the 19th Sound and Mu-sic Computing Conference, June 5-12th, 2022, Saint-Etienne (France)*. 6 2022, Zenodo.

[7] Anne Sèdes, “Approche musicale de la décorrélation mi-crotemporelle dans la bibliothèque HOA,” in *Journées d’Informatique Musicale 2015*, université de Montréal, Ed., Montréal, Canada, May 2015, Université de Montréal.