DOI: 10.4119/unibi/3003418
title: Supplementary Material for "pya AGen - Audio Generators for Sonification in Python"

description: img {
 max-width: 100%;
 height: auto;
}
video {
 max-width: 100%;
 height: auto;
}
audio {
 width: 100%;
}
/* .scrollx {
 overflow-x: scroll;
} */



# PyA - AGen - Sonification Examples 

**Supplementary Material for L. Born & T. Hermann (2025). "pya AGen - Audio Generator Extensions for Sonification in Python", Proc. ICAD 2025**
<br>

To install pya AGen, install the feature-agen branch of the pya repository using this command:
```
pip install git+https://github.com/interactive-sonification/pya.git@feature-agen[agen]
```

- pya_agen is an extension for [pya](https://pypi.org/project/pya/), the python audio coding package.
- pya_agen provides flexible ways to generate audio signals using AGens, i.e. audio generators.
- AGens 'lazily' generate (i.e. compute when needed) audio samples on demand.
- For documentation of AGens check out the [example notebook](https://github.com/interactive-sonification/pya/blob/feature-agen/examples/pya-examples-agen.ipynb).
- This Jupyter notebook contains supplementary material for the paper.
- The examples demonstrate the use of pya and Agen in particular for coding sonifications.

Load Datasets for sonification examples

<pre class="github-light" style="background-color:#fff;color:#24292e;" tabindex="0"><code>data_path = "https://raw.githubusercontent.com/interactive-sonification/sonecules/main/notebooks/data"
# load eeg dataeeg_df = pd.read_csv( f"{data_path}/epileptic-eeg.csv", delimiter=",", header=None)
# load ecg dataecg_df = pd.read_csv( f"{data_path}/ecg-200Hz-10s-6channels.csv", delimiter=" ", header=None)
# load the penguin data setpenguins_df = sns.load_dataset("penguins")penguins_df = penguins_df.dropna(subset=["bill_length_mm", "bill_depth_mm", "flipper_length_mm", "body_mass_g", "sex"]).reset_index(drop=True)
# load the building data setbuilding_df = pd.read_csv(f"{data_path}/building.csv", delimiter=",", names=[ "monday", "tuesday", "wednesday", "thursday", "friday", "saturday", "sunday", "hour_from_noon", "hour", "am_pm", "temperature", "humidity", "solar_radiation", "wind_speed", "hc_wb_electrical", "hc_wb_cold_water", "hc_wb_hot_water"])</code></pre>

## 1. Audification with pya Agen
Audification is so straightforward that it already works with pya (without Agen): as such

<pre class="github-light" style="background-color:#fff;color:#24292e;" tabindex="0"><code># load data into Asiga1 = Asig(eeg_df[5], sr=250)
# plot and audify (aka: play) data at given compression ratea1.plot(lw=1).norm().stereo().fade_out(0.01).play(rate=20) </code></pre>


Your browser does not support the audio element.


<br>

Using AGens we can achieve it using the PlayAsig AGen:

<pre class="github-light" style="background-color:#fff;color:#24292e;" tabindex="0"><code># load data into Asigdasig = Asig(eeg_df[[3,5]], sr=250).resample(target_sr=44100)
# define Agen to play data at constant rate asyn = PlayAsig(dasig, rate=20)
# render sound signal (aka play asyn), and plot and play asig = asyn.gen_asig().plot(lw=0.5, offset=2).play(onset=0.5)</code></pre>
![](https://pub.uni-bielefeld.de/download/3003418/3003710.svg)



Your browser does not support the audio element.


As reward we can now easily solve more complicated audification needs.
For example
- audification of EEG data, but only time segment [3sec, 40sec] with channels 3 left, channel 5 right
- audify the data 'looped'
- the compression rate should jump (at 5 Hz) between 30 and 60 so that we can hear patterns in the data at two rates in a multiplexed way
- render, plot and play 3.4 seconds for the user, with a 0.1s fade in/out to avoid pops and clicks 

<pre class="github-light" style="background-color:#fff;color:#24292e;" tabindex="0"><code># prepare data data = Asig(eeg_df[[3,5]], sr=250)[{3:40}].resample(target_sr=44100)
# define AGen involving a modulator for raterate_agen = LFPulse(freq=5, width=0.5).linlin(-1,1,30,60)asyn = PlayAsig(data, rate=rate_agen, done="loop").fade_in(0.1).fade_out(0.1)
# render sound signal (aka play asyn) and plot and playasyn.gen_asig(seconds=3.4).plot(lw=0.5, offset=2).play(onset=0.5)</code></pre>

![](https://pub.uni-bielefeld.de/download/3003418/3003714.svg)


Your browser does not support the audio element.


## 2. Parameter Mapping Sonification with pya AGen
### 2.1. Discrete Parameter Mapping Sonification
For this demo we use the penguins dataframe containing attributes bill length bill depth flipper length and body mass for 330 penguins

<pre class="github-light" style="background-color:#fff;color:#24292e;" tabindex="0"><code>df = penguins_dffig, axes = plt.subplots(1, 3)for i, col in enumerate(['bill_length_mm','bill_depth_mm','flipper_length_mm']): df.plot(x=col, y="body_mass_g", kind="scatter", s=2, ax=axes[i])</code></pre>

![](https://pub.uni-bielefeld.de/download/3003418/3003721.svg)

As minimal example let's play the middle plot with bill_depth as onset and body_mass_g as frequency.
- Note that such as PMSon, including synth definition, mapping and rendering is a two-liner.

<pre class="github-light" style="background-color:#fff;color:#24292e;" tabindex="0"><code>SeqAGen([ (pam.linlin(x, 13, 23, 0, 5), (SinOsc.ar(freq=y*0.1) * 0.05 * Line(1,0,0.3))|Pan2.p(pos=0)) for x, y in df[['bill_depth_mm', 'body_mass_g']].to_numpy()]).gen_asig().play()</code></pre>


Your browser does not support the audio element.


<br>

For an example of a more generic and complete discrete parameter mapping sonification, let's
- map body mass to onset
- map bill length to pitch
- map bill depth to stereo position
- map flipper length to duration

For that we define a function asyn that returns a custom Agen.
We can regard it as an synth.

<pre class="github-light" style="background-color:#fff;color:#24292e;" tabindex="0"><code>def asyn(pitch=40, amp=0.2, att=0.001, dur=0.2, pos=0): sig = SinOsc.ar(pam.midi_to_cps(pitch)) env = Env([0, 1, 0], [att, dur], done="stop") agen = Pan2(sig*env*amp, pos) return agen </code></pre>

With the above-defined Agen we can render the sonification.

Different from above, let's add all rendered Agens into a sound canvas,
- using pyas .x (extend) array access
- which extends the canvas dynamically

In result we have the full sonification in a multi-channel Asig ason, which we can
- play via the Aserver.
- or e.g. for replaying at different rates (one-liner) 
- or to save our result to disk (one-liner).


<pre class="github-light" style="background-color:#fff;color:#24292e;" tabindex="0"><code>duration = 3
# mappingspitches = pam.linlin(df.bill_length_mm.to_numpy(), 30, 60, 0, 48) + 40posses = pam.linlin(df.bill_depth_mm.to_numpy(), 10, 24, -1, 1)durs = pam.linlin(df.flipper_length_mm.to_numpy(), 0, 300, 0.1, 0.4)onsets = pam.linlin(df.body_mass_g.to_numpy(), 2800, 6500, 0, duration)
# render agens into an Asig asonason = Asig(0, channels=2, label="PMSon")for i, onset in enumerate(onsets): ason.x[{onset:None}] += asyn(pitch=pitches[i], amp=0.05, dur=durs[i], pos=posses[i]).gen_asig()
# plot and play the sonificationason.plot(offset=2,lw=1).play(onset=0.3) </code></pre>
![](https://pub.uni-bielefeld.de/download/3003418/3003716.svg)

Your browser does not support the audio element.


### 2.2. Continuous Parameter Mapping Sonification
Let's sonify environmental data from the building data set, which contains the hourly readings of temperature, humidity, solar radiation etc.

![](https://pub.uni-bielefeld.de/download/3003418/3003720.svg)

Let's prepare for modulation (continuous PMSon is mostly about modulating parameters over time).
- it is useful to turn the multivariate time series in a multi-channel Asig.
- the data is hourly for 6 months, let's choose the sampling rate so that 1s audio is 1 month of data

<pre class="github-light" style="background-color:#fff;color:#24292e;" tabindex="0"><code>data_asig = Asig(df, sr=24*30, cn=['temp', 'humidity', 'solar', 'wind']) # 1 month= 1sec (unit)plt.figure(); data_asig.plot(offset=2); plt.grid()plt.xlabel('time [1s=1month] before resampling')ds = data_asig.resample(44100)</code></pre>

![](https://pub.uni-bielefeld.de/download/3003418/3003719.svg)

Let' assume we want to create a PMSon which
- maps temperature to pitch of a sine tone as MIDI note range [60, 84] 
- maps solar radiation to amplitude [0, 1]
- maps humidity to stereo position [left, right]
- maps wind speed to the amplitude of added white noise

We furthermore want 
- to render the time interval from [2.2, 4.5] (months)
- at a rate of 0.5, i.e. 1 month in 2 seconds

Here we go: the implementation is even shorter than the specification, and readible

<pre class="github-light" style="background-color:#fff;color:#24292e;" tabindex="0"><code># namespace idx for more readable channel referencingidx = SimpleNamespace(**{name: i for i, name in enumerate(ds.cn)})
ds_agen = PlayAsig(ds[{2.2:4.5}], rate=0.5)agsig = SinOsc.ar(freq=ds_agen[idx.temp].linlin(0, 1, 60, 84).midi_to_cps())agpan = Pan2.ar(agsig, pos=ds_agen[idx.humidity].linlin(0,1,-1,1))agall = agpan * ds_agen[idx.solar].linlin(0, 1, 0.2, 1) + WhiteNoise() * ds_agen[idx.wind]ason = agall.gen_asig().plot(offset=2, lw=0.1).play(onset=0.5)
</code></pre>

![](https://pub.uni-bielefeld.de/download/3003418/3003718.svg)



Your browser does not support the audio element.



## 3. Earcons, Spearcons, Auditory Icons with pya AGen
### Earcons

Earcons are short motifs, i.e. sequences of pitched tones, so earcons are best composed from its scores.

Here is a rather basic earcon specification
- for simplicity using a single synth mimicking steel drums
- If needed PlayAsig can be used as Sampler on musical tones

<pre class="github-light" style="background-color:#fff;color:#24292e;" tabindex="0"><code>def asyn(note=60, lvl=1, dur=1, pos=0): """a simple steel drum""" f0 = pam.midi_to_cps(note) agsig = klang([ (f0, 1), (f0*3.01, 0.2), (f0*5.3, 0.1), (f0*5.13, 0.1)]) return Pan2(agsig * pam.db_to_amp(lvl) * XLine(1, 0.01, dur=dur, done="stop"), pos)
def earcon(notes, bpm=90, base_note=60, play_onset=0): """a simple earcon player for a list of note dicts""" aear = Asig(1, channels=2, label="earcon") for ev in notes: ev = SimpleNamespace(**ev) aear.x[{ev.at * 120/bpm:None}] += asyn(note=ev.note+base_note, lvl=ev.lvl, dur=ev.dur).gen_asig() return aear.play(onset=play_onset)</code></pre>

<pre class="github-light" style="background-color:#fff;color:#24292e;" tabindex="0"><code># an earcon playing g - c - d - ee1 = earcon(notes=( dict(at=0, note=7, dur=1.2, lvl=4, pos=-1), dict(at=0.2, note=0, dur=0.1, lvl=1, pos=-0.5), dict(at=0.3, note=2, dur=0.1, lvl=1, pos= 0.5), dict(at=0.4, note=4, dur=0.6, lvl=5, pos= 1)), bpm=90, base_note=60)
# an earcon playing g - d# - c (c-minor downward arpeggio) - 1 octave lowere2 = earcon(notes=( dict(at=0, note=7, dur=2, lvl=1, pos=0), dict(at=0.05, note=3, dur=1.5, lvl=1, pos=0), dict(at=0.1, note=0, dur=1, lvl=1, pos=0)), bpm=90, base_note=48, play_onset=2)
# to render both earcons as sequence into an audio file useearcon_sequence = e1.append(e2)earcon_sequence.plot(lw=0.5)</code></pre>

![](https://pub.uni-bielefeld.de/download/3003418/3003717.svg)

Your browser does not support the audio element.


### Auditory Icons

- Often, everyday sounds are used, e.g. crushing paper to represent file deletion.
- The ideal way to implement these is to simply load the audio files using Asig() 
 and play that either directly via 
 
 ```PlayAsig(Asig("data/klirr.wav"), rate=0.5).gen_asig().play()```


## 4. Model-based Sonification with pya AGen

The cell below is a minimalistic implementation of the data sonogram model. It includes:
- loading of data
- feature selection / filtering
- scatter plot of data
- definition of a sonification model implementation as constrained mapping
- Mouse click interaction event anchored to the scatter plot

<pre class="github-light" style="background-color:#fff;color:#24292e;" tabindex="0"><code>fig = plt.figure(figsize=(4,4))df = penguins_dfdf.plot(x="bill_depth_mm", y="bill_length_mm", kind="scatter", s=3, ax=plt.gca());data = df.loc[:, ["bill_depth_mm", "bill_length_mm", "flipper_length_mm"]].to_numpy()
velocity = 3
def son_fn(click_pos): (WhiteNoise(200) * Line(0.6,0,0.2)**4).gen_asig().play() # initial noise
 distances = np.linalg.norm(data[:,:2] - click_pos, axis=1) # distances to clickpos for i in np.argsort(distances)[:25]: # dispatch some neighbors' sounds # sonification model constraints / defines sonic parameters onset = distances[i] / velocity freq = pam.midi_to_cps(pam.linlin(data[i, 2], 150, 250, 50, 90)) amp = pam.db_to_amp(pam.linlin(distances[i], 0, 3, -6, -40)) dur = 0.8 pan = 1 if data[i,0]-click_pos[0] &gt; 0 else -1 # generate data point sound ag = SinOsc(freq) * Line(1, 0, dur)**4 Pan2(ag * amp, pan).gen_asig().stereo().play(onset=onset)
def on_click(event): if event.inaxes is not None: # if inside plot area son_fn(np.array([event.xdata, event.ydata]))fig.canvas.mpl_connect('button_press_event', on_click)</code></pre>


Your browser does not support the video element.


Notice that 
- most of the steps (data handling, plotting, interaction, mapping, 
parameter control) are done best in a programming environment anyway. 
- the need for a sound rendering comes in at two locations in the son_def function:
 - the noise signal to provide feedback that and when the model has been excited
 - the data point sounds as the expanding wave reaches them (in the for loop)
- for both these cases, a 1-2 line pya-Agen specification does the job and forgoes
 the need to involve an extra dependency.