A07:
MUSICAL ACOUSTICS

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022 in Gyeongju, Korea

ABS-0031

Numerical investigation of effects of mouthpiece geometry on
flow and sound generation in a single-reed instrument
Tsukasa YOSHINAGA1*; Hiroshi YOKOYAMA1; Tetsuro SHOJI2; Akira MIKI2; Akiyoshi IIDA1
1

Department of Mechanical Engineering, Toyohashi University of Technology, Japan
2

Research and Development Division, Yamaha Corporation, Japan

ABSTRACT
Single-reed instruments produce sounds with fluid-structure-acoustic interactions in a mouthpiece, and it has
been known that a small geometrical change may affect the sound quality. In this study, we investigate theeffects of mouthpiece geometry on the flow and sound generation in a single-reed instrument by conducting
numerical flow simulations coupled with a structural analysis of reed vibration. The effects of mouthpiecegeometry were explored by changing the mouthpiece outlet diameter. The flow and sound characteristics
were predicted by solving the compressible Navier-Stokes equations with high-accuracy finite difference
schemes, while the interaction with the reed vibration was simulated by solving a one-dimensional dynamic
beam equation. The results showed that the waveform of the reed tip opening was changed by the mouthpiece
diameter, and the amplitudes of even number harmonics were reduced by the inwards tapers and matchedmouthpiece diameter to the resonator inlet. The flow visualization revealed that the vortex rings are generated
at the mouthpiece outlet, and the expansion rate of air in the mouthpiece increased earlier with the inwards
tapers, suggesting that the timing of air expansion changed the reed tip waveforms and the sound quality ofthe single-reed instrument.
Keywords: Single-reed instrument, CFD, Aeroacoustics, Fluid-structure interaction

1. INTRODUCTION
The single-reed instruments, like a clarinet and saxophone, produce the sound with the interactions
among the airflow from the mouth, reed oscillation, and acoustic resonance in the resonator. To
investigate the nature of sound generation, many numerical simulations have been conducted for many
years from one-dimensional modeling (e.g., 1-3) to three-dimensional flow simulations (e.g., 4-6).
These numerical studies mainly focused on the comparison with the experiments and investigation of
the player’s characteristics like blowing pressure and lip forces (5-6). Even portato and staccato
articulations were reproduced in the simulation (3). However, a few studies examined the effects of
instrument geometry on sound quality with the numerical simulations. Although da Silva et al., (4)performed the flow simulations on the different tip shapes of the mouthpiece geometry, the resulting
sound characteristics are still not clear.
It has been known that the instrument geometry, especially mouthpiece geometry, has a great
impact on sound generation (7) and its effects were widely discussed (8-11). Pillinger (11)investigated the effects of internal and external dimensions of mouthpieces on the quality of soundgenerated by an artificial blower. In addition, Ozdemir et al., (12-13) parametrically changed thedimensions of the saxophone mouthpiece with a 3D printer and investigated the sound quality as wellas the blowing resistance and flexibility. However, detailed mechanisms of how the sound quality waschanged are still unclear without the numerical simulation.
Therefore, in this study, we constructed five mouthpieces of Saxonett, one of the simplest singlereed instruments, with different bore diameters and conducted the flow simulation for the entire
instrument geometry to clarify the effects of the mouthpiece geometry on the flow and soundgeneration. By calculating the compressible flow equations coupled with a one-dimensional dynamic
beam equation, the physical phenomena occurring around the instrument are simulated in the computer* yoshinaga@me.tut.ac.jp

with different mouthpiece dimensions. The information about a tendency with the parametrical changeof mouthpiece dimensions will contribute to the better design of instrument geometry.

2. METHODS
2.1 Single-reed Instrument
We use a Saxonett (JRS700, Jupiter) to simulate the sound generation of a single-reed instrument.
The Saxonett consists of a clarinet mouthpiece and recorder-like resonator, and the mouthpiece is set
in the artificial blower to mimic the player’s mouth. The Saxonett with the artificial blower is depicted
in Fig. 1 (a). The pressure chamber of the artificial blower has dimensions of 70 × 40 × 43 mm (V =120 cm3 ) which reproduces the average volume of the human vocal tract and trachea. On the far sidefrom the mouthpiece, the inflow region is set with the constant flow supply. The tone holes werecovered by the cylinder and the instrument produces the lowest note of C4. The resonator’s innerdiameter is 13.3 mm, and the total length from the mouthpiece tip to the resonator outlet yields 307
mm. The axis x1 is set in the longitudinal direction of the resonator, and the origin is set at the tip ofthe mouthpiece. The reed is fixed to the mouthpiece at the point of ligature (x1 = 38.5 mm), and thelip force is set at x1 = 10.5 mm with the lip width of 4.1 mm.
Details of the mouthpiece geometry are shown in Fig. 1 (b). From the inlet of the tip opening, theside walls of entry gradually expanded to the throat, and the throat connects to a cylindrical bore. The
total length of the mouthpiece is 89.6 mm, and the bore has a length of 48.7 mm. In this study, thediameter of the bore outlet is changed from d = 11.3 to 15.3 mm. As a result, the taper angle of thebore increased from −1.176° to 1.176° (minus value indicates the inwards taper). The dimensions are
summarized in Table 1. The original diameter of the Saxonett mouthpiece bore is approximately 15
mm. However, the bore outlet smoothly connects with the resonator when d = 13.3 mm, because theinner diameter of the resonator is 13.3 mm. This smooth connection more accurately mimics theconnection of the mouthpiece and barrel for the standard clarinet. There is a cavity with a length of
1.8 mm between the mouthpiece and the resonator for all cases.

Figure 1 – Schematic of single reed instrument. (a) Resonator and mouthpiece of Saxonett andpressure chamber of artificial blower. (b) Mouthpiece geometry with different diameters. The dottedline shows the cross-section of d = 11.3 and 15.3 mm.

Total length ofmouthpiece (mm)89.6

Table 1 – Dimensions of mouthpieces
Bore length
Outlet Diameter d
Cross-sectional area
(mm)
(mm)
A (mm2)
11.3
100.3
48.7

Taper angle(deg)−1.176

12.3

118.8

−0.588

13.3

138.9

0

14.3

160.6

0.588

15.3

183.9

1.176

2.2 Numerical Simulation
To consider the fluid-structure-acoustic interactions in the single-reed instrument, we numericallysolve the three-dimensional compressible Navier-Stokes equations. The compressible flow simulation
enables to calculate the sound generation from the flow passing through the oscillating reed. In
addition, a one-dimensional beam equation was solved to consider the fluid-structure interaction of
the reed oscillation. The deforming reed with the surrounding airflow is expressed by the volume
penalization (VP) method, one of the immersed boundary methods (14). The VP method expresses thewall with porous material in the structured grids, and we fix the parameter of porosity to realize theacoustic reflectivity of 99%. The spatial derivatives are solved by a sixth-order-accuracy compact
finite difference scheme, while the time integration is performed using the third-order-accuracy
Runge-Kutta method. To simulate the turbulent flow in the mouthpiece, the large eddy simulation(LES) is applied with tenth-order-accuracy spatial filter. These methodologies enable to simulate theinteractions between jet flapping and acoustic resonance in a recorder (15).
The reed oscillation is simulated by solving a dynamic Euler-Bernoulli beam equation with theexternal force of airflow, lip, and contact forces. The lip force is calculated as a linear stiffness which
is proportional to the distance between the lip position and the reed surface. The spatial derivatives
are solved by a second-order accuracy finite-difference scheme, whereas the time integration is
performed using the implicit θ-scheme (16). The material constants like Young’s modulus and reed
density are chosen from the experimental values, while the damping and internal loss coefficients aremanually tuned by comparing the reed tip waveforms of the experimental measurements (17). The
details are presented in our previous paper (6). In addition, the lip stiffness is adjusted to obtain the
self-sustained oscillation with the typical tip opening.
The computational grids are shown in Fig. 2. The minimum grid size is 0.025 mm at the tip of themouthpiece, and the grid size is gradually expanded towards the outlet to reduce computational costs.
To resolve vortices generated near the bore outlet of the mouthpiece, the grid sizes around theconnecting part between the mouthpiece and resonator are kept smaller than 0.2 mm. The total grid
number is approximately 1.72 × 108 .
As a boundary condition, a uniform flow of flow rate of 138 cm3 /s and pressure of 5.5 kPa wereset at the inflow region of the artificial blower. We confirmed that the mean pressure inside thechamber was kept at 5.5 kPa for the entire simulation time for all mouthpiece geometry. At the outletof the computational domain, a non-reflecting boundary with a buffer region is set to prevent thereflection from the outlet. The far-field sound was evaluated by sampling the pressure at 100 mm fromthe outlet of the resonator. The time step for the time integration was set to 2.68 × 10-8 s. As the initialcondition, the pressure values in the artificial blower, mouthpiece, and resonator are set to 5.5 kPa,
and the initial tip opening was set to 1.1 mm which is the rest position of the reed for the original
Saxonett mouthpiece.

Figure 2 – Computational grids around the mouthpiece. The brown and yellow parts show the regions of therigid wall and oscillating reed, respectively.

3. RESULTS AND DISCUSSION
The waveforms of the tip opening h are plotted from the initial position to t = 52 ms formouthpieces with d = 11.3, 13.3, and 15.3 mm in Fig. 3. At the initial state, no lip force was applied
for three cycles (up to t = 14 ms) and added the lip force with the stiffness of Klip = 2.3 × 105 N/m2 at
t = 14 ms. Then, the lip stiffness was slightly decreased to Klip = 2.2 × 105 N/m2 at t = 28 ms to keepthe amplitudes of h around 0.5 mm for the rest of the time steps. The stiffness of Klip = 2.3 × 105 and2.2 × 105 N/m2 can be converted to approximately 2.24 and 2.14 N, respectively, based on the
maximum of h. By keeping the lip stiffness, the self-sustained oscillation was obtained with almost
the same displacement amplitudes. The oscillation frequency was the same with different diameters
until the lip force was applied, and then, the frequency increased with the lip force for the smaller
diameter (d = 11.3 mm) whereas the frequency decreased with the larger diameter (d = 15.3 mm),compared to d = 13.3 mm.
The last three cycles of each mouthpiece diameter are plotted in Fig. 4 with normalized time by its
period. The time t = 0 is defined at the beginning of the closed phase. The fundamental frequenciesof the mouthpiece with d = 11.3, 12.3, 13.3, 14.3 and 15.3 mm were f0 = 271.8, 269.0, 268.3, 265.7,
and 264.5 Hz, respectively. With the increase of d from 11.3 to 13.3 mm, the maximum of h decreased
from 0.55 to 0.47 mm. Then, with the increase of d from 13.3 to 15.3 mm, the maximum of h increased
from 0.47 to 0.51 mm. The maximum of h appeared in the earlier phase when d < 13.3 mm, whereas
the maximum of h appeared in the later phase when d > 13.3 mm. With the smooth connection fromthe mouthpiece to the resonator (d = 13.3 mm), the waveform became a rectangular shape in the open
phase, and this waveform was similar to those observed in the previous experiments of the clarinet
with relatively low tones (18-19).

Figure 3 – Waveform of tip opening h with different diameters d = 11.3, 13.3, 15.3 mm.

Figure 4 – Waveform of tip opening h with normalized time for the last three cycles.
The spectra of sound pressure sampled at 100 mm from the resonator outlet are plotted in Fig. 5(a) for d = 11.3, 13.3, and 15.3 mm. The frequency is normalized by the fundamental frequency f0 of
the reed oscillation. The maximum amplitude of approximately 99 dB appeared at the fundamental
tone for all cases, and the amplitude decreased with the increase of frequency. The even number
harmonics (i.e. second and fourth harmonics) were 10 to 30 dB smaller than the other odd number
harmonics (i.e. first, third, and fifth harmonics), and this result indicates that the Saxonett in thecurrent setup produced the clarinet-like sounds (7) in the same way as the experiment (17).
The discrepancies in sound pressure levels (SPLs) of each harmonic from that of the fundamental
tone (SPLf0 ) are plotted in Fig. 5 (b). The second harmonic decreased from SPL−SPLf0 = −30 to −39
dB with the increase of d from 11.3 to 13.3 mm and increased from SPL−SPLf0 = −39 to −34 dB with
the increase of d from 13.3 to 11.3 mm. These values are correlated with the shape of the tip waveform
h, indicating that the rectangular shape of the waveform in h with the smooth connection of d = 13.3mm reduces the amplitude of the second harmonic. While the third and fifth harmonics were almostthe same values with different d, the fourth and sixth harmonics increased with the increment of d.
These results suggest that the second harmonic is correlated with how the mouthpiece smoothly
connects with the resonator, whereas higher even harmonics (fourth and sixth) are correlated with theangle of taper of the mouthpiece. If we assume that the typical clarinet tone is formed by the smaller
amplitudes of even harmonics (7), the taper angle should be as small as possible with the smoothconnection to the resonator in the current setup. Meanwhile, the higher amplitudes in higher harmonics
of d = 15.3 mm indicate that clearer and brighter tones are generated with the larger bore diameters,
and this result is consistent with the previous experiment (11).

Figure 5 – Sound pressure levels sampled at 100 mm from the resonator outlet. Spectra of sound pressures
are plotted in (a) with different diameters d = 11.3, 13.3, and 15.3 mm. The frequency is normalized by thefundamental frequency f0. The differences of amplitudes of harmonics from the fundamental tone are plottedin (b).
The instantaneous flow pressures and vortex structures in the instrument are visualized in Fig. 6 for d =11.3, 13.3, and 15.3 mm at t/T = 0.9. To visualize the vortex structures, the second invariant of velocity
gradient tensor Q is calculated, and iso-surfaces of Q = 107 s-2 are depicted with a contour plot of the pressure
distribution on the medial plane of the instrument. At t/T = 0.9, the reed tip started closing and the velocity
at the mouthpiece outlet was the maximum. The turbulent jet entered through the tip opening, and the vortices
are dissipated at the throat of the mouthpiece for all cases. When d = 11.3 mm, the inwards taper increased
the velocity in the bore, and vortex rings were generated from the outlet edge of the bore. On the contrary,
the vortex rings were generated at the resonator edge with d = 15.3 mm, because the flow directly impinged
on the resonator edge due to the larger diameter of the mouthpiece. The vortex tubes were also generated at
the cavity of tone holes. The pressure contours showed that the pressure decreased earlier with the smaller d,whereas the higher-pressure values remained in the resonator with the larger d.
The waveforms of pressure and velocity in the longitudinal direction u1 at the center of the mouthpieceoutlet (x1 = 89.6 mm) are shown in Fig. 7 for one cycle. The pressure of d = 11.3 mm increased earlier thanthat of d = 13.3 mm, whereas the pressure of d = 15.3 mm increased later than that of d = 13.3 mm in thereed’s opening phase. The maximum and minimum values of pressure waveforms were almost the same fordifferent mouthpiece diameters. In contrast, the velocity at the mouthpiece outlet started increasing at almost
the same time for different diameters. However, the slope of the increase was different, and the velocity
reached the maximum of 11.1 m/s for d = 11.3 mm, while the maximum was 9.1 m/s for d = 15.3 mm.
To elucidate the relationship between the flow and reed tip waveform, the expansion rate ρU2 − ρU1 in
the mouthpiece was calculated with mass flow rate ρU1 at the tip opening and mass flow rate ρU2 passing
through the bore outlet at x1 = 89.6 mm. The waveforms of the expansion rate with three diameters are plottedin Fig. 8. The gray part in Fig. 8 indicates the positive pressure period in the mouthpiece. When the pressure
became positive at around t/T = 0.4, the expansion rate started decreasing, and the air in the mouthpiececompressed in all cases. The compressed air pushes the reed from the inside, and the reed tip started opening.
Then, at t/T = 0.55, the expansion rate started increasing, pushing the reed further to open. This expansion
rate increased earlier with the smaller diameter d = 11.3 mm, whereas the expansion rate increased later withthe larger diameter d = 15.3 mm. These timings of increase in the expansion rate are correlated with those of
the reed tip waveforms, indicating that the different bore diameters changed the expansion rate of air in themouthpiece and changed the reed tip waveforms. This tip opening waveform changed the frequency

characteristics of sound sources near the reed tip and resulted in different far-field sound quality. The threedimensional fluid-structure interaction simulation enabled to show how the flow and sound generations werechanged in the mouthpiece with different dimensions, and this helps makers to improve the design of thesingle-reed instrument.

Figure 6 – Instantaneous pressure distributions and vortex structure visualized by the iso-surfaces of second
invariant of velocity gradient tensor Q = 107 s-2. The pressure and vortices are depicted with d = 11.3 mm (a),13.3 mm (b) and 15.3 mm (c).

Figure 7 – Waveforms of pressure (a) and velocity u1 (b) at the center of the mouthpiece outlet (x1 = 89.6mm). The time is normalized by the period of one cycle.

Figure 8 – Expansion rate ρU2 − ρU1 in the mouthpiece. The mass flow rate ρU1 is measured at tip opening
whereas ρU2 is measured at the mouthpiece outlet (x1 = 89.6 mm). The time is normalized by a period of one
cycle. The gray part shows the positive pressure period, while the other parts show negative pressure periods.

4. CONCLUSIONSIn this study, the effects of mouthpiece geometry on the flow and sound generation in a singlereed instrument were investigated by conducting compressible flow simulations. By changing thediameter of the bore outlet, the phase of the maximum tip opening was shifted, and the amplitudes ofeven number harmonics were changed. The results suggest that the smooth connection between themouthpiece and resonator reduces the amplitude of second harmonics, while the increase of theinwards taper angle reduces the amplitudes of the other even harmonics and produces more clarinetlike sounds. The flow visualization revealed that the vortex rings were generated at the edge of themouthpiece connection, and the expansion rate increased earlier with inwards tapers, whereas the
expansion rate increased later with the outwards tapers. These results suggest that the expansion rateof air in the tapered mouthpiece changes the reed tip waveforms and resulting sound quality. In future
work, we need to evaluate the blowing resistance and flexibility from the current analysis for thedesign of better instruments. In addition, further analysis of the relationship between the harmonic
amplitudes and the expansion rate in different playing conditions is expected.

ACKNOWLEDGEMENTS
This work was partly supported by JSPS Grant-in-Aid for Early-Career Scientists (KAKENHI,
Grant number: JP20K14648). We acknowledge H. Onitsuka, K. Arimoto, and R. Baba of Yamaha
Corporation for helpful comments and discussion.

REFERENCES
1. Backus, J. Small‐vibration theory of the clarinet. J Acoust Soc Am. 1963; 35(3): 305-313.
2. Stewart, S. E., Strong, W. J. Functional model of a simplified clarinet. J Acoust Soc Am. 1980; 68(1):109-120.
3. Chatziioannou, V., Schmutzhard, S., Pàmies-Vilà, M., Hofmann, A. Investigating clarinet articulation
using a physical model and an artificial blowing machine. Acta Acust United Acust. 2019; 105(4): 682694.
4. da Silva, R. A., Scavone, G. P., van Walstijn, M. Numerical simulations of fluid-structure interactions
in single-reed mouthpieces. J Acoust Soc Am. 2007; 122(3): 1798-1809.
5. Giordano, N., Thacker, J. W. Navier-stokes-based model of the clarinet. J Acoust Soc Am. 2020; 148(6):3827-3835.
6. Yoshinaga, T., Yokoyama, H., Shoji, T., Miki, A., Iida, A. Global numerical simulation of fluidstructure-acoustic interaction in a single-reed instrument. J Acoust Soc Am. 2021; 149(3): 1623-1632.
7. Fletcher, N. H., Rossing, T. D. The physics of musical instruments. Springer, New York, USA; 2012.

8. Wehner, W. L. The effect of interior shape and size of clarinet mouthpieces on intonation and tone quality.
J Res Music Educ. 1963; 11(2): 131-136.
9. Gibson, L. Fundamentals of Acoustical Design of the Soprano Clarinet. Music Educators Journal, 1968;
54(6): 113-115.
10. Backus, J. The clarinet reed and its relation to the clarinet mouthpiece. J Acoust Soc Am., 1978; 64:
S170-S171.
11. Pillinger, E. The effects of design on the tone and response of clarinet mouthpieces. Doctoral dissertation,
London Guildhall University, 2000.
12. Ozdemir, M., Cascini, G. An experiment-driven mass-personalisation model: Application to saxophonemouthpiece production. Proc of the Design Society: DESIGN Conference, May 2020; pp. 1037-1046.
13. Ozdemir, M., Chatziioannou, V., Verlinden, J., Cascini, G., Pàmies-Vilà, M. Towards 3D printed
saxophone mouthpiece personalization: Acoustical analysis of design variations. Acta Acust, 2021; 5:
46.
14. Liu, Q., Vasilyev, O. V. A Brinkman penalization method for compressible flows in complex geometries.
J Comput Phys. 2007; 227(2): 946-966.
15. Yokoyama, H., Miki, A., Onitsuka, H., Iida, A. Direct numerical simulation of fluid–acoustic interactions
in a recorder with tone holes. J Acoust Soc Am, 2015; 138(2): 858-873.
16. Avanzini, F., Van Walstijn, M. Modelling the mechanical response of the reed-mouthpiece-lip systemof a clarinet. Part I. A one-dimensional distributed model. Acta Acust united Acust, 2004; 90(3): 537547.
17. Yokoyama, H., Kobayashi, M., Iida, A. Analysis of flow and acoustic radiation in reed instruments bycompressible flow simulation. Acoust Sci Tech. 2020; 41(5): 739-750.
18. Backus, J. Vibrations of the reed and the air column in the clarinet. J Acoust Soc Am. 1961; 33(6): 806809.
19. Idogawa, T., Kobata, T., Komuro, K., Iwaki, M. Nonlinear vibrations in the air column of a clarinet
artificially blown. J Acoust Soc Am. 1993; 93(1): 540-551.

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022 in Gyeongju, Korea

ABS-0055

Spectral composition of the attack portion of a recorder tone:
How the player can affect the tone color
Nicholas GIORDANO1; Jared W. THACKER
Auburn University, USA

ABSTRACT
We have used Navier-Stokes-based simulations to study the spectral content of tones produced by a soprano
recorder. Our focus is on the attack portion of a recorder tone and how it can be influenced and controlled by
the player. We show that both the blowing speed and the time taken to initiate blowing can be used to control
the amplitudes and time variations of the upper partials during the attack. Our simulations are consistent withprevious experimental results reported in the literature for flue instruments and with a new analysis of tones
produced by a soprano recorder. The results illustrate how a player can control the expressiveness of arecorder tone.
Keywords: Musical instruments, recorder, Navier-Stokes

1. INTRODUCTION
Physical modeling has yielded important insights into tone production by a wide variety of musicalinstruments. Wind instruments have been especially challenging for such studies, since thefundamental equations of fluid mechanics, the Navier-Stokes (NS) equations, are a complicated set of
nonlinear equations that require a computational approach when applied to a realistic instrument. Evenso, available computer resources now make it possible to study a variety instruments using the NS
equations. During the course of such simulations we have observed interesting that interesting spectral
content is predicted during the attack phase of tones produced by flue instruments. This behavior was
first noted some time ago in NS simulations in (1); in this paper we report further studies which givesome qualitative insights into the behavior and also show that similar spectra features are found insimple tones produced by a recorder played by a beginner (i.e., one of the authors). It is also worth
noting that unknown to the authors at the time of reference (1), the same basic behavior has beenreported in a number of previous studies of flue instrument tones.

2. Modeling Details
We use an explicit finite-difference-time-domain algorithm to numerically solve the compressible
3D Navier-Stokes equations for a soprano recorder. Simulation details are described elsewhere (1).
The numerical model had dimensions similar to the Yamaha YRS23 soprano recorder, with a chamfer
on the lower edge of the windway (Fig. 1) and an inner diameter that tapered as one moves from thelabium to the open end. The simulations positioned the instrument inside a closed “box” (which could
be visualized as a very small room) with walls that reflect and damp sound produced by the instrument.
The calculations yielded the air density and velocity inside and outside the recorder. Variations in thedensity are proportional to the variations of the pressure, and the results below shown below show thesound pressure at a location above the labium. The sound pressure beyond and off the axis of the open
end exhibited similar behavior. In previous work we have verified [1] that the absolute magnitude of
the sound pressure inside the instrument was approximately 130 dB, which is comparable to that foundin real instruments. The calculated value of the sound pressure outside the recorder and about 10 cm
distant and off axis from the open end was smaller by typically two orders of magnitude.
All of the results in this paper were obtained with the tone holes in Fig. 1 all closed so that the1

njg0003@auburn.edu

instrument produced its lowest note. For the soprano recorder this is C5, which has a fundamental
frequency of approximately 525 Hz. These tones also contained components (higher partials) at
frequencies that are approximately integer multiples of the fundamental frequency.
labium and window

tone holes

Figure 1. Model of the recorder studied in the simulations. The distance from the exit of the channel thelabium was 4.0 mm, and from the end of the channel to the end of the resonator was 28 cm. The model
contained tone holes that could be open or covered as if by a player. For all the results in this paper the tone
holes were all closed.

3. Simulated Tones
Figure 2a shows the sound pressure outside the instrument for the note C5 when the recorder plays
at a relatively low volume. Specifically blowing was initiated at t = 0 with the blowing speed ramped
up uniformly to its final value of 10 m/s (in the windway) at t = 5 ms. This choice of ramp up time isat the low end of times reported in experiments in the literature but seems consistent with what canbe achieved by human players. This blowing speed is then maintained until 250 ms at which time theblowing speed is ramped to zero and the tone dies away. From Fig. 2b it is seen that a steady stateoscillation with a frequency of about 525 Hz (corresponding to the pitch of C5) is reached by about t= 60 ms and maintained thereafter. The time it takes to reach steady state is much g reater than theramp up time due to the time it takes for the sound to reach steady state inside the recorder tube andin the somewhat reverberant enclosure than surrounds the instrument.
Figure 3 shows spectra analysis of the waveform in Fig. 2. These d ifferent spectra were calculatedusing Fast Fourier transforms of data segments extending over about 20 ms centered at times given in
the caption. The blue spectrum, calculated at 80 ms, shows the steady state behavior. This spectrum
is dominated by the first partial at frequency f1 = 525 Hz, with only a very small peaks at the second
partial (f2 approximately 1050 Hz) and the third partial (f3 approx. 1600 Hz). The position of thesecond partial is extremely weak so we have assumed its frequency is twice f 1. Interestingly, theposition of the third partial at the earliest time appears to be a bit higher than found at longer times; asimilar shifting of the partial frequencies will be seen at other blowing pressures below.
1.0

1.0(b) sound pressure vs. time

0.5

sound pressure (arb. units)

sound pressure (arb. units)(a) sound pressure vs. time

0
-0.5
-1.0
-1.5
0

0.1

0.2
t (s)

0.3

0.5
0
-0.5
-1.0
-1.5
0

0.02

0.04

0.06

0.08

0.1

t (s)

Figure 2. Simulated sound pressure versus time for the note C5 played by a soprano recorder at a blowing
speed of 10 m/s and a ramp up time of 5 ms. (a) Full note. (b) Early portion of the tone.

Figure 3. Spectra for the sound pressure waveform in Fig. 2 showing the approximate positions of the first,
second, and third partials of the tone at different times.Black: Fourier spectrum calculated using data
centered at t=20 ms using data extending approximately 10 ms before and after that time in Fig. 2b. Red:
Spectrum at t=40 ms. Blue: Spectrum at t=80 ms. Note that the vertical scale is logarithmic.
Another way to view how the spectral composition of the tone in Fig. 2 varies during the courseof the tone is to plot the peak powers of the different partials as functions of time; see Fi g. 4. The firstpartial dominates during the entire course of the tone, with the power at the second and third partials
being typically three orders of magnitude smaller at all times. In other words, the amplitudes of thepartials are typically smaller by a factor of at least 30. This tone, produced at a low blowing pressure,is thus very close to being a pure tone.

Figure 4. Power at the peaks of the first, second, and third partials in Fig. 3 as a function of time during thefull course of the tone. Black: First partial. Red: Second partial. Blue: Third partial.
The spectral properties of the simulated tones change as the recorder is blown harder. Figure 5 shows
results for the early part of a tone produced with a steady state blowing speed of 12 m/s. The only difference
between this tone and the one in Fig. 2 is that the steady state blowing speed was increased from 10 to 12
m/s. There is now a region between about 30 and 60 ms in which there seems to be a new spectral component(in addition to the first partial near 525 Hz). Figure 3 shows spectra analysis of the waveform in Fig. 2. These
different spectra were calculated using Fast Fourier transforms of data segments extending over about 20 ms
centered at times given in the caption. The orange spectrum, calculated at 100 ms, shows the steady statebehavior. This spectrum is dominated by the first partial at frequency f1 = 525 Hz, with only a very small
peaks at the second partial (f2 approximately 1050 Hz) and the third partial (f3 approx. 1600 Hz). At earlytimes, especially at 30, 40 and 60 ms, the third partial is quite significant. Also apparent at early times aresubstantial components around 800 and 1300 Hz; these frequencies do not appear to be related to the base(f1) component of the tone and we refer to them as inharmonic partials. We will consider their origin below.

sound pressure (arb. units)

0.5

sound pressure vs. time

0

-0.5

-1
0

0.02

0.04
t (s)

0.06

0.08

Figure 5. Simulated sound pressure versus time during the initial (attack) portion of the tone for the note C5
played by a soprano recorder with a blowing speed of 12 m/s and a ramp up time of 5 ms.

Figure 6. Spectra for the sound pressure waveform in Fig. 5 showing the approximate positions of the first,
second, and third partials of the tone at different times. Also indicated are spectral components referred toas inharmonic partials in the text. Black: Fourier spectrum calculated using data centered at t=20 ms using
data extending approximately 10 ms before and after that time in Fig. 5b. Red: Spectrum at t=30 ms. Blue:
Spectrum at t=40 ms. Green: Spectrum at t=60 ms; Orange: Spectrum at 100 ms.
Figure 7 shows how the power at the different partials vary with time for the tone produced byblowing at 12 m/s. While the first partial again dominates throughout the tone, at early times (before
about 30 ms) the third and inharmonic partials are about an order of magnitude weaker than the firstpartial, corresponding to an amplitude smaller by about a factor of 3-5. This is consistent with what
is seen by eye in the waveform in Fig. 5.
We next consider a simulated tone produced by blowing at the even larger blowing speed of 15
m/s. Again, all the other conditions and parameters are the same as for the two tones considered earlier.
The resulting waveform at the first part of this new tone is shown in Fig. 8. While the behavior at longtimes is again nicely periodic with the expected frequency (525 Hz) a range of times during which
other spectral components are important is again seen. In addition, the time required to reach steadystate is significantly longer than found at a blowing speed of 12 m/s, which was itself longer thanfound at 10 m/s. The length and spectral nature of the attack portion of the tone thus varysystematically as the blowing speed is increased.

Figure 7. Power at the peaks of the first, second, and third partials (labeled as f1, f2, and f3), and theinharmonic partial near 800 Hz (fin) in Fig. 6 as functions of time.

sound pressure vs. timesound pressure (arb. units)

1

0

-1

-2
0

0.05

0.1
t (s)

0.15

0.2

Figure 8. Simulated sound pressure versus time during the initial (attack) portion of the tone for the note C5
played by a soprano recorder with a blowing speed of 15 m/s and a ramp up time of 5 ms.
Spectra at several times during the course of this tone are shown in Fig. 9. The first partial is seen
to increase with time, while the second partial initially increases and then decreases as the steady stateis reached. The third partial behaves in an opposite manner, being small initially and then increasing
at long times. In addition, the second partial seems to be somewhat smaller than 2xf1 at early times
and then shifts to a slightly higher frequency at longer times. The uncertainties for the third pa rtial
are greater, but it too seems to shift some as time progresses.
The behavior of the different partials is shown in Fig. 10 and we see that the second partial now
dominates until abut 80 ms before becoming smaller than the third partial after about 1 60 ms. The
spectral composition of the tone is thus complex and varies substantially during the course of thistone, which a listener would presumably still recognize as C5.

Figure 9. Spectra for the sound pressure waveform in Fig. 5 showing the approximate positions of the first,
second, and third partials of the tone with blowing speed 15 m/s at different times. Black: Fourier spectrum
calculated using data centered at t=30 ms using data extending approximately 10 ms before and after thattime in Fig. 5b. Red: Spectrum at t=80 ms. Blue: Spectrum at t=100 ms. Green: Spectrum at t=200 ms.

Figure 10. Power at the peaks of the first, second, and third partials in Fig. 9 as a function of time during
the full course of the tone. Black: First partial. Red: Second partial. Blue: Third partial.
The results in Figs. 2-10 show that our simulations predict a systematic variation of the tone color
as the recorder is blown harder, with an essentially pure tone at low blowing speeds to a tone in which
the attack is dominated by the second partial. We should note that these tones are all in the blowing
range in which C5 is produced; the regime change transition to C6 occurs at a higher blowing speed
near 18 m/s. That transition and a fuller discussion of these ton es will be discussed in a future paper.

Figure 10. Simulated sound pressure versus time during the initial (attack) portion of the tone for the note
C5 with a blowing speed of 12 m/s and a ramp up time of 10 ms.

Figure 12. Spectra for the sound pressure waveform in Fig. 11 showing the approximate positions of thefirst (f1), third (f3), and inharmonic (fin) partials of the tone with blowing speed 12 m/s and ramp up time
10 ms, at different times during the tone (Fig. 11). Black: Fourier spectrum calculated using data centered
at t=50 ms using data extending approximately 10 ms before and after that time in Fig. 11. Red: Spectrum
at t=70 ms. Blue: Spectrum at t=100 ms.

4. EXPERIMENTAL OBSERVATIONS
Tones produced by flue instruments – organ pipes, recorders, and flutes – have been studied formany years by many researchers. The phenomena of interest to us in this paper have been observed
previously (see for example Refs. [2-7], which consider both organ pipes and recorders), but to thebest of our knowledge have not been analyzed in quite the ways or in the kind of detail for a particular
instrument as required for comparison with the modeling work presented earlier in this paper. We
therefore next present an analysis of a single tone produced by a real soprano recorder and analyze
that tone in the same way as we have analyzed the simulated tones discussed earlier. The recorder isa Yamaha model YRS25 (a student quality instrument composed of plastic), and we consider thelowest note produced by this instrument with all tone holes closed, C5, the same tone as consideredin our simulations.
Figure 11 show the waveform for a typical C5 tone produced by the YRS25 blown by one of theauthors; (a) shows the entire tone while (b) shows the initial, attack portion in more detail. An attempt
was made to ramp the blowing speed smoothly to a final value that was then maintained until the endof the tone. The periodicity achieved around 0.08 ms continued until the tone finished.

2(b) sound pressure vs. timesound pressure vs. timesound pressure (arb. units)

sound pressure (arb. units)

(a)
1

0

-1
0

0.2

0.4
t (s)

0.6

1

0

-1
0

0.05
t (s)

0.10

Figure 11. Sound pressure versus time waveform for the note C5 produced by a soprano recorder as playedby one of the authors. (a) Full note. (b) Attack portion of the tone. Note that t=0 in (b) has been shifted tobe nearer to the start of the tone.
An interesting feature seen in Fig. 12 is the way the frequencies of the second and third partials
vary small but significant amounts during the course of the tone. We believe that these frequency
shifts are real, and note that they resemble similar shifts seen in th e simulations.
Figure 13 shows how the power in each partial varies with time. It is notable that the second partial
is as large or nearly as large as the first partial until about 80 ms, while the first partial dominates by
about an order of magnitude or more at longer times. This again is similar to the behavior of thesimulated tones at the larger blowing speeds.

Figure 12. Spectra showing the first, second, and third partials of the tone during different portions of theattack. Black: Fourier spectrum calculated using data centered at t=10 ms in Fig. 1b. Red: Spectrum
calculated at t=30 ms. Blue: Spectrum calculated at t=100 ms. The approximate positions of the first (f1),second (f2), and third (f3) partials are indicated.

Figure 13. Power of the first, second, and third partials of the tone as functions of time during the fullcourse of the tone. Black: First partial. Red: Second partial. Blue: Third partial.
It is seen that the second partial is essentially equal in power to the first partial f or the first 50 msof the tone with the first partial then dominating at longer times. It is well known (see for example [8 10]) that a human listener is able to perceive pitch and timbre during portions of a tone as short as 20
ms, so the change in timbre evidenced in Fig. 13 is certainly perceptible.
One other feature of Fig. 12 is also worth noting. The frequencies of the different partials appear
to also vary some with time. This is most noticeable with the second partial which moves to higher
frequency at long times, while the frequency of the third partial seems to fall and then increase during
the course of the tone.

5. CONCLUSIONSIn a broad sense, the simulation results are quite similar to that found in the real tone consideredin Figs. 11-13. During the attack the higher partials are as large or larger than the first partial, thus
contributing very significantly to the timbre. It is well known (see for example [8 -10]) that a humanlistener is able to perceive pitch and timbre during portions of a tone as short as 20 ms, so the changes
in timbre we have observed are certainly perceptible. For the real tone it is the second partial thatcompetes for early dominance while for the simulated tone it is the third partial. This difference iscertainly significant, but here we have considered simulation results for only one note at one blowing
speed and with one ramp up time. Other simulations, which will be reported in the future, show thatthe behavior during the attack phase depends on those parameters. Our simulations have also foundthat the behavior can depend on the specific instrument geometry, so while the simulation model inFig. 1 is intended to resemble the Yamaha soprano recorder, the effect of remaining differences
between the geometry of the actual instrument and the model still need to be explored and understood.
Our results also clearly show the effect of the player. The recorder is often the first instrument
taught to a young student, presumably because it is easy for a beginner to produce a nice tone, incontrast to many other instruments, such as the violin. However, as with other instruments, it isimportant for the player to have some ability to affect the timbre. For the case of the recorder,
simulations like those presented in this paper show that the blowing speed and the ramp up time canboth profoundly affect the timbre, especially during the attack portion of the tone.
As a general result, our results suggest that Navier-Stokes-based simulations can be used study
rather detailed aspects of flue instruments, and address subtle but important questions concerning thetones produced by real instruments. We plan to report more extensive results of this kind in the future.

ACKNOWLEDGEMENTS
This work was supported in part by the U.S. National Science Foundation through grant PHY1806231.

REFERENCES
1. Giordano N, J Acoust Soc Am. 2014; 135(2): 906-916.
2. Verge MP, Fabre B, Mahu WEA, Hirschberg A, van Hassel RR, Wijnands APJ, de Vries JJ, Hogendoorn
CJ, Jet Formation and jet velocity fluctuations in a flue organ pipe. J Acoust Soc Am. 1994; 95(2): 11191132.
3. Nolle AW, Finch TL, Starting transients of flue organ pipes in relation to pressure rise time. J Acoust Soc
Am. 1992; 91(4): 2190-2202.
4. Verge MP, Hirschberg A, Caussé R, Sound production in recorderlike instruments. II. A simulation model.
J Acoust Soc Am. 1997; 101(5): 2925-2939.
5. Castellengo M, Acoustical analysis of initial transients in flute like instruments. Acustica-Acta Acustica
1999; 85(2), 387-400.
6. Miklós A, Angster J, Properties of the sound of flue organ pipes. Acustica-Acta Acustica 2000; 86(4):611-622.
7. Ségoufin C, Fabre B, Verge MP, Hirschberg A, Wijnands APJ, Experimental study of the influence ofmouth geometry on sound production in a recorder-like instrument: Windway length and chamfers.
Acustica-Acta Acustica 2000; 86(4), 649-661.
8. Patterson RD, Peters RW, and Milroy R, in Klinke R, Hartmann R, editors. Hearing - Physiological Bases
and Psychophysics, Proceedings of the 6th International Symposium on Hearing. Berlin: SpringerVerlag; 1983. p. 321-326.
9. Suied C, Agus TR, Thorpe SJ, Mesgarani N, Pressnitzer D, Auditory gist: Recognition of very shortsounds from timbre cues. J Acoust Soc Am. 2014; 135(3): 1380-1391.
10. Ogg M, Slevc, LR and Idsardi WJ, The time course of sound category identification: Insights fromacoustic features. J Acoust Soc Am. 2017; 142(6), 3459-3473.

ABS-0100

Numerical study of a French horn mouthpiece accompanied by vibrating lips
and an oral cavity with compressible direct numerical simulation
Rei SUMITA(1) , Ryoya TABATA(1) , Sho IWAGAMI(1) , Yuki NAKAHARA(1) , Takeshi NANRI(2) , Taizo KOBAYASHI(2) , Yuji
HATTORI(3) , Kin’ya Takahashi(1)(1) Kyushu Institute of Technology, Japan, takahasi@phys.kyutech.ac.jp(2) Kyushu University, Japan, tkoba@cc.kyushu-u.ac.jp(3) Institute of Fluid Science, Tohoku University, JAPAN, yuji.hattori@tohoku.ac.jp

ABSTRACT
A two-dimensional model of a French Horn mouthpiece is numerically studied with a 2D direct numerical simulation(DNS) of the compressible Navier-Stokes equations to investigate the sound generation mechanism from the viewpoint of aeroacoustics. That is, we consider the sounding mechanism of buzzing, when the mouthpiece without abore is played. Our numerical tool is highly accurate due to the minimum mesh size of the order of the micro-meter,and details of ﬂuid motion and acoustic oscillation inside and near the mouthpiece are successfully reproduced. In
particular, we focus on the roles of vibrating lips and an oral cavity in the sound generation mechanism. When themouthpiece without lips and an oral cavity is driven by a periodic ﬂow with a certain frequency, a single tone without
overtones is observed. On the other hand, when the mouthpiece is driven by vibrating lips with an oral cavity, agenerating sound includes rich overtones and its waveform is similar to that observed experimentally.
Since thebore is a linear element and cannot generates overtones from a single tone by itself, the sound of a horn including richovertones is generated by a mouthpiece with the vibrating lips and oral cavity.
Keywords: French Horn Mouthpiece, Buzzing, Aerodynamic sound, compressible DNS

1INTRODUCTION

The sounding mechanism of brass instruments is a long-standing important
problem in the ﬁeld of musicalacoustics and many authors have contributed to this problem [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14].
The dynamics of brass instruments is well modeled by delay differential equations under the one-dimensional
approximation, for example, the Adachi-Sato models[1, 7, 8], and such models well reproduce oscillating motions in the mouthpiece and even in the bore. The vibrating lips of a player, namely lip reeds, are the key forunderstanding the sound generation mechanism. The motion of lips is roughly categorized into two types ofreed valves, the so-called “outward-swinging door” and “sliding door”, which affect characteristics of generating sound, e.g. behavior of the sound frequency with respect to the lip resonance frequency [1, 5, 6, 7, 8]. In
reality, the lips of a player behave as a mixture of these models. For the models of wind instruments with an
oscillating reed(s), we merely assume that a volume ﬂow injected through the oscillating reed slit U is changed
to acoustic pressure p in the mouthpiece as p = Z0U, where Z0 is the characteristic acoustic impedance, andthis assumption apparently works well[1]. However, we do not have a clear answer to the problem of how
and where an acoustic motion is generated from a ﬂuid ﬂow injected through the oscillating reed slit. This is
a problem of aeroacoustics and the generating sound is regarded as an aerodynamic sound[15, 16]. Thus, weneed to take an approach based on aeroacoustics to reproduce and understand the sound generation process[14].
In this study, we focus on the sound generation process observed in a French horn mouthpiece detachedfrom a resonance tube from the viewpoint of aeroacoustics. Actually, brass players use the practice of playing
a mouthpiece alone, called mouthpiece buzzing, and can play notes on a scale as possible. Note that themouthpiece on brass instruments works as a Helmholtz resonator and enhances resonance peaks of the aircolumn, i.e., notes, in a middle range of its register[1]. Actually, we numerically reproduce the sound generation

(a)

(b)

sl

ol

oh

Oral cavity

sh

H

Dm

D
L

Figure 1. French horn mouthpiece and 2D model. (a) YAMAHA TMPHR Mouthpieces for Horn Practice. (b)Two-dimensional model: H = 16mm, L = 61.4mm, D = 6mm, Dm = 3mm, sh = 1mm, sl = 4mm, oh = 11mm,
ol = 20mm.
process as accurate as possible by using a compressible ﬂuid solver, which simultaneously reproduces ﬂuid andacoustic motions. To do this, we adopt a compressible Direct Numerical Simulation (DNS), which is considered
the most accurate and reliable scheme, although other less accurate schemes, e.g. Large Eddy Simulation (LES)and Lattice Boltzmann Method (LBM), are often used and provide some successful results [14, 17, 18, 19, 20].
Due to the ability of supercomputers at present, we adopt a two-dimensional (2D) model and consider theeffects of vibrating lips together with an oral cavity on the characteristics of ﬂuid motion and generating sound[14, 17, 18]. In the previous work[14], we brieﬂy reported the result of the model with vibrating lips without
an oral cavity. In this paper, we discuss the effect of the vibrating lips in detail and also discuss the effectof the oral cavity combined with the vibrating lips, because the existence of the oral cavity should affect thesound generation process to a greater and less extent.

2

MODEL AND NUMERICAL METHOD

Figure 1 shows the cross-section of a French horn mouthpiece that we adopt and the dimensions of its 2Dmodel to which an oral cavity is connected through a lip reed slit. We prepare three models which are different
in the geometry of the lip reed slit and oral cavity: the model without the lip reed slit and oral cavity called“Mouthpiece”, the model with the lip reed slit called “Mouthpiece+lip” and the model with the lip reed slit andoral cavity called “Mouthpiece+lip+cavity”.
We consider the following four cases. In the “Case 1”, the “Mouthpiece” model is driven by a Poiseuilleﬂow with the maximum velocity of V = 10m/s, and a frequency of damped oscillation of pressure in the mouthpiece is observed to detect the resonance frequency of the mouthpiece, the Helmholtz resonance frequency. In“Case 2”, to mimic a ﬂow injected through the vibrating lip reeds, the “Mouthpiece” model is driven by aperiodic ﬂow like an alternating current with velocity given by
V = V0 (1 − cos ωH t)/2,(1)

where V0 = 10m/s and ωH is the resonance angular frequency obtained in the “Case 1”. In “Case 3”, the effectof the vibrating lips is considered with the “Mouthpiece+lip” model. The lip reed slit is changed alternatively
opening and closing with period TH = 2π /ωH while a constant ﬂow with a velocity of V = 10m/s is injectedfrom the left end of the slit. In “Case 4” the effect of the oral cavity together with the vibrating lips isconsidered with the “Mouthpiece+lip+cavity” model. The lip reed slit is changed as “Case 3”, while a constant
ﬂow with a velocity of V = 1m/s is injected from the left end of the oral cavity so that the ﬂow velocity
approximately reaches 10m/s at lip reed slit when it is opened.
To perform numerical calculation, we adopt a compressible Direct Numerical Simulation (DNS) developed
by Komatsu et al.[17]. By using the higher-order ﬁnite difference schemes with the volume penalization (VP)
method, the two-dimensional (2D) compressible Navier-Stokes equations are solved[17, 18]. Figure 2 (a) and(b) show the numerical grids around the mouthpiece for the “Mouthpiece” model and “Mouthpiece+lip+cavity”
model, respectively. The points Pi and Po indicated by the ﬁlled yellow circles are observation points inside andoutside the mouthpiece, respectively. For the “Mouthpiece” model and “Mouthpiece+lip” model, the mouthpieceis embedded in a solid box. However, for the “Mouthpiece+lip+cavity” model, the outer shape of the mouthpiece is reproduced to explore a more realistic performance. The minimum grid size near the mouthpiece is

(a)

(b)

Po

Pi

Po

Pi

Figure 2. Mesh near the mouthpiece. The observation points are incated by “Pi” and “Po”. (a) Mouthpiecemodel. (b) Mouthpiece+lip+cavity model.
of 0.737 × 10−2 mm, which is enough to reproduce ﬂows in boundary layers near the solid walls inside themouthpiece, lips and oral cavity. The entire computational domain is of 2000 × 2000mm2 for all cases. The
calculation is done up to 0.01sec for each case. Table 1 shows mesh parameters and time step for the “Mouthpiece” model and “Mouthpiece+lip+cavity” model. Table 2 shows physical parameters of compressible ﬂuid.
The simulations were performed on the high-performance computing cluster ITO (subsystem A) with 1 node of
36 cores (36 parallel threads) in Kyushu University taking dozens of days for each calculation.
Table 1. Mesh parameters and time step
Model

Number of grids

Minimum grid size

Time step ∆t

Mouthpiece

2806 × 2480

0.737 × 10−2 mm

1.8 × 10−9 s

Mouthpiece+lip+cavity

2807 × 3096

0.737 × 10−2 mm

1.8 × 10−9 s

Table 2. Physical parameters of compressible ﬂuid

3

parameters

Density at rest ρ0

Viscosity µ

Sound speed c0

values

1.162kg/m3

1.846 × 10−5 Pa · s

347.127m/s

Numerical results

3.1 Case 1: Mouthpiece model driven by a constant ﬂow
The horn mouthpiece is a Helmholtz resonator, but it is not easy to obtain its resonance frequency theoretically,
because the neck cannot be clearly distinguished from the body. Thus, we estimate the resonance frequency
from a damping oscillation of pressure in the mouthpiece, when a constant ﬂow is injected into it.
Figure 3 (a) shows the velocity distribution at t = 0.01s and ﬁgure 3 (b) shows the change in pressure at
the observation point Pi in the mouthpiece. As shown in Figure 3 (a), an air-jet goes straight for a short timeand is rolled up into an eddy near the throat entrance. Then, the rolled-up eddy is divided into two parts:
the greater part goes along the upper or lower wall of the cup, and the remaining part enters the throat, ﬁrst
forming a wavy jet and then seemingly changing to an acoustic wave.
However, as shown in Figure 3 (b), the pressure oscillation, which is regarded as an acoustic oscillation,
decays in a short time. This is because a constant ﬂow cannot sustain permanent oscillation with a deﬁnite
pitch. Next, we estimate the resonance frequency without using the Fourier transform as follows. If T ′ denotes

(a)

(b)

Figure 3. Results of Case 1. (a) Velocity distribution at t = 0.01s. (b) Pressure at the observation point Pi.
the time interval of N cycles of pressure ﬂuctuation, the resonance frequency fH is approximated as fH = N/T ′ .
Then, the estimated resonance frequency is fH = 1108Hz. Note that it is approximately 75 Hz lower than thetheoretically estimated resonance frequency of the Helmholtz resonator (1184 Hz).
3.2 Stationary oscillations for Case 2, Case 3 and Case 4
For Case 2, Case 3 and Case 4, stationary oscillations of acoustic waves are observed on the inside and outsideof the mouthpiece. Let us see ﬂuid motions in the mouthpiece at ﬁrst. Figure 4 shows the velocity distributions at t = 0.01s near the mouthpiece for the three cases. In Case 2 shown by Figure 4 (a), the periodicallyinjected ﬂow changes to vortices in the cup and the major parts travel into the throat giving rise to an acoustic
oscillation. Compared with Case 1, fewer vortices are observed in the cup and the wavy jet almost disappears
in the throat. In the Case 3 shown by Figure 4 (b), the air jet injected through the vibrating lips is broken
into small pieces of vortices distributing irregularly in the cup and some parts go into the throat causing an
acoustic oscillation. In Case 4 shown by Figure 4 (c), the air jet injected through the vibrating lips is broken
into small pieces of vortices, which distribute more densely in particular on the left side than those of Case 3.
Small pieces of vortices are also observed on the right side of the oral cavity. However, an acoustic oscillation
together with a few vortices is observed in the throat.
Figure 5 shows the pressure distributions at t = 0.01s in the whole calculation region for the Case 2 andCase 4. For Case 2, a stationary spherical wave is radiated from the open end of the mouthpiece. For Case 4,
a stationary wave is also observed, but the radiated wave is not a sinusoidal-type wave and is rather modulated.
Furthermore, it is smaller in amplitude than that in Case 2.
Figure 6 shows pressure oscillations at the observation points Pi and Po for the three cases. For Case 2,
almost sinusoidal oscillations are observed in the stationary states at both points. However, the oscillation at
the point Pi is much larger in amplitude than that at the point Po, because a small piece of the resonancewave leaks as a radiation wave from the open end. For Case 3, the pressure oscillation at the point Pi isalmost sinusoidal, while the oscillation at the point Po is apparently modulated. The individual oscillations at
the points Pi and Po are slightly smaller in amplitude than the corresponding ones for Case 2. For Case 4,
the pressure oscillation at the point Pi is apparently modulated due to the effect of the oral cavity. Indeed, apressure oscillation is also observed in the oral cavity and affects the motion in the mouthpiece through theoscillating reed slit. Similar pressure oscillations were observed in the mouthpieces of Brass instruments inexperiments[1, 4]. The oscillation at the point Po is much modulated including a large amount of the 2nd and3rd harmonic components. The individual oscillations at the points Pi and Po are much smaller in amplitude
than the corresponding ones for Case 2.
Figure 7 shows the power spectra of the pressure oscillation at the observation point Po for the three cases.
For Case 2, ignoring a tiny 2nd harmonic component, only the fundamental component dominates the oscillation
so that an almost sinusoidal wave is observed. For Case 3, the fundamental component is still dominant, but the
3rd harmonic component is secondly dominant and the 2nd and 4th harmonic components appear with smaller

(a)

(b)

(c)

Figure 4. Velocity distributions at t = 0.01s. (a) Case 2. (b) Case 3. (c) Case 4.
(a)

(b)

Figure 5. Pressure distributions at t = 0.01s. (a) Case 2. (b) Case 4.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 6. Pressure oscillations at the observation points Pi and Po. (a) Observation point Pi of Case 2. (b)Observation point Po of Case 2. (c) Observation point Pi of Case 3. (d) Observation point Po of Case 3. (e)Observation point Pi of Case 4. (f) Observation point Po of Case 4.

pressure [dB]

Case 2
Case 3
Case 4frequency [Hz]Figure 7. Power spectra of the pressure oscillations at the point Po.
amplitudes; thus, the wave is apparently deformed as shown Figure 6 (d). Although the result is not shown
here, the power spectrum of the oscillation at the point Pi includes some amount of the harmonics, in particular
lower-order harmonics. Thus, we think that these 2nd to 4th harmonic components are relatively enhanced in
the oscillation at the point Po due to the characteristic of the radiation impedance of the mouthpiece. For Case4, the fundamental and the 3rd harmonic are in the same order in the spectrum and the 2nd harmonic also takea large value in amplitude; thus, a wave is periodic but includes 2nd and 3rd harmonic components as shown
Figure 6 (f). Even for the power spectrum of the oscillation at the point Pi, lower-order harmonic components
take large values in amplitude and the wave is apparently deformed as shown Figure 6 (e). Thus, such harmoniccomponents are relatively enhanced in the oscillation at the point Po due to the characteristic of the radiation
impedance of the mouthpiece. Therefore, the nonlinear interaction between the mouthpiece and the oral cavity
through the vibrating lips plays an important role for creating an acoustic wave with rich harmonics.

4

CONCLUSIONSIn this paper, we numerically studied the sound generation process of the French horn mouthpiece detachedfrom a resonance tube, namely mouthpiece buzzing, with the 2D compressible DNS. In particular, we focusedon the inﬂuence of the oral cavity combined with the vibrating lips on the ﬂuid and acoustic motions in themouthpiece as well as on the radiated acoustic wave. To do this, we treated the three models, which areindividually different in geometry and driven by different ways, and found the following facts concerning theconstruction of harmonic components in the acoustic oscillation.
When the mouthpiece is driven by a periodic ﬂow with the resonance frequency of the mouthpiece, theacoustic oscillation involves negligible harmonic components except the fundamental. Even only the model
including the vibration of lip reeds however induces some amount of lower-order harmonic components on the
acoustic oscillation in the mouthpiece. Since the radiation impedance of the mouthpiece relatively enhancesthese harmonic components, the radiated wave observed in a far ﬁeld is considerably deformed. The existenceof the oral cavity combined with the vibrating lips signiﬁcantly enhances harmonic components in the acoustic
oscillation in the mouthpiece, which is similar to those observed in experiments on brass instruments[1, 4].
Since the radiation impedance of the mouthpiece relatively enhances the harmonic components of the radiatedwave, an acoustic wave with rich harmonics is observed in a far ﬁeld. Therefore, the nonlinear interaction

between the mouthpiece and the oral cavity through the vibrating lips plays an important role in the soundgeneration process.
Unfortunately, we did not deal with the problem of how and where an acoustic oscillation is generated froma ﬂuid ﬂow injected through vibrating lips. To attack this problem, we need to take an approach based on the
aeroacoustic theory, e.g. Lighthill’s acoustic analogy [15, 16]. For another, to explore the interaction between
the mouthpiece and oral cavity through the vibrating lips in detail, we also need an analysis of the interaction
between compressible ﬂuid and vibrating lips. Further developments in these directions are left for future work.

ACKNOWLEDGEMENTS
The present work was supported by Grant-in-Aid for Scientiﬁc Research (C) Nos. 19K03655 from the Japan
Society for the Promotion of Science (JSPS) and “Joint Usage/Research Center for Interdisciplinary Large-Scale
Information Infrastructures” and “High Performance Computing Infrastructure” in Japan (Project ID: jh220001).
Part of the work was carried out under the Collaborative Research Project of the Institute of Fluid Science,
Tohoku University.

REFERENCES[1] Fletcher NH, Rossing TD. The Physics of Musical Instruments, 2nd ed. New York, USA: Springer-Verlag;
1998.
[2] Backus J. Input impedance curves for the brass instruments. J Acoust Soc Am. 1976; 60: 470-480.
[3] Fletcher NH. Excitation in woodwind and brass instruments. Acoustica. 1979; 43: 63 ‒ 72.
[4] Elliot SJ, Bowsher JM. Regeneration in brass instruments. J Sound Vib. 1982; 83:181 ‒ 217.
[5] Fletcher NH. Autonomous vibration of simple pressure-controlled valves in gas ﬂows. J Acoust Soc Am.
1993, 93: 2172 ‒ 2180.
[6] Yoshikawa S. Acoustical behavior of brass player s lips. J. Acoust Soc Am. 1995; 97: 1929 ‒ 1939.
[7] Adachi S, Sato M. Time-domain simulation of sound production in the brass instrument. J Acoust Soc Am.
1995; 97: 3850-3861[8] Adachi S, Sato M. Trumpet sound simulation using a two-dimensional lip vibration model. J Acoust Soc
Am. 1996; 99: 1200-1209.
[9] Gilbert J, Ponthus S. Artiﬁcial buzzing lips and brass instruments: Experimental results. J Acoust Soc Am.
1998; 104: 1627-1632.
[10] Cullen, JS Gilbert, J Campbell DM. Brass Instruments: Linear Stability Analysis and Experiments with an
Artiﬁcial Mouth. Acta Acustica united with Acustica. 2000; 86: 704-724.
[11] Campbell M. Brass instruments as we know them today. Acta Acustica united with Acustica. 2004; 90:
600 ‒ 610.
[12] Myers A, Pyle RW Jr, Gilbert J, Campbell DM, Chick JP, Logie S. Effects of nonlinear sound propagationon the characteristic timbres of brass instruments. J Acoust Soc Am. 2012; 131: 678-688.
[13] Fréour V, Guillot L, Masuda H, Usa S, Tominaga E, Tohgi Y, Vergez C, Cochelin B. Numerical continuation of a physical model of brass instruments: Application to trumpet comparisons, J Acoust Soc Am,
2020; 148: 748-758.
[14] Sumita R, Tabata R, Kobayashi T, Takahashi K, Hattori Y. Numerical Study of a French Horn Mouthpiecewith Compressible Direct Numerical Simulation. Proc AFI 2021; 27-29 October 2021; Sendai, Japan 2021.
p.175-177.

[15] Lighthill MJ. On sound generated aerodynamically. Part I: General theory. Proc. Roy. Soc. London A.
1952; 211: 564-587.
[16] Howe MS. Acoustics of Fluid-Structure Interactions. Cambridge, UK: Cambridge Univ. Press; 1998.
[17] Komatsu R, Iwakami W, Hattori Y. Direct numerical simulation of aeroacoustic sound by volume penalization method. Comp Fluids. 2016; 130: 24-36.
[18] Iwagami S, Ryoya T, Kobayashi T, Hattori Y, Takahashi K. Numerical Study on Edge Tone with Compressible Direct Numerical Simulation: Sound Intensity and Jet Motion. Int J Aeroacoust. 2021; 20: 283-316.
[19] da Silva AR, Scavone GP. Numerical simulations of ﬂuid-structure interactions in single-reed mouthpieces.
J Acoust Soc Am. 2007; 122: 1798-1809.
[20] Miyamoto M, Ito Y, Iwasaki T, Akamura T, Takahashi K, Takami T, Kobayashi T, Nishida A, Aoyagi M.
Numerical study on acoustic oscillations of 2D and 3D ﬂue organ pipe like instruments with compressible
LES. Acta Acustica united with Acustica. 2013; 99: 154-171.

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022 in Gyeongju, Korea

ABS-0112

Developing an interactive drum model: Using the discrete cosine
transform
Tim KIRBY1, Mark SANDLER2
1
2

QMUL, UK, t.c.kirby@qmul.ac.uk

QMUL, UK. mark.sandler@qmul.ac.uk

ABSTRACTThis paper expands upon the analysis and synthesis technique proposed by Kirby and Sandler in 2021 (1),which investigates how the evolution of drum modes change with strike intensity. This was observed in thediscrete cosine transform (DCT) of unwindowed drum sounds and enabled the synthesis of drum sounds
whose intensity is intermediate between those analysed. In this paper, the 67 modal features of a tom-tom
drum are modelled analytically, to move towards a complete synthesis. The phase function of each modal
feature is modelled with a linear rational function. The amplitude function of the modal feature is firstdecomposed into odd and even components, and each component fitted using functions such as an inverse
Gaussian. The evolution of the parameters of these analytical functions can be mapped onto a single
controllable parameter that represents strike intensity. This leads to an analytical model of a modal featurethat can be used to build a physical model of a tom-tom, which will be illustrated by an interactive
demonstrator.
Keywords: Analysis, Synthesis, Drum, Modelling

1. INTRODUCTION
Music production settings are becoming increasingly reliant on virtual instruments to augment or
replace traditional acoustic setups. Of particular interest here are drum synthesis techniques thatincorporate physical modeling. These instruments are not limited to prerecorded samples; model
parameters can be continuously varied through user automation or randomisation to improve realism
or create experimental sounds. Physical drum models can provide the user with complete control overthe specifications of the instrument and its surroundings, and these instruments can be extremely
responsive in terms of strike position, velocity, gesture, and tool.
At the same time, synthesising realistic drum sounds is still an open problem. Finite difference
methods, such as those generated in the NESS project (2), have produced some of the most realistic
results, but these models are computationally intensive and can’t be run in real time. Hybrid methods
have been used to bridge this gap. IK Multimedia’s Modo Drums (3) is one example, which
manipulates existing samples using modal synthesis in real-time. This is an interesting approach,though it is much more limited than a full physical model and the manipulated samples can still soundsynthetic.
This research aims to develop a real-time interactive drum model that is informed by reference
samples, but is not reliant on them. Here we dynamically model modal features to move towards afull physical drum model. The aim is to show that it is possible to analytically model modal features
accurately enough that expected time domain behaviours can be generated.
1.1 Framework Recap
In (1), it was shown that the discrete cosine transform of an unwindowed tom-tom sound reliably
contains modal features, Xm,n (ω), which encode the full time domain activity of a given mode. A
representative modal feature is shown in Figure 1. These modal features have a chirp like form which
can be decomposed into instantaneous amplitude and phase functions, also shown in Figure 1.

Figure 1. (Top) DCT representation of the fundamental frequency of a 9x10" Yamaha Beech
Custom tom-tom, struck at the moderately high velocity of 102 out of 127. This is X0,1 (ω), a
representative example of a chirp like modal feature, with an unmodulated fundamental frequency of
131 Hz. (Middle) Instantaneous amplitude (envelope) of the above modal feature, A0,1 (ω), extractedusing the Hilbert transform. (Bottom) Unwrapped instantaneous phase of the modal feature, φ0,1 (ω),
also extracted via the Hilbert transform.
An ideal circular membrane has Bessel-function solutions in the radial direction, and cosine
function solutions in the azimuthal direction. These solutions correspond to modes, which areclassified by their number of nodal diameters, m, and their number of nodal circles (4). Modes can be
written in the form (m, n), with the fundamental frequency being (0, 1). The instantaneous amplitude,
Am,n (ω) and the instantaneous phase, Φm,n (ω), evolve with strike velocity in a clear fashion. Thisevolution was demonstrated by interpolating the amplitude and phase functions extracted from real
drum samples and using these to synthesise modes of intermediate velocity.
Here, these functions are modelled analytically, to work towards a complete modal model that
doesn’t rely on reference samples. This type of model would be much more flexible, as it can be fully
parameterised to model drums of arbitrary specification. This research represents the first attempt at
modelling these functions analytically. The aim is to investigate whether the evolution of Am,n (ω) andΦm,n (ω) can be modeled with simple functions, to a degree that is sufficient to return the expectedbehaviour in the time domain. The analytical functions used here are not intended to capture every
single nuance found in the reference data, but rather to model the general evolution of these referencefunctions, as a first approximation. The same goes for the time domain activity, the aim is not tosynthesise a exact copy of the original time domain signal, but to synthesise a version that has a very
similar pitch trajectory and envelope, and is perceptually convincing.
If the expected time domain activity is recovered, it will serve two purposes. Firstly, it would
provide a proof of concept for a novel drum synthesis technique. Secondly, it would help us to better
understand how the IDCT representation is linked to the time domain activity, by understanding therelative perceptual importance of different aspects of the IDCT representation. There has been little
prior work in this type of synthesis. The closest method is arguably IFFT synthesis, which wasintroduced in 1980 (5). This has been used to generate large ensembles of sinusoids with fixed
frequencies (6), but it hasn’t been used to model the more complex non-linear modal activity discussed
here.

2. METHOD
2.1 Data Set
Tom-tom samples were extracted from Superior Drummer 3 by Toontrack (www.toontrack.com)
by triggering every sample from 1 to 127 midi velocity with all velocity layers loaded and all hit
variation features turned off. Each sample is 8s long to ensure a full decay. The following analysis isbased on the 9x10" Yamaha Beech Custom tom-tom but is applicable to any tom-tom and is expected
to transfer to any drum that exhibits pitch glide.
Central strikes were used as this is the best strike position to excite the fundamental mode. Out of
the 68 unique samples obtained, only 1 was deemed to be anomalous as the modal feature did not fit

the trend and it is audibly off-centre. The integrated loudness, as defined by (7), was also calculated
for each drum sound using the “integratedLoudness” command in MATLAB (The MathWorks, Natick,MA).
2.2 Modeling the Phase Function
The phase function is the most interesting component of a modal feature, as it encodes the time
domain pitch glide of the drum mode. The unwrapped phase function of the fundamental, Φ0,1 (ω), wasextracted as before, as shown in in Figure 2. The active regions of the phase functions were estimatedto be within (130 < f < 180Hz). This only includes the varying parts of the function. The plateaus
found at (f ≲ 130Hz) merely indicate the lack of sinusoidal oscillation that is found within the active
region of a modal feature.

Figure 2. Extracted phase functions for the fundamental of a 9x10" Yamaha Beech Custom tomtom (top), for comparison with the linear rational function model (bottom). The phase functions of allunique sample in the data set are overlaid. The maximum magnitude of phase in the curved section
increases with strike velocity, as does the frequency width. The active regions were estimated to bewithin 130 < f < 180Hz.
The active region was modelled with a linear rational function:

Where am,n , bm,n , and cm,n are real-valued constants, for a given strike velocity, v.
The parameters were fitted using the ’rat11’ option in MATLAB. The parameters were first fittedusing random start points, which is the default option. This provided the expected results for the vast
majority of strike velocities. The fitting was then repeated using a starting point of the median valueof each parameter from the original fits (am,n = 0.5095, bm,n =−483.9, cm,n =−854). This method
provided repeatability and provided the expected results for all 67 samples.
This is illustrated in Figure 2, where the model is compared to reference data. The active region of
the model appears to be virtually indistinguishable by eye, with the exception of the transition fromthe plateaus into the active region of the model feature f ≈ 130Hz). The was deemed negligible in thismodel, but this could be smoothed out or modeled analytically if necessary.
2.3 Evaluation of Phase Function Model
Figure 3 displays the coefficient of determination, R2 , values obtained when the model phasefunctions are compared to their respective reference function. The mean value is 0.9811 and thestandard deviation is 0.0240, which supports the idea that the model is a good fit of the referencefunction. The model performs particularly well for the range of strike velocities where there issignificant pitch glide. This is reassuring as this is arguably the most important region. The R2 values
decrease somewhat for the lowest strike velocities, though this at least partly due to the change inshape of the reference phase function.

Figure 3. (Top) Goodness of fit analysis using the coefficient of determination, R2 . Comparison ofeach model phase function with the respective reference. (Middle) Pitch trajectories from referencemodes obtained using the pYIN method (8) in Sonic Annotator (9). (Bottom) Pitch trajectories
obtained with linear rational function model. All strike velocities are overlaid. The magnitude of thepitch glide increases with strike velocity for both the reference and model.
The phase function encodes the time domain pitch glide of the drum mode. It is therefore worth
comparing the pitch trajectories of the reference data with that obtained by replacing Φm,n (ω) with thelinear rational function model. This pitch trajectories are shown in Figure 3. As expected, the pitch
trajectories of the model are not identical to the reference pitch trajectories, however, they do follow
a similar evolution, with both the magnitude of the pitch glide and it’s duration, increasing with strikevelocity, to the same approximate values. This is further demonstrated in Figure 4, where themagnitude of the pitch glide for both reference and model data, is plotted against strike velocity.

Figure 4. The estimated change in pitch of the fundamental, as a function of strike velocity.
Integrated loudness is used as a proxy for strike velocity. Pitch estimates were obtained using thepYIN method (8) in Sonic Annotator (9). The blue circles are for the reference data, the red circles
are for the model data.

The linear rational function model has 3 parameters (a,b,c). The evolution of these parameters withstrike velocity is shown in Figure 5. Each parameter was clearly correlated with the integrated
loudness of the drum sample, and in each case, this correlation was modelled with a dual exponential.
This means that these three parameters can be controlled by a single parameter which represents strikevelocity.
As the model phase function was able to recover the expected pitch glide behaviour in the time
domain, and the model parameters evolved in a smooth and continuous fashion, the linear rational
function model is deemed suitable as a first approximation.

Figure 5. Evolution of phase function parameters with strike velocity. Integrated loudness is used
as a proxy for strike velocity.
2.4 Modeling the Amplitude Function
The amplitude function contributes to the envelope of the time domain signal. At low strike
intensities, the amplitude function is a bell-shaped curve. As strike intensity increases, the area under
the curve increases, and the peak becomes increasingly asymmetrical. A characteristic dip to the leftof the peak becomes present, as shown in Figure 6.

Figure 6. Evolution of the decomposed amplitude function with strike velocity, for the fundamentalof a 9x10" Yamaha Beech Custom tom-tom. The amplitude function components of all unique samples
between midi velocity 1-66 are overlaid, as these illustrate the general trend in a clear manner. (Top)
Evolution of the amplitude function, A0,1 (ω). (Middle) Evolution of the even component of theamplitude function, 𝐴$!,# (ω). The area under 𝐴$!,# (ω) increases with strike velocity. (Bottom)
Evolution of the odd component of the amplitude function 𝐴%!,# (ω). The area under 𝐴%!,# (ω) increases
with strike velocity.

No single function was found that could adequately model the evolution of this function, so theamplitude function was decomposed into its odd and even components. This was done by taking thediscrete cosine transform of the amplitude function. This yields a vector that has two distinct trends,
one found in the odd numbered samples, and one in the even numbered samples. These trends wereseparated by duplicating this vector and setting the unwanted (odd or even) samples to zero. These
vectors can then be transformed back to the original domain using the inverse discrete cosine
transform, to yield the odd and even components of the amplitude function. The odd and evensymmetry is about the line f=131Hz, which is the unmodulated fundamental frequency. These
components are shown in Figure 6.
The addition of these components returns the full amplitude function, A0,1 (ω). The even component,
𝐴$!,# (ω), is the dominant component at all strike velocities. At the lowest strike velocities the odd
component, 𝐴%!,# (ω), is negligible as A0,1 (ω) is itself is essentially an even function. As strike velocity
increases, 𝐴%!,# (ω) becomes increasingly significant, increasing the asymmetry of A0,1 (ω). The odd
and even components are simple enough to model, and modeling them separately provides the user
with a high degree of control over the shape of the amplitude function. Both components are centredon the unmodulated fundamental frequency which is intuitive and allows the unmodulated
fundamental frequency to easily be parameterised, to model a membrane of arbitrary tension or radius.
Each component exhibits (odd or even) symmetry about this central frequency, which means that there
is redundancy, so only half of each function needs to be modelled.
There is some distortion that occurs to the right of the peak of the amplitude function at the highest
amplitudes. This was previously shown to have little perceptual effect on the time domain signal (1).
Nonetheless, it complicates the decomposition of the amplitude function, so these samples are notincluded in this first approximation modeling of the amplitude function. This distortion could be
included in later models.
2.5 Modeling the Odd Component
The right half of 𝐴%!,# (ω) resembles a probability distribution. This was originally modeled as aninverse Gaussian, but this has now been superseded by a linear rational function:

Where dm,n , em,n , fm,n , and gm,n are real-valued constants, for a given strike velocity, v.
The parameters were fitted using the ’rat12’ option in MATLAB. The parameters were first fittedusing random start points, which is the default option. The fitting was then repeated using a starting
point of the median value of each parameter from the original fits (dm,n = 1.2219, em,n = 0.0338, fm,n=−6.5728,and gm,n = 69.7205).
2.6 Modeling the Even Component
It is easier to model 𝐴$!,# (ω) in the transformed domain, before the inverse discrete cosine
transform step described in Section 2.4. The transformed vector alternated between positive andnegative values. A dual exponential was used to model the magnitude of this vector:

Where hm,n , im,n , jm,n , and km,n are real-valued constants, for a given strike velocity, v. Ω is thetransformed version of ω after the DCT has been performed on the amplitude function.
2.7 Evaluation of Amplitude Model
Figure 7 displays the modeled versions of the amplitude components, along with theirsuperposition which yields the full amplitude model.

Figure 7. Modeled evolution of the amplitude function (top) produced from the superposition of
the even component model (middle) and the odd component model (bottom).
Figure 8 displays the coefficient of determination, R2 , values obtained when the amplitude function
model is compared to the reference function. The top graph shows the values for the full amplitude
model. This comparison is made in the time domain, to assess whether the model recovers the expected
time domain behaviour. The mean R2 value is 0.9970 with a standard deviation of 0.0048, whichsuggests a very good fit across all tested strike velocities (The majority of values are even larger, butthey begin to decrease at higher strike velocities). This supports the conclusion that the amplitude
model is sufficient to capture the expected time domain behaviour.

Figure 8. Goodness of fit analysis using the coefficient of determination, R2 . (Top) Comparison in
the time domain of a mode synthesised using the full amplitude model, against the reference mode.
(Bottom) Comparison of the modeled odd and even components against their respective referencecomponent. This comparison was made in the domain in which each component is modeled.
The bottom graph compares the separate odd and even component models, in the domain in which
they were modelled. The dominant even component values are all above 0.9963, which again suggests
a good fit across all strike velocities. The odd component has much lower values at the lowest strikevelocities. This is because the odd component is almost negligible at these strike velocities, asevidenced by the fact that the values for the full amplitude model appear unaffected.

3. CONCLUSION
This model was intended as a proof of concept. Both the phase function and the amplitude function
have been suitably modeled to return modal behaviour in the time domain, that is consistent with that
expected for a tom-tom, while not quite identical to the reference data. It is unclear whether any
differences are perceptible; a listening test could be used to investigate this. This model can beparameterised to model modal behaviour in a dynamic fashion. Each parameter can be mapped to asingle master parameter which represents strike velocity. This facilitates the creation of an interactive
demonstrator that could be triggered by a control surface such as an electronic drum kit.
Transformations can also be used to apply this model to drums of different specification. It may
also be possible to link the model functions and their parameters to the underlying physics of a drum.
The phase model could be improved by modelling the transition from the plateau into the active
region. This could be done analytically, or a machine learning model could be trained on the
differences between this analytical model and the reference data.
The amplitude model appears to perform extremely well over the tested data. Some samples were
excluded from the amplitude model as these over complicated the decomposition and analysis of thisinitial proof of concept modeling. Now that this proof of concept has been achieved, these samples
could be modeled in detail.

ACKNOWLEDGEMENTS
This research was funded by Queen Mary University of London. The author would also like tothank Toontrack, and all those involved in creating the meticulously sampled library of drums sounds
found in Superior Drummer 3, which has served as an invaluable dataset. Thanks are also given to all
those in the Centre for Digital Music for their valued support.

References
1. Kirby T, Sandler M. The evolution of drum modes with strike intensity: Analysis and synthesis using thediscrete cosine transform. The Journal of the Acoustical Society of America. 2021 Jul 12;150(1):202-14..
2. Bilbao S, Desvages C, Ducceschi M, Hamilton B, Harrison-Harsley R, Torin A, Webb C. Physicalmodeling, algorithms, and sound synthesis: The NESS project. Computer Music Journal. 2019 Jun 1;43(23):15-30.
3. IK Multimedia Modo Drum. Available from: https://www.ikmultimedia.com/products/mododrum/
4. Errede S. Vibrations of ideal circular membranes (eg drums) and circular plates. UIUC Physics 406
Acoustical Physics of Music. 2002.
5. Chambelin, H. Musical Applications of Microproccessors, 1st ed. Haydens Books, New York. 1980
6. Rodet X, Depalle P. Spectral envelopes and inverse FFT synthesis. InAudio Engineering Society
Convention 93 1992 Oct 1. Audio Engineering Society.
7. Series BS. Algorithms to measure audio programme loudness and true-peak audio level. International
Telecommunication Union. ITU-R BS.1770-4.
8. Mauch M, Dixon S. pYIN: A fundamental frequency estimator using probabilistic threshold distributions.
In IEEE international conference on acoustics, speech and signal processing (icassp) 2014 May 4 (pp.
659663).
9. Cannam C, Sandler M, Jewell MO, Rhodes C, d’Inverno M. Linked data and you: Bringing musicresearch software into the semantic web. Journal of New Music Research. 2010 Dec 1;39(4):313-25.

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022 in Gyeongju, Korea

ABS-0243

Modeling and measurements of organ pipe sound radiation
Samuel D. BELLOWS1; Timothy W. LEISHMAN2
1,2Brigham Young University, USA

ABSTRACT
Pipe organs have played an important role in Western musical genres for centuries and are unique in theiracoustic radiation as sound is produced from numerous pipes. At low frequencies, the directivity of anindividual organ pipe can be roughly modeled using one or two point sources, whose source strengths can be
determined, for example, using transmission matrix methods. This work studies the limitations of thisapproach in predicting the directivities of organ pipes at higher frequencies involving diffraction effects.
Modeled directivities are compared to those measured with a spherical array and with a near-field cylindricalarray to illustrate the differences.
Keywords: Directivity

1. INTRODUCTION
The transmission matrix method (TMM) represents a complex system as a one-dimensionaltransmission line, allowing a simple yet robust approach to modeling many musical instruments (1).
Because the one-dimensional model is approximately valid as long as the instrument bore remains
small compared to the acoustic wavelength, this approach is particularly useful in predicting thefrequencies produced using various fingerings of wind instruments even for higher frequencies. While
it is common to utilize the TMM to predict played frequencies and input impedance curves, theapproach also helps predict the directional characteristics of musical instruments. Nonetheless, theeffectiveness of the TMM in predicting directivities have remained inconclusive, partly due to thelack of high spatial resolution directivity measurements for comparisons.
This work contrasts TMM-based directivity models to measured spherical directivities. In
particular, this investigation applies the TMM to organ pipes due to their simple natures. Results showthat the TMM-based techniques successfully predict directivities at low frequencies. However, BEM
simulations and near-field cylindrical scans reveal that the neglecting of diffraction effects limits theeffectiveness of the approach.

2. THEORETICAL MODELING
At low frequencies, a simple point-source model reasonably describes the radiation of sound fromthe organ pipes (2, 3). For open pipes, two sources are necessary: one at the mouth and one at the open
end. For a closed pipe, only a single source at the mouth is necessary. For time-harmonic radiation,
the pressure at a point r is𝑁𝑝(𝒓, 𝑘) = 𝑖𝑧0 𝑘 ∑ 𝑈𝑛 𝐺(𝑟, 𝑟𝑛 ) ,

(1)

𝑛=1

where N is the number of openings, 𝑈𝑛 is the frequency-dependent volume velocity at the nth opening,samuel.bellows11@gmail.com
tim_leishman@byu.edu

12

𝑖 is the imaginary unit, 𝑧0 = 𝜌0 𝑐 is the characteristic specific acoustic impedance of the fluid, 𝑘 is thewavenumber, and𝐺(𝑟, 𝑟𝑛 ) =𝑒 −𝑖𝑘𝑅𝑛
4𝜋𝑅𝑛(2)𝑅𝑛 = |𝒓 − 𝒓𝒏 |(3)is the free-space Green’s function withbeing the distance from the observation position 𝒓 to the location 𝒓𝒏 of the nth opening.
When considering normalized far-field directivities, the volume velocity of the mouth isarbitrary for a closed pipe with only one opening. However, for an open pipe, the impedance
translation theorem relates the volume velocity between the two openings (1, 3). The TMM
simplifies the numerical implementation for more complex sources with many openings (3). The
acoustic radiation impedance of the open end of the pipe can be modeled as an unflanged tube:
𝑧0(4)
𝑍𝐴𝑅,𝑂 = (0.25(𝑘𝑎)2 + 𝑖0.6𝑘𝑎).
𝑆
The acoustic radiation impedance of the mouth can be roughly modeled as a baffled circular
piston:
𝑍𝐴𝑅,𝑀 =𝑧0
𝐽1 (2𝑘𝑎) 𝑖𝐻1 (2𝑘𝑎)
(1 −
+
).
𝑆
𝑘𝑎
𝑘𝑎

(5)

Translating the impedance of the open end through length L to the mouth opening yields
𝑍𝐴 = 𝑍𝐴0𝑍𝐴𝑅,𝑂 + 𝑖𝑍𝐴0 tan (𝑘𝐿),
𝑍𝐴0 + 𝑖𝑍𝐴𝑅,𝑂 tan (𝑘𝐿)(6),where 𝑍𝐴0 = 𝑧0 /𝑆 and S is the pipe’s cross-sectional area. A current divider circuit gives thevolume velocity at the mouth as
𝑍𝐴(7)
𝑈𝑚 = 𝑈𝑖𝑛,
𝑍𝐴 + 𝑍𝐴𝑅,𝑀
where 𝑈𝑖𝑛 is an arbitrary input volume velocity. Finally, the volume velocity at the end of the pipe
is given by another current divider circuit:
𝑈𝑂 = (𝑈𝑖𝑛 − 𝑈𝑚 )−𝑖𝑍𝐴0 csc(𝑘𝐿).
𝑖𝑍𝐴0 (tan(𝑘𝐿⁄2) − csc(𝑘𝐿)) + 𝑍𝐴𝑅,𝑂(8)

Thus, a simple model can estimate the volume velocities at the pipe openings for predicting farfield directivities using Eq. (1).
2.1 Kirchhoff-Helmholtz Integral Equation
A more complete representation of the pressure field follows from the Kirchhoff-Helmholtz
Integral Equation (KHIE) (4):𝜕(9)𝑝(𝑟) = 𝑖𝑧0 𝑘 ∬ 𝑢𝑛 (𝑟𝑠 )𝐺(𝑟, 𝑟𝑠 )𝑑𝑆 + ∬ 𝑝(𝑟𝑠 ) 𝐺(𝑟, 𝑟𝑠 )𝑑𝑆,
𝜕𝑛where u n is the normal particle velocity, the integral is taken over the boundary S and n is the normal
direction into the domain. In this form, the first integral of the KHIE can be roughly interpreted as theradiation of sound from the source due to surface vibrations, whereas the second integral can beinterpreted as the pressure field due to diffraction and scattering effects of the boundary.
If one considers the boundary surface S to fully enclose the exterior of the pipe, it is evident that

Eq. (1) approximates the first integral of the KHIE with
𝑈𝑛 = ∬ 𝑢𝑛 (𝑟𝑠 )𝑑𝑆𝑛 ,(10)

where 𝑆𝑛 is the surface bounding the nth pipe opening. With this approach, the openings of the pipe
behave as vibrating pistons of effective areas S n. This approximation should remain valid provided
that the acoustic wavelength is larger than the pipe openings. However, Eq. (1) neglects the second
integral in the KHIE. Thus, when this term becomes important, deviations between the modeled andmeasured directivities will arise. Section 5 explores this limitation by comparing measured resultswith a BEM simulation, which numerically evaluates Eq. (9).

3. MEASUREMENTS
The authors employed spherical and cylindrical arrays to measure the pressure fields produced by
two organ pipes: one open metal pipe and closed wood pipe. The spherical array, shown with theclosed wooden pipe in Fig. 1(a), consisted of 36 12.7 mm (0.5 in) microphones placed in 5-degreeincrements in the polar angle at a radius of a = 0.97 m. The cylindrical array, shown with the open
metal pipe in Fig. 1(b), consisted of 36 6.35 mm (0.25 in) microphones placed in 5 cm increments at
a circular radius of ρ = 0.25 m. A turntable rotated each array in 5-degree azimuthal steps. Relative
calibrations for each microphone array and a near-field reference microphone ensured normalization
between the repeated measurements using frequency-response functions (FRFs) [(5)].

(a)

(b)

Figure 1. Spherical (a) and cylindrical (b) microphone arrays used to assess the radiated fields of twoorgan pipes.

4. RESULTS
4.1 Theoretical Prediction
Figure 2 shows simulated directivity balloons for the first through sixth partials of the open metal
pipe using Eq. (1) with L = 0.46 m. Both radius and color indicate levels on a 40 dB scale. The mouthof the pipe faces the 𝜙 = 0° azimuthal marker. For the fundamental, the directivity appears as a single
disc-like shape. As anticipated for a two point-source model, as frequency increases, the number oflobes also increases. The directivity patterns are nearly axisymmetric about the line connecting thecenter of the top opening of the pipe with the mouth. Because the wooden pipe has only one radiating

opening, the simulated directivity pattern for all frequencies is omnidirectional.

Figure 2. Simulated directivity balloons of the open metal pipe using Eq. (1) for the first six partials: (a) 331
Hz, (b) 662 Hz, (c) 993 Hz, (e) 1323 Hz, (f) 1654 Hz, and (f) 1985 Hz.

4.2 Directivity Measurements
Figure 3 shows the directivity balloons of the measured metal pipe. The patterns are remarkably
similar, especially for the first three partials below 1 kHz. As for the simulated case, the balloons areroughly axisymmetric about the axis connecting the openings. However, at higher frequencies,
reduced levels appear behind the pipe (near 𝜙 = 180°) that reduce the axial symmetry.

Figure 3. Measured directivity balloons of the open metal pipe for the first six partials: (a) 331 Hz,(b) 662 Hz, (c) 993 Hz, (e) 1323 Hz, (f) 1654 Hz, and (f) 1985 Hz.
Figure 4 shows the measured closed wooden pipe. While for the fundamental, the radiation is roughly
omnidirectional, at higher frequencies, particularly above 1 kHz, the radiation is significantly reduced
behind the instrument. Small undulations appear, although they are weak and not nearly as pronounced
as for the open pipe.

Figure 4. Measured directivity balloons for the measured closed wooden pipe for the first six partials: (a)335 Hz, (b) 670 Hz, (c) 1005 Hz, (e) 1340 Hz, (f) 1675 Hz, and (f) 2010 Hz.

5. ANALYSIS
As suggested by the measurement results, simple point-source models provide reasonable estimatesof source directivities at lower frequencies. However, reduced levels behind the pipes suggest thatdiffraction around the pipe itself may play an important role in the radiation patterns, particularly athigher frequencies. An enhanced model using a BEM evaluation of the KHIE provides additional
insights into this assertion.
To simulate the pressure while maintaining similarity to the modeling approach used in Eq. (1), asingle boundary enveloping the exterior of the pipe describes the source geometry. This approach
divides the pipe into an exterior and interior portion so that the openings of the pipes become
inhomogeneous Neumann (velocity) boundary conditions. Then, as shown in Eq. (10), the net volume
velocity used in Eq. (1) is the surface integral of the particle velocity at each opening. The TMM
method predicts the particle velocity at each opening by dividing the predicted values from Eqs. (7)and (8) by the area S n of the opening. While a more accurate model would follow from treating thepipes as thin-walled structures and allowing the computational domain to include both the interior andexterior of the pipe, this method is useful because it maintains a consistent approach between thesimple point-source model of Eq. (1) and the first term of the KHIE. The second term of the KHIE,
numerically evaluated using BEM, can then be seen as a correction term to the original model.
Because the field produced by the second term decays at a faster rate than the 1/r dependence ofthe first term due to the normal derivative of the Green’s function, this field is more apparent in thenear-field of the source. Consequently, measurements produced by the cylindrical array are beneficial
for identifying the effects of the second term in the KHIE.
Figure 5 plots results for the fundamental of the closed wooden pipe. Figure 5(a) shows the pressure
evaluated on the cylindrical sampling surface as simulated by Eq. (1) with the point source centeredat the mouth of the instrument. As anticipated, the pressure is nearly uniform for constant height z,
although the levels slightly increase in front of the pipe mouth and decrease behind the pipe as themouth is not directly at the center of array. Figure 5(b) shows the simulated pressure using only the
first term of the KHIE. Because the wavelength is much larger than the opening, Eq. (10) remains areasonable approximation and the pressure remains similar to that predicted by the point-source model.
Figure 5(c) shows the simulated pressure when both terms in the KHIE are included. The inclusion of
the second term significantly decreases the levels behind the instrument, and the area in front of themouth is clearly seen as a red circular patch. The BEM simulation agrees well with the measured data
shown in Fig 5(d), which also has a similar red circular patch in front of the pipe mouth. Thus,
including the second term would improve the accuracy of the TMM-based point-source model.

Figure 5. Pressure on the cylindrical scanning surface for the fundamental of the closed wooden pipe.
(a) TMM-based point-source model simulation. (b) BEM simulation using only the first term of the
KHIE. (b) BEM simulation using the full KHIE. (c) Measured pressure.

6. CONCLUSIONSThis work explored limitations in modeling the directivities of organ pipes using equivalent pointsources combined with the TMM. The results show that while there is excellent agreement between
simulations and measurements at low frequencies, significant deviations occur at higher frequencies.
Near-field cylindrical scans and BEM simulations reveal that neglecting diffraction effects around thepipe can cause these deviations. Future work could include applying the method to more complex
sources such as woodwind instruments and developing simplified approaches to model diffraction
effects.

ACKNOWLEDGEMENTS
The authors express appreciation for funding from the William James and Charlene Fuhriman
Strong Family Musical Acoustics Endowed Fellowship.

REFERENCES
1. Plitnik GR, Strong WJ. Numerical method for calculating input impedances of the oboe. J Acoust Soc
Am. 1979;65(3):816-825.
2. Northrop, PA. Problems in the analysis of the tone of an open organ pipe. J Acoust Soc Am. 1940;
12(1):90-94.
3. Chaigne A, Kergomard J. Acoustics of Musical Instruments. New York, USA: Springer; 2016.
4. Pierce, AD. Acoustics. Springer International Publishing; 2019.
5. Leishman TW, Rollins S, Smith HM. An experimental evaluation of regular polyhedron loudspeakers as
omnidirectional sources of sound. J Acoust Soc Am. 2006;120(3):1411-1422.

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022 in Gyeongju, Korea

ABS-0313

Numerical simulation of the acoustic field around a violin
Masao YOKOYAMA1; Yuya ISHIGAKI1; Amane TAKEI2; Genki YAGAWA31
2
3

Meisei University, Japan

University of Miyazaki, Japan

Toyo University and University of Tokyo, Japan

ABSTRACT
We numerically simulated the sound pressure field around the violin body using the finite-element method.
Although there are many studies on the vibration and sound radiation of violins, obtaining results with a neatinterpretation from a quantitative and qualitative standpoint remains a challenge. Herein, the highly precise
geometry of the violin made by Stradivari was scanned with a micro-computed tomography scanner, where
the material of the violin body was wood (spruce and maple) with orthotropic properties. This study aimed
to clarify the relationship between the properties of wood and acoustic radiation around a violin vibrated bya force at the bridge.
Keywords: Sound, Insulation, Transmission

1. INTRODUCTION
We conducted numerical simulations of the vibration and sound radiation of violins made by old masters,such as the Antonio Stradivari and Guarneri families. Analyses using numerical simulations have been
actively promoted since the beginning of the 21st century.1 Computed tomography (CT) scanners have also
been used to observe the internal structure of violins 2,3 and to investigate the material properties of wood.4,5
Furthermore, methods using numerical simulation and/or CT scanners have been investigated as non-invasive
and non-destructive methods for the analysis of historical assets such as violins.6
Previously, we conducted a coupled numerical simulation of violins using the finite-element method (FEM)
and modeled the effects of wood properties on the vibration mode of the violin body.7 This paper reports themeasurement of the geometric data of a violin made by Stradivari using a micro-CT scanner, the relationshipbetween the mode vibration and properties of wood, and the results of numerical simulations coupling thevibrations by a forced oscillation on a bridge with the acoustic field pressure around the violin.

2. GEOMETRIC DATA AND NUMERICAL SIMULATION
2.1

Three-dimensional geometry using a micro-CT scanner

The geometry of the violin made by Antonio Stradivari (1719) was scanned from the tailpiece to the scrollof the neck using a micro-CT scanner with a precision of 0.1 mm, as illustrated on the left side of Figure 1.
The image on the right side of Figure 1 is a snapshot visualizing the interior of the violin body using scanned
1

masao.yokoyama@meisei-u.c.jp

geometric data with computer-assisted design (CAD) software. As the scanned raw-geometric data include
many fragments and holes, we must clean the data using CAD software before numerical simulation andvisualization.
The import of geometric data, meshing, and FEM calculations were conducted using COMSOLMultiphysicsTM. The geometric data were divided into various parts, such as top and back plates, ribs, soundposts, and bass bars, to set different physical properties for each of them (Figure 2), and then saved as STEP
files (standard for the exchange of product model data). The STEP files were imported into COMSOLMultiphysics as geometric objects. In addition, we set the spherical area of air surrounding the violin (Figure
2, right side). Points A–D represent the positions where the sound pressure was calculated in Section 3.2. In
COMSOL Multiphysics, the mesh generator discretizes the domains into tetrahedral second-order meshelements using the free-mesh method.8 In total, there are approximately two million elements, including
violins and air. The eigenmode frequency, displacement of the body, and sound pressure were calculatedusing the FEM via the acoustic-structure interaction module of COMSOL Multiphysics.

Figure 1 – View using the micro-computed tomography scanner and visualization of the interior of theviolin using computer-aided design software

A

x
y

D

B

C

Figure 2 – Meshes of numerical simulation using the auto-mesh function of COMSOL Multiphysics. Points
A–D is the position where the sound pressure is calculated in Section 3.2.

2.2

Parameter setting of the numerical simulation

The physical characteristics of wood can be set in three orthogonal directions in COMSOL Multiphysics: thelongitudinal-grain direction (x-axis), radial annual-ring direction (y-axis), and direction tangential to theannual ring (z-axis). The typical values of the physical properties, such as Young’s modulus, rigidity modulus,
and Poisson’s ratio, have been measured by Green et al.9 The representative values of the density for maple
and spruce are 0.63 and 0.36, respectively, and Young’s moduli, Ex, are 12.6 GPa and 9.9 GPa, respectively.
EY/EX, EZ/EX, and the rigidity modulus (G/EX) are the ratios of the longitudinal Young’s modulus (EX) (Table
1).
Table 1 Values of orthotropic properties employed in the numerical simulation9.
Ex value of maple is 12.6 GPa and that of spruce is 9.9 GPa.
Property
Maple Spruce
Young’s modulus EY / EX
0.132
0.078
EZ / EX
0.065
0.043
Rigidity modulus GXY / EX
0.111
0.064
GYZ / EX
0.021
0.003
GXZ / EX
0.063
0.061
Poisson’s ratio μXY
0.424
0.372
μYZ
0.774
0.435
μXZ
0.476
0.467
However, these values differed from the actual scanned values of the violins. We cannot disassemble andmeasure the properties of a cultural asset such as the Stradivari violin. Thus, we estimated the values for thenumerical simulation using the frequency response function (FRF) of the scanned violin, as described in thenext section.

3. SIMULATION RESULTS
3.1

Mode vibration and estimation of values of orthotropic properties

In this study, we estimated the orthotropic properties of wood to be set in the numerical simulation bycomparing important eigenmode frequencies obtained by numerical simulation with those measured from thereal Stradivari violin (1719) by FRF. In other words, this is an inverse problem where we estimate the valuesof the real properties of the wood by changing parameters such as density and Young’s modulus in the
numerical simulation and matching the eigenmode frequencies of the numerical simulation with those of the
FRF.
The eigenmodes at low frequencies under 1,000 Hz, called A0, B1−, and B1+, were investigated in thevibration analysis of violins 1,2. Figure 3 shows the FRF of the violin proposed by Stradivari (1719). Using a
miniature hammer, we tapped the sides of the bridge on both the E and G strings and measured the results
using an FFT analyzer. The peak on the left side of the graph (265 Hz) corresponds to the first air cavity
mode, called A0. The next peaks were B1− (428 Hz) and B1+ (511 Hz), which were purported to influence
the timbre of the sound.
Figure 4 shows the numerical simulation of the changes in the eigenfrequency owing to the changes in density

and EX. The middle value in each graph represents the frequency calculated using the representative values.9
This graph depicts eigenfrequencies with varying densities and EX ranging from −20% to +20%. The freevibrations of the violin body were calculated without any constraint points.
Figure 4 shows that the eigenfrequency increased as EX increased and decreased as the density increased. As
shown in Table 2, the eigenfrequencies obtained by the numerical simulation using representative values
were lower than those obtained by the FRF of the scanned violin. For instance, if we decrease the density ofboth maple and spruce by 10% and increase Ex by 10% (also EY, EZ, and G), the eigenfrequencies obtainedby the numerical simulation approach those obtained by the FRF.

G string side

E string side

35
30

Pa/N

25
20
15
10
5
0
0

100

200

300

400

500

600

700

800

900

1000

f Hz

Figure 3 – Frequency response function of Stradivari (1719) from the hammering test.

A0, maple
B1-, maple
B1+, maple

550

A0, maple
B1-, maple
B1+, maple

550

A0, spruce
B1-, spruce
B1+, spruce

500

450

450

eigenfrequency, Hz

eigenfrequency, Hz

500

A0, spruce
B1-, spruce
B1+, spruce

400
350
300

400
350
300

250

250

200
0

5

10

Ex, GPa

15

20

200
0.2

0.4

0.6

Density

0.8

Figure 4 – Eigenfrequencies in the A0, B1−, and B1+ modes. The eigenfrequency increases with Young’s
modulus and decreases with the increasing density.

Table 2 – Comparison of the eigenfrequency (Hz): with the frequency response function (FRF) of
Stradivari, numerical simulation with the representative value of the wood property9, and with a decrease indensity of 10% and increase in Ex of 10%.
Experiment

Numerical simulationMode

FRF (Strad. 1719)

Representative9

Density −10%, Ex +10%

A0

265

258

265

B1−

428

399

431

B1+

511

475

516

Figure 5 shows the displacement in the z-axis direction of the top/back plate in A0, B1 −, and B1+, which
was calculated using the modified density and EX, as mentioned above. The red area indicates where thedisplacement in the z-axis direction is positive, and the blue area indicates where it is negative.
In the A0 mode, the top plate bends widely on the bass bar side (left side of the violin in Figure 6 (a)). In theB1 mode, the top plate bends in the horizontal direction, and the back plate bends in the vertical direction. In
contrast, the top plate bent in the vertical direction and the back plate bent in the horizontal direction in the
B1+ mode. The displacement of the violin becomes asymmetrical because there is a sound post and bass bar
inside the violin body. Other studies have found similar displacement patterns.2
Figure 6 shows the acoustic pressure fields around the violin in the numerical simulations. The radiation
above the f-hole of the top plate was significant. The radiation under the back plate was also significant in
the B1 mode. The sound radiation from the body appears to depend significantly on the bass bar and bend in
the long-side direction (upper- and lower-bout directions).

x
y

x
y

(a) mode A0, 265.0 Hz(b) mode B1−, 431.4 Hz(c) mode B1+, 516.1 Hz

Figure 5 – Displacement in A0, CBR, B1−, B1+ mode.

yx(b) mode B1−, 431.4 Hz(a) mode A0, 265.0 Hz(c) mode B1+, 516.1 Hz

Figure 6 – Sound radiation around the violin in the A0, B1−, and B1+ modes.
3.2

Acoustic pressure field around the violin in forced vibrations on the bridge

This section presents the simulation results when a forced sinusoidal oscillation is applied to the bridge ofthe violin. Figure 7 depicts the displacement of the body in the z-axis direction by forced vibrations. The
right side of Figure 7 shows the oscillating position of the G string. The frequency of the sinusoidal function
at 196 Hz (G3, the fundamental frequency of the G string) was inputted in the y-axis direction. On the lefthand side of Figure 7, the red area indicates where the displacement is positive and the blue area indicateswhere the displacement is negative.
The bridge alternately oscillated from side to side in the y-axis direction, and alternate vibrations of the top
plate were induced by the bridge oscillation. In particular, the magnitude of the displacement on the bass bar
side (the left side of the violin in Figure 7) was significant. We also speculated that the vibrations of the scroll
and fingerboard do not significantly influence the sound volume and timbre; however, these vibrations are
not negligible (see the video of detailed simulation results, https://youtu.be/m3cgG-TJs9Q).

zysound post
bass barbass barx

sound postzyFigure 7 – Displacement of the violin by the forced vibrations on the bridge where the G string is placed(196 Hz). The bass bar side has significant vibrations. The scroll and fingerboard also vibrate.

The acoustic pressure fields in the y–z plane(including the bridge and sound post) and in thex–y plane (30 mm above the arch of the topzyplate) are shown in Figures 8 and 9, respectively.
We simulated the vibration of the top plate by thesinusoidal

oscillation

of

the

bridge andconcentric sound radiation from the f-hole andC-bouts. Similar to the experimental results
obtained by Wang,10 sounds at a low pitch
radiated concentrically from the violin body. By
contrast, the directivity of the sound wasobserved at a higher pitch. In the near future, wewill numerically analyze the characteristics ofFigure 8 – Sound pressure field in the y–z plane in theforced vibrations with a sine wave of 196 Hz at the G

string position on the bridge.
sound radiation by varying the frequency and position of forced vibrations.

yxt=0.1T (T=1/196 s)

t=0.15Tt = 0.6T

t = 0.8T

t=0.3T

Figure 9 – Sound pressure fields around the body of the violin in the x–y plane 30 mm above the top of thearch.
Figure 10 shows the temporal change in acoustic pressure around the violin. Each line depicts the transitionof the acoustic pressure at points A–D, as shown in Figure 2, during approximately two cycles of thesinusoidal function from the start of the forced oscillation (t = 0–0.01 s). The dotted line in the graph indicatesthe sinusoidal function of forced vibration on the bridge. The acoustic pressure at each point was similarly
varied. Thus, we can also infer from the graph that the sound radiates concentrically when the bridge vibrates
at 196 Hz with an open G string.
For instance, we can numerically simulate the directivity of sound radiation for the pitch difference. This

analysis of the acoustic field around theforced vibration on bridge(y-axis direction)

violin can be conducted experimentallyin an anechoic chamber using arraymicrophones. However, there are only alimited number of laboratories andfacilities with anechoic chambers, andthe cost is high. Thus, if we can substitutethenumerical

simulation

for

the

experiment on sound radiation, coupled(second)simulation can become an extremely
useful tool.

Figure 10 – Temporal change of sound pressure aroundthe violin body at points A–D (in Figure 2).

4. CONCLUSIONS
A coupled simulation of the mode vibrations and sound radiation of a violin scanned by a micro-CT scanner
was conducted using the FEM. We analyzed the relationship between the properties of wood and theeigenfrequency and visualized the vibration of the violin body and sound radiation in mode vibrations.
Furthermore, we demonstrated that coupled simulation can analyze sound radiation. In future work, we intend
to conduct a numerical simulation of sound radiation caused by string oscillations and analysis of soundradiation in a concert hall using a large-scale parallel computer.

ACKNOWLEDGEMENTS
The authors would like to thank Bunkyo-Gakki Co. Ltd. for providing violins. This study was
supported by JSPS KAKENHI (grant number 21K11969).

REFERENCES
1. Gough, C. (2021). Acoustic characterization of string instruments by internal cavity measurements. The
Journal of the Acoustical Society of America, 150(3), 1922–1933.
2. Bissinger, G. and Oliver, D. (2007). 3-D laser vibrometry on legendary old Italian violins. Sound andVibration, 41(7), 10–15.
3. Borman, T. and Stoel, B. (2009). Review of the uses of computed tomography for analyzing instrumentsof the violin family with a focus on the future. J Violin Soc Am: VSA Papers, 22(1), 1–12.
4. Stoel, B. C., Borman, T. M., and De Jongh, R. (2012). Wood densitometry in 17th and 18th century Dutch,
German, Austrian and French violins, compared to classical Cremonese and modern violins. PLoS One,
7(10).
5. Stanciu, M. D., Mihălcică, M., Dinulică, F., Nauncef, A. M., Purdoiu, R., Lăcătuș, R. and Gliga, G. V.
(2021). X-ray Imaging and computed tomography for the identification of geometry and construction
elements in the structure of old violins. Materials, 14(20), 5926.
6. Sodini, N., Dreossi, D., Chen, R., Fioravanti, M., Giordano, A., Herrestal, P. and Zanini, F. (2012). Noninvasive microstructural analysis of bowed stringed instruments with synchrotron radiation X-ray
microtomography. Journal of Cultural Heritage, 13(3), S44–S49.
7. Yokoyama, M. (2021). Coupled numerical simulations of the structure and acoustics of a violin body. The
Journal of the Acoustical Society of America 150(3), 2058–2064.
8. Acoustic-Structure Interaction, COMSOL Homepate, https://www.comsol.jp/model/acoustic-structureinteraction-417.
9. Green, D. W., Winandy, J. E. and Kretschmann, D. E. (1999). Mechanical properties of wood. Wood
handbook: Wood as an engineering material. Madison, WI: USDA Forest Service, Forest Products
Laboratory. General technical report FPL; GTR-113: 4.1-4.45, 113.
10. Wang L. M. and Burroughs C. B., (2001). Acoustic radiation from bowed violins, Journal of the
Acoustical Society of America. 110(1), 543–555.

ABS-0399

Combination of deep-learning-based audio separation and speechenhancement for noise reduction of extracted signal from polyphonic music
Soichiro KOBAYASHI(1) , Takashi NOSE(2) , Akinori ITO(3)(1) Faculty of Engineering, Tohoku University, Japan(2) Graduate School of Engineering, Tohoku University, nose@tohoku.ac.jp(3) Graduate School of Engineering, Tohoku University, aito.spcom@tohoku.ac.jp

ABSTRACTThe music source separation technique has been dramatically improved with the development of deep learning,but the quality of the separated signal is still to be improved. In this work, we cascaded music source separationand enhancement models to suppress the noise included in the separated signal. We used Demucs, a timedomain music separation model, as the separation model. Demucs is a decoder-encoder model that takes anensemble music waveform as an input and converts the waveform into the waveform of the desired part using
the 1D-CNN and GLU. In addition, we developed a U-net-based denoising autoencoder that worked in the timefrequency domain as the enhancement model. As an experiment, we used mixed music signals in MUSDB18
and separated them into four parts: Drums, Bass, Vocals, and Other. We used the signal-to-distortion ratio(SDR) as an objective metric. The average SDR of the separated four signals using Demucs was 6.08 dB,
while connecting the denoising autoencoder gave an SDR of 6.26 dB. Furthermore, we jointly re-trained theconnected models, which gave the SDR of 6.28 dB. This result showed that cascading a signal separation modeland a signal enhancement model improved the quality of the separated signal.

Keywords: Music signal separation, Speech enhancement, Deep learning, Demucs

1

INTRODUCTION

Sound source separation is a technology to separate a specific sound from a mixed acoustic signal [1]. There
are two kinds of source separation techniques: one is the single-channel method [2], and the other is the multichannel method [3]. In this paper, we focus on the single-channel method and aim to apply the method tomusic signal separation, where musical sound signals are separated into individual parts such as drums, bass,
and vocals.
Speech enhancement is a technique applied to noisy speech signals to emphasize speech sound and suppress
noise [4]. The speech enhancement can be viewed as a kind of sound source separation, separating the soundinto speech and other sounds.
Until less than a decade ago, source separation and speech enhancement techniques were based on mathematical signal modeling, such as the non-negative matrix factorization (NMF) [2] or the independent low-rankmatrix analysis (ILRMA) [5]. Recently, deep learning techniques have been extensively studied [6, 7, 8]. However, it is still impossible to separate each part completely, even with the deep-learning-based sound source
separation model. The idea proposed in this paper is to suppress the leakage of the other part using a speechenhancement model as post-processing. According to this idea, we verify that the proposed method improves
the quality of separated music sound signals by combining sound source separation and speech enhancement
models.

Drums Wave

Bass Wave

Other Wave

Vocals Wave

Decoder1 (ConvTranspose1D)
Decoder2 (ConvTranspose1D)
・・・
Decoder6 (ConvTranspose1D)
Linear
BiLSTM
Encoder6 (Conv1D)・・・
Encoder2 (Conv1D)
Encoder1 (Conv1D)
Mixture Wave

Figure 1. The model structure of
Demucs2

Drums Wave

Bass Wave

ISTFT

ISTFT

ISTFT

ISTFT

U-NetDrums

U-NetBass

U-NetOther

U-NetVocals

Other Wave

Vocals Wave

2

Mixer

Time - Frequency domain

STFT
Mixture Wave

Figure 2. The time-frequency-domain part of the KUIELab-MDX-Net

RELATED WORKS

2.1 Music source separation
The music source separation is a kind of sound source separation [9, 10]. Music source separation models
based on deep learning are roughly classified into time-frequency-domain and time-domain models. The timefrequency domain models, such as Open-Unmix [11] and D3Net [12], receive an amplitude spectrogram obtained by short-time Fourier transformation (STFT) of the original signal and output amplitude spectrograms ofindividual parts. The time-frequency domain models are good at separating time-varying sounds, such as vocal
signals [13].
On the other hand, time-domain separation models such as Wave-U-Net [14] and Demucs [13] input theoriginal signal’s waveform into the model, and the waveform of each part is obtained as the output of themodel. Since these models take the waveform as the input and output, it does not need STFT, and the phase
information is incorporated into the model automatically. It is said that this kind of model is good at separating
sounds with strong attacks, such as drums and bases [13].
Demucs is a model with a U-Net [15] structure for one-dimensional convolution connecting six encoder anddecoder pairs. The encoder performs convolution on the input signal’s waveform, and one convolution layer
doubles the number of channels. Similarly, the decoder’s transposed convolution layer halves the number ofchannels, then outputs the specific part’s waveform. In addition to the model structure, the masking structureusing the Gated linear units (GLU) is adopted, and a two-layer Bidirectional LSTM (BiLSTM) is used for thebottleneck part. As a result, Demucs gives a high-quality separation in the time domain.
The above-mentioned methods such as Open-Unmix and Demucs only consider a specific part when extracting the signal of that part. On the other hand, KUIELab-MDX-Net [16] introduces the “Mixer” module that
exploits information of instruments of other parts when extracting the signal of a specific instrument. It is oneof the state-of-the-art methods in the competition for music source separation [17] in 2021.
This model is a combination of the time-domain and time-frequency-domain networks. The time-domainpart is the same as Demucs, and the time-frequency-domain part combines U-Net-based signal extractors andthe Mixer module. Figure 2 shows the network structure of the time-frequency-domain part of KUIELab-MDXNet. In the module shown in Figure 2, separation into each part is performed by a separate model, and thenthe separated signal is enhanced using a network called Mixer. The Mixer module is the signal enhancement
mechanism that emphasizes a signal of a specific part, exploiting the information of other parts and the original
signal.

2.2 Speech enhancement using deep learning
For speech enhancement, which is a kind of sound source separation, the models that use deep learning areclassified into two types: one is a time-frequency domain model, and the other is a time-domain model. An
example of speech enhancement in the time-frequency domain is speech enhancement using a mask. First, anamplitude spectrogram mixed with noise is input to the model, and the model estimates which part of the input
is the noise to be removed. Then, a mask of the same size as the input is outputted, acting as a filter to hide
the noise. The noise can be removed from the amplitude spectrogram by multiplying the input by this mask.
Macartney and Weyde [18] proposed a deep-learning-based speech enhancement model that employed the
U-Net structure, similar to the Wave-U-Net. Since then, a number of U-Net-based speech enhancement models
have been proposed so far [19, 20].
Recently, it has been proven that the model’s performance can be improved by combining the time-frequency
domain and time domain, including speech enhancement, such as Multi-Domain Processing via Hybrid Denoising Networks (MDPhD)[21]. This model connects the sub-networks of both domains, and the network is trained
considering the features of both domains. Another example is the Hybrid Demucs [22], which adds waveforms
separated in both domains.

3

PROPOSED MODEL

The idea of the proposed method is to combine the audio source separation method (Demucs) and other methods
such as speech enhancement or music separation methods to enhance the result of the first separation.
We examined two models. The first is the cascade model, which combines Demucs and the speech enhancement models. We intended to verify whether post-processing could improve the quality of the separated signal.
The second one is a combination of Demucs and Mixer part of the KUIE-MDX-Net. Using this model, we
examined how using other parts improved the quality of the specific part’s signal.
3.1 The cascade model
In this model, the separation model (Demucs) and the enhancement model are trained independently, and theseparated signal is input to the enhancement model to reduce noise. Figure 3 shows the network structure of
the entire model. As shown in the figure, this model first separates the input waveform into individual parts
using Demucs, and the waveform of each part is enhanced using an enhancement model trained part by part.
Figure 4 shows the model structure of the speech enhancement model. It has a U-Net structure, with an
encoder and a decoder with five two-dimensional convolution layers. This network calculates the time-frequency
mask, and the estimated mask is multiplied by the original spectrogram to reduce noise [20].
The enhancement models were prepared part by part. Let the amplitude spectrograms separated into fourparts (Drums, Bass, Other, and Vocals) by Demucs be A p (k,t), where 1 ≤ p ≤ 4 is the part number. Then
we calculate the log spectrogram as log(1 + A p (k,t)), and this log spectrogram is input to the enhancement
model. The enhancement model calculates the binary mask M p (k,t). We trained the model so that log(1 +A p (k,t))M p (k,t) becomes similar to the reference signal.
The detailed specification of the model is shown in Table 1. We used the Leaky ReLU as the activation
function, and we used the sigmoid function as the activation of the output layer of the decoder so that the output
values of the network (the time-frequency mask) were between 0 and 1. When we transformed the amplitude
spectrogram into a waveform, we used the phase of the original signal since the amplitude spectrogram lacked
the phase information.
3.2 The mixer-combined model
The second model combines Demucs and Mixer in KUIELab-MDX-Net. Using this model, we investigate how
the use of other parts improves a specific part’s speech enhancement model. Figure 5 shows the structure of
the second model. Its basic structure is a combination of Demucs and Mixer; the difference is that the originalMixer (as shown in Figure 2) receives the time-domain signal of each part and the original waveform, whereas
the proposed model does that in the time-frequency domain. We employed this structure aiming to combine thetime-domain-based and time-frequency-domain-based models to compensate for the weak points of those models

Figure 3. Structure of the cascade model

Figure 4. Structure of the speech enhancement model[21]. Figure 6 shows the structure of the mixer network, which is the same as that of the enhancement network(Fig. 4), except that it has five input channels (four parts and the original signal) and four output channels (fourparts).
When training this model, we first trained Demucs using the training data, and then we trained the entire
model. We used the loss function combining two kinds of losses, the loss after separation (Lwave1 ) and afteremphasis (Lwave2 ), and the total loss was the sum of these loss functions.
L = Lwave1 + Lwave2(1)

We used the weighted-SDR (wSDR) loss [23] as a loss function.

4

EXPERIMENT

4.1 Materials
We used MUSDB18 corpus [24] for training and testing the models. We used 86 songs for training, 14 fordevelopment, and 50 for testing. Five signals, Mixture (the original signal), Drums, Bass, Other, and Vocals, areprepared for one song. The sampling frequency of the files is 44100 Hz, and the quantization is 16 bit/sample.
When training a model, we extracted 11-second segments by shifting 1 second and used them as individual
data. In addition, we augmented the training data with the following augmentation methods:• Randomly mixing instruments from different songs [25]• Randomly swap left and right channels of instruments [25]• Randomly scale the signal from 0.25 to 1.25 [25]• Randomly change the phase by multiplying −1 [26]• Random pitch shift from −2 to 2 halftones [27]• Ramdom tempo change from 0.88 to 1.12 [27]Signals in MUSDB18 have two channels. We input the stereo signal to Demucs part of the model, then we
split two channels and input them individually to the enhancement model.

Table 1. Network conditions of the cascade model
Cascade model

Mixer-combined model(21, 5), (2, 2)

Kernel size and stride
Channel size of the encoder

1 − 45 − 90 − 90 − 90 − 90

5 − 45 − 90 − 90 − 90 − 90

Channel size of the decoder

90 − 90 − 90 − 90 − 45 − 1

90 − 90 − 90 − 90 − 45 − 4

STFT Frame size and shift4096, 1024

Window function

Hann window

Batch size

12

6

Maximum number of epochs

50

Loss function

MSE

wSDR

Activation

Leaky ReLU / Sigmoid (the last layer)

Optimizer and learning rate

Adam, 2 × 10−4

Table 2. The separation results evaluated by SDR [dB]
Drums

Bass

Other

Vocals

All

IRM

10.04

9.33

7.78

10.36

9.38

Demucs

6.59

7.04

4.19

6.48

6.08

Cascade model

6.63

7.11

4.40

6.88

6.26

Mixer-combined model

6.49

7.22

4.48

6.92

6.28

4.2 Result
Table 2 shows the summary of the objective evaluation for 50 pieces. We used the signal-to-distortion ratio(SDR) [28], representing the ratio of other sounds. The IRM (Ideal Ratio Mask) in the table represents theresult when we use the ratio of the clean and noisy spectrograms as the time-frequency mask. This result is anupper limit of the separation quality when using the mask-based speech enhancement.
When we compare Demucs and the cascade model, we can confirm that the enhancement model improves
the SDR for all parts. Especially, “Other” and ”Vocals” showed larger improvement than “Drums” and “Bass.”
The use of the mixer-combined model gave more improvement for three parts except for Drums. The SDRof the Drums was slightly lower than that of the Demucs separation.
4.3 Examples
Figure 7 shows examples of spectrograms for vocal signal separation. Figure 7 (a) is the reference signal. Figure
7 (b) is the output from Demucs. The circled part in (b) has a straight striped pattern derived from the soundof Other part (guitar). Comparing the figures (a) and (b), the separation result by Demucs is insufficient tosuppress the instrumental sound. The instrumental sound is suppressed in the results (c) and is almost removed
in the result (d).
Figure 8 shows the examples for drum signals. We can see the horizontal line-like patterns in the parts
surrounded by the ellipses derived from the other instrumental sound. This noise is suppressed in (c) and (d).
On the other hand, the part surrounded by the dashed circle is the noise from vocal signal. We can see that thisℒ!"#$%
Drums Wave

Bass Wave

ISTFT

ISTFT

Other Wave

Vocals Wave

ISTFT

ISTFT
STFT

Mixer

Time - Frequency domain
STFT

STFT
Mid Drums Waveℒ!"#$&Mid Bass Wave

STFT

Mid Other Wave

STFT
Mid Vocals Wave

DemucsTime domain
Mixture Wave

Figure 5. The mixer-combined model

Figure 6. The mixer network

noise is suppressed in (c). However, in (d), the harmonic pattern of the vocal signal is over-suppressed, whichcauses additional distortion of the separated signal. This distortion is audible as a kind of noise.

5

CONCLUSION

In this paper, we propose a method to improve music source separation performance using speech enhancement.
We used the time-domain audio separation method Demucs as a baseline model. We introduced two models;
one is the cascade model, where a simple U-Net-based model is used that suppresses noise part by part. Anotheris the mixer-combined model, in which all parts are considered simultaneously for enhancement.
We conducted an experiment to compare Demucs and the proposed methods, and the result showed the
proposed methods improved the SDR over the baseline method. In addition, the mixer network was better
than the part-by-part enhancement on average, but it over-suppressed the vocal signal, introducing additional
distortion.
In future work, we need to compare the proposed methods to the existing methods that combine the timedomain and time-frequency domain signals, such as Hybrid Demucs. In addition, we plan to incorporate theadversarial generative network technology [29, 30] for further improvement.

ACKNOWLEDGMENT
Part of this work was supported by JSPS KAKENHI Grant Number JP21H00895.

References[1] Pal M, Roy R, Basu J, Bepari MS. Blind source separation: A review and analysis. In: Proceedings of International Conference Oriental COCOSDA held jointly with 2013 Conference on Asian Spoken Language
Research and Evaluation (O-COCOSDA/CASLRE); 2013. p. 1–5.
[2] Févotte C, Vincent E, Ozerov A. Single-Channel Audio Source Separation with NMF: Divergences, Constraints and Algorithms. In: Makino S, editor. Audio Source Separation. Springer; 2018. p. 1–24.
[3] Yellin D, Weinstein E. Multichannel signal separation: Methods and analysis. IEEE Transactions on Signal
Processing. 1996;44(1):106–118.

(a) Reference(b) Separation by Demucs(c) Result by the cascade model(d) Result by the mixer-combined model

Figure 7. An example of enhancement result (Vocals)[4] Chaudhari A, Dhonde S. A review on speech enhancement techniques. In: Proceedings of 2015 International Conference on Pervasive Computing (ICPC); 2015. p. 1–3.
[5] Sawada H, Ono N, Kameoka H, Kitamura D, Saruwatari H. A review of blind source separation methods:
two converging routes to ILRMA originating from ICA and NMF. APSIPA Transactions on Signal andInformation Processing. 2019;8:e12.
[6] Huang PS, Kim M, Hasegawa-Johnson M, Smaragdis P. Deep learning for monaural speech separation.
In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP);
2014. p. 1562–1566.
[7] Wang D, Chen J. Supervised speech separation based on deep learning: An overview. IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2018;26(10):1702–1726.
[8] Xu Y, Du J, Dai LR, Lee CH. A regression approach to speech enhancement based on deep neuralnetworks. IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2014;23(1):7–19.

(a) Reference(b) Separation by Demucs(c) Result by the cascade model(d) Result by the mixer-combined model

Figure 8. An example of enhancement result (Drums)[9] Liu R, Li S. A review on music source separation. In: Proceedings of IEEE Youth Conference on
Information, Computing and Telecommunication; 2009. p. 343–346.
[10] Rafii Z, Liutkus A, Stöter FR, Mimilakis SI, FitzGerald D, Pardo B. An overview of lead and accompaniment separation in music. IEEE/ACM Transactions on Audio, Speech, and Language Processing.
2018;26(8):1307–1335.
[11] Stöter FR, Uhlich S, Liutkus A, Mitsufuji Y. Open-Unmix – a reference implementation for music source
separation. Journal of Open Source Software. 2019;4(41):1667.
[12] Takahashi N, Mitsufuji Y. D3Net: Densely connected multidilated DenseNet for music source separation.
arXiv preprint arXiv:201001733. 2020.
[13] Défossez A, Usunier N, Bottou L, Bach F. Music Source Separation in the Waveform Domain. arXiv
preprint arXiv:191113254. 2019.
[14] Stoller D, Ewert S, Dixon S. Wave-U-Net: A multi-scale neural network for end-to-end audio source
separation. In: Proceedings of ISMIR Conference; 2018. p. 334–340.

[15] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In:
International Conference on Medical image computing and computer-assisted intervention. Springer; 2015.
p. 234–241.
[16] Kim M, Choi W, Chung J, Lee D, Jung S. KUIELab-MDX-Net: A Two-Stream Neural Network for Music
Demixing. arXiv preprint arXiv:211112203. 2021.
[17] Mitsufuji Y, Fabbro G, Uhlich S, Stöter FR.
arXiv:210813559. 2021.
[18] Macartney C, Weyde T.
arXiv:181111307. 2018.

Music demixing challenge 2021.

arXiv preprint

Improved speech enhancement with the Wave-U-Net.

arXiv preprint[19] Giri R, Isik U, Krishnaswamy A. Attention Wave-U-Net for speech enhancement. In: Proceedings ofIEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA). IEEE; 2019. p.
249–253.
[20] Tolooshams B, Giri R, Song AH, Isik U, Krishnaswamy A. Channel-attention dense U-Net for multichannel speech enhancement. In: Proceedings of IEEE International Conference on Acoustics, Speech andSignal Processing (ICASSP). IEEE; 2020. p. 836–840.
[21] Kim JH, Yoo J, Chun S, Kim A, Ha JW. Multi-domain processing via hybrid denoising networks forspeech enhancement. arXiv preprint arXiv:181208914. 2018.
[22] Défossez A. Hybrid Spectrogram and Waveform Source Separation. arXiv preprint arXiv:211103600. 2021.
[23] Choi HS, Kim JH, Huh J, Kim A, Ha JW, Lee K. Phase-aware speech enhancement with deep complex
U-Net. In: Proceedings of International Conference on Learning Representations; 2018. .
[24] Rafii Z, Liutkus A, Stöter FR, Mimilakis SI, Bittner R. The MUSDB18 corpus for music separation; 2017.
Available from: https://doi.org/10.5281/zenodo.1117372.
[25] Uhlich S, Porcu M, Giron F, Enenkl M, Kemp T, Takahashi N, et al. Improving music source separation
based on deep neural networks through data augmentation and network blending. In: Proceedings of IEEE
International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2017. p. 261–265.
[26] Nachmani E, Wolf L. Unsupervised Singing Voice Conversion. In: Proceedings of Interspeech 2019; 2019.
p. 2583–2587.
[27] Cohen-Hadria A, Roebel A, Peeters G. Improving singing voice separation using deep u-net and wave-unet with data augmentation. In: Proceedings of 27th European Signal Processing Conference (EUSIPCO);
2019. p. 1–5.
[28] Vincent E, Gribonval R, Févotte C. Performance measurement in blind audio source separation. IEEE
Transactions on Audio, Speech, and Language Processing. 2006;14(4):1462–1469.
[29] Fan ZC, Lai YL, Jang JSR. SVSGAN: Singing voice separation via generative adversarial network. In:
Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).
IEEE; 2018. p. 726–730.
[30] Stoller D, Ewert S, Dixon S. Adversarial semi-supervised audio source separation applied to singing voice
extraction. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP). IEEE; 2018. p. 2391–2395.

ABS-0467

Automatic classification of blowing properness in flute sounds*
Sai OSHITA(1) , Tetsuro KITAHARA(1)(1) Nihon University, Japan, {ohshita,kitahara}@kthrlab.jp

ABSTRACTOur goal is to develop a system that evaluates the technique of novice flute players when playing their instrument. This paper addresses automatic evaluation of flute sounds. We collected flute sounds played with a properand improper blowing styles. Using the sounds, a subjective evaluation with a five-graded scale were shown toparticipants. Then, automatic evaluation of those sounds (that is, the prediction of the degree of the appropriateness and the classification of appropriate and inappropriate sounds) was conducted with linear regression anddecision trees using results of the subjective evaluation. The results showed that (1) the root mean squared errorof the regression-based prediction was 0.65, and (2) the accuracy of the decision-tree-based classification was
93%. On the other hand, the accuracy of the classification of blowing styles using the decision tree was lower(about 70% for two-class (normal/non-normal) classification, 20–40% for five-class classification).

Keywords: flute, wind instrument, acoustic analysis, automatic evaluation

1

INTRODUCTION

The flute is an instrument whose sound changes greatly when the direction of the breath and the size of the mouth
are changed. Therefore, it is necessary to carefully control the size of the mouth and the direction and strength of thebreath in order to play the flute with appropriate tone quality. However, although there are many books on the market
instructing how to play the flute, not many of them clearly describe these points. Even if one reads a detailed book, it
is not easy for a beginner to listen to his/her own sound and judge how to improve it by themselves.
Various studies have been conducted on the acoustic analysis of flutes. Ando[1, 2, 3, 4] analyzed the differencesin the range of sound and the number of overtones in different registers when the blowing angle and breath strength
are changed, as well as the differences among multiple flutes. Onogi et al. [5] analyzed how to make the machine
performance more like human performance. Nishimura et al. [7] analyzed the differences in tone preferences between
professional and amateur flutists. Yorita et al. [9] and Fletcher [10] analyzed how air pressure, lip shape, and overtones
were related to years of experience. These were mainly acoustic analyses and did not map the relationship with theblowing style.
On the other hand, Wilcocks [6] presented practice routines including methods of controlling the tone of the flute.
Without taking into account whether or not good sounds were being produced, the paper focused on methods forstudents to find the proper way to blow by trial-and-error.
Yoonchang Han[8] had created a system to judge whether the player was playing apppropriate sounds by judging
the head tube relationship, air pressure, and fingering from the sound, but not whether the sound was good or not.
Therefore, our goal is to develop a system that informs novice flute players whether they are producing appropriate
sounds. For this aim, we first collected audio samples of flute tones with proper and improper blowing styles. Then,we extracted acoustics features from those, and collected participant responses to the tone qualities. Using those data,
we attempted automatic classification of proper and improper tones using linear regression and decision trees.
* This work was supported by JSPS Kakenhi Nos. 22H03711 and 21H03572.

2

METHOD

2.1 Used Data
In order to collect the sounds of the improperly played music, we collected the performance sounds on a
crowdsourcing site. They were asked to play the score shown in Figure 1. and we instructed the performers
not to play vibrato. In order to reduce the burden on individual performers, we divided the collection into thefollowing two pattern.
1 [normal] [larger mouth] [smaller mouth] [breath upward]
2 [normal] [breath downward] [breath stronger] [breath weaker]
The number of performers and the number of performance sounds collected are shown in Table 1. However,since it is difficult to distinguish [breath stronger] and [breath weaker] from forte and piano as a musicalexpression, we decided not to use them in the following analyses. In order to compensate for the fact that thesound volume varies depending on the recording conditions, we corrected the amplitudes so that the temporalmean values of the amplitudes are equal.

Figure 1. Note performed by participants

Table 1. Participants (performers) for flute sound collection# of performers

flute experience (years)# of sounds per performer

Pattern 1

12

0.1∼10

about 76

Pattern 2

12

0.1∼10

about 88

2.2 Subjective Evaluation
Using a Web-based crowdsourcing service, subjective evaluations of the collected sounds were conducted. Participants were limited to current or former students of flute majors in music colleges or high school music
departments, and those who have played the flute for at least 12 months. As a result, three participants from
Table 2 participated.
When a participant opened the designated web page, 20 randomly selected performance sounds were displayed. They listened to them one by one and entered their answers to the questions in Table 3. The choices
for choice-type questions are Table 4 and Table 5.
2.3 Acoustic Analysis Methods
2.3.1 Silence interval removal
In order to remove the sections before and after blowing from the collected acoustic signals, the following
process is performed. First, the root mean square (RMS) is calculated for each of the 512 points for a given
acoustic signal. Let a1 a2 · · · aT be the sequence of the RMS. The maximum value of RMS is amax . Then, we
find the smallest value from nS satifying ∀n ∈ [nS , nS + α] : an > amax /4 (n = 1, 2, · · · T ), which is regarded as
the onset time of the sound. Similarly, we find the largest value from nE satisfying ∀n ∈ [nE − α, nE ] : an >amax /4 (n = 1, 2, · · · T ), which is regarded as the offset time of the sound.

Table 2. Participants (evaluators) for subjective sound evaluation
Participant# of participation

Flute experience (years)

Other instrument experience

A

1

3

Sax, piano

B

10

5

Piano, harp

C

1

6

Electric organ, percussion, etc.

D

1

22

Piano, percussion, piccolo, etc.

E

1

28

Piano

F

1

28

Piano

Table 3. Questions used in the subjective evaluation
1

How good did it sound?

option1

2

how clear was it

option13

How small was the volume swing?

Response type 1

4

How little was the pitch swing?

Response type 1

5

How much breath did you hear?

Response type 1

6

Were there applicable problems with the blowing method?

Response type 2

7Things I noticed other than the above

description

Then, the acoustic signal from nS + β to nE − β is extracted and used in the following analysis, The acoustic
signal extracted from the interval from nS + β to nE − β is used for the subsequent analysis. The reason for
narrowing the interval to be extracted from [nS , nE ] by only beta is to avoid the situation where the fundamental
frequencies in the vicinity cannot be extracted due to calculation errors at the start and end points. We set
α = T /19 and β = (nE − nS )/50.
2.3.2 Features related to temporal variation of amplitude and fundamental frequency
It is important for beginners to be able to produce sounds with little variation in amplitude and fundamental frequency with time. so we focus on acoustic features that represent temporal variation of amplitude andfundamental frequency.
First, the RMS and fundamental frequency (F0) are obtained for each of the 512 points of the acoustic
signal after removing the silence interval. Let a1 a2 · · · aTa and f1 f2 · · · fT f be the sequence of the RMS and F0,
respectively. To extract F0, we used PyWorld, a speech analysis library for Python. From { fn }, we excluded
the frames whose F0 was not found. Then, the following four features are extracted.
a|an − an−1 |• Average of time variation of amplitude: dv = T1a ∑Tn=2

Tf| fn − fn−1 |• Average of time variation of fundamental frequency: d f = T1f ∑n=2

• Amplitude range: rv = maxn an − minn an• Fundamental frequency range: the quartile range fr of f1 f2 · · · fT f . The silent frames were excluded because the F0 calculated from them may be −∞ or NaN in PyWorld.
2.3.3 Features related to the amplitude spectrum
According to Ando’s analysis [1], the better the sound is, the more higher harmonics there are. In addition, themore breath sounds are included, the more they appear as non-harmonic components. Therefore, the following
features are extracted.
• Number of harmonic components (including fundamental frequency components) at the beginning of blowing:Table 4. Response type 1 for subjective evaluation

Table 5. Response type 2 forsubjective evaluation

Below beginner level. Seen as just starting level.

1

Breathing too strong

2

Beginner level. There are some areas that need improvement.

2

Breathing too weak

3

Intermediate level. Some improvement is needed. In general,
the student’s performance is satisfactory.

3

Mouth size too large

4

Mouth too small

4Intermediate to advanced level. There are some points to be
improved, but the performance is acceptable for an amateur
concert.

5

Breath too upward

6

Breath too downward

Advanced level. There is nothing to be improved at all.

7

No problem

8

I don’t know

1

5

From the first 1-sec interval (after removing the silence interval), the amplitude spectrum is obtained, andis denoted by S[ f ] ( f : frequency). Let F0 be the fundamental frequency. Then, The amplitude of the k-th
harmonic, sk , is calculated as sk = max{S[kF0 − r], · · · , S[kF0 + r]}. The upper bound of k is defined sothat (sk + sk+1 )/2 > 2sk+1/2 is not satisfied or sk < 0.01 · max f S[ f ]. Let os be the number of overtones
extracted. The value of F0 is calculated based on the equal temperament (C5≈523.25[Hz]).
• Percentage of overtones (non-fundamental components) in all harmonics at the beginning of blowing:
fs = ∑Kk=1 sk s1 , where s1 , · · · , sk are the amplitudes of the extracted harmonics.
• Percentage of overtones in the entire spectrum at the beginning of blowing:
ns = ∑Kk=1 sk ∑ f S[ f ]. This feature takes a lower value when more non-harmonic components such as
breath sounds are included.
• Number of overtones (calculated from the middle interval),• Percentage of non-fundamental frequency components in all overtones (calculated from the middle interval),• Percentage of overtone components in the whole spectrum (calculated from the middle interval):
For extracting these features, the same process as above is applied to the central one second of theacoustic signal. These features are denoted by oc , fc , and nc , respectively.
2.4 Automatic Evaluation
From the collected performance sounds, we constructed a linear regression and a decision tree. The average
responses to the subjective evaluation were used as the objective variable and the acoustics features mentioned
above were used as the explanatory variables. In addition, we conducted a decision tree experiment when thetype of blowing styles were used as the objective variable. For all evaluations, half of the data were used fortraining the models, and the remaining were used for testing them.
2.4.1 Linear regression
With linear regression, the objective variable y is approximated as y = w0 dv +w1 rv +w2 d f +w3 r f +w4 os +w5 oc +w6 ns + w7 nc + w8 fs + w9 fc + γ, where the values of wn and the constant (γ) were estimated. The same linear
regression was applied to the data without the outliers. Accuracy was evaluated as the root mean square error(RMSE).
2.4.2 Decision tree
To predict the subjective evaluation with a decision tree, we have to discretize evaluators’ responses. We attempted two different ways of discretization: two-class [lower than 2 / 2 or higher], and three-class [lower than

2 / 2 (incl.) to 3 (not incl.) / 3 or higher]. For sounds that have more than one evaluation, their average wasused. The same experiment was also conducted after removing outliers.

3

RESULT

3.1 Results of acoustic feature extraction and subjective evaluation
The results of acoustic feature extraction (standardized) are shown in Figure 2. This figure shows the soundsof [normal], [large mouth], [small mouth], [upward breath], and [downward breath] from left to right, and thevertical lines indicate the boundaries of those blowing styles. The figure suggests the following features do notdepend clearly on the blowing styles:• the mean value of amplitude fluctuation (dv ),• the number of overtones in the first second (os ),• the number of overtones in the middle second (oc ),• the percentage of non fundamental frequency components in the first second ( fs ),• the percentage of non fundamental frequency components in the middle second ( fc ),• the percentage of overtones in the first second (ns ), and• overtones in the middle 1 second (nc ).
On the other hand, the following features have no outliers for the normal blowing style while there are someoutliers for other blowing styles:• the mean value of fundamental frequency fluctuation (d f ),• the range of amplitude (rv ), and• the range of fundamental frequency (r f ),If such sounds with outliers of acoustic features actually produce inappropriate sounds, these acoustic features may be used to classify the appropriateness of the sounds. We therefore analyzed the sounds with outliers
and their subjective evaluation (Table 6). The results show that outliers do not appear in the normal blowing
style while outliers appear in all other blowing styles. Also, the sounds with outliers were evaluated low (e.g.,
1 or 2 for responses to the subjective evaluation, commented like “almost airy sounds”). Above, we regarded
features satifying the following conditions as outliers: d f ≥ 1.5, rv ≥ 1.0, and r f ≥ 1.0.
Regarding the expected problems of blowing (Table 3, Question 6), the option of too weak breath wasselected for all but one of the tones played with the breath facing upward. For all but two of the tones thatwere blown with the breath upward, the option that the direction of the breath was too downward was selected.
3.2 Results of acoustic feature analysis and subjective evaluation
Figure 3 shows the amplitudes, fundamental frequency, and amplitude spectra (the first 1-sec, the central 1sec) of the sounds that were subjectively evaluated highly (sound ID: d19, downward breath) and low (sound
ID: b17, loud mouth). Looking at the amplitude and fundamental frequency in Figure 3, it can be seen thatsounds with higher ratings have less temporal variation. In fact, temporal-variation-related features for Soundd19 (d f = −0.193, rv = −0.271, r f = −0.274) are smaller than those for Sound b17 (d f = 0.248, rv = −0.167,
and r f − 0.252). Note that some of these values are negative because they have been normalized so that the
average among all sounds is zero. As for the amplitude spectrum, it can be seen that the higher-rated have highamplitudes in higher-order harmonics, which is consistent with the finding by Ando [1].
On the other hand, no difference in the distribution of features was observed between the different blowing
styles. That means that it is difficult to identify the cause of the problem in the blowing style, even if it it ispossible to identify whether the blowing is appropriate or not. .

Table 6. Performance sound with outlier feature and subjective evaluation of it (Blank: not rated)
outlier
Performance method

large mouthsmall mouth

breath upward

breath downward

question numbers

sound ID

feature value

1

2

3

4

5

6

7

b10

rv

2

2

2

2

2

2, 4, 6

Unstable and not blowing properly.

b11

rv

1

1

1

1

1

2,6

The sound is muffled and not properly reproduced.

b12

df ,rf

1

1

1

1

1

1, 2, 4,6

Out of the question, Air noise

s9

rf

1

1

1

1

1

1, 2, 6just starting out, and don’t seem to understand anything.
can’t play properly at all. haven’t got the knack of making notes.

s13

rf

1

1

1

1

1

2, 6

still haven’t gotten the hang of how to make the sound.

s14

rv

1

1

1

1

1

2, 6

broken sounds15

df ,rf

1

1

1

1

1

2, 6

The place where the breath hits is not right. sound likea person who blew for the first time today.

s16

df ,rf

1

1

1

1

1

2, 4, 6The sound is not right. breath is not in the right place.
There is no sound at all.

s17

rf

1

1

1

1

1

2, 3, 4, 6

Not a problem, just getting started. No sound was coming out. felt like blowing into it, not forming a mouth
shape.

u6

rv

2

2

2

2

2

1, 2

The breath was too strong and the sound be interrupted
in the middle. The sound was choppy and unsteady.

u8

rv

u9

rf

1

1

1

1

1

2, 4, 6just starting out and are still getting used to it.

u10

df ,rf

1

1

1

1

1

2, 5, 8The sound is not coming out properly. don’t know
where to breathe and how much to breathe. You are
not blowing.

u11

df ,rf

1

1

1

1

1

3, 6

Like someone who blew for the first time today. Thereis no sound at all.

u12

df ,rf

1

1

1

1

1

2

Almost air sound.

u13

df

d9

df ,rf

1

1

1

1

1

2, 6

Sounds terrible. Like someone who blew for the firsttime today.

d13

df ,rf

d17

df

1

1

1

1

1

2, 6

can’t get the sound right. not hitting the right spot withbreath, not getting the hang of it.

3.3 Linear regression
The root mean square error (RMSE) for the prediction of the subjective evaluation using linear regression is0.670 (rounded to the nearest third decimal place). When the outliers are removed, the RMSE is is 0.642(rounded to the nearest third decimal place).
The coefficients w0 , · · · , w9 and the constant γ estimated with the linear regression, where the approximation
equation is y = w0 dv + w1 rv + w2 d f + w3 r f + w4 os + w5 oc + w6 ns + w7 nc + w8 fs + w9 fc + γ are shown in Table 7.
Comparing these coefficients with and without outliers, the weight of r f has changed significantly. This could
be because r f is negative for non outliers but is positive for some outliers. There was little difference in theroot mean square error (RMSE) when outliers were removed or not. Figure 4 shows a comparison between
predicted and actual values of the subjective prediction. The figure shows that, even though the actual value ofthe highest subjective evaluation is 4.75, its predicted value is 2.38. When the outliers are removed, the sounds
where actual subjective evaluation are greater than 3 have lower predicted values than the actual evaluation.

Table 7. Coefficients of linear regression equation (Rounded to the nearest third decimal place)
w0

w1

w2

w3

w4

w5

w6

w7

w8

All data−0.202−0.168−0.239−0.1040.519−0.202Outliers excluded−0.204−1.525−0.606

8.312

0.219

0.091

w9

0.015

0.056

−0.449

0.240

2.049

0.009

−0.014−0.137−0.0663.916

γ

3.4 Decision tree
3.4.1 Objective variable: subjective evaluation
As described above, the prediction of subjective evaluation with decision trees were conducted as two-class andthree-class classification. The results of these classification are shown in Figure 5. The depth of the decision
tree is 2. Table 8 shows the classification accuracy when the depth of the decision tree is 2, which has the
maximum classification accuracy among different tree depths.
Table 8. Decision trees’ depth and classification accuracy (in parentheses: outliers removed)
Classification

Maximum(Depth 2)

Maximum

Depth(Maximum time)Two-class (lower than 2 / 2 or higher)

0.93 (0.94)

0.93 (0.94)

2, 3, 4 (2)Three-class (lower than 2 / 2 to 3 / 3 or higher)

0.83 (0.84)

0.86 (0.84)

3 (2)The classification accuracy was about 80∼90%, regardless of whether outliers were included or not. For
top-level decision rule, d f was often used, while only amplitude- and F0-related features such as dv , rv , andr f were used for other-level decision rule. This is considered to be consistent with the prediction that a soundwith less temporal variation in amplitude and fundamental frequency is a good sound. It also indicates that
amplitude and fundamental frequency related features are important in predicting subjective evaluation.
3.4.2 Objective variable: type of blowing
We conducted different classification tasks: two-class [normal / other], three-class [normal / mouth-size-related /breath-direction-related] and five-class [normal / mouth big / mouth small / breath upward / breath downward].
The results of those tasks are shown in Figure 6, where the depth of the decision tree is 2. Table 9 lists theclassification accuracy with the depth of the decision tree of 2, which has the maximum classification accuracy.
On average, the accuracy was lower than 20% for the five-class classification, about 30% for the three-classclassification, and about 40% for the two-class classification (even though the maximum classification accuracy
is about 70%.). The classification accuracy decreased when the outliers were removed from the prediction ofclassification.
Table 9. Decision Tree Depth and Discrimination Rate 2 (in parentheses: outlier removal)
Classification

Maximum (Depth 2)

Maximum

Depth(Maximum time)

two-class (normal, others)

0.7 (0.41)

0.71 (0.75)

3 (5)three-class (normal, oral, breath)

0.39 (0.31)

0.49 (0.44)

8 (5)five-class (each blowing style)0.36 (0.19)

0.36 (0.19)

2 (2)Features related to the fundamental frequency such as d f , r f were often used in the top-level decision rule,
while features other than os , oc were often used in the other-level decision rule. The ns and nc were used more
frequently in the two-class classification than the other classifications. Compared to the classification of thesubjective evaluation, features related to the amplitude spectrum were used more frequently. These imply that
amplitude, fundamental frequency, and amplitude spectrum are effective for the prediction of the blowing styles.

4

CONCLUSION

The purpose of this study is to investigate whether it is possible to identify blowing styles (especially improper
blowing styles) from performed sounds. Our results imply that it is possible to classify sounds with proper andimproper blowing with accuracy of 80∼90% , whereas it is difficult to identify the cause of inproper blowing
from acoustic features.
However, the number of responses to the subjective evaluation is still small, so the results may be affectedby individual differences among the evaluators. We have to increase the number of responses to obtain morereliable analysis. In addition, we have to analyze various performances including long tones, melodies, andarpeggio, which will be important to achieve a practical performance suppoer system.

REFERENCES[1] Ando Y, Music·musicology and acoustics of musical instruments, Acoustical Society of Japan, 59 (3),pp.146-152, 2003
[2] Kato M, Nishimura A, Ando Y, Frequency measurements of flute tones by analytic signal, information
Processing Society of Japan, 2000-MUS-039 (16), pp.43-50, 2001
[3] Ando Y, Drive Conditions of a Flute and their Infiuences upen and Sound Pressure Level and Fundamental
Frequency of Generated Tone, Acoustical Society of Japan, 26 (6), pp.253-260, 1970
[4] Ando Y, Drive Conditions of the Flute and their Influences upon Harmonic Structure of Generated Tone,
Acoustical Society of Japan, 26 (7), pp.297-305, 1970
[5] Onogi K, Yokoyama H, Iida A, Mechanism analysis of timbre change by blowing angle of flute (inJapanese), Supercomputing Division, 22 (Special Issue 1), 2020
[6] Wilcocks. G, Improving Tone Production on the Flute with Regards to Embouchure, Lip Flexibility, Vibrato
and Tone Colour, Thesis, University of Pretoria, 2006.
[7] Nishimura A, Kato M, Ando Y, Effect of the Fluctuations of Harmonics on the Subjective Quality of FluteTone, Proceedings of Forum Acusticum Sevilla, MUS-01-004, 2002
[8] Yoonchang H, Kyogu L, Hierarchical Approach to Detect Common Mistakes of Beginner Flute Players,
Proc. of ISMIR, 2014
[9] Ron Y, and John C, Using Spectral Analysis to Evaluate Flute Tone Quality, Proc. of Meetings on Acoustics, 23 (035001), 2015[10] Fletcher. N. H, Acoustical Correlates of Flute Performance Technique, The Journal of the Acoustical Society of America 57, 233, 1975

Mean value of amplitude swing (dv )Mean value of fundamental frequency (d f )Amplitude range (rv )fundamental frequency range (r f )Overtone number (first second) (os )Overtone number (middle 1 second) (oc )Percentage of non-fundamental frequency components
in all harmonics (first second) ( fs )Percentage of non-fundamental frequency components
in all harmonics (middle 1 second) ( fc )Ratio of overtones to the rest (first second) (ns )Ratio of overtones to the rest (middle 1 second) (nc )Figure 2. Distribution of the feature values (standardized) for each performance note (from left to right: [Normal], [Large mouth], [Small mouth], [Breath upward], [Breath downward])

Rating Good:Amplitude (rms)

Rating Good:fundamental
frequency

Rating Good:amplitude
spectrum (first second)

Rating Good:amplitude
spectrum (middle 1 second)

Rating bad:amplitude (rms)

Rating bad:fundamental
frequency

Rating bad:amplitude
spectrum (first second)

Rating bad:amplitude
spectrum (middle 1 second)

Figure 3. Amplitude, fundamental frequency, and amplitude spectrum of the performance sounds that were
good/bad rated.

Outliers included

Outliers removded

Figure 4. Comparison of actual subjective evaluation and its prediction with linear regression (horizontal: actual
evaluation, vertical: predicted)

Two-class (lower than 2, 2 or higher)

Two-class (lower than 2, 2 or higher), outliers removed

Three-class (lower than 2, 2–3, 3 or higher)

Three-class (lower than 2, 2–3, 3 or higher), outlier removed

Figure 5. Decision trees obtained with predicted subjective evaluations

Two-class (normal, others)

Two-class (normal, others), outliers removed

Three-class (normal, oral, breath)Three-class (normal, oral, breath), outliers removed

Five-class (each blowing style)

Five-class (each blowing style), outliers removed

Figure 6. Decision trees obtained for predicting blowing styles

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022 in Gyeongju, Korea

ABS-0516

Effects of the temperature on the vibration duration of tuning forks
Kyota NOMIZU1; Sho OTSUKA2,3,4; Seiji NAKAGAWA2,3,4
1

Dept. of Medical Eng., Grad. School of Sci. & Eng., Chiba Univ., Japan
2

Ctr. for Frontier Medical Eng., Chiba Univ., Japan
3

4

Grad. School of Eng., Chiba Univ., Japan

Med-Tech Link Ctr., Chiba Univ. Hospital, Japan

ABSTRACT
Tuning forks are used in the fields of music, clinical tests of vibrotactile sensation/audition, and healing. For
the tuning fork, longer duration of the fundamental tone is required as well as accuracy of the frequency of
the fundamental tone. Additionally, it is desirable that the harmonic tone decreased immediately after the
stroke. However, in the manufacturing process of the tuning forks, only the fundamental frequency is tuned,and durations of tones have not been evaluated. In addition, although previous studies have revealed theeffect of temperature on the frequency, the effect of temperature on durations of tones has not been
investigated. In this study, we developed a tuning-fork-striking device and evaluated effects of temperatureon the effective durations of the fundamental and harmonic tones of several tuning forks made of different
metallic materials. As results, the effective durations of fundamental tone varied with change of thetemperature and the variations tended to depend on materials, although the magnitudes of the variation weredifferent among individual tuning forks.
Keywords: Tuning fork, Vibration, Effective duration, Fundamental, Harmonic

1. INTRODUCTION
The tuning fork is a U-shaped metal instrument that emits sounds of a specific frequency whenstruck. It has been used for a variety of purposes; music, medicine, healing, and others. Each fork is
made manually by artisans, and the frequency is adjusted by polishing. Therefore, the production
process of tuning forks are still dependent on individual skills of the artisan.
Sounds produced by a tuning fork is divided into a fundamental tone (sound at the resonant
frequency) and harmonics. Immediately after a strike of a tuning fork, a harmonic called "clang" tone
is produced in abundance. The fundamental vibration mode, in which the tines move symmetrically,
produces the fundamental tone, and the second mode produces the clang tone that has approximately
six times higher frequency of the fundamental tone (1). Considering the purpose of using a tuning
fork, it is desirable that the fundamental tone persists as long as possible and the clang tone decays as
quickly as possible.
In the manufacturing process of the tuning forks, the fundamental frequency is tuned by hardening
and polishing, however, the vibration duration is not evaluated. Further, while there have been somestudies on relationships between the temperature and the frequency (2,3) and the radiated sound field(4) of tuning forks, there have been few studies on the vibration duration. Based on the above, wehave been conducting research on the vibration duration of tuning forks and evaluated individual
differences and the effects of holding strength (5).
Tuning forks are made of various metal materials such as aluminum, carbon steel, and stainlesssteel. The frequency of tuning forks depends on the properties of the metallic materials, such asYoung's modulus and density, and varies with temperature (6). Therefore, it is highly likely that thevariation of the vibration duration with temperature change will also vary depending on the material.
In this report, we struck several tuning forks made of different metallic materials at different
temperatures and compared the variation of the vibration duration with temperature change.
All experiments were approved by the Institutional Review Board of the Life Science Research of
Chiba University. Necessary information about the experiments was provided to the participants andinformed consent was obtained from each participant before the experiments.

2. EXPERIMENTS
2.1 Tuning ForkIn this experiment, we used four aluminum tuning forks with a resonance frequency of 440 Hz(Niti-On), four carbon steel tuning forks with a resonance frequency of 440 Hz (Niti-On), and four
stainless steel tuning forks with a resonance frequency of 528 Hz (Niti-On 04-040). All forks areshown in Figure 1, and the length and mass of each fork are shown in Table 1.
2.2 Basic Procedure
First, the tuning forks were heated to 70 °C in a thermostatic bath. After a sufficient heating time,
the tuning forks were placed in a tuning-fork-striking device in an anechoic room. Tuning forks placedat room temperature gradually decreased in temperature. A thermocouple temperature sensor (A&D
AD-1214) was attached to the tuning fork holder of the tuning-fork-striking device to measure thetemperature. We struck the tuning fork at 2 °C intervals as the temperature decreased. Aluminumtuning forks were measured at 11 temperature levels ranging from 26 to 46 °C, carbon steel tuning
forks at 12 levels ranging from 28 to 50 °C, and stainless steel tuning forks at 8 levels ranging from26 to 40 °C. The above procedure was repeated four times for each tuning fork.
2.3 Tuning Fork Striking
The tuning fork was struck using a newly-devised tuning-fork-striking device, as shown in Figure
2. The tuning-fork-striking device consists of a tuning fork holder and a mallet driving unit with aspring. The tuning fork holder was made by using a 3D printer to match the shape to the stem of eachtuning fork. The mallet strike position was 1/3 to 1/4 of the tuning fork tine length from the free endof the tine, which is considered a desirable strike position because it corresponds to the node of thesecond mode and thus reduces the clang tone (7,8). In this experiment, the mallet drive intensity was
kept constant for all measurements.

MalletSpringTuningfork
holderFigure 1 - Tuning forks used in the experiment.

Figure 2 – Tuning-fork-striking device.

Table 1 - Tuning forks parameters.
Material

Aluminum

Carbon steel

Stainless steel

Number

1

2

3

4

1

2

3

4

1

2

3

4

Length [cm]

18.2

18.3

18.3

18.3

16.8

16.7

16.8

16.8

13.3

13.4

13.4

13.2

Mass [g]

64

65

63

64

143

141

143

143

55

55

54

52

2.4 Sound Recording
The sound produced by the tuning fork was recorded for 30 s each in an anechoic room using a
1/4-inch free-field microphone (Brüel & Kjær 4939-A-011) and a conditioning amplifier (Brüel &Kjær 2690-0S4). The microphone was placed at the same height as the fork. The distance between the
microphone and the fork was kept at 10 cm. The tuning fork and microphone were placed at a sufficientdistance from the inner walls of the anechoic room.
2.5 Vibration Evaluation
In sound analysis, we focused only on the fundamental and clang tones of the tuning fork. Fast
Fourier Transformation (FFT) and Short-time Fourier Transformation (STFT) were applied to theobtained acoustic signal data to estimate the frequencies of the fundamental and clang tones. Next,
we applied a band-pass filter to the time waveform to extract each tone. We extracted the amplitude
maximums of both tones from the absolute values of the peaks observed immediately after the tuning
fork was struck, and we calculated the time required for the amplitude to reach 1/10 of the maximum
amplitude (we defined this as "vibration duration").
Figure 3 shows an example of vibration evaluation using the above method. The acoustic signal
data was recorded as shown in Figure 3(a), the Short-time Fourier Transformation (STFT) was applied
as shown in Figure 3(b), and the tone was extracted by band-pass filtering as shown in Figure 3(c)based on the frequency analysis results. Figure 3(d) shows the calculation of the vibration duration
based on the absolute values of the peaks of the tone.

3. RESULTS

Amplitude [a.u.]Frequency [Hz]3.1 Frequency
The frequencies of the fundamental and clang tones decreased with increasing temperature fortuning forks of all materials.
For the aluminum and carbon steel tuning forks, the effect of temperature was almost the sameamong tuning forks of the same type. For the aluminum tuning fork, the fundamental frequency
decreased by approximately 2.8 Hz, and the clang frequency decreased by approximately 18 Hz for atemperature increase of 20 °C. For the carbon steel tuning fork, the fundamental frequency decreased

Time [s]Time [s](a) Recorded sound time waveform.

(b) Example of STFT result.

Amplitude [a.u.]

Amplitude [a.u.]

Maximum amplitude
1/10

Vibration duration

Time [s]Time [s](c) Extracted fundamental tone.

(d) Calculation of vibration duration.

Figure 3 – Vibration evaluation.

by approximately 1.6 Hz, and the clang frequency decreased by approximately 11 Hz for a temperature
increase of 22 °C.
On the other hand, the stainless steel tuning forks showed more significant individual variability,with fundamental and clang frequencies decreasing by 1.2 to 2.2 Hz and 8 to 13 Hz, respectively, witha temperature increase of 14 °C. Considering the temperature range, the carbon steel tuning forkclearly showed a smaller decrease in frequency with increasing temperature than the tuning forks
made of other metals.
3.2 Vibration Duration of the Fundamental Tone
Figure 4 shows the relationship between the temperature and the vibration duration of thefundamental tone of each tuning fork.
For aluminum and stainless steel tuning forks, the vibration duration of the fundamental tone
decreased with increasing temperature, except for the stainless steel TF4, which recorded an extremely
short fundamental tone. There were individual differences in the magnitude of the effect of increasing
temperature on both materials.
On the other hand, for the carbon steel tuning fork, the vibration duration gradually increased withincreasing temperature, with the longest vibration durations recorded at around 44°C for all forks.
There was a tendency for vibration duration to shorten at higher temperatures. However, the vibration
duration at 50°C, the highest temperature measured in this experiment, exceeded the vibration
duration at the lowest temperature for all of the tuning forks.
3.3 Vibration Duration of the Clang Tone
Figure 5 shows the relationship between the temperature and the vibration duration of the clang
tone of each tuning fork.
The clang tone of the aluminum tuning fork tended to prolong as the temperature increased. In
contrast, the carbon steel and stainless steel tuning forks did not show any common variation due tothe temperature effect.
TF1

TF2

TF3

TF4

18
16
14
12

12

Vibration duration [s]

20

10

TF1

TF2

TF3

TF4

8
6
4
2
0

10
26 28 30 32 34 36 38 40 42 44 46
Temperature [℃]

28 30 32 34 36 38 40 42 44 46 48 50
Temperature [℃](a) Aluminum tuning fork.

(b) Carbon steel tuning fork.

12

Vibration duration [s]Vibration duration [s]

22

10

TF1

TF2

TF3

TF4

8
6
4
2
0
26

28

30 32 34 36 38
Temperature [℃]40(c) Stainless steel tuning fork.
Figure 4 – Vibration duration of the fundamental tone.

4

TF1

TF2

TF3

TF4

3
2
1

4

Vibration duration [s]Vibration duration [s]

5

3

TF1

TF2

TF3

TF4

2
1
0

26 28 30 32 34 36 38 40 42 44 46
Temperature [℃]

28 30 32 34 36 38 40 42 44 46 48 50
Temperature [℃](a) Aluminum tuning fork.

(b) Carbon steel tuning fork.

Vibration duration [s]

4
3

TF1

TF2

TF3

TF4

2
1
0
26

28

30 32 34 36 38
Temperature [℃]40(c) Stainless steel tuning fork.
Figure 5 – Vibration duration of the clang tone.

4. DISCUSSION
Tuning forks of all materials showed a change in the vibration duration of the fundamental tone as
the temperature increased. However, the variation in vibration duration varied with the tuning forkmaterial. The fundamental tone of the aluminum and stainless steel tuning forks tended to shorten as
the temperature increased, and the effect was greater for the aluminum fork. On the other hand, thecarbon steel tuning fork tended to prolong the fundamental tone as the temperature increased, up toaround 44°C, a result different from that observed for other metals. Material proper ties such asYoung's modulus and coefficient of thermal expansion are considered to cause these differences in theeffect of temperature. Also, the significant difference in the size of the tuning forks used in theexperiments (Table 1) may have influenced the results.
The effect of temperature on the frequency and duration of the fundamental vibration showed thatthe carbon steel tuning fork exhibited minor variations with the temperature of any of the tuning forks.
In addition, there was a tendency for the fundamental tone to prolong with an increase in temperature.
These results suggest that carbon steel tuning forks are relatively more suitable for use at hightemperatures than other materials, although a 1°C increase results in a decrease in the fundame ntal
frequency of approximately 0.07 Hz. On the other hand, aluminum tuning forks, which showed asignificant decrease in frequency and shortening of the fundamental tone with increasing temperature,
are not considered suitable for use at high temperatures in terms of both frequency and vibration
duration.
It should be noted that polishing, which is an integral part of the manufacturing process for carbon
steel and stainless steel tuning forks, generates a great deal of heat. This result suggests that evalu ating
the vibration duration during the manufacturing process is not easy, which involves repeated polishingand tuning. In particular, the stainless steel tuning forks recorded almost identical vibration durations
at high temperatures. However, at room temperature, differences were observed (Figure 4). This means

that the tuning of tuning forks at high temperatures after polishing does not necessarily result in
optimal operation at room temperature, and it is necessary to take countermeasures.

5. CONCLUSIONSIn this research, we compared the effect of temperature on the vibration duration of tuning forks
made of aluminum, carbon steel, and stainless steel. As a result, the vibration duration of thefundamental tone of the tuning folks made of aluminum and stainless steel tended to shorten as thetemperature increased. On the other hand, the vibration duration of the tuning folk made of carbon
steel gradually increased with increasing temperature, with the longest vibration duration recorded ataround 44°C. The differences in material properties such as Young's modulus and coefficient ofthermal expansion are considered to be the cause of these differences in the effect of temperature
among the metallic materials. This temperature effect may be an issue in the manufacturing process
to optimize the operation at room temperature.
Based on this relationship with temperature, we are aiming to establish a method for evaluating
the vibration duration in the tuning fork manufacturing process. For this purpose, it is necessary tocomprehensively examine the effects of shape parameters and other environmental conditions, inaddition to holding strength and temperature, which have been investigated so far. By examining eachof these parameters, we hope to contribute to change manufacturing technology of tuning forks, which
relies on the experience of artisans now, to explicit knowledge.

ACKNOWLEDGEMENTS
We are grateful to NITI-ON Co. Ltd. for their cooperation. A part of this work was supported by
JSPS KAKENHI Grant Numbers JP19K22950 and JP20H04497, and a Research Grant for the Next
Generation Research Incubator from the Institute for Global Prominent Research, Chiba University
for SN.

REFERENCES
1. Rossing TD, Russsell DA, Brown DE. On the acoustics of tuning forks. Am J Phys. 1992; 60(7): 620626.
2. Greenslade TB, Jr. The acoustical apparatus of Rudolph Koenig. Phys Teacher, 1992; 3(12): 518-524.
3. Greer ZL. Temperature, frequency, and Young’s Modulus of an aluminum tuning fork. ISB J Phys. 2011;
5(1): 1-4.
4. Russell DA. On the sound field radiated by a tuning fork. Am J Phys. 2000; 68(12): 1139-1145.
5. Nomizu K, Otsuka S, Nakagawa S. Evaluation of individual differences of vibration duration of tuning
forks. J Acoust Soc Am. 2021; 149(4): A69.
6. Miller DC. The Science of Musical Sounds, 2nd ed. New York, USA: Macmillan, 1926. p. 29-33.
7. Yüksel M, Kemaloğlu YK. Acoustic analysis of used tuning forks. J Int Adv Otol. 2017; 13(2): 239-242.
8. Russell DA. The tuning fork: an amazing acoustics apparatus. Acoust Today. 2020; 16(2): 48-55.

ABS-0523

Visualizing mode shape of a drumhead under non-uniform tension for
estimating tuning condition using Fourier transform profilometry
Ryota HASHIMOTO(1) , Kohei YATABE(2) , Yasuhiro OIKAWA(3)(1) Department of Intermedia Art and Science, Waseda University, Japan, hashimo_108@fuji.waseda.jp(2) Department of Electrical Engineering and Computer Science, Tokyo University of Agriculture and Technology, Japan,

yatabe@go.tuat.ac.jp(3) Department of Intermedia Art and Science, Waseda University, Japan, yoikawa@waseda.jp

ABSTRACTIn this paper, vibration modes of a musical drumhead were visualized using Fourier transform profilometry(FTP). Purpose of the visualization is to estimate tuning condition of a drumhead under non-uniform tension.
Since drums are often tuned independently of a musical scale, their tuning heavily relies on each drummer’s taste
and experience. If modes of drumhead can be visualized, it will assist beginners in judging the uniformness oftension, which should enable them to tune drums more easily. To realize such visualization, we applied FTP,
which is an optical method to measure the displacement of a surface, to a vibrating drumhead. Experimental
tuning condition was set adjusting bolts tightness around a drumhead. Then, a video of a vibrating drumhead wasshot with a high-speed camera. Time variation of the displacement and a mode shape were obtained applying
FTP to each frame of the video. Comparing the visualized mode shape and tuning condition, correspondence
between them was shown.

Keywords: Drumhead, Fourier transform profilometry, Tuning, Vibration mode, Visualization

1

INTRODUCTION

In typical music genres such as rock and jazz, drums are necessary instruments to control the playing speed andgive thickness to sounds. For this reason, drums are used very frequently in most popular songs. However, sincenon-professional drummers often do not have their own drums, they use drums installed in a music rehearsal
studio for practice. The use of the same drums by some drummers may cause their tuning to shift, or they may
change the tuning. Due to this situation, the tuning of the drums varies each time, so it is necessary to tune
them many times. A drummer tunes a drum adjusting the tightness of tension bolts around a drumhead. Since
they are often tuned independently of a musical scale, their tuning depends largely on the taste and experienceof each drummer. Therefore, it is not easy to tune it quickly. Furthermore, beginners do not know the rightsounds for a music, so it is more difficult for beginners. To make the tuning easier and quicker for them andexperienced drummers alike, estimating the tuning condition of a drumhead is needed. If it is achieved, themethodology can be applied to practical tuning equipment.
Non-uniform drumhead vibration has been analyzed by several numerical methods. For example, Mei calculated nodal patterns of a circular membrane under various tension conditions using the finite element method [1].
As another example, Sathej and Adhikari proposed a numerical method based on Fourier-Chebyshev collocation for the popular Indian drums tabla, which is eccentrically loaded [2]. In addition, a drumhead under
non-uniform tension was modeled and analyzed by Samejima and Fukuda using a Fourier-Chebyshev spectral
collocation technique, which is one of the spectral method [3]. They showed that under non-uniform tension,
the degeneracy could be lifted and thus each mode split into two frequencies. They also mentioned that whether
only one or both of the two degenerate modes are excited depends on the position of the striking.
For a real drum, some optical methods have been proposed to visualize the vibration. Worland measured

Figure 1. First 12 modes of an ideal membrane.

(a) The reference modes.

(b) The degenerate modes.

Figure 2. Examples of degenerate mode pair.
the effect of tuning conditions on drumhead vibration by electronic speckle pattern interferometry (ESPI), whichuses laser interference [4]. It is shown that not only degeneracy can be lifted by the non-uniformities, but alsothat the angle of the nodal line changes or curves depending on bolts tightness. However, the laser method
require delicate equipment such as beam splitters and several mirrors. Although a single sensor head composedof the necessary equipment is sufficient for ESPI, it is hardly simple enough for actual drum tuning in any
case. Another example is the use of Fourier transform profilometry (FTP), a technique that uses only a camera
and a video projector. By applying it to each picture frame, the 3D shape of the drumhead is reconstructed
and a vibration animation is generated [5]. However, the measurement does not take into account non-uniform
tension.
In this paper, we apply FTP to a non-uniform tension drumhead to visualize the effect of its tuning conditions on vibration. Experimental tuning conditions are set adjusting the tension bolts of a snare drum. The
amplitude distributions for each conditions are obtained and their shapes are compared. Paying attention to theshape change of the (1, 1) vibration mode, we clarify the corresponding relationship with the tuning condition.
The results shows regularity in the angle and bending of nodal lines, consistent with the previous experiments.
Visualizing the relationship by FTP, which is easier to set up than the laser method, will improve the convenience of tuning estimation and its applicability to tuning tools.

2

FUNDAMENTAL CONCEPTS

2.1 Ideal circular membrane
An ideal membrane is assumed to have uniform areal density and tension, and there is no air loading or
stiffness. The general solution of the wave equation for such an ideal circular membrane is given by cos (mθ ) zm,n (r, θ ,t) = Ĉm,n Jm (km,n r)e jωm,n t ,(1)
 sin (mθ ) where Ĉm,n is displacement amplitude, Jm (·) is the Bessel functions of the first kind of order m, the θ is anangle in the polar coordinates with an antinode position as zero, and the km,n is a constant that makes Jm (km,n R)
equal to the nth zero at the circumference (r = R), giving the angular frequency ωm,n [6,7]. These form a (m, n)Figure 3. Schematic diagram of FTP.
mode which has m nodal diameters and n nodal circles including the circumference.
Figure 1 shows an example of (m, n) mode of an ideal circular membrane. When the number of nodal
diameter is greater than zero (m > 0), Eq. (1) has two solutions for one m because it contains both cos (mθ )and sin (mθ ). The pairs of modes corresponding to these solutions have the same frequency and are doubly
degenerate. Their nodes and antinodes positions are interchanged (rotated by an angle π/2m) as shown in
Fig. 2. The mode degeneracy can be lifted by perturbations such as non-uniform tension and areal density andbecome separately observed. The spectral method and ESPI visualizes such degenerate modes and shows thatthey change the angle and bending of their nodes.
2.2 Fourier transform profilometry
FTP is one of the optical methods to acquire 3-D object shapes. A sinusoidal grating of a certain frequency is
projected on an object surface and photographed by a camera from directly above. The surface displacement isobtained by analyzing the distortion of the stripes caused by the unevenness of the surface as phase modulation [8]. Using a high-speed camera and processing one frame at a time, the system can be applied to dynamic
objects [9–12]. It can be conducted using only a camera and a video projector, which eliminates the need for
detailed and delicate equipment arrangement unlike the laser methods.
Figure 3 shows a schematic diagram of FTP. To begin, a sinusoidal grating is projected onto the drumhead.
Let f0 be the spatial frequency of the grating image in the reference plane, and given that the light source ofthe grating is at infinity E∞ . The observed luminance value is expressed as∞gT (x, y) =∑ An r0 (x, y)e2πin f0 x ,

(2)

n=−∞

where An is a Fourier coefficient and r0 (x, y) is a non-uniform distribution of reflectivity on the reference plane.
However, the actual light source is at Pr, not at infinity E∞ . Since there is a phase modulation distribution
φ0 (x, y) = 2π f0 BC(3)

due to the projection angle in practice, the fixed reference plane is as follows:∞

g0 (x, y) =∑ An r0 (x, y)ei{2πn f0 x+nφ0 (x,y)} ,

n=−∞

(4)

Figure 4. Experimental setup for FTP.
which represents a non-vibrating drumhead image.
When the drumhead is vibrating, an image observed at time t is∞g(x, y,t) =∑ An r(x, y,t)ei{2πn f0 x+nφ (x,y,t)} ,

(5)

n=−∞

where r(x, y,t) is the reflectivity distribution due to the height distribution of the drumhead, and the φ (x, y,t) is
the phase modulation due to both the height and the projection angle at time t,φ (x, y,t) = 2π f0 BD.

(6)

Computing the space-domain 2-D Fourier transform of g0 (x, y) and g(x, y,t), only the n = 1 components corresponding to the fringe frequency f0 are extracted. Applying the inverse Fourier transform to the extracted
components, we obtainĝ0 (x, y) = A1 r0 (x, y)ei{2π f0 x+φ0 (x,y)} ,i{2π f0 x+φ (x,y,t)}ĝ(x, y,t) = A1 r(x, y,t)e

(7)
.

(8)

Then, calculating the product of ĝ and the complex conjugate of ĝ0 yieldsĝ(x, y,t)ĝ∗0 (x, y) = |A1 |2 r(x, y,t)r0 (x, y)ei∆φ (x,y,t) ,∆φ (x, y,t) = φ (x, y,t) − φ0 (x, y),

(9)
(10)

and the phase difference at each frame is given by∆φ (x, y,t) = atan2 {Im [ĝ(x, y,t)ĝ∗0 (x, y)] , Re [ĝ(x, y,t)ĝ∗0 (x, y)]} ,(11)

where the function atan2(b, a) is the unique phase of a complex number a + bi. The Im[·] and Re[·] represent
the imaginary part and real part, respectively. The phase is often wrapped in the range of [−π, π], therefore,
3-D phase unwrapping is necessary for practical purposes. In this research, the branch cut method which is oneof 2-D phase unwrapping algorithm was applied to the final frame. After that, 1-D unwrapping was performed
toward the front in the time direction, resulting in phase unwrapping of all frames [13].

Table 1. Experimental equipment and conditions.
High-speed camera

MEMRECAM HX-3 (nac)Frame rate

2000 fps

Projector

VPL-PHZ10 (SONY)

Drum tuner

DATK Torque Key (EVANS)

L01145 mmd

755 mmf00.1408 line/mm(a) A frame where the drumstick was photographed.

(b) A vibrating drumhead frame.

Figure 5. Example images of a snare drum taken.
The height distribution is obtained from the phase difference ∆φ (x, y,t). From Eqs. (3) and (6), Eq. (10)gives the following:∆φ (x, y,t) = φ (x, y,t) − φ0 (x, y)= 2π f0 BD − BC
= 2π f0CD.

(12)

Furthermore, similarity relation between △PrHCa and △CHD givesd : CD = {L0 − Z(x, y,t)} : {−Z(x, y,t)},−dZ(x, y,t)CD =.
L0 − Z(x, y,t)

(13)
(14)

Substituting Eq. (14) into Eq. (12), we obtain time variation of the height distribution,
Z(x, y,t) =L0 ∆φ (x, y,t).
∆φ (x, y,t) − 2π f0 d

(15)

3

EXPERIMENT

3.1 Equipment and tuning condition
The measurement object is the only batter side of a snare drum with six tuning bolts around it. The batter sideis the side that is struck when playing the drums. The system was set up as shown in Fig. 4, and Table 1
shows the conditions for FTP. The projected sinusoidal grating consists of sine waves with 100 gradations fromthe minimum to maximum. Experimental tuning conditions were set adjusting bolts tightness using a type ofdrum tuners, “Torque Key." It idles when a certain amount of torque is applied, allowing bolts to be tightened
evenly. Besides, the user can decide the tightening strength with a number from 0 to 20 written on the tuner.
We assigned numbers 10 and 20 to loosen and tighten bolts respectively, and set several tuning conditions
depending on their position. All bolts around the drumhead on the back side were set to number 10.
At the start of the measurement, the center of the drumhead was struck with a wooden drumstick, and at
the same time, a high-speed camera began taking video. Three measurements were taken per tuning condition
to examine FTP stability. To avoid using frames where the drumstick was shot, we selected 1000 frames fromthe 100th to the 1100th frame after the striking. The vibrating drum images of 1920 × 2560 pixels was thenobtained. Moreover, they were reduced to 1800 × 1800 pixels by removing pixels outside the plane. Figure 5
shows the examples of the taken images.

(a) Schematic diagrams of tuning condition. The white and black points are the loosened and tightened bolts respectively.

(b) Linear (1, 1) mode shapes.

(c) Curved (1, 1) mode shapes.

Figure 6. Variation of (1, 1) mode shapes with tuning conditions.

Figure 7. (1, 1) mode shapes obtained in three trials for one condition.

3.2 The (1, 1) mode
The (1, 1) mode shape can be obtained from the calculated time variation of the height. For each pixel (x, y) of
the height distribution Z(x, y,t), the amplitude spectrum was obtained by Fourier transform in the time domain.
The frequency whose amplitude is the second largest was obtained in each pixel because the first may include
the (0, 1) mode, which is not the objective. We counted the number of pixels where the same frequency is
obtained and listed ten frequencies with large pixel count. Then, we saved ten amplitude distribution images
corresponding to them. Finally, the (1, 1) mode shapes were obtained in the saved images.
3.3 Comparing mode shapes
Figure 6 shows the (1, 1) mode shapes obtained for each tuning condition. The white and black points in
Fig. 6(a) refer to the positions of the loosened and tightened bolts, respectively. The amplitude ranges from 0
mm to 1.1 times the maximum of each distribution.
Figure 6(b) shows amplitude distributions of linear nodal lines. The line rotated as the number of tightenedbolts increases. Also, the nodal lines tended to appear as if they connect black and white points located at
opposite sides of the circumference. This may be due to the geometric symmetry of the tension distribution.
Figure 6(c) shows the other (1, 1) mode, which was degenerate. These became able to be observed separately
because the degeneracy was lifted by the non-uniform tension. Arc-shaped nodal lines appeared around regionsof low tension. This result is consistent with the calculation by the spectral method and the ESPI observation.
In addition, the curve of the nodes was more gentle in the condition with more black or white points. Thismeans that as the tension becomes more uniform, the arc node approaches a linear shape such as the (1, 1)
mode in Fig. 1.
Figure 7 shows the mode shapes obtained from three measurements for one condition. Although the struck
spots were not exactly the same throughout the three trials, all the mode shapes looked alike. Thus, the changes
in mode shape in Fig. 6 can be directly attributable to the tuning conditions.

4

CONCLUSIONSIn this paper, the (1, 1) mode of a non-uniform tension drumhead is visualized with only a camera and a
projector by FTP. The stability of the method was demonstrated by making three measurements per tuning
condition. Moreover, this allowed us to compare the mode shapes under each tuning condition. Particularly,
two regular features were visualized: linear nodes follow geometric symmetry and arc nodes enclose regions of
low tension. This indicates that the FTP can provide information that may lead to tuning estimation. In the
future, we will investigate specific methods for estimating unknown tuning states from mode shapes.

REFERENCES[1] C. Mei, “Free Vibrations of Circular Membranes under Arbitary Tension by the Finite-Element Method," J.
Acoust. Soc. Am., vol. 46, no. 3, pp. 693–700, Feb. 1969.
[2] G. Sathej and R. Adhikari, “The eigenspectra of Indian musical drums," J. Acoust. Soc. Am., vol. 125, no.
2, pp. 831–838, Feb. 2009.
[3] T. Samejima, and R. Fukuda, “Vibration analysis of a musical drum head under nonuniform density andtension using a spectral method," Acoust. Sci. & Tech., vol. 37, no. 6, pp. 295–302, 2016.
[4] R. Worland, “Normal modes of a musical drumhead under non-uniform tension," J. Acoust. Soc. Am., vol.
127, no. 1, pp. 525–533, Jan. 2010.
[5] Q. Zhang and X. Su, “High-speed optical measurement for the drumhead vibration," Opt. Express, vol. 13,
no. 8, pp. 3110–3116, May. 2005.
[6] N.H. Fletcher and T.D. Rossing, “Circular Membranes," in The Physics of Musical Instruments, 2nd ed. New
York, NY, USA: Springer Sci. Bus. Media, Inc., 1998, ch. 3, sec. 3, pp. 73–75.

[7] S.L. Garrett, “Circular Membranes," in Understanding Acoustics: An Experimentalist’s View of Sound andVibration, 2nd ed. Berlin, Germany: Springer Nat., 2020, ch. 6, sec. 2, pp. 294–309.
[8] J.C.P. Ortega1, J.W.R. Moreno, L.B. Rodriguez, E.G. Hurtado, T.S. Jimenez, J.M.R. Arreguin, and A. Rivas,“Image Processing for 3D Reconstruction Using a Modified Fourier Transform Profilometry Method," inProc. 6th MICAI, Aguascalientes, Mexico, Oct. 2007, pp.705–712.
[9] X. Su, and W. Chen, “Fourier transform profilometry: a review," Opt. Lasers Eng., vol. 35, no. 5, pp.
263–284, May. 2001.
[10] M. Takeda, and K. Mutoh, “Fourier Transform Profilometry for the auto measurement of 3-D object
shapes," Appl. Opt., vol. 22, no. 24, pp. 3977–3982, Dec. 1983.
[11] S. Fenga, C. Zuoa, L. Zhanga, T. Taoa, Y. Hua, W. Yina, J. Qiana, and Q. Chen, “Calibration of fringe
projection profilometry: A comparative review," Opt. Lasers Eng., vol. 143, Aug. 2021.
[12] H. Zhang, Q. Zhang, Y. Li and Y. Liu, “High Speed 3D Shape Measurement with Temporal Fourier
Transform Profilometry," Appl. Sci., vol. 9, no. 19, Oct. 2019.
[13] R.M. Goldstein, H.A. Zebker, and C.L. Werner, “Satellite radar interferometry: Two-dimensional phaseunwrapping," Radio Sci., vol. 23, no. 4, pp. 713–720, July. 1988.

ABS-0552

Difference in timbre of violin in a seat and on
stage
Yuya IHIGAKI1, Masao YOKOYAMA1
1

Meisei Univercity, Japan

ABSTRACT
Generally, it is speculated that the tone of Stradivarius is superior to those of other violins, and it sounds farther.
However, many points regarding the difference between the acoustic features of Stradivarius and that of theothers are still unclear. In this study, we recorded the sounds from six violins from old violin to brand new violinon a stage and a seat. We then compared the frequency spectrum and the signal-to-noise ratio (S/N ratio) of
those sound sources. Based on the results, the difference in the power spectrum was observed at approximately
3–5 kHz. And the difference of S/N ratio was also appeared in some frequency bands. In the higher harmonic
overtone around 16 kHz, we found that there are violins which have the peaks of harmonic overtone and ones
which do not have. We investigated whether the difference in the shapes of these frequency spectrums and S/Nratios were related to the tone color of violin and the so-called distant peal phenomenon.

Keywords: Violin, timbre, Frequency spectrum, S/N ratio

1.

INTRODUCTION

Studies on the sound of antique violins such as Stradivarius have been conducted extensively.1-5 We analyzed thetone color with focusing on a phenomenon called the “distant peal” in this study. “Distant peal” is a phenomenon thataudience can hear the sound well and clearly even if the sound volume of performance is not so loud. In contrast, aphenomenon called “side rumble” which the sound is not sufficiently audible even when violinist performs violin
loudly.6 Generally, the sound by distant peal is considered good so that players keep in mind in playing andmaintaining the distant peal. In addition, it is said that there are some violins which sound in distant peal and somewhich sound in side rumble. Therefore, in this study, we compared the power spectrum and S/N ratio on a stage andseat using six violins and examined the existence of the distant peal of the violins.

2.

EXPERIMENTThe recording was conducted in a concert hall (Toppan hall, Tokyo) without an audience. The player was aprofessional violinist. The six violins used in this study were Stradivarius (1705, violin S), Del Gesu (1742, violin D),
the beginner’s violin (violin P) made in Japan, Ferdinand (2011, violin F) made in Germany, and two other brandnew violins manufactured by two Japanese violin makers (violin H and N). The microphones used for measurements
were two nondirectional microphones (Earthworks M23). As shown in figure 1, we placed the microphone on thestage (over a 20 cm from the bridge of violin) and the seat (15 m from the stage). We instructed the player to play G
major scale (three octaves) with each musical instrument. We performed spectrum analysis after recording the sounds
and compared the power spectrum and cross-spectrum for each violin. In addition, we calculated the S/N ratio in the
1/1 octave band around A4 (band 1), at approximately 3-5 kHz (band 2), and 10-12 kHz (band 3).

S⁄N ratio =Average of peaks
Average of noises(1)Figure 1. Snapshot of Experiment

3.

RESULTS

3.1 Comparison of the power spectrums
We present the results for the power spectrum (time average) in the A4 sound of each violin in Figures 2-7.

Figure 2. Spectrum in the A4 sound of violin S

Figure 3. Spectrum in the A4 sound of violin D

Figure 4. Spectrum in the A4 sound of violin P

Figure 5. Spectrum in the A4 sound of violin FFigure 6. Spectrum in the A4 sound of violin N

Figure 7. Spectrum in the A4 sound of violin H

We focus on the difference between the power spectrum of the stage and seat. First, for the spectrum on the stage,some strong peaks appeared from a fundamental frequency (f0) to 60 kHz. Second, while the peaks of violins P and
F decreased gently over f0, the peaks of violins D, N, and H are significant at approximately 3 kHz. For the spectrum
recorded on the seat, some peaks exceeding 40 dB were observed for violin S in the 3-5 kHz compared with theother violins. In addition, the peaks are not seen at approximately 16 kHz for violin N.
Next, we present the results of the cross-spectrum in the A4 sound of each violin in Figures 8-13.

Figure 8. Cross spectrum in the A4 sound of violin S

Figure 9. Cross spectrum in the A4 sound of violin D

Figure 10. Cross spectrum in the A4 sound of violin P

Figure 11. Cross spectrum in the A4 sound of violin FFigure 12. Cross spectrum in the A4 sound of violin N

Figure 13. Cross spectrum in the A4 sound of H

First, the peak of the fundamental frequency of violin S was not significant compared to those of the other violins.
Second, some peaks exceeding 2.5 kdB appeared in violin S from the fundamental frequency to 3 kHz. Third, gentle
disruptions of the peaks appeared from 10 to 12 kHz for violins S and D. According to these results, the difference ofpower spectrum is observed in the A4 sound from a fundamental frequency to 6 kHz. It is seemed that thisphenomenon is similar to the singers’ formant which we often observe singers voice in opera. It is reported that thesound which the opera singer sings with emphasizing a specific frequency (2.5 kHz – 3.5 Hz) are not buried among
the orchestra sounds. In addition, regarding the higher harmonic overtone over 16 kHz, the difference in the peak of
harmonic overtone was observed among 6 violins, and it seems to be a characteristic of violin concerning the easeof distant peal.
3.2 Comparison of the S/N ratios
We show the difference in the S/N ratios of the 6 violins in the A4 sound in Tables 1 and 2.
Table 1. The S/N ratios on stage of each violin in A4
Band 1

Band 2

Band 3

Instrument S

1.55

1.56

1.36

Instrument D

1.62

1.36

1.27

Instrument P

1.66

1.57

1.35

Instrument F

1.53

1.35

1.23

Instrument N

1.68

1.57

1.37

Instrument H

1.71

1.60

1.42

Table 2. The S/N ratios on seat of each violin in A4
Band 1

Band 3

Band 3

Instrument S

1.76

1.75

1.42

Instrument D

1.65

1.47

1.37

Instrument P

1.70

1.58

1.35

Instrument F

1.62

1.40

1.26

Instrument N

1.85

1.40

1.26

Instrument H

1.91

1.73

1.33

In band 1, the values of the S/N ratio on seat increased for all instruments comparing with that of on stage. In particular,
the values significantly increased in violins S, N, and H. In band 2, the values of the S/N ratio increased for all violins
except instrument N; the increase for violins S, D, and H are significant. In band 3, the S/N ratio of violins S, D, andF increased, and the increase for violin D was significantly high. Based on these results, the S/N ratio of violins S, D,
and F increased in all bands. Especially, that of violins S and D increased significantly. Therefore, it may be thereason that audience at a seat hear the sound well and clearly.

4.

CONCLUSIONSIn this study, we observed the differences in the frequency spectrum and the S/N ratio on a stage and a seat andanalyzed the tone color of violins. We analyzed the property of each violin by comparing the peaks and S/N ratios insome frequency bands. In particular, the phenomenon similar to the singers’ formant was appeared in 3-5 kHz.
This might be the acoustic explanation to the distant peal of violin. In a future study, we will compare the sound of
violins in terms of various pitch and analyze other acoustic characteristics.

REFERENCES
1. Buen A. A brief introduction into the violin acoustics history. Proceedings of the Baltic-Nordic Acoustics Meeting.
2006:8-10.
2. Woodhouse J.The acoustics of the violin: a review. Rep Prog Phys. Vol 77, No.11, 2014.
3. Gough CE. Violin acoustics, Acoustic Today. Vol 12, No.2, p.22-30, 2016.
4. Obata T. Why is Stradivarius expensive? The history of violins and the prices. IPSJ Technical Report. vol111, No.12,
2016.
5. Yokoyama M. Possibility of distinction of violin timbre by spectral envelope. Applied Acoustics. Vol 157, 2020.
6. Kobayashi M, Ueno K. Hearing impression of the flute performance sound in the hall -Inspection about distant pealand side rumble as the individuality of the musical instrument-. ASJ. Vol 73, No.4, p.212-220, 2017.

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022 in Gyeongju, Korea

ABS-0762

Personal learning portfolio for music university students based on
acoustic analysis of performance soundMasanobu MIURA1
1

Kunitachi College of Music, JAPANABSTRACT
University students of music performance are usually under severe training, but they cannot receive objective
judgment since the evaluation factors are not clarified. The author proposed a personal learning portfolio forthe students based on acoustic analysis of performance sound. In this project, a marimba performance is dealt
with to analyze. Several styles of grip for two mallets in a single hand are investigated. Several marimba
players play several stroking patterns on the mallet. Sound spectrogram and several MIR parameters (suchas Spectral Flux and Spectral Centroid) were used to describe their performance features. Subjective
judgements are conducted for the recorded sounds, where marimba players were asked to answer subjective
impressions for each sound. The author designed a panel on display where the sound’s waveform,
spectrogram and MIR parameters are drawn. By observing the panel, the players are possible to understandtheir own performances visually. Students are also possible to compare their playing sounds with otherstudents, including their own playing done several years ago. At the conference, the author will demonstrate
a concrete example of MIR parameters on the students’ personal learning.
Keywords: Musical Performance, Sound Analysis, Learning Portfolio, MIR Parameters

1. INTRODUCTION
The recent development of IT provides many opportunities for university education, such as online
resources for university education and SNS (social network services) styled learning have been
provided where students can access the educational resources and can learn wit hout person-to-person
instructions from teachers. In the case of musical university students, such kind of development has
not been seen so much since many studies of music training are based on personal instructions.
Moreover, from the student’s perspective, an ad-hoc style of learning is widespread in musicalperformance training since measuring the extent of music learning is essentially difficult. In order toimprove the educational environment of musical university students, a personal learning portfol io is
proposed here, where the musical university student’s performance sound will be accumulated andthen compared year by year by him/herself. Here, percussion playing is targeted. The marimba playing
is dealt with. The performance was previously recorded by other literature [1,2]. Here the recorded
sound is employed to develop a methodology of percussion playing’s learning portfolio.

2. ACOUSTIC ANALYSIS
2.1 Recording Experiment [1,2]Two students (P1 and P2) majoring in percussion performance participated in this test. The
recorded playing is from the former study of the author and Mr. Takeshita, a PhD candidate for
percussion playing at Kunicathi College of Music. The instrument used was a Yamaha YM -5000, andmallets were made by Lesta Jay MG-04B and MG-05B with modifications. The performance sound
was recorded using two RODE NT2A microphones placed 13 cm from the marimba parallelly locatedto the change of pitch and a Roland Quad Capture mixer connected to a computer. The sheet music
played was the most standard interval in the performance, shown in Figure 1, which includes thewidths of the major-third interval. The tempo was set to 100 bpm, and the participants were asked to1miura.masanobu@kunitachi.ac.jp

play a pattern in which they played two notes simultaneously with one hand, then alte rnatively stroked
each of two mallets. All audio signals are recorded by 44.1kHz, 24bit sampling.

Figure 1 – Sheet music of recorded performance [1,2]

2.2 Analysis
For the recorded performance sound, MIR (Music Information Retrieval) parameters, such as
Spectral flux Fluxn (1) is calculated using the following equations.
𝐾

𝐹𝑙𝑢𝑥𝑛 = ∑|X 𝑛 (𝑘) − X 𝑛−1 (𝑘)|

(1)

𝑘=1

, where n represents the index of frame on short time Fourier transform, k represents bin index for
windowed waveform, K is the total number of bin index or half of window length, and X represents
power spectrum of audio signal x(t). The Fast Fourier Transform is performed with the length of 4096
samples with hanning window. In general, the spectral flux is said to represent the fluctuation of thesound spectrogram in conjunct two windowed signals. At here, the flux is used to represent timbrechange when the stroke occurs.
Spectral Centroid SC n (2) is calculated as follows
𝑆𝐶𝑛 =

∑𝐾
𝑘=1 log 2 𝑓(𝑘) 𝑋𝑛 (𝑘)∑𝐾
𝑘=1 𝑋𝑛 (𝑘)(2), where the f(k) represents the centre frequency of the k-th bin index. The spectral centroid is themoment centre of gravity on the sound power spectrum, representing the sensation of brightness on
current sound.
2.3 Example of calculated results
Figure 2 shows an example of a recorded performance for the sheet music in Figure 1. The results
show four different figures, where the first and second are acoustic signals from two separate
microphones. The first is derived from the relatively higher pitch region, whereas the second is fromthe relatively lower pitch region. The third one represents the change of Spectral Flux, where the blue
is from the microphone from the higher pitch region and the red curve is from a relatively lower area.
The fourth figure shows the spectral centroid, where the abscissa represents time in music, and theordinate represents the pitch as the order of power of two. Due to the essentials of the spectral centroid,
when no performance sound is on the recorded audio, the spectral centroid provides no practicalinformation about the sound. At this figure, SC reaches 2^11 to 2^13(Hz), which is not reflected in
the real spectrogram. Figure 3 shows the sound spectrogram of Marimaba’s stroke, from which thespectrum centroid will likely be around 2^8 or 2^9.

Figure 2 – Example of result for sheet music in Fig.1, played by P1.

Figure 3 – Sound spectrogram for the first recoded note in Fig.2
2.4 Example of calculated results
Figure 4 shows another example for sheet music in Fig.1. Even if the P1 and P2 use the same
marimba and mallet, the performance is not the same. Concerning the first six strokes, those on P1
are played in a clear timbre, whereas those on P1 give a more robust, impulsive and rugged impression.
The impression is undoubtedly tricky between the two, but the use of only sound spectrogram is notefficient since the difference is difficult to be displayed. Figure 5 shows the sound spectrogram playedby P2 for the first note in Fig.2, which can be compared with Figure 3. As can be seen from Fig.3 andFig.5, no dramatic difference is observed, but the timbre is inconsistent. In practical situations in
training and self-learning, musical university students only have to judge it according to their senses.
Following the red and blue curve on the first note in Figure 2 and Figure 4, that on Figure 2 (, or P1’s
playing) gives no such difference between the blue and red curves. However, Figure 4 (or P2’s playing)provides a substantial difference in time between red and blue. The blue curve is delayed from the red
curve. Since the blue show a higher-pitch microphone, the change of spectrum is different between
P1 and P2. More specifically, the playing in P2, the higher-pitch component, stays longer than the P1.

Figure 4 – Another example of Figure 3, played by P2.

Figure 5 – Sound spectrogram for the first recoded note in Fig.4

3. OUTLOOK OF PORTFOLIO FOR MUSICAL STUDENTS
3.1 Why needed?
Students of musical universities usually only rely on their sense or intuition when evaluating theirplaying. It is inefficient because it is not easy for students to judge their playing. The recorded acoustic
signal is helpful since they can play back it, but it is time-consuming. The proposed personal portfolio
does not always require the student to understand their playing but provides some informative features
or aspects of their performance. Therefore, for music university students, the proposed portfolio will
be effective.
3.2 Possible method to use as portfolio
Following is the possible flow of the portfolio of students. Firstly standard sheet music is set to beplayed several times over four years. They played it with the same instrument and mallet, with thesame microphone and exact place to record. The recorded acoustic signal is represented as a panel
like Figure 2. The recorded acoustic signal and MIR parameters are shown graphically, and byobserving and listening to their playing, students are possible to recognize their playing's detailed

features. The recorded playing with the analyzed result is accumulated while the students belong to
the university, and parametric comparison is conducted to the students' feedback.

4. CONCLUSIONSIn this presentation, a method to evaluate the timbre of a played sound is explained. Moreover, thepossible usage of the result of MIR parameter as the descriptor of musical playing. Finally, the use of
MIR parameters to realize the student’s portfolio that enables students to think about th eir own playing
more is introduced. In the near future, the proposal of this system to my affiliation will be discussed.

ACKNOWLEDGEMENTS
This study is partly supported by Research Fund on Kunicathi College of Music.

REFERENCES
1. Takeshita K., Basic study for the four mallet performance on Marimba using motion capture and acoustic
analysis, Report on Master Course Study in Graduate School of Kunitachi College of Music (2022.2, inJapanese)
2. Takeshita K., and Miura M., " Basic Study on Motion Analysis of Four-Mallet Marimba Performance using
Motion Capture System ", Report of Technical Committee of Musical Acoustics, Acoustic Society ofJapan, MA2021-40, pp.51-56 (2021.10, in Japanese).

ABS-0800

Music signal analysis in the Large Time Frequency Analysis Toolbox
Clara HOLLOMEY∗ , Nicki HOLIGHAUS, Peter BALAZS
Acoustics Research Institute, Academy of Sciences, Austria*clara.hollomey@oeaw.ac.at

ABSTRACTThe Large Time Frequency Analysis Toolbox (LTFAT) is a toolbox for Matlab and GNU Octave that is dedicated to the time-frequency analysis of audio signals. Based on mathematical frame theory, the generalizationof vector bases to redundant sets of vectors, LTFAT provides a variety of time-frequency representations that are
particularly suitable for analyzing music signals. Besides an implementation of the classical short-time Fouriertransform, it comprises invertible auditory filterbanks using the mel-, Bark- and ERB-scale, an invertible implementation of the constant-Q transform, and algorithms for calculating the wavelet transform. Those signal
representations can be readily used for audio applications such as denoising, audio data compression, and theseparation of tonal and transient components, each of which being exemplified with software demonstrationsin the toolbox. LTFATs blockprocessing framework allows for their real-time application and playback, directly from Matlab and Octave. The toolbox can, along with its extensive documentation, be downloaded from
ltfat.org.

Keywords: Audio signal analysis, Filter banks, open source

1

INTRODUCTION

The Large Time-Frequency Analysis Toolbox (LTFAT) has been designed as both an educational and a computational tool for applied harmonic analysis and audio signal processing [19, 14]. Besides the well known
windowed Fourier transform, LTFAT contains implementations of perceptually motivated [12], constant-Q [7],and wavelet transforms [15]. All those routines operate on nonlinear frequency scales, and implementing themsuch that one is still able to reconstruct the time-domain signal from the time-frequency representation can bechallenging, even in oversampled scenarios. Moreover, particularly in applications where the reduction of (audio)
data, or conversely, increased robustness to noise is required, being able to adjust the resulting time-frequency
representation’s redundancy is desirable.
In LTFAT, functionality for achieving time-frequency representations that fulfil those requirements is implemented as filter banks. Embedded in an efficient framework for signal analysis, processing, and synthesis, they
are versatile tools for a wide range of applications in music signal analysis and processing. In the following, wesummarize the mathematical background of LTFAT’s filter bank framework, describe its structure, and introduce
its most useful transforms for music signal analysis.

2

MUSIC SIGNAL ANALYSIS VIA FILTER BANKS

2.1 Mathematical Background
Filter banks split an input signal x[n] into a set of signals x1 [n], x2 [n], . . ., such that each of them corresponds
to a different frequency region of x[n]. Often implemented as an array of modulated lowpass (i.e. bandpass)filters complemented with a downsampling unit for preventing excessive redundancy, they are an establishedtool for working with music signals, with applications comprising their coding [13], compression [17], andprocessing [16].

To ensure that the time domain signal can be recovered, one often imposes strong constraints on the filters
and their spacing in frequency, by either requiring the filters’ frequency response to be zero at integer multiplesof the downsampling rate 1a [21], or, for greater robustness against modifications and noise, to be altogether
band limited consistent with a [18].
Those constraints, however, can impede the analysis of music signals. Music often covers a large frequency
range, and consequentially, comprises frequency components with a variety of wavelengths. Moreover, it is nonstationary in nature, and changes in the high frequencies may occur on much shorter time scales than those in
the low frequency range. Yet, at the same time, a great deal of information, e.g. on the musical instruments
used, their relative intensity, or even the playing style can be conveyed by fine grained changes in the timefrequency properties. Hence for music signals, flexibility in assigning the frequency resolution to time-frequency
representations is of particular interest. However, the design of filter banks that allow for such flexibility requires the relaxation of the above mentioned constraints. To still obtain invertible time-frequency coefficientmatrices, strategies are needed that take variations over the whole time-frequency plane covered by the filterbank into account.
Such strategies are provided by frame theory. A finite set of vectors Φ = {φ1 , φ2 , ..., φk } is called a frame [6]when there exist positive real numbers λmin and λmax such thatK

λmin ∥x∥2 ≤ ∑ |⟨x, φk ⟩|2 ≤ λmax ∥x∥2 , for every x.

(1)

k≥1

Frame theory provides a way to mathematically describe redundant time-frequency representations where thedownsampling in time does not necessarily equal that in frequency, such as filter banks [2]. Frames generalize
the concept of bases in that they may comprise more vectors than are strictly needed to span the vector space,thus providing a higher degree of flexibility for the design of time-frequency representations while simultaneously ensuring their invertibility. Thus, frame theory provides the mathematical foundation for LTFAT’s filterbank framework. In the following, we detail the framework’s structure and outline the functionality that is particularly suitable for analyzing music signals.

2.2 Filter banks in LTFAT
LTFAT provides the filterbank function, implemented in Matlab/GNU Octave with a backend in C for greater
numerical efficiency, for calculating filter bank coefficients according toL−1

cm [n] = ∑ x[l]gm [am n − l],

(2)

l=0

with x the input signal, gm the mth filter, am , its associated downsampling factor, and L the length of the filterbank. filterbank accepts the input signal along with the filter type and downsampling factors as input arguments. The filter bank coefficients, the frequency responses of its filters, and the filter bank’s overall frequency
response can be visualized using the functions plotfilterbank, filterbankfreqz, and filterbankresponse.
Moreover, there is functionality for calculating the instantaneous time and frequency (filterbankphasegrad),conducting phase-based spectrogram reassignment (filterbankreassign), and reconstructing the phase fromthe filter bank’s magnitude coefficients (filterbankconstphase). The frame bounds can be retrieved forcomplex and real signals via filterbankbounds and filterbankrealbounds respectively. Their ratio indicates whether direct inversion using the ifilterbank function, and possibly a dual window generated via
filterbankwin, or iterative inversion via ifilterbankiter is more appropriate for a given time-frequency
coefficient matrix.
Moreover, LTFAT provides functionality for generating filter functions that are specifically matched to therequirements of certain types of time-frequency representations. In their minimal configuration, most filter generators accept the sampling frequency, the desired filter bank length, and the frequency range to be covered

as input arguments and output the appropriately scaled filter functions, their downsampling factors and center
frequencies. Thus, they can be readily used in conjunction with filterbank. There are filter generators fordesigning filter banks in the style of the STFT, the constant-Q transform, the wavelet transform, for perceptually motivated filter banks, and a warped filter bank, the latter accepting nearly arbitrary center frequency tobandwidth ratios. The generic signal flow for signal analysis via filter banks in LTFAT is, along with the most
relevant parameters, depicted in Figure 1.

Figure 1. LTFAT’s signal flow for filter bank processing. The filter generators are named after the type of filter
they yield (i.e. either gabfilters, audfilters, cqtfilters, waveletfilters, warpedfilters) and output the desired filtersand downsampling factors such that they can be readily processed, along with the input signal, via the filterbank
function. Depending on the framebounds of the resulting coefficients, either direct or iterative inversion can beapplied.

2.2.1 Gabor filter banks
The short time Fourier transform (STFT), where the signal of interest is segment-wise expanded in terms of its
Fourier coefficients
N−1

2πn

c[n] = ∑ x[n]e− j N

(3)

n=0

provides a uniform and fixed resolution for each time-frequency point and is probably the most commonlyused time-frequency representation. To reduce redundancy, downsampling in both, the time and the frequency
domain is often applied. However, Fourier theory lacks concise criteria for the reconstruction of such downsampled STFTs. In line with the concept of frame theory, Gabor theory views time-frequency representations
as expansions of the time-domain signal in terms of shifted and modulated windows
gm,n [t] = e−2 jπmbt g[t − na],(4)

where a and b are the downsampling factor in time and frequency, respectively. The resulting coefficients havethe structure
K−1c[m, n] = ∑ x[k]gm,n [k].
k=0

(5)

Thus, Gabor theory provides a unified mathematical framework that encompasses the STFT and provides clear
definitions for downsampling in the time-frequency domain. For this reason, in LTFAT, the filter generator targeting time-frequency analysis on equidistantly spaced time-frequency grids is termed gabfilters.

2.2.2 Perceptually motivated filter banks
The audfilters generator provides a linear representation with resolution evolving across frequency. It matches
the human auditory time-frequency resolution, both in terms of frequency spacing and bandwidth.
Supported scales comprise two variants of the mel scale [20], the critical band rate in bark [22], and twovariants of the "equivalent rectangular bandwidths" (ERB) scale [11], An overview of all auditory scales available in LTFAT is depicted in Figure 2. To ensure the filter bank’s invertibility, low- and highpass filters, centeredaround 0 Hz and the Nyquist frequency, respectively, are part of the filter generator’s output.

Figure 2. The auditory scales available in LTFAT.
2.2.3 Constant-Q transform
The constant-Q transform has been specifically designed with musical instruments in mind [4]. Based on theobservation that the absolute positions of harmonics depend on the fundamental frequency while their relative
distance remains constant on a logarithmic scale, it allows for the concise representation of time-frequency
patterns, largely invariant to their fundamental frequencies. An example of that relative frequency invariance of
instrumental patterns is depicted in Figure 3. A further benefit of logarithmically spaced filter center frequencies
arises when audio data covers several octaves, since, as compared to equidistant frequency spacings, fewer
frequency bins may be required to cover a given frequency range.
In LTFAT, the cqtfilters generator provides an invertible constant-Q transform. Similar as for the audfilters
function, additional low- and highpass filters ensure stable invertibility.
2.2.4 Wavelet transform
Wavelets are an alternative to Fourier coefficients for mapping a time-domain signal to the time-frequency domain. LTFAT provides waveletfilters, a generator for redundant wavelet filter banks as introduced in [8].
Considering wavelet filters of the form
n − d(l + δ j )1(6)
ψl, j [n] = √ ψsjsjwith ψ the mother wavelet, s its dilation, d(l + δ j ) its translation parameter, and with δ j corresponding to asmall delay generated by a low discrepancy sequence (δ0 , δ1 , δ2 , . . .), allows for greater freedom in adjusting the

(a)

(b)

Figure 3. Constant-Q transform of a glockenspiel signal with fundamental frequency 1000 Hz (left) and 500 Hz(right). The overtone structure is similar in both cases, favoring the detection of musical instrument patterns.
frequency resolution than is usually provided by dyadically sampled transforms. At the same time, the resulting
transform fulfills the frame property, allowing for flexible adjustments of its redundany. Lowpass filters ensuring
waveletfilters’s invertibility are added to the generator’s output.
2.2.5 Warped filter banks
The warpedfilters function proposed in [9] allows for the direct provision of a function freqtoscale thatdefines the center frequency to bandwidth ratio of the filterbank. Therefore, it offers the greatest amount of
freedom in filter bank design. The function freqtoscale can be freely chosen, provided that it is an invertible,
increasing real function. However, care needs to be given to the scaling of that function, which needs to takethe desired number of frequency channels into account, and for the function not being too steep locally, as thismay yield filters with very low bandwidths, potentially causing unsatisfactory results.

3

FINAL REMARKS

While LTFAT’s main focus is on invertible representations, undersampled representations are equally easily constructed when desired. Besides most of the transforms presented here that emphasize freedom in choosing thecenter frequency to bandwidth ratio of the filters, LTFAT features a transform that allows the window to changewith time [1]. This non-stationary Gabor transform provides flexibility in the time-domain while still allowing
for an efficient fast Fourier transform implementation, and is available via the nsdgt function call. Moreover,many of LTFAT’s time-frequency transforms can be used in real time settings via LTFAT’s block processing
framework, directly from Matlab and Octave. In addition to offering freedom in the design of time-frequency
representations for music analysis, LTFAT also provides algorithms for their processing. They comprise efficient
time-varying filtering via Gabor multipliers [5], tonal-transient separation via the time-frequency jigsaw algorithm [10] and the lasso shrinkage [3], as well as approaches for audio denoising and compression. Demos andexamples are available directly from within the toolbox and on the homepage ltfat.org.

ACKNOWLEDGEMENTS
This work was partly supported by the Austrian Science Fund (FWF): LOFT (FWF - P 34624, Localized,
Fusion and Tensors of Frames) and NoMASP (FWF P 34922-N, Nonsmooth Nonconvex Optimization Methods
for Acoustic Signal Processing).

REFERENCES[1] P. Balazs, M. Dörfler, F. Jaillet, N. Holighaus, and G. Velasco. Theory, implementation and applications of
nonstationary Gabor frames. Journal of computational and applied mathematics, 236(6):1481–1496, 2011.
[2] P. Balazs, N. Holighaus, T. Necciari, and D. Stoeva. Frame theory for Signal Processing in Psychoacoustics. In Excursions in Harmonic Analysis, Volume 5, pages 225–268. Springer, 2017.
[3] A. Beck and M. Teboulle. A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems.
SIAM journal on imaging sciences, 2(1):183–202, 2009.
[4] J. Brown. Calculation of a Constant Q Spectral Transform. Journal of the Acoustical Society of America,
89:425–, 01 1991.
[5] M. Dörfler and B. Torrésani. Representation of operators in the time-frequency domain and generalized
Gabor multipliers. Journal of Fourier Analysis and Applications, 16(2):261–293, 2010.
[6] R. J. Duffin and A. C. Schaeffer. A class of nonharmonic Fourier series. Transactions of the American
Mathematical Society, 72(2):341–366, 1952.
[7] N. Holighaus, M. Dörfler, G. A. Velasco, and T. Grill. A framework for invertible, real-time constant-Q
transforms. IEEE Transactions on Audio, Speech, and Language Processing, 21(4):775–785, 2012.
[8] N. Holighaus, G. Koliander, C. Hollomey, and F. Pillichshammer. Grid-based Decimation for Invertible
Wavelet Transforms in Audio Processing. submitted, 2022. https://ltfat.org/notes/ltfatnote057.
pdf.
[9] N. Holighaus, C. Wiesmeyr, and Z. Prša. A class of warped filter bank frames tailored to non-linear
frequency scales. arXiv preprint arXiv:1409.7203, 2014.
[10] F. Jaillet and B. Torrésani. Time-frequency jigsaw puzzle: Adaptive multiwindow and multilayered Gaborexpansions. International Journal of Wavelets, Multiresolution and Information Processing, 5(02):293–315,
2007.
[11] B. C. Moore and B. R. Glasberg. Suggested formulae for calculating auditory-filter bandwidths and excitation patterns. Journal of the Acoustical Society of America, 74(3), 1983.
[12] T. Necciari, N. Holighaus, P. Balazs, Z. Prša, P. Majdak, and O. Derrien. Audlet filter banks: A versatile
analysis/synthesis framework using auditory frequency scales. Applied Sciences, 8(1):96, 2018.
[13] T. Painter and A. Spanias. Perceptual Coding of Digital Audio. Proceedings of the IEEE, 88(4):451–515,
2000.
[14] Z. Průša, P. L. Søndergaard, N. Holighaus, C. Wiesmeyr, and P. Balazs. The Large Time-Frequency Analysis Toolbox 2.0. In International Symposium on Computer Music Multidisciplinary Research, pages 419–442.
Springer, 2013.
[15] Z. Průša, P. L. Søndergaard, and P. Rajmic. Discrete Wavelet Transforms in the Large Time-Frequency
Analysis Toolbox for MATLAB/GNU octave. ACM Transactions on Mathematical Software (TOMS),
42(4):1–23, 2016.
[16] M. Schroeder. Vocoders: Analysis and synthesis of speech. Proceedings of the IEEE, 54(5):720–734, 1966.
[17] D. Sinha and J. D. Johnston. Audio compression at low bit rates using a signal adaptive switched filterbank. In 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference
Proceedings, volume 2, pages 1053–1056. IEEE, 1996.
[18] J. O. Smith. Spectral Audio Signal Processing. W3K, 2011.

[19] P. L. Søndergaard, B. Torrésani, and P. Balazs. The Linear Time Frequency Analysis Toolbox. International
Journal of Wavelets, Multiresolution and Information Processing, 10(04):1250032, 2012.
[20] S. S. Stevens and J. Volkmann. The Relation of Pitch to Frequency: A Revised Scale. The American
Journal of Psychology, 53(3):329–353, 1940.
[21] P. P. Vaidyanathan. Multirate Systems and Filter Banks. Pearson Education India, 2006.
[22] E. Zwicker, G. Flottorp, and S. Stevens. Critical Band Width in Loudness Summation. Journal of the
Acoustical Society of America, 29:548—-557, 1957.

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022, in Gyeongju, Korea
Extended abstractABS-0844

Measurement of the ringing sound of Sacred Bell of Great King
Seongdeok using multi-channel and an acoustic camera
Youngsun KIM1; Mina KIM2; Jiyoung OH3; Jeongho JEONG4
1

4

Graduate School of Seoul National University, Seoul National University, Korea
2

Diplom Tonmeister, MBI, Korea

3

Sound Engineer, Sgosda, Korea

Fire Safety & Building Environment Research Center, Korean Fire Protection Association, Korea

ABSTRACTThe Sacred Bell of Great King Seongdeok is a cast bronze bell for Buddist temples, completed in AD 771
measuring 3.66 m in height, 2.27 m in diameter (at the rim), 11-25 cm in thickness (of the wall), weighs 18.9
tons, and an acoustical cylinder particularly found in Korean bells is on top.
This research project employed multi-channel microphones at several points around the bell for measuring
the variations of sound pressure levels over the time at each frequency band from the moment of tolling thebell and changes in the central axis of sound pressure was observed with an acoustic camera.
Commissioned by Gyeongju National Museum, the measurement was conducted as a part of the examinationof the bell, so the ringing was taken with much less force than ordinary. As a result of the experiments, it isobserved that the audible lasting time of the bell is 1 minute and 53 seconds, and the central axis of soundpressure moves over time.
Keywords: Korean Bell, Pavillion, Acoustic camera, Multi-channel microphone measurement

1. INTRODUCTION
The Sacred Bell of Great King Seongdeok was completed in 771 by King Hyegong of Silla as thelargest bell in Korea. This bell is hung on a pavilion and makes a sound by bell tolling with a bellpounder by beating the spot called “Dangjwa,” decorated with lotus patterns.
The first place to be dedicated and installed was Bongdeoksa temple in Gyeongju, but it was moved
several times when Bongdeoksa temple disappeared due to flood damage in 1460. Currently, it has
been displayed at the Gyeongju National Museum since 1975. Unfortunately the original design of thepavilion of Bongdeoksa temple has not remained. For the design of the pavilion and the hemisphericalcylindrical cavity system on the ground, a more specific academic basis is needed to restore theoriginal construction.

2. CHARACTERISTICS
2.1 Shape
Measuring 3.66 m in height, 2.27 m in diameter (at the rim), 11 ~ 25 cm in thickness (of t he wall),weighs 18.9 tons. An acoustical cylinder on top and a resonator at the bottom of the ground appear
particularly in Korean bells as tradition.

1

youngsunkim@snu.ac.kr
min_a_kim@hotmail.com
3
jyo100@gmail.com
4
jhjeong92@daum.net
2

Figure 2. Cylinder on top of the bell
Top diameter: 148, Height:770, Bottom diameter: 82(Unit: mm)

Figure 1. The Sacred Bell of Great King Seongdeok‘(Cultural Heritage Administration, http://english.cha.go.kr)
2.2 Sound
The acoustic characteristic of The Sacred Bell of Great King Seongdeok yields 64.06 Hz as thelowest frequency, and 64.38 Hz is generated at the same time which yields beat. Table 1 shows the beatfrequency of each natural frequency and the number of modes in each direction.
Table 1 Natural frequencies of the Sacred Bell of the Great King Seongdeok (Kim S-H, Jeong W-T, Kang Y-J, 2012)
SequenceMode(m, n)

Frequencies (Hz)(0, 2)

64.07(0, 2)

64.42(0, 3)

168.52(0, 3)

168.63(1, 2)

189.34(1, 2)

190.55(1, 3)

227.99

(1, 3)

228.34(0, 4)

281.93(0, 4)

282.61(1, 4)

349.4(1, 4)

350.1

1

2

3

4

5

6

2.3 Duration of the audible bell soundAs a result of the measurement at dawn (03:20 AM) on October 28, 2020, the duration of the
audible bell sound was 1 minute 53 seconds, and the background noise was about 30 dB at 500Hz, as
shown in Figure 3. As the purpose of the test was to examine whether the bell was damaged, theintensity of the ringing was generally fragile compared to the intensity of striking the bell. As a result,the measured audible ringing time is relatively short.

Figure 3. Background noise in the space around the pavilion

2.4

Reverberation time of the pavilion

The reverberation time of the pavilion is shown in Figure 4. As the reverberation time of thepavilion, the rapid increase in the reverberation time below 125Hz at T20 is presumed to be due to the resonance of the bell.

Figure 4. The reverberation time of the pavilion of the Sacred Bell of Great King Seongdeok

2.5

Acoustic camera measurement results

In order to examine the movement path of sound energy, an acoustic camera was used to takepictures, and the results are shown in Figure 4.
There is no change in position from 4 sec. to 32 sec. of the ringing. After 32 sec. the peak energy
location changes, but the exact cause is a matter that requires further research.

(a) Immediately after the bell tolling(b) Peak energy location after 1 sec.
of the bell tolling(c) Peak energy location after 1.5 sec.
of the bell tolling(d) Peak energy location after 3 sec.
of the bell tolling(e) Peak energy location after 4 sec.
of the bell tolling(f) Peak energy location after 32 sec.
of the bell tolling(g) Peak energy location after 38 sec.
of the bell tolling(h) Peak energy location after 39 sec.
of the bell tolling(i) Peak energy location after 40 sec.
of the bell tolling

Figure 5. Peak energy location movements over time after the bell tolling3. DISCUSSIONThe Sacred Bell of Great King Seongdeok is the largest in the Shilla period in Korea The acoustic
function of the sound tube is known as the resonance transmission and the high -frequency soundattenuator, and it is known that it has a function to increase the amplification to sound last longer intime by using hemispherical cylindrical cavity system on the ground.
This study introduces the central movement of negative energy shown in acoustic cameras, and it isconsidered that research on the structure of the species, the design of the pavilion, and the direct
relationship with sound is needed.

ACKNOWLEDGEMENTS
This research was supported by the Ministry of Culture, Sports, and Tourism of Korea throughGyeongju National Museum.

REFERENCES
1. Seong, Myeong-Je, Haan, Chan-Hoon, Presentation of the Restoration of a Bell Pavilion of KingSong-Dok Bell, The Regional Association of Architectural Institute of Korea, Fall conference[Internet],
2007:p.386-391.
2. Do, hakhoe, The Study on Bell Head of Sangwonsa bell and The Bell of King Sungdeuk, Korea Art
Education Association[Internet], 2011;25(3):289–314.
3. Yang-Hann Kim. The sound of the Sacred Bell of Great King Seongdeok Broadcasting and MediaMagazine [Internet], 1998;3(1):28-38.
4. Kim Seong-hae. Silla Instruments of Archaeological Remain. The Society for Korean
Historico-Musicology, 2017;59(0):5-42.
5. Kim S-H, Jeong W-T, Kang Y-J. Design of a Variable Resonator for the Sacred Bell of the Great King
Seongdeok. The Journal of the Acoustical Society of Korea [Internet], 2012 Jul 31;31(5):288–97.

ABS-0872

A Custom Feature Set For Analyzing Historically Informed Baroque
Performances
Nolan ELEY(1,2) , Théodora PSYCHOYOU(3) , Catherine LAVANDIER(1) , Brian F.G. KATZ(2)(1) ETIS Laboratory, CY Cergy Paris University, ENSEA, CNRS, France, nolan.eley@dalembert.upmc.fr(2) Sorbonne Université, CNRS, Institut Jean Le Rond d’Alembert UMR7190, Paris, France, brian.katz@sorbonne-universite.fr(3) Sorbonne Université, Institut de Recherche en Musicologie UMR 8223, France

ABSTRACT
A common strategy for analyzing music performances is to extract a high number of low-level features followedby dimensionality reduction. While this can be an effective way of identifying differences in performances, interpreting these differences in a musically meaningful way is difficult. An alternative strategy is to select or design
features which are appropriate for the task at hand, however, this must be done carefully and in accordance withmusicological principles. In this study, a custom feature set was designed to capture baroque expressivenessin music performances, motivated by principles of historically informed performance practice. This feature set
was verified on a small set of annotated performances then applied to a larger group of recordings of historicallyinformed solo baroque performances in two halls, one historically appropriate and one modern. These features
were examined to study the effect of room acoustics on the baroque expressiveness of the performances in orderto gain insight into the role of acoustics in historically informed performances.

Keywords: Music performance analysis, chamber music, room acoustics, historically informed performance

1

INTRODUCTION

There is often a large variety among expressive musical performances of a single composition. The field ofmusic performance analysis is concerned with systematically identifying and assessing these variations. There
have been many strategies for analyzing different aspects of musical performances, which are well described in[1]. Because each genre may have a different set of intrinsic mechanisms which contribute to an idealized performance within that genre, it is important that the analysis framework take into consideration the musicologicalcontext of the task at hand.
One way to address this is through careful selection of low-level features. However, the available low-levelfeatures may be limiting and often lack a direct musical meaning. Furthermore, it is not always known whichfeatures will be most discriminative and one runs the risk of leaving out meaningful features whose capabilities
may be currently unknown. Another strategy, therefore, is to extract a large number of low-level features thenapply dimensionality reduction, followed by statistical analysis [2, 3]. While this can be effective at identifying
similarities and differences among performances, one disadvantage is that the resulting dimensions can often be
difficult to interpret, especially in a way that is musically meaningful. One way to overcome these problems is
to design custom features for identifying specific musical parameters which are known to be of interest to thecurrent task, based on musicological principles.

2

CONTEXT

There are many factors which can affect musical performance, one of which is the acoustic environment [4].
This study takes place as part of a larger effort to study the interactions between acoustics and musical performance from a historical perspective, merging the study of acoustics with the fields of historically informedperformance (HIP) and historical musicology [5, 6]. More specifically, this study focuses on baroque HIP andwhether the acoustics of spaces of the era facilitate HIP musicians’ play. To better understand this problem,
it is helpful to know which musical parameters are most salient in the genre. Through the study of period
musical treatises and manuals, musicologists have identified a number of characteristics which are responsiblefor defining baroque performance style and serve as the basis for HIP [7, 8, 9].
Musicologists have adopted the term “stylishness” as a way to describe musical expressiveness in terms of
its appropriateness for a specific musical and historical context [10]. For example, the parameters which define
an expressive performance of 19th century romantic music are very different from those which define an equally
expressive performance of 18th century baroque music. While both performances may be perceived as equallystylish, their perceived expressiveness may vary widely due to this term’s strong association with mainstream
romantic musical gestures. For this reason, researchers have started utilizing the term “baroque expressive” inorder to differentiate a type of expressive performance which is deemed as being baroque stylish [11].
The goal of this paper is to design an analysis framework for identifying the parameters which make up
baroque expressiveness among other performance styles of baroque music. This framework will be verified
against a small set of recordings which have already been authenticated as representing distinct performancestyles (see Section 3). Following this verification, the designed features will be applied to a set of recordings from a previous study [6] to shed light on the role of acoustics in facilitating baroque HIP practice (seeSection 4).

3

FEATURE DESIGN AND VERIFICATION

3.1 Dataset
Three distinct schools of baroque performance practice are identified in [11], and we refer the reader to that
paper for more information. The three schools are referred to as expressive-emotional, modern-literalistic, andbaroque-expressive. In that study, listeners were able to identify these three schools of performance as evidenced
by ratings in several musical categories. Relying only on the musical example used in that study (the first 8 barsof J.S. Bach’s Sarabanda from the D minor Partita for solo violin) to verify our analysis framework would not
suffice as that would not provide enough data to yield meaningful results. So, the number of recordings was
expanded to three full solo violin pieces within each performance school. These pieces, all composed for solo
violin by J.S. Bach, were the aforementioned Sarabanda, the Gavotte en Rondeau from the E major Partita,
and the Largo from the C major Sonata. While these additional recordings have not been verified by listeners
as embodying specific performance parameters, it is assumed that additional recordings by the same musicians
would carry similar performance characteristics to the one previously examined.
Below follows the development of features to identify baroque expressiveness in three major areas which
listeners have already identified in [11], namely, phrasing, tone quality, and vibrato. The verification for eachfeature set is done using support vector machines (SVM) using a one versus one approach with cross validation(10 random folds) to predict the musician which serves as a proxy for performance style.
3.2 Phrasing
The phrasing style in a baroque expressive performance is often described as rhetorical, since it tends to mimic
the patterns found in the art of oratory, in that it is varied, locally nuanced, and rhythmically flexible [12]. By
contrast, other performance styles tend to use legato, or continuous phrasing, in order to highlight cadences andlarge phrase boundaries.
Researchers have found that phrasing is closely related to changes in tempo and loudness, and correlations
are typically found between these two musical parameters [13, 14]. Our methodology for capturing phrasing
was to separate the recordings into segments of different lengths (1, 2, 4, 8, and 16 bars) and extract featuresfrom the tempo and loudess curves of these segments as well as the entire recordings. In total, the dataset
yielded 1029 segments. The tempo curves were made up of the note-level tempo values smoothed with amoving average filter since smoothed tempo curves have been found to be helpful in modeling expressive timing[15, 16]. The loudness curves were calculated using the frame-wise root-mean square (RMS) with a window
size of 0.05 s and a hop size of 0.02 s. The features extracted from these curves are range, standard deviation

(SD), and coefficients of a parabolic function (2nd-order polynomial) which have been shown to be effective inmodeling expressive performance [17, 18].
Using these features as input to an SVM tasked with predicting the musician, we were able to achieve
an average F-score (the geometric mean of precision and recall) of 72%. As further exploration, the sameclassifier was used to predict the composition, and this achieved an average F-score of 87%. As the different
compositions are stylistically different, they should exhibit very different phrasing characteristics, meaning thisF-score may reflect somewhat the upper bound of the discriminability of this feature set.
3.3 Tone Quality
Tone quality or timbre is multidimensional and complex. One dimension of tone quality which was found to
be representative of baroque expressive play is “lightness of tone.” Our methodology for identifying this tone
quality was to use mel-frequency cepstral coefficients (MFCC) on the harmonic and percussive components ofthe signal which were extracted using the median filter approach [19]. This approach applies a median filter
to the spectrogram either across successive bins or frames which enhances and suppresses either the harmonicor percussive components of the signal. A typical use case for such an algorithm might be to separate thedrums in a pop song to facilitate remixing. However, when applied to solo instrumental music, the percussive
components end up highlighting the nontonal, transient, and stochastic aspects of the signal which translate toelements such as bow and articulation technique which are likely correlated with one’s perception of tone intensity [20]. Additionally, the nontonal spectrum has previously been identified as being particularly discriminativein instrument classification [21]. MFCCs were extracted from these harmonic and percussive signals in addition
to the original, unprocessed audio. The MFCCs were calculated frame-wise using a window size of 0.04 s anda hop size of 0.01 s, and 21 coefficients were used.
To quantify the effectiveness of these various MFCC groups in identifying the performer, four classification
tasks were considered. One using each the percussive, harmonic, and unprocessed audio, and one which used
the MFCCs from both the harmonic and percussive audio. Using the unprocessed audio, the classifier was able
to achieve a class-weighted average F-score of 68%. Using the harmonic and percussive signals improved theclassification to 72% and 77%, respectively. And finally, using MFCCs from both the harmonic and percussive
signals together achieved the highest class-weighted average F-score of 83%. These results suggest that the harmonic, and especially the percussive components of the signal can be more discriminative than the unprocessedsignal, although more testing is needed to see how robust these components are for different types of signals.
3.4 Vibrato
Vibrato is a musical ornamentation made up of semi-periodic fluctuations of pitch and is often described by two
features: rate and extent [22]. The rate refers to the frequency of modulation which is typically in the range of
4 Hz to 8 Hz [23], while the extent refers to the depth or intensity of the vibrato, measured in cents. Baroqueexpressive performances tend to use less intense vibrato, less frequently than mainstream performance styles.
Most methods calculate extent in reference to a pitch center, usually the mean [24, 23, 25, 26]. However,this assumes that the fundamental frequency remains unchanged throughout the duration of the note. While
this may be the case in controlled laboratory experiments and therefore would not pose a problem to thoseexamining vibrato parameters under such conditions [27], in real performance situations, performers do notalways start and end each note at the precise pitch center and may adjust their intonation while maintaining
vibrato. One previous attempt to control for this used linear regression to model the pitch center [28].
The proposed methodology aims to improve upon previous methods in two ways. First, the mean removal
step is modified by modeling the change in fundamental frequency over the duration of the note with a polynomial curve, aiming to improve the accuracy of the measured vibrato extent. Second, it models the evolution of
the vibrato rate as it changes throughout the duration of the note using polynomial fit coefficients, rather thanrelying on a single measure (such as the median) for the vibrato rate.
The first step in the process is to extract pitch contours [29]; this was done using the Melodia plug-in1
for Sonic Visualiser2 . Pitch contours are continuous curves of varying length and can be output as multi-pitch
1 https://www.upf.edu/web/mtg/melodia
2 https://www.sonicvisualiser.org/

10
900

800

9

700
600
500
400
300

8

790

10-cent bin

Frequency (Hz)Frequency (Hz)

800

780

770

6
5
4

760

200

3
0.5

1

1.5

0.5

Time (s)

1

1.5

2

0

(b) A selected candidate exhibiting achange in F0 throughout the note

90

90

20

80

80

50

70
60
50

40

40

30

30

20
1

1.5

Time (s)(d) Upsampled and smoothed contour

2

10
5
0
-5
-10
-15

20
0.5

2

15

Pitch (cents)

Pitch (cents)

25

60

1.5

(c) Raw, quantized contour

100

70

1

Time (s)

100

0

0.5

Time (s)(a) Pitch contour candidates for a single
note that contains multiple pitches

Pitch (cents)

7

-20
0

0.5

1

1.5

2

Time (s)(e) Smoothed contour with overlaid estimated pitch center

0

0.5

1

1.5

2

Time (s)(f) Contour with pitch center removed

Figure 1. Vibrato feature extraction.
representations giving them an advantage to other fundamental frequency algorithms such as pYin [30] whenapplied to polyphonic contexts. The pitch contours were separated by note using note onset information. In
cases where multiple contours were detected during the span of one note, the best pitch contour was chosen bysumming the energy in each bin (representing 10 cents) and then selecting the bin with the most energy. Thisbin, along with the six bins above and below (equivalent to a range of 130 cents) were selected for further
processing (see Fig. 1b).
Because the raw contour was quantized to the nearest 10-cent bin (see Fig. 1c), the contour was upsampled,smoothed using a moving average filter, and factored by a gain of 10 so that each bin would represent one
cent (see Fig. 1d). The presence or lack of vibrato was detected if the signal met three conditions: a minimum
length, zero crossing rate (on the pitch-centered signal), and extent. These thresholds were arrived at empiricallythrough preliminary tests.
If the presence of vibrato was detected the pitch center was approximated using a polynomial curve ofdegree 5 (see Fig. 1e). This pitch center was then subtracted from each value, resulting in the final signal
seen in Fig. 1f from which the vibrato parameters were extracted. The extent was calculated by taking themean of the doubled absolute value of each peak and trough. The rate was calculated frame-wise by taking an
auto-correlation of the signal followed by peak picking with a peak distance threshold set to stay within the
typical rate of vibrato. The median value across all frames was selected as the rate. Polynomial fit coefficients

Table 1. Vibrato characteristics for each performance style.
Style

Prevalence

Rate (mean & SD)

Extent (mean & SD)

Expressive-emotional

32%

7.1 Hz ±0.75

16.8 cents ±5.8

Modern-literalistic

22%

7.3 Hz ±0.73

19.1 cents ±7.3

Baroque-expressive

13%

6.8 Hz ±0.87

12.7 cents ±3.9

of order 3 were taken of the frame-wise rate to model the evolution of the rate across the duration of the note.
In order to quantify the discriminability of these proposed vibrato features, four classification tasks wereconsidered. Two tasks used mean removal while the other two used the polynomial curve modeled pitch center
removal. These two groups either used only the baseline features (rate and extent) or the baseline features in
addition to polynomial fit coefficients. Overall, 972 notes with vibrato were identified out of 4374 played notes.
In general, the different classification tasks did not yield very different results. Using the baseline features(vibrato rate and extent) yielded class-weighted average F-scores of 47% and 49% using the mean and polynomial curve as pitch center estimation, respectively. Adding the additional features meant to model the changeof rate yielded class-weighted average F-scores of 46% and 52% using the mean and polynomial curve as pitch
center estimation, respectively. These F-scores are all significantly above random chance (33%) and, overall, theproposed changes in methodology and additional features yielded an approximately 5% increase in classification
accuracy.
Various vibrato characteristics are shown in Table 1 and reveal that, as expected, the baroque-expressive styleexhibits the least amount of vibrato, in terms of prevalence (the ratio of notes with vibrato to those without),
rate, and extent. The expressive-emotional style predictably shows the most prevalent vibrato, and both it andthe modern-literalistic style show somewhat comparable rates and extents.

4

APPLIED ANALYSIS

The dataset that this analysis framework was developed for was part of a larger study intended to help understand the role of acoustics in historically informed performance. A broad overview of the study can be foundin [6], but a brief description follows. Musicians specializing in historically informed baroque performance
practice played in two rooms, one historically appropriate to the musical era, and one modern. The historicalroom is the Salon des Nobles in Versailles, and the modern hall is the amphitheater from the Cité de la Musiquein Paris. 10 musicians participated, 4 violists da gamba, 3 flutists, and 3 theorbists. The musicians each played
3 pieces of music which were chosen for each instrument by musicologists, and they repeated each piece around3 times in each room, rendering a total of more than 180 recordings. The same classification exercises used
to verify the features were used when applying the analysis framework to these performances, but instead ofbeing tasked with predicting the musician, they were tasked with predicting the room in which the performances
occurred.
Using the phrasing set of features yielded average F-scores of 90%, 95%, and 83% for the viol, flute,and theorbo, respectively. While this is fairly high, it is possible that some of the features which make up
the phrasing may have been influenced by the sound of the room captured by the microphone. Even thougheach instrument was recorded with a directional microphone situated 1 m away, resulting in a high ratio of
instrument sound compared to room sound, the sound of the room can still be identified in most recordings.
Some of the features which make up the phrasing set are related to the RMS level, which could be influenced
by this incidental room sound. It is difficult to tell exactly which identified differences are due to this roomsound and which are due to actual differences in the performance. In order to clarify this, the classification
task was repeated on these three instruments using only the tempo-related features in the phrasing set. This
yielded average F-scores of 64%, 73%, and 53%, respectively. This indicates that at least some of the phrasing
expressiveness was changed due to the room, at least for the viol and flute. It was previously reported in [6]that the performances by the theorbists included many mistakes, so it is not surprising that the results using
only tempo-related features resulted in an accuracy that is only slightly above random chance.
The vibrato analysis is limited to the viol since it is the only instrument used in the study which is capableof vibrato. In total, 1145 notes were identified with vibrato out 6888 played notes, meaning almost 17% of theplayed notes were embellished with vibrato, compared to the 22% in the test set of Bach recordings. The resultsof the classifier using the vibrato feature set performed no better than random chance, indicating that the vibratostyle did not change as a function of the room. This is not very surprising as it was the least discriminative
feature in the test set. Despite the poor performance of vibrato as a predictor of room, some interesting findings
can still be reported. 54% of the vibrato occurred in the Salon des Nobles. Furthermore, about 30% of thevibrato can be attributed to one of the four violists who had 3 years of performing experience, compared toan average of 37.3 (SD: 5.5) years of experience the other 3 violists had. This implies that perhaps the more
liberal use of vibrato may be correlated with having less experience or different training. Lastly, the averagevibrato rate was 6.8 Hz and the average extent was 15.2 cents.
In general, the features designed to describe tone quality were able to predict the room with fairly highaccuracy, with class-weighted average F-scores among the 90% range using the normal, harmonic, or percussive
components of the signal. There was not much of a difference between the unprocessed and the harmonic
signals, while the percussive signal resulted in an F-score of about 5-7 points lower. However, it should be
noted that these MFCCs are likely influenced by the room sound to some degree though it is difficult to say
precisely how much. The percussive signal, with its emphasis on transients, is likely influenced by the room
less, and therefore, its slightly lower F-scores may more accurately reflect an actual difference in tone quality.

5

Conclusions

The goal of this study was to simplify the analysis of music performance to a few musically meaningful dimensions which are relevant to the task at hand, in this case, historically informed baroque performance. We
identified three major areas which are important to baroque expressiveness, namely, phrasing, vibrato, and tone
quality, and developed an analysis framework for identifying differences in these areas. This framework was firstsuccessfully verified on recordings from a published study in which musicologists and listeners had identified
significant differences among baroque performance styles.
Subsequently, this analysis framework was applied to an unrelated set of recordings of baroque performances, with the goal of identifying significant differences among the performances which may have been
influenced by the room and its acoustics. Judging the success of the analysis framework on this dataset wasdifficult, as it was not previously known how these performances differed, nor to what degree. Overall, thephrasing and tone quality features identified significant differences between performances in the two rooms.
However, it was noted that some of this may be due to the sound of the room incidentally picked up by the
microphone. The vibrato feature set did not show any significant differences between the two rooms, however,it did reveal a difference between performers which may be correlated with their experience or training.
While this method of performance analysis was able to reduce the problem to a few components, thesecomponents are still described by high-dimensional feature sets which exhibit problems of interpretability. Morework (and larger datasets) would be needed to reduce the dimensionality of these feature sets in a way thatmakes the results easier to interpret. For example, we are able to identify differences in phrasing, but cannot
say definitively what this means on the continuum of “legato” to “separated”. A complementary listening test
performed on a subset of these recordings may help identify what these differences are.
One issue with developing an analysis framework on a few recordings of Bach violin pieces is the major riskof over-fitting to such a small and unrepresentative dataset. Ultimately, without larger datasets of performancesof baroque music annotated along specific dimensions of performance, it is difficult to create a truly robust
analysis framework. Still, the approach outlined in this paper suggests that a tailored method of analyzing
music performance can be effective and yield meaningful results that are more interpretable than broad methodsof low-level feature extraction.

ACKNOWLEDGEMENTS
This work was supported by the Paris Seine Graduate School Humanities, Creation, Heritage, Investissement
d’Avenir ANR-17-EURE-0021 – Foundation for Cultural Heritage Science. This project takes place as partof EVAA_Ver: Expérience Virtuelle en Acoustique Archéologique. We would also like to thank the research
center of the Château de Versailles and the Musée de la Musique of the Philharmonie de Paris, as well as themusicians who participated in the study.

REFERENCES[1] Lerch A, Arthur C, Pati A, Gururani S. An Interdisciplinary Review of Music Performance Analysis.
Transactions of the Intl Soc for Music Information Retrieval. 2020 Nov;3(1):221-45.
[2] Schärer Kalkandjiev Z. The Influence of Room Acoustics on Solo Music Performances. An EmpiricalInvestigation [PhD Dissertation]. Berlin: Technischen Universität; 2015.
[3] Amengual Garí S, Kob M, Lokki T. Analysis of trumpet performance adjustments due to room acoustics.
In: Proc of the Intl Symposium on Room Acoustics. Amsterdam; 2019. p. 65-73.
[4] Kob M, Amengual Garí SV, Schärer Kalkandjiev Z. Room Effect on Musicians’ Performance. In: Blauert
J, Braasch J, editors. The Technology of Binaural Understanding. Springer Intl Publishing; 2020. p. 223-49.
Series Title: Modern Acoustics and Signal Processing.
[5] Eley N, Mullins S, Stitt P, Katz BFG. Virtual Notre-Dame: Preliminary results of real-time auralization
with choir members. Immersive and 3D Audio: from Architecture to Automotive (I3DA). 2021.
[6] Eley N, Lavandier C, Psychoyou T, Jossic M, Katz BFG. Performance analysis of solo baroque music
played in a period and modern hall. 16th french acoustics congress of the french acoustics society. 2022
Apr.
[7] Lawson C, Stowell R. The historical performance of music: an introduction. Repr ed. Cambridge handbooks to the historical performance of music. Cambridge: Cambridge University Press; 2003.
[8] Haynes B. The End of Early Music: A Period Performer’s History of Music. New York, NY: Oxford
University Press; 2007.
[9] Donington R. Baroque Music Style and Performance: A Handbook. W.W. Norton & Company; 1982.
[10] Schubert E, Fabian D. The dimensions of baroque music performance: a semantic differential study.
Psychology of Music. 2006 Oct;34(4):573-87.
[11] Fabian D, Schubert E. Baroque expressiveness and stylishness in three recordings of the D minor Sarabanda for solo violin by J.S. Bach. Music Performance Research. 2009 Jan;3:36-56.
[12] Ponsford D. Instrumental performance in the seventeenth century. In: Lawson C, Stowell R, editors. The
Cambridge History of Musical Performance. Cambridge University Press; 2012. p. 421-47.
[13] Palmer C. Mapping Musical Thought to Musical Performance. Journal of Experimental Psychology: Human Perception and Performance. 1989;15(12):331-46.
[14] Palmer C. Music Performance. Annual Review of Psychology. 1997;48:115-38.
[15] Cambouropoulos E, Dixon S, Goebl W, Widmer G. Human Preferences for Tempo Smoothness. In: Procof the VII Intl Symposium on Systematic and Comparative Musicology; 2001. p. 18-26.
[16] Schreiber H, Zalkow F, Müller M. Modeling and estimating local tempo: A case study on Chopin’s
Mazurkas. In: Proc of the 21st Intl Soc for Music Information Retrieval Conf; 2020. p. 773-9.

[17] Li S, Dixon S, Plumbley MD. Clustering Expressive Timing With Regressed Polynomial Coefficients
Demonstrated By A Model Selection Test. In: Proc of the 18th Intl Soc for Music Information Retrieval
Conf; 2017. p. 457-63.
[18] McAngus Todd NP. The dynamics of dynamics: A model of musical expression. The Journal of the
Acoustical Soc of America. 1992 Jun;91(6):3540-50.
[19] Fitzgerald D. Harmonic/Percussive Separation using Median Filtering. 13th Intl Conf on Digital Audio
Effects (DAFx-10). 2010 Jan.
[20] Schoonderwaldt E, Guettler K, Askenfelt A. Effect of the Width of the Bow Hair on the Violin String
Spectrum. In: Proc of the Stockholm Music Acoustics Conf. Stockholm, Sweden; 2003. p. 91-4.
[21] Fragoulis D, Papaodysseus C, Exarhos M, Roussopoulos G, Panagopoulos T, Kamarotos D. Automatedclassification of piano-guitar notes. IEEE Trans Audio Speech Lang Process. 2006 May;14(3):1040-50.
[22] Seashore CE. Psychology of Music. New York and London: McGraw-Hill Book Company; 1938.
[23] Weninger F, Amir N, Amir O, Ronen I, Eyben F, Schuller B. Robust feature extraction for automatic
recognition of vibrato singing in recorded polyphonic music. In: Intl Conf on Acoustics, Speech andSignal Processing (ICASSP). Kyoto, Japan: IEEE; 2012. p. 85-8.
[24] Herrera P, Bonada J. Vibrato Extraction and Parameterization in the Spectral Modeling Synthesis Framework. Proc of the Digital Audio Effects Workshop (DAFX98). 1998.
[25] Pang HS, Yoon DH. Automatic detection of vibrato in monophonic music. Pattern Recognition. 2005
Jul;38(7):1135-8.
[26] Regnier L, Peeters G. Singing Voice Detection in Music Tracks using Direct Voice Vibrato Detection. In:
Intl Conf on Acoustics, Speech and Signal Processing (ICASSP). Taipei, Taiwan; 2009. p. 1685-8.
[27] Brown JC, Vaughn KV. Pitch center of stringed instrument vibrato tones. The Journal of the AcousticalSoc of America. 1996 Sep;100(3):1728-35.
[28] De Muynke J, Eley N, Ferrando J, Katz BFG. Preliminary analysis of vocal ensemble performances inreal-time historical auralizations of the Palais des Papes. In: Proc on the 2nd Symposium of The Acoustics
of Ancient Theatres. Verona, Italy; 2022. .
[29] Salamon J, Gomez E. Melody Extraction From Polyphonic Music Signals Using Pitch Contour Characteristics. IEEE Transactions on Audio, Speech, and Language Processing. 2012;20(6):1759-70.
[30] Mauch M, Dixon S. PYIN: A fundamental frequency estimator using probabilistic threshold distributions.
In: Intl Conf on Acoustics, Speech and Signal Processing (ICASSP). Florence, Italy: IEEE; 2014. p. 65963.

ABS-0886

Pipe organ buffet radiation patterns under different excitation strategies
Gonzalo VILLEGAS CURULLA(1) ; Elliot K. CANFIELD-DAFILOU(1) ; Piergiovanni DOMENIGHINI(2) ; Benoit FABRE(1) ;Christophe d’ALESSANDRO(1) ; Brian F. G. KATZ(1) *(1) Sorbonne Université, CNRS, Institut Jean Le Rond d’Alembert, F-75005 Paris, France(2) Università degli Studi di Perugia, Italy

ABSTRACT
Studying the directivity of musical instruments such as pipe organs is challenging because of their great sizeand because they typically cannot be moved from the locations where they are built to the laboratory. This
study compares the radiation pattern of organ buffets under different excitation conditions, using a 19th century
French pipe organ. We investigate the positive and the great organ independently due to their spatial separationand size. The excitation strategies were comprised of cylindrical and an omnidirectional electroacoustic sources.
Measurements were carried out on each organ buffet along a horizontal line spanning the width of the church.
Results in octave bands are shown and compared with a discussion on possible causes for observed differences.
Significant variations in directivity were observed for the 2 kHz to 4 kHz octave-band regions where scatteringfrom pipes is expected to have a predominant effect, while little variation from omnidirectional was observed
for lower frequency bands for both sources.

Keywords: Pipe Organ, Source Directivity, Musical Acoustics

1

INTRODUCTION AND MOTIVATION

The interest of studying the radiation and directivity patterns of pipe organs is framed within the broader scopeof modeling the acoustic behavior of individual components of pipe organs (e.g., pipes, windchests, buffets)
both alone and in combination with one another. These goals are both theoretical—for understanding the underlying behavior of the instrument, and applied to case of modeling historical instruments such as the Dalleryorgan of Chapelle Sorbonne and the instrument in Notre-Dame by Thierry, Clicquot, and Cavaillé-Coll. An
overview of the broader context of pipe organ modelling can be found in [1]. Results from recent campaignsof measurements and simulations are presented here as well as in [2, 3, 4, 5].
The current study considers the organ buffet as an acoustic obstacle (diffraction/scattering element). Horizontal directivity patterns of the positive and great organ of St. Élisabeth are presented when excited by idealized
electroacoustic sources located within the buffets. These sources include an omnidirectional point-source (3D)
and a linear-array cylindrical source (2D). These two approaches offer simplifications of the acoustic radiationof pipes within the organ buffets.
Different time scales are involved in the production and radiation of sound produced by pipe organs. Startingfrom the aero-acoustic sound production in each pipe, the sound is first radiated inside the buffet, where thecharacteristic time is given by Lbu f /c (∼5 ms in the current case), where Lbu f is the characteristic length of thebuffet (height of the facade’s slit) and c the propagation velocity of the sound . The next time scale is givenby the room.
In a more detailed analysis, multiple regimes can be extended to the field inside the buffet with the additional aspects of: the establishment of modes, complex scattering due to the presence of hundreds or thousands* gonzalo.villegas_curulla@upmc.fr; piergiovanni.domenighini@studenti.unipg.it; elliot.canfield-dafilou@dalembert.upmc.fr;
benoit.fabre@sorbonne-universite.fr; christophe.dalessandro@sorbonne-universite.fr; brian.katz@sorbonne-universite.fr.

of pipes of different heights, and the establishment of a diffuse incident field onto the facade from the inside,
where the facade constitutes the interface between the interior and the exterior of the organ buffet. This facade can be initially described as an array of air-masses trapped between cylinders (pipes) that are brought tooscillation.
In [6], the explicit problem of an evenly-distributed array of cylinders is discussed. These cylinders canrepresent organ pipes with known diameters and center-to-center spacing in the facade, where a first-order estimation is proposed to predict the scattered pressure. In [2], a comparative study of the 1-slit system was
proposed using a laboratory proxy organ buffet, with respect to theoretical models informing directivity andfull-range SPL-vs-frequency about the slit and the cylindrical source used in the current study of the positive.
The presence of the facade is bound to contribute as a frequency dependant boundary condition, resulting
in a reflection or transmission with frequency dependant behaviour and with small losses. As a first approximation, it may well be considered almost transparent in the low frequencies. In [4], indices are shown that the
protruding towers of the facade of the positive (see Section 1.1) constitute a scattering contribution already forthe region of middle center frequencies of 1 kHz and 2 kHz.
1.1 The Organ in St. Élisabeth, Paris
The organ in St. Élisabeth was dedicated on 28 April, 1853 and restored in 1999. It is a large historicalorgan, with a total facade area of approximately 10 m × 10 m. The organ has three levels and hosts a totalof 2322 pipes, of which 143 are visible in the facade. The organ has 42 stops on three manuals and pedal,
with tracker action [7]. Figure 1 shows the locations of the three main radiating cavities and their wind-chest
distributions.
The positive (420 pipes) stands on the floor of the organ loft, separated and in front of the main caseand corresponds to the first manual keyboard. Its facade (inscribed in a 2.62 m × 2.64 m surface) contains
33 pipes, organized in 3 towers of 5 pipes each, and 2 flat sets of 9 pipes. The tallest pipe in the facadeis 6 feet, sounding F1 . It is relatively narrow and protrudes into the nave, radiating into free field without









∗
∗





∗
∗
(a) Facade.

(b) Zoom on facade pipe details.

2m
1.4 m

(c) Face (upper) and overhead (lower) views of
the organ with position of the wind chests. 2D(■) and 3D (∗) source locations and receiver positions (–•–) are indicated.

Figure 1. The organ in St. Élisabeth, Paris and measurement schematics.

nearby reflective surfaces. Above the positive resides the main case 5.35 m above the floor level. It contains thegreat organ (GO) in the center, and the pédale on each sides behind the highest towers. The GO (1014 pipes)
and pédale (180 pipes) correspond to the second manual keyboard and pedal board, respectively. Its facade isapproximately 9.39 m × 9.6 m, and contains 75 pipes, organized in 7 towers and 8 flat segments. The tallest
pipe in the facade is 12 feet (≈3.7 m), sounding F0 . The récit (708 pipes) crowns the instrument; it is enclosed
in a swell-box and corresponds to the third manual keyboard. Its facade (inscribed in a 2.95 m × 3 m square)contains 35 mute pipes. The récit is not studied here.
The acoustics of the church show moderate reverberation times (T20 = 2.14 s to 3.68 s across the 125 Hz to4000 Hz octave bands, see [8]).
1.2 Theory and Approximations
We make use of a number of approximations and assumptions in order to decompose the problem into smaller,
less complex systems. Namely, we try to isolate and simplify the organ into studies on the sources, the pressure
field inside the cavity, and the transmission and radiation into the building; the last of these being the subjectof the current study. The sound-production sources are organ pipes with a non-flanged termination on one end(the so–called passive end of the resonator) and a mouth or window opening at the other end. The latter canbe approximated in the low frequencies as a second monopole, but more elaborate characterizations are readily
available: filtering has been described in [9] and general intensity around the pipe as a whole in [10]. The
majority of pipes are assumed to be cylindrical and of open-open terminations. Each of them is not only asound source, but also an obstacle to the propagation front of other sources within the buffet. In the conditionof a fully closed buffet, the pressure field inside is mostly transmitted through the front face of the organ, thefacade. The pipe-to-pipe spacings in the facade are the most salient transmission aperture outwards: they canbe represented by an array of slits of resilient air-masses. Said slits are considered in the finite and infinitesimal
thickness, and ideally as infinite in vertical length condition [6, 11, 12] or finite length, amounting to the fullheight of the cavity. These apertures become larger at the very bottom of the facade pipes due to the conicityof their foot, but this is considered irrelevant here. If acquisitions of the radiated field into the nave are madeclose enough to the facade, the infinitely tall slit [13] is a plausible approximation.
The effects of scattering will be expected to become apparent for ka ≫ 1, where a is the half-width of
the spacing between facade pipes, and k = ω/c is the wavenumber. For comparability purposes, the cavity of
the St. Élisabeth’s positive organ was chosen as a feasible approximation of the proxy organ used in previous
laboratory studies [2, 3, 4].

2

Methods

The organ buffets under examination were excited by 6 s logarithmic sweeps spanning 60 Hz to 20 kHz, played
from electroacoustic sources [14]. Signal acquisitions were made in several points inside the buffets for reference. The measurements outside the buffet were made along a set of equally-spaced locations in a straightline spanning the width of the nave (10 m). Acquisitions were made using the following audio hardware: RME
UFX+, RME Octamic, omnidirectional measurement microphones BAMT1 class 1, and a Carver PM–175 Amplifier. The 3D source was a dodecahedron omnidirectional source, Dr. Three 3d-032. For the 2D cylindricalsources, the small 2D array was comprised of 18 Aurasound NSW2-326-8A speakers, with a 50 mm center-tocenter spacing; the large 2D array was comprised of 10 Monacor SPH 165CP, with a 170 mm center-to-center
spacing. The speakers were positioned near the center of the positive, and near the center of the left-half ofthe great organ. Groups of 4 to 5 microphones were arranged with 20 cm spacing on a transversal tension line,
manually positioned using a pull-cord and measuring tape (see Figure 2(a)). See Figure 1(c) for source andreceiver locations.
Deconvolution of the acquired signals was performed. Temporal alignment of the resulting IRs, compensating for arrival times, were calculated based on microphone positions along the acquisition lines, parallel to themeasured facade (see Figure 1(c)), converting the linear measurement array to a polar propagation estimation
relative to the source position. Octave-band frequency filtering of data was performed and relative pressure levels were calculated using fourth order Butterworth filters given by MATLAB’s octaveFilterBank. Subsequent(a) Transversal microphone measurement installation (b) 2D source configuration for the positive, showing 2D (c) 2D source used in thefor the positive.
cylindrical source (left, grey column) and 3D omnidirec- GO, in the anechoic room.
tional source (centre, red).

Figure 2. Measurement setup example in St. Élisabeth and source.
amplitude correction of time aligned IRs was applied dependant on a reference distance r′ /r, with r the shortest
path at 0◦ . For the 3D omnidirectional source a −6 dB per doubling distance was employed. For the 2D cylindrical source, the correction was more complicated, employing a simplified model of −3 dB or −6 dB for nearor far-field corrections, based on an idealized finite array (see [15]) with a 2Lc2 /λ rule for the cut-off, where
Lc is a characteristic length, here the height of the source. The IRs were windowed in time to avoid contributions from from wall reflections. The length of this window was determined based on the distance between the
microphones closest to the side-walls and the length of the direct path. An adequate estimation was determined
to be 4.7 ms between the direct path to the microphone position closest to wall and its first reflection arrival,
applied after time alignment of each position’s deconvolved IR. Results are presented for octave bands from125 Hz to 4 kHz. Due to the microphone spacing (20 cm), spatial aliasing limits the interpretability of higher
frequency data.

3

RESULTS AND DISCUSSION

The octave band directivity pattern results for the positive are show in Figure 3, and for the GO in Figure 4, forthe two source excitation conditions (3D omnidirectional and 2D cylindrical). Note the increasing measurement
point density with increasing angle, due to the projection of the linear measurement setup to polar coordinates.
Results are first examined for each buffet separately, then cross-compared. Closer inspection of the positive
microphone array showed a variance in calibration level of 0.87 dB (B&K calibrator type 4231); accounting forthis would likely resolve the observed minor fluctuations. Similar variations were not observed for the grandorgan microphone array.
3.1 Positive Directivity
Inspection of the results for the positive identify a relatively omnidirectional directivity pattern across the octave band frequencies of 125 Hz to 1000 Hz (Figures 3(a) to 3(d)) for both 3D and 2D sources. Some minor
attenuation can be observed at the extreme angles (|θ | > 50◦ ), likely due to the enclosure effect of the buffet windowing the energy generated within the buffet, redirecting it to the more central region. Patterns are
relatively smooth, with measurement variations on the order of a few dB.
Directivity in the 2000 Hz octave band (Figure 3(e)) shows a significant broadening of the directivity forthe 3D source, enforcing radiation at ±45◦ relative to on-axis. This effect is less pronounced for the 2D
source, where the directivity is more erratic compared to lower frequency bands. This effect is attributed to thesignificant scattering effect of the pipes and facade interface.
Results for the 4000 Hz octave band (Figure 3(f)) are similar to those at the low frequency bands, thoughwith higher variance. This variance is expected due to the effects of spatial aliasing at higher frequencies.

30

0

60
0
0
-9

30

0

60

-3

0
0

-20
-15
-10
-5
0
5
10 9

0

(d) 1kHz

-9

-20
-15
-10
-5
0
5
10 9

0
-9

-20
-15
-10
-5
0
5
10 9

0

-6
0

-6
0

0

60

60

-3

-3

0

30

0

(c) 500Hz

0

0

30

0

(b) 250Hz

-6
0
-9

-20
-15
-10
-5
0
5
10 9

0

-6

0

60

-3

0

30

0

-20
-15
-10
-5
0
5
10 9

0
-9

-20
-15
-10
-5
0
5
10 9

0

-6

0

60

-3

0

30

0
0
-3
0
-6
0
-9

(a) 125Hz(e) 2kHz(f) 4kHz

30

0

60
0
30

0
0
0

(e) 2kHz

-20
-15
-10
-5
0
5
10 9

0

-6
-20
-15
-10
-5
0
5
10 9
0
-9
0

-9
0

0

60

60

-3

-3
-6
0

60
-20
-15
-10
-5
0
5
10 9
(d) 1kHz

-20
-15
-10
-5
0
5
10 9

0

0

-3
0
0
-6
-9
0

(c) 500Hz300(b) 250Hz

300(a) 125Hz

-9

-20
-15
-10
-5
0
5
10 9

0

-6
0

60

-3

0

30

0
0
-3
-6
0
0
-9

-9

0

-20
-15
-10
-5
0
5
10 9

0

-6
0

60

-3

0

30

0

Figure 3. Directivity (dB) of positive organ for the 3D ( • ) and 2D ( • ) source excitation conditions, 4.7 ms
window, and normalized to data at 0◦ .

(f) 4kHz

Figure 4. Directivity (dB) of the GO organ for the 3D ( • ) and 2D ( • ) source excitation conditions, 4.7 ms
window, and normalized to data at 0◦ . Note the off-center position of the source relative to the building (seeFigure 1(c), with the sources closer to the side wall (positive angles).)
3.2 Great Organ Directivity
Directivity patterns for the octave band frequencies of 125 Hz to 500 Hz (Figures 4(a) to 4(c)) show relatively
omnidirectional directivity for both source types, with pattern fluctuations occurring at the extreme angles. The
slight increase at the extreme (θ ≥ 40◦ ) could be attributed to the contribution of minor reflections from theside wall at such proximity to the boundary. At the other extreme positions (θ ⪆ −60), one can observe an
increase in energy relative to the on-axis response. For 125 Hz to 250 Hz, the 2D source directivity exhibits a
discontinuity, likely attributed to the idealized cylindrical propagation correction for mapping the measurement
points from their linear configuration to polar coordinates relative to the source position (see Section 2). Asmoother, more sophisticated estimation of propagation properties for the truncated linear source array could
improve results, though it is noted that the corrected values asymptotically converge to the 3D source responsein those frequency bands. A general increase in energy is seen with negative angle progression (approaching
5 dB), more pronounced for 125 Hz to 250 Hz with the 3D source and 500 Hz with the 2D source.
Results for the 125 Hz to 1000 Hz octave bands (Figures 4(a) to 4(d)) show increasing emphasis for theextreme negative angles, (on the order of 5 dB). It is worth noting that the GO has horizontal platforms separating levels. It is plausible that with increasing angle, and hence increased distance to the receiver array,reflections from these floor/ceiling surfaces arrive with diminishing delays relative to the direct sound, thereby
becoming more prevalent within the 4.7 ms analysis window. As this behavior appears common to both sources,
this seems a reasonable assumption.
Results at the higher octave bands, 2000 Hz to 4000 Hz (Figures 4(e) to 4(f)), exhibit large oscillations in
directivity patterns for both sources, though the patterns do not align at all angles. It would seem plausible thatsuch variations in directivity could be attributed to the scattering properties of the large pipes making up the
GO in general.
3.3 Positive and Great Organ Directivity Comparison
Comparing the results between the two measured organ buffets (the positive and GO), similar behaviour can beobserved at the lower frequency regions, with markedly less measurement variance (noise) for the GO directivity patterns. This observation holds true for both sources, for which it is noted that the 3D was the same source
while the 2D source was different due to the different dimensions/scale of the two organ buffets. For both buffets, the 2 kHz octave band showed the most pronounced deviations from free-field, omnidirectional directivity
patterns, with patterns exhibiting increased variance at higher frequencies. The observed differences in thesepatterns between 3D and 2D sources suggest that the different excitation methods (spherical versus cylindricalwaves) as well as the slight positional differences within the buffets and of the host of interior scatter objects
induced different but still pronounced patterns.
While the GO contains more larger-diameter, low frequency pipes than the positive on the whole, examination of the two facades shows very similar pipe diameters in the center region (see Figure 1(b)), which
would lead to an expectation of similar frequency behaviour with regards to scatter properties. However, whilethese patterns were clearly evident for the GO in the 2 kHz to 4 kHz octave bands, they are only observed in
the 2 kHz octave band for the positive. The complex response of the positive could be attributed to the generally small pipe diameter and spacing within the buffet, producing scattered patterns with spatial resolutions
exceeding the current measurements spatial aliasing limits. Reproducing these measurements with a finer spatial
resolution would allow further investigation of this phenomenon at these frequency ranges.

4

CONCLUSIONS AND FUTURE WORKThis study has presented measurement results of an investigation into the diffraction/scattering effects of theorgan buffet and facade on the directivity pattern of the radiated sound field. Two ideal sound sources were
employed, an omnidirectional point source and a linear (truncated) cylindrical source, as approximations of anorgan pipe source. Two sections of the church organ at St. Élisabeth were studied, the small and compact positive and the larger great organ. Measurements were made linearly, spanning the nave of the church at a distanceof roughly 1.4 m to 2 m, and subsequently processed to obtain polar directivity data, excluding contributions ofwall reflections through short time windowing.
Octave band results highlighted generally omnidirectional directivity in the frequency range of 125 Hz to1000 Hz. Prominent variations were observed in the 2 kHz octave band, attributed to the diffraction and scattering effects of the pipes comprising the interior and facade of the organ. Similarity between the two organ
sections in frequency appears reasonable as similar pipe dimensions are present in both the positive and greatorgan, despite the presence of larger pipes in the great organ. Differences in the details of the pattern are
expected due to differences in the pipes distribution patterns within the organ.
While some differences observed between the two excitation sources can be attributed predominantly to assumptions regarding corrections for near-field versus far-field propagation correction with the cylindrical source,these results suggest the source model is not a predominant factor in the directivity behaviour. Other variations
are attributed to slight positional differences (not precisely coincident) of the two sources within the organ.
It is interesting to note that the dissymmetry in the GO patterns, due to the lateral position of the source
excitation (particularly in the 1 kHz octave band), is consistent with the auditory perception of the diatonic
disposition of the instrument. In St. Élisabeth, as is typical in large instruments, the wind-chests are diatonic,
divided in C and C# sides. This means that half of the notes are on the left (in whole tones, C - D - E etc.)
and half on the right (C# - D# - F etc.). This is contrary to, for instance, piano strings which are chromaticallyordered (in semi-tones C - C# - D - D# etc.). This ping-pong effect is clearly audible (when one pays attention toit) and is part of the organ listening experience. Here, the sources were positioned on the C side of the organ.
Directivity patterns suggest that the buffet may emphasize this effect by enhancing lateral perception.
Further studies shall examine the case using real organ pipe excitation sources within different regions of theorgan, rather than the idealized electroacoustic sources in the current study, providing a more realistic context
for the results [5]. These results could also be further enhanced by analyzing in finer frequency resolution, suchas 1/3-octave bands and by exploration with finite difference and finite volume scheme simulations [16, 17],well-suited to scattering and diffraction problems.
Higher spatial resolution would be necessary to provide further conclusions regarding higher frequency
bands. Such high resolution measurements have been carried out on a proxy organ, comparable to the positive of the current study, comprised of a homogeneous collection of simulated pipes and using the same sources[3, 4]. These studies examined in more detail the effect of pipe density in an attempt to simplify the diffraction/scattering problem.
The perceptual relevance of various organ properties in the context of auralizations is a primary applicationof the current results. Directivity perception is currently being evaluated through a series of listening tests, followed by a study which will examine the number of spatially distributed virtual sources required for plausible/natural auralization renderings of the largest indoor musical instrument [18]. Such studies compliment previous
works investigating the applicability of spatial perception metrics, such as the inter-aural cross correlation, as it
pertains to the perceptual notion of apparent source width of such and extended instrument [8].

ACKNOWLEDGEMENTS
Funding for G.V.C. has been provided by the ISCD (grant no. FED 3 – 2019/7/2). Funding for E.C.D. and
B.K. has been provided by the European Union’s Joint Programming Initiative on Cultural Heritage project PHE(The Past Has Ears, phe.pasthasears.eu, Grant No. ANR-20-JPIC-0002), and the French project PHEND (The
Past Has Ears at Notre-Dame, Grant No. ANR-20-CE38-0014, phend.pasthasears.eu, Grant No. ANR-20-CE380014).

REFERENCES[1] Angster J, Rucz P, Miklos A. 25 years applied pipe organ research at Fraunhofer IBP in Stuttgart. In:
Proc. Int. Symp. Musical Acoust. September; 2019. p. 1-14.
[2] Dal Moro P, Villegas Curulla G, Fabre B, d’Alessandro C. La boîte expressive de l’orgue: étude acoustiquede la densité de tuyaux et d’une façade à ouverture variable. In: Proc. Congr. Fr. d’Acoust; 2022. p. 1-4.
Available from: https://hal.archives-ouvertes.fr/hal-03673985.
[3] Villegas Curulla G, Dal Moro PMM, Fabre B, Katz BFG. Radiation patterns of a multiple slit system
and applications to organ buffet modeling. In: Proc. Congr. Fr. d’Acoust.; 2022. p. 1-4. Available from:https://hal.archives-ouvertes.fr/hal-03700814.

[4] Villegas Curulla G, Domenighini P, Katz BFG, Canfield-Dafilou EK. Directivity of a small pipe organ buffet. In: Proc. Acoust. Ancient Theatres; 2022. p. 1-4. Available from: https://www.
archives-ouvertes.fr/hal-03725002/.
[5] Villegas Curulla G, Domenighini P, Katz BFG, Canfield-Dafilou EK, Fabre B, d’Alessandro C. Preliminary
analysis of organ buffet radiation under musically realistic excitation mechanism. In: Vienna Talk. Vienna;2022. p. 1.
[6] Berry DL, Taherzadeh S, Attenborough K. Acoustic surface wave generation over rigid cylinder arrays on
a rigid plane. J Acoust Soc Am. 2019;146(4):2137-44. doi:10.1121/1.5126856.
[7] d’Alessandro C. Orgues, Musiques et Musiciens à Sainte-Élisabeth. La flûte harmonique. 2010;(91):220.
Available from: https://hal.archives-ouvertes.fr/hal-02360434.
[8] Katz BFG, d’Alessandro C. Apparent source width and the church organ. In: Proc. Congr. Fr. d’Acoust.
and De. Gesellschaft Akust.; 2004. p. 1235-6. Available from: https://hal.archives-ouvertes.fr/
hal-01789803.
[9] Ernoult A, Fabre B. Window impedance of recorder-like instruments. Acta Acustica united with Acustica.
2017;103(1):106-16. doi:10.3813/AAA.919037.
[10] Odya P, Kotus J, Szczodrak M, Kostek B. Sound Intensity Distribution Around Organ Pipe. Arch Acoust.
2017;42(1):13-22. doi:10.1515/aoa-2017-0002.
[11] Park HH, Eom HJ. Acoustic scattering from a rectangular aperture in a thick hard screen. J Acoust Soc
Am. 1997;101(1):595-8. doi:10.1121/1.417971.
[12] Pàmies T, Romeu J, Genescà M, Balastegui A. Sound radiation from an aperture in a rectangular enclosure
under low modal conditions. J Acoust Soc Am. 2011;130(1):239-48. doi:10.1121/1.3596465.
[13] Mellow T, Kärkkäinen L. On the sound fields of infinitely long strips. J Acoust Soc Am. 2011;130(1):15367. doi:10.1121/1.3596474.
[14] Farina A. Advancements in impulse response measurements by sine sweeps. In: Audio Eng. Soc. Conv.
122. vol. 3; 2007. p. 1626-46.
[15] Chaigne A, Kergomard J. Acoustics of Musical Instruments. Springer; 2016.
[16] Bilbao S, Hamilton B. Passive Volumetric Time Domain Simulation for Room Acoustics Applications. J
Acoust Soc of Am. 2018 10;145. doi:10.1121/1.5095876.
[17] Weber A, Katz BFG. Sound scattering by the Gothic piers and columns of the Cathédrale Notre-Damede Paris. Acoustics. 2022;4:679-703. SI:Acoustics, Soundscapes and Sounds as Intangible Heritage.
doi:10.3390/acoustics4030041.
[18] Canfield-Dafilou EK, Domenighini P, De Muynke J, d’Alessandro C, Katz BFG. Perceptual Influence of
Directivity on Pipe Organ Auralization. In: Vienna Talk. Vienna; 2022. p. 1.

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022, in Gyeongju, Korea
Extended abstractABS-0887

Investigation of relationshipbetween beat-timing and acoustics-parameterfor method of beat-timing and tempo estimation
Madoka OKEMOTO1 and Masanobu MIURA2
1

Graduate School of Kunitachi College of Music, Japan
2

Kunitachi College of Music, Japan

ABSTRACT
Research on beat-timing and tempo estimation has been studied in the field of musical acoustics. Nowadays,
researchers are trying to use deep learning technology to explore the estimation of beat-timing and tempo. It
is hard to explain the result of deep learning since it is an obscure method of factors. However, research on
beat-timing and tempo is deep involvement with human perception and cognition of beat-timing and tempo.
A computational method of beat-timing and tempo by explaining the factor of understanding the results cancontribute to the investigation into a human's perception and cognition of beat-timing and tempo. The authors
will investigate the relationship between beat-timing and acoustics parameters for beat-timing and tempo
estimation. Beat-timing is given as timing data of classical music labeled by a student at the KunitachiCollege of music graduate school in Japan. Acoustics parameters were calculated with the length of the time
window by one-hundredth of a second. The results suggest that the most significant acoustic-parameter is
RMS.
Keywords: beat-timing, acoustic-parameter, deep learning, beat-timing estimation, LSTM

1. INTRODUCTION
In this report, "beat" refers to the beat in music. The beat in music is a unit of musical rhythm. The
beat-timing is treated as the switching time between beats when a piece of music is divided by beats.
A musical term similar in nuance to beat, the tempo is a measure of the speed of the music andrepresents the number of beats present in a minute.
Methods for estimating beat-timing and tempo have been proposed in the field of musical acoustics.
Nowadays, deep learning methods have been proposed to solve this problem. It is hard to explain the
result of deep learning since it is an obscure method of factors. However, research on beat-timing andtempo is deep involvement with human perception and cognition of beat-timing and tempo. A
computational method of beat-timing and tempo by explaining the factor of understanding the results
can contribute to the investigation into a human's perception and cognition of beat-timing and tempo.

2. AIMSIn this report, we focus on beat-timing, investigate the relationship between beat-timing andacoustic-parameters, and examine the acoustic-parameters that are significant for beat-timing
estimation.

3. METHOD
The first method is to confirm the correlation between beat-timing and acoustic-parameters. The
1
2

okemoto.madoka.ra4@st.kunitachi.ac.jp
miura.masanobu@kunitachi.ac.jp

second method uses an LSTM (Long Short-Term Memory) network [1] to examine the estimation
results when each acoustic-parameter is predicted as an explanatory variable and beat-timing data as
an objective variable.

4. USED DATA
The beat-timing data used in this report is the beat-timing data obtained by labeling beat times by
students of the Graduate School of the Kunitachi College of Music, and then using the data to assign
a flag of 1 if it is a beat and 0 if it is not a beat for every one-hundredth of a second.
The acoustic-parameter data in this report are the data of RMS (rms), zero -crossing (zcs),
spectrum-centroid (cnt), and spectrum-flux (flx), which are calculated every one-hundredth of asecond.
The music pieces for which beat-timing data were labeled and acoustic-parameters were calculated
were taken from 36 classical music pieces.

5. RESULTThe correlation

5.1 The correlation between beat-timing and acoustic-parameters
Figure 1 shows the correlation between beat-timing data and acoustic-parameters data. Figure 1
confirms that all four acoustic parameters have a low correlation with beat-timing data. Although allcorrelations are low, the correlation between rms and zcs tends to be high among these four.
0.008
0.006
0.004
0.002
0
-0.002-0.004-0.006-0.008
-0.01

rms

zcs

cnt

flx

The acoustic-parameters

Figure 1 - The correlation between beat-timing data and acoustic-parameters data
5.2 LSTM network to examine
This section describes the estimation results when using an LSTM network to predict each acoustic
parameter as an explanatory variable and beat-timing as an objective variable. The feedback of the
LSTM network is 50. Since it is 50 in one-hundredth of a second division, it is 0.5 seconds in time.
Table 1 shows the classification results using the LSTM network. Table 1 summarizes theclassification results into True Positive (TP), False Positive (FP), False Negative (FN) , and True
Negative (TN). Table 2 shows the values of Accuracy, Precision, Recall, Specificity, and F -measurecalculated from Table 1.
Table 2 confirms that the highest F-measure values are obtained when rms is used as the
explanatory variable. Although Precision is highest for cnt, F-measure tends to be low because Recall
is low.
Table 1 - The classification results using the LSTM network
TP

FP

FN

TN

rms

390

34

6644

390456

zcs

248

20

6786

390970

cnt

198

11

6836

390979

flx

236

26

6798

390964

Table 2 - Accuracy, Precision, Recall, Specificity, and F-measure calculated from Table 1
Accuracy

Precision

rms

0.983200.91981zcs

0.982900.92537cnt

0.98280flx

0.98286

RecallSpecificity

F-measure0.055440.99991

0.10459

0.03526

0.99995

0.067930.94737

0.028150.99997

0.054670.90076

0.03355

0.999930.06469

6. CONCLUDING REMARKSThis report researched relationship between beat-timing and acoustics-parameter for method ofbeat-timing and tempo estimation. As a result, the results of 5.1 and 5.2 suggest that rms may be theacoustic-parameter most closely related to beat-timing data among the four acoustic-parameters.
Future work will verify the effects of combinations of acoustics-parameter on the performance ofbeat-timing estimation.

ACKNOWLEDGEMENTS
This work was supported by JSPS KAKENHI Grant Number JP21J13539.

REFERENCE[1] Hochreiter, S., and J. Schmidhuber. "Long short-term memory." Neural computation. Vol. 9, Number 8,
pp.1735–1780 (1997).

ABS-0997

timbre.fun:
A gamified interactive system for crowdsourcing a timbre semantic vocabulary
Ben HAYES(1) , Charalampos SAITIS(2) , György FAZEKAS(3)(1) Queen Mary University of London, United Kingdom, [b.j.hayes, c.saitis, gyorgy.fazeaks]@qmul.ac.uk

ABSTRACT
We present timbre.fun (https://timbre.fun/), a web-based gamified interactive system where users create sounds in
response to semantic prompts (e.g., bright, rough) through exploring a two-dimensional control space that maps nonlinearly
to the parameters of a simple hybrid wavetable and amplitude-modulation synthesizer. The current version features 25
semantic adjectives mined from a popular synthesis forum. As well as creating sounds, users can explore heatmaps generated
from others’ responses, and fit a classifier (k-nearest neighbors) in-browser. timbre.fun is based on recent work, including by
the authors, which studied timbre semantic associations through prompted synthesis paradigms. The interactive is embedded
in a digital exhibition on sensory variation and interaction (https://seeingmusic.app/) which debuted at the 2021
Edinburgh Science Festival, where it was visited by 197 users from 21 countries over 16 days. As it continues runningonline, a further 596 visitors from 35 countries have engaged. To date 579 sounds have been created and tagged, which will
facilitate parallel research in timbre semantics and neural audio synthesis. Future work will include further gamifying thedata collection pipeline, including “leveling-up” to unlock new words and synthesizers, and a full open-source release.

Keywords: Timbre, Synthesis, Crowdsourcing, Gamification, Timbre semantics

1 INTRODUCTION
The scientific study of musical timbre has historically focused primarily on understanding the perceptual phenomenon itself(28). This has variously involved seeking acoustic explanations for experimental data (13; 19; 33), studying its cross-modal
associations (38; 15), and subjectively constructing systems for its description (25; 30). More recently, however, anotherapproach has emerged which seeks to understand the relationship between timbre and sound production (15; 34) throughprompted synthesis tasks. In these studies, participants are asked to produce a sound using a synthesiser in response toa prompt, such as a descriptive adjective. This yields a dataset of prompt-sound pairs which can be analysed to study
the effects of various prompts on the sounds created.
This approach is of particular interest from the perspective of synthesiser control. The problem of providing meaningful
controls for synthesisers has recurred in the literature since the late 70s (36; 27; 26), and has recently become of particularinterest with the advent of complex neural audio synthesisers capable of being used in music production and sounddesign workflows (14; 21; 7; 2). However, the data collected in these studies have been limited in both scope and scale.
Paired with the heterogeneity observed between timbral datasets (33), this has limited the usefulness of this data in such
downstream tasks. To address these issues, we propose an approach for crowdsourcing timbre semantic data in a mannerthat is amenable to scaling and gamification. Through an exploratory analysis, we demonstrate that the data collected ina preliminary run of this study exhibit an emergent structure which is broadly congruent with the findings of prior timbre
semantic research. We further observe, through the application of simple machine learning techniques, that the data collectedin this manner hold sufficient predictive power to allow the affective connotations of semantic prompts to be classified
from synthesiser parameters and acoustic features.

2 RELATED WORK
2.1 Prompted Synthesis
The standard paradigm for timbre semantic research involves listeners rating sounds along scales defined by descriptive
adjectives (24). Controlling characteristics of the stimuli allows their perceptual influence on timbre semantic associations to
be studied. This approach does not, however, provide insight into the inverse relationship: the influence of timbre perception
and its semantic associations on the process of sound design. Despite significant effort having been invested into developingadjective-controlled systems for these tasks (11; 12; 3; 32; 31; 9), this relationship has only recently begun to receive
attention in the psychoacoustic literature (34; 15).
In the present authors’ prior study (15), experienced sound designers programmed an FM synthesiser in response to
adjectival prompts, and rated their created sounds on semantic scales. In Wallmark, et al.’s study (34), classically trained
musicians interacted with a simplified 2D control space mapped to FM synthesiser parameters. In both studies, clear
structure emerged when the created sounds were analysed in terms of their acoustic characteristics, synthesis parameters,
and relationships to prompt words. Both similarities to and deviations from the findings of conventional timbre studies wereobserved, suggesting that the prompted synthesis approach is a viable method for studying the specifics of the “inverted”
relationship between timbre and sound design.
2.2 Crowdsourcing
Estellés-Arolas and González-Ladrón-de-Guevara (10), through a detailed meta-analysis, defined crowdsourcing as “a typeof participative online activity in which an individual, an institution, a non-profit organization, or company proposes to agroup of individuals of varying knowledge, heterogeneity, and number, via a flexible open call, the voluntary undertaking ofa task.” In this work, we view crowdsourcing as an alternative to lab-based or structured online methods of data collection,
where increasing the flexibility of participation enables the collection of data at much greater scale.
Adjerid & Kelley (1) discuss how the allure of “big data” makes crowdsourcing attractive in experimental sciences,where logistical constraints often limit sample sizes. Where large scale datasets were once out of reach to all but the mosthighly resourced labs, they point out that a new generation of tools paired with the ubiquity of internet connections hasbrought this approach to data collection within the realm of possibility of more modestly financed projects. They argue
that there are many benefits to collecting data at scale for multiple sub-fields of psychology, but go on to raise concerns
around the ethics, quality, and appropriateness of these approaches to data collection.
Nonetheless, crowdsourced data has been shown to have benefits that extend beyond those induced simply by the sizeof the dataset. Casler, et al. (4) demonstrated that crowdsourcing a dataset resulted in greater socio-economic and ethnic
diversity in the participant pool, whilst yielding almost indistinguishable test results from participants recruited in personor on social media. This suggests that crowdsourcing may be of particular interest in fields where cultural bias introduced
through the homogeneity of participants is impactful. In timbre research, for example, it cannot be safely assumed thatany perceptual phenomena observed are entirely devoid of cultural influence.
The similarity in group responses in Casler, et al.’s (4) study is encouraging in terms of the quality of data collected
online. Nonetheless, the relince of psychoacoustic research on consistent audio reproduction sets the bar somewhat higher.
To address this concern, Zacharakis, et al. (37) compared responses from a timbre dissimilarity study conducted in alaboratory, to those collected online. Timbre spaces computed from the two datasets showed very high configurational
similarity, suggesting that both groups responded to similar acoustic cues despite variability in listening apparatus.
Crowdsourcing has previously been successfully applied to sourcing acoustic-semantic vocabularies in the context ofmusic production. Specifically, Cartwright & Pardo (3) collected a set of descriptors of 40-band equalizer (EQ) curves fromwithin an interactive EQ interface. Subsequently, Stables, et al. (32) captured descriptors from within the music production
workflow by providing fully functional compressor, reverb, and EQ audio plugins.
2.3 Gamification
The term gamification broadly refers to the practice of imbuing a platform, application, or interface with motivational design
features borrowed from games (20). It is of particular interest in the design of crowdsourced experiments, as these rely
on a large body of participants being sufficiently motivated to provide responses. To address the ambiguity in terminology
and praxis in crowdsourcing and gamification, Morschheuser, et al. (20) proposed a simple system of four archetypes forclassifying gamified crowdsourcing systems, within which the present work would be described as a crowd-creating task.
Morschheuser, et al. state that a central challenge in designing a gamified crowdsourcing system is providing incentives
that promote “the formation of positive motivations towards crowdsourcing work” whilst also fitting the type of activity.
They further acknowledge that intrinsic motivation, such as that induced by tasks that “allow a participant to be creative andexperience autonomy”, can be dominated by the extrinsic motivation of social or financial reward, or perceived motivational
affordances (e.g. credits or in-game rewards) of the task. For these reasons, we were careful in designing the present study
to avoid rewards that might overshadow the central activity, or incentivise users to act quickly and carelessly. We took
particular inspiration from the approach of Stables, et al. (32), who motivated users to interact with their audio plugins
by providing access to the responses of other users. Further details of the choices we made are given in section 3.3.
With the growing number of online platforms for gamified crowdsourced data collection, the popularity of this approachis rapidly growing despite limited data as to its effectiveness (17). One study even found that gamification did not improveparticipant attrition rates for “effortful” and “unengaging” tasks (18). Further, Deterding, et al. (5) point out challenging,unanswered ethical questions, including whether the “playful veneer” of games and gamified systems leads users to share
more data than they otherwise would, and to what extent users’ effort might be considered an unfair use of their labour.
Despite these concerns, however, Keusch & Zhang (17) did observe a clear effect throughout the literature on psychologicaloutcomes such as fun, interest, and satisfaction.

3 METHOD
3.1 Participants
Due to the casual, gamified nature of the experiment, participants were not subject to any selection or screening process,nor was any demographic information collected. Each participant was, however, assigned a unique anonymous user ID
on loading the page which was stored with each response they submitted. On their first visit, users were asked if theywould allow a cookie to be stored on their computer to allow this user ID to persist between sessions. If the cookie wasdeclined, a new ID would be generated at each visit. At the time of writing, a total of 95 unique user IDs have beenrecorded, and a total of 579 responses submitted.
3.2 Web interface
The prompted synthesis interface was hosted online1. On arrival at the web page, visitors were given a brief explanationof the interface in informal language. Those who wished to learn more about the motivations behind the page were able toclick a link to read some more detaihled information. After reading the introductory directions, participants were presented
with a rectangular control space containing a black square. Clicking and dragging the black square around the space
produced a sound from a synthesiser built on the WebAudio API. As the square moved around the space, the sound of thesynthesiser changed in accordance with a control mapping. Above the control space was a short textual prompt, instructing
participants to create a sound matching a given adjective. The created sound was stored in a database alongside a timeseries of points representing the path taken through the control space.
3.3 Gamification & rewards
As discussed in section 2.3, we opted for a “light touch” in terms of gamification and rewards. In particular, to createa more relaxed, game-like experience, we opted for a more colourful user interface with informal language, including
the use of emoji, in the explanatory text. To maximise the intrinsic motivation of the task itself, we used language thatemphasised its creative and exploratory nature, and designed the control space mapping to be sufficiently nonlinear as
to prevent users from easily employing prior knowledge of sound synthesis. Finally, to provide a reward for creating sounds,
two further features were unlocked once five trials had been completed: (i) explore the responses of other users via heatmap
visualisations, (ii) fit a kNN classifier in-browser to predict the prompts associated with different sounds.
3.4 Word stimuli
The descriptive adjectives used in the textual prompts were taken from the ModWiggler corpus collected in a prior promptedsynthesis study (15). As the sounds visitors created lacked a clear amplitude envelope or sub audio rate modulation andwere thus effectively time invariant, we removed two words from the original list of 27 that clearly implied a particular
temporal evolution: percussive and plucky. For a detailed explanation of how these were sourced, see (15).
3.5 Synthesiser
To encourage participants to explore the synthesiser control space by listening to the sounds rather than relying on priorexpectations, we aimed to create a synthesiser and accompanying parameter mapping that fulfilled the following criteria:
1. sounds do not obviously resemble familiar instruments
2. control dimensions do not obviously map to familiar (e.g. FM or subtractive) synthesiser parameters
3. control dimensions do not ehibit an obvious linear mapping to any specific attribute of the soundThe resulting synthesiser design consisted of three components connected in series: an interpolated wavetable space,a ring modulator, and a bandpass biquad filter. Details of these components and the parameters they exposed are given
below, and an overall schematic is given in Fig. 1.
To allow the synthesiser to produce a variety of harmonic distributions without implying a familiar synthesis method,a wavetable oscillator was chosen as the initial sound source, with wavetables sampled using trilinear interpolation froma 2-dimensional grid. This allows for a direct mapping from the 2D control space to the wavetable.
1 The website is available at https://timbre.fun/2D wavetable space

Wavetable
co-ordinates

Ring modulator

Mod.
freq.

Mod.
gain

Bandpass filter

Centrefrequency

Figure 1. A schematic diagram of the hybrid wavetable and ring modulation synthesiser used in timbre.fun. The signal
is generated by the wavetable oscillator, and processed by the ring modulator and bandpass filter. Parameter values areprovided by nonlienar transformations of co-ordinates from the 2D control space.
This space of wavetables was sampled from a multilayer perceptron with sinusoidal activations (29) which was trained
to reproduce single cycles of audio from the NSynth dataset (8). The network’s inputs consisted of the x- and y-coordinatesin the 2D wavetable space, and a time index within the wavetable, represented in two dimensions as a point on a unit
circle to avoid discontinuities at the start and end of the wavetable. The x- and y-coordinate inputs for each training sample
were derived as a learned matrix projection from a one-hot representation of sample identity, thus effectively allowing an
unsupervised spatial organisation of wavetables to be learned. After training, the wavetable grid was sampled at a spatial
resolution of 64 x 64 and a temporal resolution of 384.
The wavetable space allowed for the production of purely harmonic spectra, but was not capable of producing inharmonic
signals. To address this, a ring modulator was added to the signal chain due to its ability to produce a finite number ofinharmonic partials and its low dimensional parameter space. In a ring modulated signal, each frequency component ωi inthe carrier signal is replaced by two frequency components ωi ±ωm where ωm is the modulator frequency. Ring modulation
can therefore be used to create a variety of effects from subtle beating through to complete destruction of the harmonic
structure of the signal. The modulation frequency and gain parameters were mapped to the 2D control space.
Prior research into the semantic correlates of timbral attributes (15; 38) suggests that a relationship exists between certain
verbal descriptors and acoustic features describing the shape or distribution of energy in the frequency spectrum. A notable
example is the relationship between descriptions of brightness and the spectral centroid (23), an audio feature describing theweighted mean of frequencies present in a signal. With the wavetable space and ring modulator alone, however, participants
would ave been unable to shape the broader distribution of spectral energy beyond the pre-existing variation in harmonic
amplitudes between
the various wavetables. For this reason, we introduced a biquadratic bandpass filter with a fixed√2
resonance of 2 to the end of the signal chain. Participants were thus able to more closely control the spectral distributionof their created sound. The centre frequency parameter was mapped to the 2D control space.
3.5.1 Control mapping
Previous prompted synthesis studies have allowed participants to interact both through full sets of synthesiser controls(15) and low dimensional mappings (34), in which synthesis parameters correspond directly to the x- or y-axis. These
approaches risk encouraging participants, especially those with existing synthesis experience, to rely on their intuition about
the workings of the synthesiser in formulating their responses. To ameliorate this effect, we opted to obscure the relationshipbetween the dimensions of the control space and the parameters of the underlying synthesiser (with the exception of thex- and y-coordinates in the wavetable space) by using a nonlinear mapping of the form:
pi =(0.5cos(ax+by+cxy)+0.5)2
where the parameters a, b, and c were unique to each synthesiser parameter. These parameters were manually tuned
to ensure a subjectively wide variety of sounds across the space, with no clear linear relationship between either axis andany synthesiser parameter. The values used in the experiment are listed in the source code repository2 alongside the final
ranges of the resulting parameters.
3.6 Experiment design
Prior timbral studies have typically followed a design that facilitates statistical hypothesis testing, or the application of otherrelevant data analysis techniques. In particular, stimuli are usually applied equivalently to all participants, or at least to
2 https://github.com/ben-hayes/crossmodal-synthesis

aggressivebigbright

clean

clear

complex

dark

deep

dull

gritty

hard

harsh

mellow

metallic

noisy

raw

rich

rough

sharp

smooth

sweet

thick

thin

warm

woody

Figure 2. Heatmaps computed from 2D kernel density estimates of participants’ responses in the control space per prompt.
those within a group. Whilst this is valuable from the perspective of drawing robust conclusions from the data, it constrains
the scope of data that can be collected due to the redundancy that is necessitated by these statistical techniques. The utilityof this data in applications that aim to use data-driven methods to build on research in timbre perception, such as synthesis
control, is therefore limited.
As the purpose of timbre.fun is not to directly test hypotheses, but rather to pilot a method for crowdsourcing timbre
data at scale with a view to supporting downstream applications, we opted to relax these usual constraints in favour ofcollecting a greater amount of more diverse data. Specifically, participants were able to complete as few or as many trials
as they wished and prompts were simply sampled from a uniform distribution, with no guarantee that each participant
responded to any specific subset of prompts.

4 RESULTS & DISCUSSION
Before analysis, the dataset was filtered to reduce the influence of low effort and accidental responses by removing any
sound for which three or fewer JavaScript mouse move events were recorded. This resulted in a dataset containing 468
responses from 93 participants. Due to the inherently unmatched and unbalanced nature of the collected data, we forgo
conventional statistical analysis in lieu of an exploratory analysis.
4.1 Control space density estimates
As an initial data exploration step, we visualised 2D kernel density estimates – with a standard bivariate normal – ofparticipants’ responses in the 2D control space, conditioned on the semantic prompt. These are shown in Fig. 2
For the majority of prompts, the resulting densities appear relatively diffuse and lack distinct modes, suggesting thatparticipants found suitable responses across large regions of the control space. Certain prompts, however, do appear tohave resulted in clearer agreement between participants: clear, dull, noisy, and sharp, for example, appear to have received
a large proportion of their responses in small regions of the control space.
High level similarities, congruent with the groupings of descriptors observed in previous timbre semantic work (15; 38; 6),
do also seem to emerge. For example, dark, deep, and dull appear to have most of their responses concentrated arounda similar region, while thin and thick primarily occupy opposing regions of the space. The nonlinear mapping of synthesis
parameters, however, means that any regions exhibiting similarity other than very distinct modes should be interpreted withcaution, as neighbouring areas of the control space could result in dramatically different sounds.
4.2 Effect of prompt on synthesis parameters
Fig. 3 illustrates the distribution of responses for two of the five synthesiser parameters – filter centre frequency andmodulator gain – conditional on the prompt given. In both cases the x-axis is sorted according to the median value per
prompt, and the colour scale is normalised to the full available range of the parameter. Plots for the remaining prompts

Filter Centre

10000

Median
3000

1000

1000
300
100

sh

ar
br p
ig
ht
th
i
ha n
m rsh
ag etagr llic
es
si
ve
no
is
y
gr
itt
ro y
ug
h
ra
w
ha
rd
co big
m
pl
ex
ric
h
th
ic
k
cl
ea
sw r
ee
t
da
rk
wa
r
w m
oo
d
cl y
ea
n
de
e
sm p
oo
th
du
m ll
el
lo
w

100

1.00

Mod GainMedian
1.00

0.75

0.75

0.50

0.50

0.25

0.25
0.00

ha
rd
no
is
ha y
rs
h
bi
g
t
co hick
ag mp
gr lex
es
si
v
ro e
ug
h
de
ep
gr
itt
y
da
rk
r ic
br h
i
m ght
et
al
lic
ra
w
sh
ar
wa p
r
w m
oo
dy
th
sm in
oo
m th
el
lo
w
cl
ea
sw r
ee
cl t
ea
n
du
ll

0.00

Figure 3. Effects of prompt on the filter centre frequency and modulator gain parameters.
are available online3.
Very pronounced differences between prompts are visible for the Filter Centre and Modulation Gain parameters. In
the case of Filter Centre, we see prompts including sharp, bright, thin, and harsh resulting in consistently high parametervalues, and deep, smooth, dull, and mellow resulting in lower ones. For the Modulation Gain, we see hard, noisy, harsh,
and big resulting in higher values, while clear, sweet, clean, and dull lead to lower values. These coarse groupings are, again,
consistent with the factor structure of our prior prompted synthesis study, in which sharp, bright, and harsh all exhibited theirhighest loadings on the same factor, and clean, clear, and sweet all exhibited their highest on another factor. The grouping
Effects on the remaining parameters (see online supplement for plots) are markedly weaker, with wider distributions
and less difference between median parameter values. Whilst the prompt effects on wavetable x- and y-coordinates appear
primarily to have resulted in diffuse distributions, some prompts do appear to have a resulted in a concentration of responses
around a particular value, suggesting that the wavetables at this point in the space may agree particularly well with thedescriptor. These prompts include sharp, gritty, and noisy.
4.3 Acoustic features analysis
Whilst synthesiser parameters offer a comprehensive description of how to reproduce a given sound from the dataset, they
provide only indirect insight into a sound’s acoustic characteristics, owing to their perceptual nonlinearity and properties
emerging from their interaction. We therefore extracted a set of acoustic features from all sounds created in the experiment
to more completely describe them.
A four second long WAV file was rendered for each sound using the same WebAudio synthesiser as found on thetimbre.fun website. Features were extracted from each audio file using the Timbre Toolbox (22). As the sounds lacked
any temporal evolution, features computed on the Temporal Energy Envelope representation were excluded. All otherrepresentations and features included in the Toolbox’s default configuration were used. By default, the Timbre Toolbox
reports two summary statistics aggregated over time: the median and interquartile range. Due, again, to the lack of temporalvariation in the sounds, we discared the interquartile range and retained only the median.
As many of the computed features measure very similar characteristics, we removed columns with high pairwise
correlations in order to avoid the analysis being biased by over-representation of certain properties. In particular, for every
pair of features with a Spearman correlation coefficient of greater than 0.9, we removed the feature with the higher mean
absolute correlation across all remaining features. The list of remaining features is given in Table 1.
Principal components analysis was then applied to the remaining features. Parallel analysis (16) was used to select anappropriate number of components. This procedure involves repeating PCA on a number of randomly sampled, uncorrelated
datasets of equivalent size, and retaining n components, where the n-th component in the real dataset is the component
with the lowest eigenvalue that still exceeds a given percentile of the eigenvalues of the n-th components of the random
3 See online supplement: https://benhayes.net/timbre-fun/Table 1. Acoustic features remaining after removing collinear pairs and their loadings onto the first three principal
components. Loadings with magnitude >0.7 are depicted in bold. See (22) for detailed descriptor explanations.
Feature

PC1

PC2

PC3

Rep.

Feature

PC1

PC2

PC3

Harm.

Spec. Crest

0.893

0.009

0.011

Harm.

F0

0.176

−0.385−0.756

Harm.

Spec. Skewness

0.858−0.135

0.054

Harm.

Spec. Slope−0.688−0.269−0.258

STFT

Spec. Skewness

0.817−0.181

0.313

ERB

Spec. Crest

0.566

0.250

−0.399Harm.

Spec. Decrease−0.7620.315−0.224Harm.

Inharmonicity−0.488

0.331

0.571

ERB

Spec. Flatness−0.715−0.466

0.058

STFT

Spec. Flux

−0.371

0.317

0.356

Harm.

Harm. Spec. Dev.

0.254

0.860

−0.033

Audio

Auto-correlation−0.3450.243−0.330Harm.

Noise Erg.

0.457

0.797

−0.125Harm.

Odd/Even Ratio

0.275−0.132

0.087

ERB

Frame Erg.

−0.3520.730−0.151

Harm.

Spec. Flux

−0.0890.249−0.286

Proportion

Rep.

0.75
Cluster

0.50

1
2

0.25

cl

bi

g
co ea
m r
pl
ex
ra
w
gr
itt
y
h
m ard
et
al
li
ha c
rs
ro h
ug
br h
ig
ht
th
in
ag no
i
gr sy
es
si
v
sh e
ar
p

m

el

lo
w
wa
rm
cl
ea
n
du
l
de l
sm ep
oo
th
da
rk
th
ic
w k
oo
dy
r ic
sw h
ee
t

0.00

Figure 4. Assignments of prompt words to each of the two acoustic clusters. The leftmost (yellow) bar for each descriptor
corresponds to cluster 1, and the rightmost (green) bar to cluster 2.
datasets. We ran the procedure for 480 iterations and found that three components exceeded the 99th percentile. Loadingsof the first three components are reported in Table 1.
The resulting principal components were then subject to k-means clustering. A two-cluster solution was selected as
optimal by computing the connectivity, Dunn index, and silhouette coefficient. For each of the 25 prompt words, we
computed the proportion of sounds created in response to that word in each acoustic cluster. These are shown in Fig. 4.
Note that the proportion was used in lieu of the raw count owing to the unbalanced design of the experiment. The majorityof prompts appear to strongly associated with a specific cluster, with 19 prompts having at least twice as many responses
in one cluster than the other.
We note that descriptors that have previously been associated with distinct semantic factors are grouped together inthese clusters. For example, cluster 2 contains the majority of responses for sharp, thin, and rough, whilst these terms load
strongly onto two distinct factors in our prior study (15) and onto three distinct factors in Zacharakis, et al.’s prior work(38). Instead of representing distinct timbral dimensions, the acoustic clusters instead appear to correspond to the opposite
ends of an aggregation of timbral scales. This emergence of two timbral “poles” in this manner is somewhat consistent
with the behaviour observed in our previous study (15) where, despite post-hoc semantic ratings supporting a five factor
solution, correlations between semantic factors and synthesiser parameters appeared to form just two groups.
4.4 Word affectUsing the word affect norms collected by Warriner, et al. (35), valence, arousal, and dominance scores were obtained
for all 25 prompts. In all cases except one the exact word form was found in the dataset. In the one case where thiswasn’t possible (woody), the closest adjectival form that shared a lemma (wooden) was used in its place. Each of thethree dimensions was then discretised into two classes: high for those above the mean, and low for those below. Two classes
were used instead of the three used by Wallmark, et al. (34) to enable both direct comparison with our acoustic clusters
and the formulation of binary classification problems (see Section 4.5).
To examine the relationship between these affect classes and the clusters obtained from acoustic features, prompts were

Table 2. Test-set accuracy for all SVM models. Columns signify input features and rows signify targets. p-values takenfrom binomial test with no-information rate as null hypothesis. *: p<0.05, ***: p<0.001
External Cluster Validation

SVM Test Accuracy (%)

Dimension

Rand

Jaccard

Fowlkes-Mallows

Synth. Params

Acoustic PCs

Valence

0.667

0.485

0.653

61.3

62.4

Arousal

0.847

0.725

0.840

73.1***

71.0***

Dominance

0.667

0.485

0.653

53.8

62.4*

assigned to the acoustic cluster which contained a greater proportion of their responses, and three external validation metrics– the Rand, Jaccard, and Fowlkes-Mallows indices – were computed between these assignments and each of the three affectdimensions in turn. The results of these metrics are presented in Table 2. Whilst low to moderate values were observed
for the valence and dominance classes, the arousal classes showed very high similarity to the separation of prompts between
acoustic clusters. Whilst neither the data nor analysis are sufficient to establish a causal relationship, these results arestrongly suggestive of an interaction between prompt arousal and the acoustic characteristics of the resulting sound.
4.5 Affect classification with support vector machines
As discussed previously, the structure of this dataset precludes conventional statistical hypothesis testing. Therefore, tofurther examine the relationship between the created sounds and the affective connotations of the prompt words we instead
opted to fit a predictive machine learning model. In particular, we used a support-vector machine (SVM) with radial basis
functions. We trained separate models on both acoustic principal components and the raw synthesiser parameters.
Data was partitioned into 80% training and 20% test subsets. Using this split, six models were fit — one for eachcombination of affect dimension and feature set (where synth parameters and acoustic principal components were the two
feature sets). SVM hyperparameters were tuned using the adaptive random search implemented in the caret package forR, following a 10-fold cross validation with 5 repeats of each fold. To ensure an even class balance, training folds wereupsampled by randomly repeating samples from the less represented class.
Table 2 lists the test-set accuracy of each trained SVM. Statistical significance of the accuracy was determined usinga binomial test, where the no-information rate of the test dataset (i.e. the raw class proportions) was used as the null
hypothesis. Consistently with the external validation metrics, the test accuracy observed for arousal classes was highly
statistically significant, further supporting an affect-timbre interaction.
These results are distinct from those of Wallmark, et al. (34), who did not observe a significant effect of arousal on
acoustic principal components. Deeper comparison of these results is challenging due to methodological differences, but sucha clear deviation from the particular timbre-affect relation observed in their work is nonetheless noteworthy. In particular,
it both lends further weight to the role of affect in prompted timbre production, whilst also suggesting that the mechanismof this interaction may to some extent be influenced by the method of synthesis available.

5 CONCLUSION
This paper presented a method for sourcing a timbre semantic vocabulary of prompt-sound pairs at scale using a gamified
crowdsourced approach. A pilot run of the method, debuted as part of a digital exhibition on sensory variation andinteraction at the 2021 Edinburgh Science Festival, resulted in a total of 579 sounds being created by 95 users. Exploratory
analysis of the collected data suggests that responses showed logical consistency with prior timbre research, and exhibit
sufficient structure to allow certain affective connotations of prompt words to be predicted from acoustic characteristicsof the created sounds. These results support further work on gamified crowdsourced collection of timbral datasets, witha view to producing large datasets for use in downstream tasks, including descriptive synthesiser control.
In future work, we will augment the data collection pipeline by introducing new sound synthesis methods, new typesof interaction (including tagging, dissimilarity rating, and more), and further gamification elements (such as a reward/points
system, and the ability to unlock new features). We will also conduct a run of the study at a larger scale, and provide
a full open-source release of the platform so that it can be adapted for other data collection tasks.

ACKNOWLEDGEMENTS
This work was supported by UK Research and Innovation [grant number EP/S022694/1].

REFERENCES[1] I. Adjerid and K. Kelley. Big data in psychology: A framework for research advancement. American Psychologist,
73(7):899–917, Oct. 2018.
[2] A. Caillon and P. Esling. RAVE: A variational autoencoder for fast and high-quality neural audio synthesis.
arXiv:2111.05011 [cs, eess], Dec. 2021. arXiv: 2111.05011.
[3] M. B. Cartwright and B. Pardo. Social-eq: Crowdsourcing an equalization descriptor map. In Proceedings of the
14th international society for music information retrieval conference, pages 395–400, Jan. 2013.
[4] K. Casler, L. Bickel, and E. Hackett. Separate but equal? A comparison of participants and data gathered via Amazon’s
MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior, 29(6):2156–2160, Nov. 2013.
[5] S. Deterding, A. Canossa, C. Harteveld, S. Cooper, L. E. Nacke, and J. R. Whitson. Gamifying Research: Strategies,
Opportunities, Challenges, Ethics. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human
Factors in Computing Systems, pages 2421–2424, Seoul Republic of Korea, Apr. 2015. ACM.
[6] T. M. Elliott, L. S. Hamilton, and F. E. Theunissen. Acoustic structure of the five perceptual dimensions of timbre
in orchestral instrument tones. The Journal of the Acoustical Society of America, 133(1):389–404, Jan. 2013.
[7] J. Engel, L. H. Hantrakul, C. Gu, and A. Roberts. DDSP: Differentiable Digital Signal Processing. In 8th International
Conference on Learning Representations, Addis Ababa, Ethiopia, Apr. 2020.
[8] J. Engel, C. Resnick, A. Roberts, S. Dieleman, M. Norouzi, D. Eck, and K. Simonyan. Neural audio synthesisof musical notes with WaveNet autoencoders. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pages 1068–1077, Sydney, Australia, Aug. 2017.
[9] P. Esling, N. Masuda, A. Bardet, R. Despres, and A. Chemla-Romeu-Santos. Flow Synthesizer: Universal Audio
Synthesizer Control with Normalizing Flows. Applied Sciences, 10(1):302, Dec. 2020.
[10] E. Estellés-Arolas and F. González-Ladrón-de Guevara. Towards an integrated crowdsourcing definition. Journalof Information Science, 38(2):189–200, Apr. 2012.
[11] R. Ethington and B. Punch. SeaWave: A System for Musical Timbre Description. Computer Music Journal, 18(1):30,
1994.
[12] A. Gounaropoulos and C. Johnson. Synthesising timbres and timbre-changes from Adjectives/Adverbs. In Applicationsof evolutionary computing. EvoWorkshops 2006, pages 664–675, Berlin, Heidelberg, Apr. 2006.
[13] J. M. Grey. Multidimensional perceptual scaling of musical timbres. The Journal of the Acoustical Society of America,
61(5):1270–1277, 1977.
[14] B. Hayes, C. Saitis, and G. Fazekas. Neural Waveshaping Synthesis. In Proceedings of the 22nd International Society
for Music Information Retrieval Conference, Online, Nov. 2021.
[15] B. Hayes, C. Saitis, and G. Fazekas. Disembodied Timbres: A Study on Semantically Prompted FM Synthesis.
Journal of the Audio Engineering Society, 70(5):373–391, May 2022.
[16] J. L. Horn. A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2):179–185, June 1965.
[17] F. Keusch and C. Zhang. A Review of Issues in Gamified Surveys. Social Science Computer Review, 35(2):147–166,
Apr. 2017.
[18] J. Lumsden, A. Skinner, D. Coyle, N. Lawrence, and M. Munafo. Attrition from Web-Based Cognitive Testing: A Repeated Measures Comparison of Gamification Techniques. Journal of Medical Internet Research, 19(11):e395, Nov. 2017.
[19] S. McAdams, S. Winsberg, S. Donnadieu, G. De Soete, and J. Krimphoff. Perceptual scaling of synthesized musical timbres: Common dimensions, specificities, and latent subject classes. Psychological Research, 58(3):177–192, Dec. 1995.
[20] B. Morschheuser, J. Hamari, J. Koivisto, and A. Maedche. Gamified crowdsourcing: Conceptualization, literature
review, and future agenda. International Journal of Human-Computer Studies, 106:26–43, Oct. 2017.

[21] J. Nistal, C. Aouameur, I. Velarde, and S. Lattner. DrumGAN VST: A Plugin for Drum Sound Analysis/Synthesis
With Autoencoding Generative Adversarial Networks, June 2022. arXiv:2206.14723 [cs, eess].
[22] G. Peeters, B. L. Giordano, P. Susini, N. Misdariis, and S. McAdams. The Timbre Toolbox: Extracting audio
descriptors from musical signals. The Journal of the Acoustical Society of America, 130(5):2902–2916, Nov. 2011.
Publisher: Acoustical Society of America.
[23] C. Saitis and K. Siedenburg. Brightness perception for musical instrument sounds: Relation to timbre dissimilarity
and source-cause categories. The Journal of the Acoustical Society of America, 148(4):2256–2266, Oct. 2020.
[24] C. Saitis and S. Weinzierl. The Semantics of Timbre. In K. Siedenburg, C. Saitis, S. McAdams, A. N. Popper,
and R. R. Fay, editors, Timbre: Acoustics, Perception, and Cognition, volume 69, pages 119–149. Springer International
Publishing, Cham, May 2019. doi: 10.1007/978-3-030-14832-4_5.
[25] P. Schaeffer, C. North, and J. Dack. Treatise on musical objects: essays across disciplines. Number 20 in California
studies in 20th-century music. University of California Press, Oakland, California, 2017.
[26] A. Seago. A New Interaction Strategy for Musical Timbre Design. In S. Holland, K. Wilkie, P. Mulholland, and
A. Seago, editors, Music and Human-Computer Interaction, pages 153–169. Springer London, London, 2013. Series
Title: Springer Series on Cultural Computing.
[27] A. Seago, S. Holland, and P. Mulholland. A Critical Analysis of Synthesizer User Interfaces for Timbre. In
Proceedings of the XVIII British HCI Group Annual Conference HCI 2004, volume 2, pages 105–108, Bristol, UK,
Sept. 2004. Research Press International.
[28] K. Siedenburg, C. Saitis, and S. McAdams. The Present, Past, and Future of Timbre Research. In K. Siedenburg,
C. Saitis, S. McAdams, A. N. Popper, and R. R. Fay, editors, Timbre: Acoustics, Perception, and Cognition, pages
1–19. Springer International Publishing, Cham, May 2019.
[29] V. Sitzmann, J. N. P. Martel, A. W. Bergman, D. B. Lindell, and G. Wetzstein. Implicit Neural Representations with
Periodic Activation Functions. In Advances in Neural Information Processing Systems, volume 33. Curran Associates,
Inc., June 2020.
[30] D. Smalley. Spectromorphology: explaining sound-shapes. Organised Sound, 2(2):107–126, Aug. 1997.
[31] R. Stables, B. De Man, S. Enderby, J. Reiss, T. Wilmering, and G. Fazekas. Semantic description of timbral
transformations in music production. In ACM multimedia, oct. 15-19, amsterdam, netherlands, pages 337–341, Oct. 2016.
[32] R. Stables, S. Enderby, B. De Man, G. Fazekas, and J. D. Reiss. SAFE: A system for extraction and retrieval ofsemantic audio descriptors. In Proceedings of the 15th international society for music information retrieval conference,
Taipei, Taiwan, Oct. 2014.
[33] E. Thoret, B. Caramiaux, P. Depalle, and S. McAdams. Learning metrics on spectrotemporal modulations reveals
the perception of musical instrument timbre. Nature Human Behaviour, Nov. 2020.
[34] Z. Wallmark, R. J. Frank, and L. Nghiem. Creating novel tones from adjectives: An exploratory study using FM
synthesis. Psychomusicology: Music, Mind, and Brain, 29(4):188–199, July 2019.
[35] A. B. Warriner, V. Kuperman, and M. Brysbaert. Norms of valence, arousal, and dominance for 13,915 English
lemmas. Behavior Research Methods, 45(4):1191–1207, Feb. 2013.
[36] D. L. Wessel. Timbre Space as a Musical Control Structure. Computer Music Journal, 3(2):45, June 1979.
[37] A. Zacharakis, B. Hayes, C. Saitis, and K. Pastiadis. Evidence for timbre space robustness to an uncontrolled online
stimulus presentation. In Proceedings of the 2nd International Conference on Timbre, Thessaloniki, Greece (Online),
Sept. 2020.
[38] A. Zacharakis, K. Pastiadis, and J. D. Reiss. An Interlanguage Study of Musical Timbre Semantic Dimensions andTheir Acoustic Correlates. Music Perception: An Interdisciplinary Journal, 31(4):339–358, Apr. 2014.

ABS-0249

Aeroacoustic analysis of oboe reeds with compressible direct numericalsimulation
Yuki NAKAHARA(1) , Rei SUMITA(1) , Ryoya TABATA(1) , Sho IWAGAMI(1) , Takeshi NANRI(2) , Taizo KOBAYASHI(2) , Yuji
HATTORI(3) and Kin’ya TAKAHASHI(1)(1) Department of Physics and Information Technology, Kyushu Institute of Technology, Japan, takahasi@phys.kyutech.ac.jp(2) Research Institute for Information Technology, Kyushu University, Japan, tkoba@cc.kyushu-u.ac.jp(3) Institute of Fluid Science, Tohoku University, Japan, yuji.hattori@tohoku.ac.jp

ABSTRACT
A two-dimensional model of an oboe reed is studied numerically with a direct numerical simulation (DNS) of
the compressible Navier-Stokes equations to investigate the sound generation mechanism from the viewpoint of
aeroacoustics. The numerical tool is extremely accurate due to the smallest mesh size on the order of micrometers and successfully reproduces the details of fluid motion and acoustic vibrations inside and outside the reed.
Particular attention is paid to the effect of reed vibration on the sound generation mechanism. When the reeds
are fixed and a periodically varying flow is injected through the fixed reed slit, an aerodynamics sound createdinside the reeds is an almost monotone including a few overtones. On the other hand, when a flow is injected
through periodically vibrating reeds from an oral cavity, more overtone components are observed and the pressure waveforms are similar to those observed in the experiment. This indicates that the richness of the overtonesof the double-reed instrument is mainly attributed to the aerodynamic sound created by the flow injected throughvibrating reeds and the bore, a linear resonator, just enhances characteristics of the instrument, e.g., formant.

Keywords:DNS,Double reed,oboe

1Introduction

The study of sound generation mechanism of double-reed instruments such as the oboe and bassoon, is an
important problem in the field of musical acoustics[1, 2, 3, 4]. However, the number of publications on
double-reed instruments is much less than those on single-reed instruments and lip-reed instruments, and noefficient model used widely has been proposed such as the Schumacher model for single-reed instruments andthe Adachi-Sato model for lip-reed instruments [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]. This is mainly
due to the complexity of sound generation process evolving nonlinear motions of compressible fluid. The double reed has a narrow channel in which an airflow must obey the governing equations of compressible fluid andthe effect of fluid motions is not negligible, for example, Bernoulli’s force acting on the inner surfaces of thereeds. An inlet flow injected from an oral cavity passes through the slit of vibrating reeds and is changed into
acoustic pressure due to the interaction with the vibrating reeds. Thus, acoustic waves generated in double-reeds
are regarded as aerodynamic sound, and one needs to address the problem of where and how acoustic vibrations
arise from a fluid flow injected through vibrating reeds.
One of the important tools for attacking this problem is a numerical analysis with a compressible fluid
scheme. Several numerical schemes for compressible fluid, e.g., Lattice Boltzmann Method (LBM), Large Eddy
Simulation (LES) and Direct Numerical Simulation (DNS), have been used for the study of wind instruments[12,13, 16, 17, 18]. Compressible DNS is the most reliable tool, but requires a huge numerical cost[19]. In thispaper, we numerically studied a two-dimensional model of an oboe reed with a compressible DNS solver toinvestigate the sound generation mechanism, when blowing a real oboe reed alone. To achieve this work, weconstructed a 2D model of an oboe reed attached to an oral cavity. First, we checked the reproducibility of ournumerical method. Then, we focused on the effect of reed vibration on the sound generation process. Thus, toclarify the effect of vibrating reeds, we investigated the fluid motion and generated sound in detail comparing
with the results when the reed was driven with a periodic flow injected through the fixed reed slit.

2Model and Numerical Method

Figure 1 shows the dimensions of the two-dimensional model of the oboe reed accompanied by an oral cavity.
We considered the following three cases by using this two-dimensional model. In Case 1, we determined theresonance frequency of the oboe reed in the following way. That is, a Poiseuille flow with a maximum velocityof V = 10 m/s was injected from the oral cavity, and the resonant frequency of the oboe reed was measured byobserving the frequency of pressure-decay oscillations in an exterior domain. In Case 2, the model was driven
by an alternating current-like periodic flow to mimic the flow injected through an oscillating reed slit. Then,the velocity of the flow at the left end of the oral cavity was given by
V = V0 (sin(2π fH t) + 1),(1)

where V0 = 10m/s and fH is the resonance angular frequency obtained in Case 1. In addition, the oboe reed was
also driven at A5 (880 Hz), which was observed when blowing a real oboe reed alone. In Case 3, the effectof reed vibration was considered. As shown in Figure 2, to mimic the oscillations of the reed plates, portionsof the plates of 1mm length from the reed tip were vibrated against each other with a period TH = 1/ fH with
opening and closing the slit between them, and a constant flow of V = 10m/s was injected from the left end of
the oral cavity.
The compressible direct numerical simulation (DNS) developed by Komatsu et al. was employed for thenumerical calculations[19]. The two-dimensional compressible Navier-Stokes equations are solved by using ahigher-order difference scheme with the volume penalization (VP) method. The entire computational domain is2000 × 2000mm2 in all cases. Calculations were performed up to 0.01 second in each case. Table 1 shows
the mesh parameters and time steps. Figure 3 shows a magnified view of the mesh near the oboe reed withobservation point P. For convenience of constructing mesh, the oboe reed and oral cavity were embedded in asolid body. Table 2 shows the physical parameters of the compressible fluid. The simulations were performed
on one node with 36 cores (36 parallel threads) of the high-performance computer cluster ITO (subsystem A)
at Kyushu University, and each calculation took approximately 40 days.

Figure 1. Two-dimensional model:Oh = 10mm,Sh = 1mm,L = 71mm,D = 5mm

Table 1. Mesh parameters and time step
Number of grids

Minimum grid size

Time step ∆t2806 × 2480

0.737 × 10−2 mm

1.8 × 10−9 sFigure 2. Vibrating portion of the reeds.

Figure 3. Model of reed embedded in the wall.
Table 2. Physical parameters of compressible fluid
parameters
values3

Density at rest ρ03

1.162kg/m

Viscosity µ

Sound speed c0

1.846 × 10−5 Pa · s

347.127m/s

Numerical results

3.1 Case1
Figure 4 shows (a) the velocity distribution and (b) the vorticity distribution near the reed at t = 0.01s. Figure 5
shows (a) the pressure distribution in the whole computational domain at t = 0.01s, (b) the change in pressure
at observation point P, and (c) a frequency-resolved graph of the pressure at observation point P. As shown in
Figures 4 (a) and (b), the incoming airflow formed a wavy jet inside the reed and vorticity took large values
near the tip of the reed. The wave jet should be transformed into a sound wave. However, as shown in Figure 5(a) and (b), the acoustic wave radiated from the open end of the reed was not a regular spherical wave and theacoustic oscillation observed at the point P was not stationary, but rather gradually decayed being accompanied
by noise components. This is because the constant flow cannot maintain a steady oscillation with a constant
pitch. Nevertheless, the resonance frequency could be calculated from the Fourier spectrum of the pressure
fluctuation and was estimated as fH = 2514Hz, which is 200Hz higher than the resonance frequency theoreticallyestimated for a conical pipe of the same length, 2314.18 Hz. Figure 6 shows the pressure distributions along thecenter line of the reed at representative observation times. Large negative values of pressure were observed near
the tip of the reed. These negative values were mostly caused by strong vortices near the tip. Since vortices
permanently existed near the tip, the envelopes of these graphs did not significantly change depending on theobservation times.
(a)

(b)

Figure 4. Velocity and vorticity distributions at t = 0.001s for Case 1. The x-axis and y-axis are in mm. (a)Velocity distribution. The scale of the color bar is in m/s. (b)Vorticity distribution. The scale of the color bar
is in 1/s.

(a)

(b)

(c)

Figure 5. Pressure oscillations and power spectra for Case 1. (a) Pressure distribution at t = 0.001s. The x-axis
and y-axis are in mm. The scale of the color bar is in Pa. (b) Pressure oscillation at the observation points P.
(c) Power spectra of the pressure oscillation observed in (b).

3000

t=0.0057
t=0.0059
t=0.0060

2000

p [pa]

1000

0

-1000

-2000

-3000

10

20

30

40

50

60

70

80

x [mm]

Figure 6. Pressure distributions along the center line of the reed for Case 1.
3.2 Case2
Figure 4 shows (a) the velocity distribution and (b) vorticity distribution near the reed at t = 0.01s, when thedriving frequency is set at fH = 2514Hz. Figure 5 shows (a) the pressure distribution in the whole computational domain at t = 0.01s and at fH = 2514Hz, (b) the changes in pressure at observation point P at fH = 2514
and 880Hz, and (c) power spectra of pressure oscillations observed in (b). Compared to Case1, no significant
differences were observed in the flow velocity and vorticity distributions, but the behavior of pressure oscillations was much different from that for Case 1. Indeed, a regular and stationary spherical wave is radiated fromthe open end of the reed. The pressure oscillations observed at the point P were regular and periodic with large
amplitude for both driving frequencies and their wave forms were similar to a sinusoidal wave. The oscillation
at fH = 2514Hz was larger in amplitude than that at fH = 880Hz, because the former was the resonance frequency of the reed. From the Fourier spectra, for the case of fH = 2514Hz, the peak of the fundamental mode
was dominant and the peak of the second harmonic was one digit less than the fundamental peak. For the caseof fH = 880Hz, the peak of the fundamental mode was dominant and the second harmonic was not observed
clearly. Figure 9 shows the pressure distributions along the center line of the reed at representative observation
times for the case of fH = 2514Hz. Except in a neighborhood of the tip, where strong vortices existed, wave
forms observed are regarded as snapshots of the standing wave of the fundamental mode.
(a)

(b)

Figure 7. Velocity and vorticity distributions at t = 0.001s for Case 2, when the driving frequency is fH =2514Hz. The x-axis and y-axis are in mm. (a) Velocity distribution. The scale of the color bar is in m/s. (b)Vorticity distribution. The scale of the color bar is in 1/s.

(a)

(b)

(c)

Figure 8. Pressure oscillations and power spectra for Case 2. (a) Pressure distributions at t = 0.001s and at
fH = 2514Hz. The x-axis and y-axis are in mm. The scale of the color bar is in Pa. (b) Pressure oscillations
at the observation points P at fH = 2514 and 880Hz. (c) Power spectra of the pressure oscillations in (b).
3000

t=0.0057
t=0.0059
t=0.0060

2000

p [pa]

1000

0

-1000

-2000

-3000

10

20

30

40

50

60

70

80

x [mm]

Figure 9. Pressure distributions along the center line of the reed for Case 2.
3.3 Case3
Figure 10 shows (a) the velocity distribution and (b) the vorticity distribution near the reed at t=0.01s and at
fH = 2514Hz for Case 3. Figure 11 shows (a) the pressure distribution in the whole computational domain at
t=0.01s and at fH = 2514Hz (b) the changes in pressure at observation point P at fH = 2514 and 880Hz, and (c)Fourier spectra of the pressure oscillations in (b). As shown in Figure 10 (a) and (b), the flow injected from theoral cavity formed a wavy jet and vortices were observed in a neighborhood of the tip of the reed, but the wave
jet gradually decayed and seemed to change into an acoustic oscillation in the middle of the reed. Actually,
vorticity decayed quickly. As shown in Figure 11 (a) and (b), an acoustic wave radiated from the open end of
the reed was stronger than that for Case 2 at fH = 2514Hz, and pressure oscillations at the point P were also
larger in amplitude than those for Case 2 at fH = 2514 and 880Hz. Note that the oscillation at fH = 2514Hz
was larger in amplitude than that at fH = 880Hz due to the same reason for Case 2. Furthermore, the wave form
at fH = 2514Hz was apparently deformed from a sinusoidal wave due to the effect of the vibrating reed plates,
and the wave at fH = 880Hz was much different from a sinusoidal wave and seemed to include the 2nd and 3rd
harmonic components. As shown in Figure 11 (c), such wave forms included rich harmonic components andare often observed for conical instruments. Thus, the vibration of reeds together with the oral cavity plays animportant role in the generation of harmonic components. Figure 12 (a) shows the pressure distributions alongthe center line of the reed at representative observation times for the case of fH = 2514Hz. Restively sharp kinksof the wave were observed and seemed to travel along a semi-oval shape in time evolution. These wave forms
are similar to the right half of the Helmholtz wave shown in Fig.12(b), which is observed in the vibration ofa string of string instruments. Such wave forms similar to the Helmholtz wave were also numerically obtainedwith the delay differential model of instruments with a conical bore [11].
(a)

(b)

Figure 10. Velocity and vorticity distributions at t = 0.001s and at fH = 2514Hz for Case 3. The x-axis andy-axis are in mm. (a) Velocity distribution. The scale of the color bar is in m/s. (b) Vorticity distribution. The
scale of the color bar is in 1/s.

(a)

(b)

(c)

Figure 11. Pressure oscillations and power spectra for Case 3. (a) Pressure distributions at t = 0.001s and at
fH = 2514Hz. The x-axis and y-axis are in mm. The scale of the color bar is in Pa. (b) Pressure oscillations
at the observation points P at fH = 2514 and 880Hz. (c) Power spectra of the pressure oscillations in (b).

(a)

(b)
3000

t=0.0057
t=0.0059
t=0.0060

2000

p [pa]

1000

0

-1000

-2000

-3000

0

10

20

30

40

50

60

70

x [mm]

Figure 12. Waves in the reed. (a)Pressure distributions along the center line of the reed for Case3. (b)Schematic view of Helmholtz wave.

4

CONCLUSIONSIn this paper, a two-dimensional model of an oboe reed was numerically studied using a two-dimensional Direct
Numerical Simulation (DNS) of the compressible Navier-Stokes equations to investigate the sound generation
mechanism in terms of aeroacoustics. Particular attention was paid to the effect of reed vibration on the soundgeneration process, and the following facts were found concerning the composition of harmonic components insound waves. When the reed was driven with a periodic flow injected through the fixed reed slit with its resonance frequency or a frequency less than it, the sound wave emitted from it contained negligibly small harmoniccomponents except the fundamental. On the other hand, when the inlet flow was injected from the oral cavity
through the vibrating reed slit, the sound wave radiated from the open end contained rich harmonic components,and similar waves are often observed for real conical instruments with a double reed. This fact indicates thatthe richness of harmonic components in sound waves generated by double reed instruments such as the oboe ismainly attributed to the nonlinear interaction between the injected flow and vibrating reed plates, which causes
an aerodynamic sound. The bore of the instrument, a linear resonator, only enhances the characteristics of the
an individual instrument such as the formant for generated tones.
Unfortunately, the problem of where and how acoustic vibrations arise from a fluid flow injected throughvibrating reeds has not been addressed. To solve this problem, an approach based on aero-acoustic theory, suchas Lighthill’s acoustic analogy, is needed. Furthermore, the vibrating parts of the reed model take unrealistic
shapes for convenience of numerical setup. Therefore, the model should be improved to reproduce more realistic
reed vibrations and to investigate sound generation processes in detail. Further development in these directions
will be the subject of future work.

ACKNOWLEDGEMENTS
The present work was supported by Grant-in-Aid for Scientific Research (C) Nos. 19K03655 from the Japan
Society for the Promotion of Science (JSPS) and “Joint Usage/Research Center for Interdisciplinary Large-Scale
Information Infrastructures” and “High Performance Computing Infrastructure” in Japan (Project ID: jh220001).
Part of the work was carried out under the Collaborative Research Project of the Institute of Fluid Science,
Tohoku University.

REFERENCES[1] Fletcher NH, Rossing TD. The Physics of Musical Instruments, 2nd ed. New York, USA: Springer-Verlag;
1998.
[2] McIntyre ME, Schumacher RT, Woodhouse J. On the oscillations of musical instruments. J Acoust Soc Am.
1983; 74: 1325-1345.

[3] Kergomard J, Weinreich G. Elementary considerations on reed-instrument oscillations in Mechanics of Musical Instruments. Springer. 1995; 230-290[4] Fabre B, Gilbert J, Hirschberg A, Pelorson X. Aeroacoustics of musical instruments. Annu. Rev. Fluid
Mech. 2011; 44: 1-25.
[5] Plitnik GR, Strong WJ. Numerical method for calculating input impedances of the oboe. J Acost Soc Am.
1979; 65: 816-825.
[6] Shimizu M, Naoi T, Idogawa T. Vibrations of the reed and the air column in the bassoon. J Acoust Soc
Jpn(E). 1989; 10: 269-78.
[7] Almeida A, Vergez C, Caussé R, Rodet X. Physical model of an oboe: comparison with experiments. Proc
Int Symp Music Acoustics. Nara, Japan. March 31st to April 3rd 2004. 2004; 112-115.
[8] Takahashi K, Goya K, Goya S, Susaki C. Numerical reproducibility of time-dependent motions of spatial
waves in air-columns of wind instruments. Proc Mtgs Acoust. 2013; 19: 035-063.
[9] Schumacher RT. Ab initio calculations of the oscillations of a clarinet. Acustica. 1981; 48: 71-85.
[10] Idogawa T, Kobata T, Komuro K, Iwaki M. Nonlinear vibrations in the air column of a clarinet artificiallyblown. J Acoust Soc Am. 1993; 98: 540-551.
[11] Takahashi K, Kodama H, Nakajima A, Tachibana T. Numerical study on multi-stable oscillations of woodwind single-reed instruments. Acta Acust. 2009; 95: 1123-1139[12] da Silva AR, Scavone GP. Numerical simulations of fluid-structure interactions in single-reed mouthpieces.
J Acoust Soc Am. 2007; 122: 1798-1809.
[13] Yokoyama H, Kobayashi M, Iida A. Analysis of flow and acoustic radiation in reed instruments by compressible flow simulation. Acoust Sci & Tech. 2020; 41: 739-750.
[14] Adachi S, Sato M. Time-domain simulation of sound production in the brass instrument. J Acoust Soc
Am. 1995; 97: 3850-3861[15] Adachi S, Sato M. Trumpet sound simulation using a two-dimensional lip vibration model. J Acoust Soc
Am. 1996; 99: 1200-1209.
[16] Miyamoto M, Ito Y, Iwasaki T, Akamura T, Takahashi K, Takami T, Kobayashi T, Nishida A, Aoyagi M.
Numerical study on acoustic oscillations of 2D and 3D flue organ pipe like instruments with compressible
LES. Acta Acustica united with Acustica. 2013; 99: 154-171.
[17] Iwagami S, Ryoya T, Kobayashi T, Hattori Y, Takahashi K. Numerical Study on Edge Tone with Compressible Direct Numerical Simulation: Sound Intensity and Jet Motion. 2021; 20: 283-316.
[18] Sumita R, Tabata R, Kobayashi T, Takahashi K, Hattori Y. Numerical Study of a French Horn Mouthpiecewith Compressible Direct Numerical Simulation. Proc AFI 2021; 27-29 October 2021; Sendai, Japan 2021.
175-177.
[19] Komatsu R, Iwakami W, Hattori Y. Direct numerical simulation of aeroacoustic sound by volume penalization method. Comp Fluids. 2016; 130: 24-36.

ABS-0259

Numerical approach for aerodynamics around two tone holes of woodwind
instruments
Shota TAKANAMI(1) , Ryoya TABATA(1) , Sho IWAGAMI(1) , Taijiro OHNO(1) , Takeshi NANRI(2) , Taizo KOBAYASHI(2) ,Kin’ya TAKAHASHI(1)(1) Kyushu Institute of Technology, Japan, takahasi@phys.kyutech.ac.jp(2) Kyushu University, Japan, tkoba@cc.kyushu-u.ac.jp

ABSTRACTIn this paper, we discuss the numerical reproducibility of the compressible fluid behavior around two tone holesof woodwind instruments by using compressible Large Eddy Simulation (LES). In particular, we focus on thesituation that the tone holes are opened and closed with moving pads above the tone holes, which is regarded
as a moving boundary problem with topology change, and reproduce the change of the pitch when opening andclosing the tone holes. Our two-dimensional model of a "recorder" has two tone holes. To reproduce the opening
and closing the tone holes, the pads are moved continuously. That is, the position of the pads were continuously
changed in the order of "open - close". Our numerical results are consistent with the Keef’s experimental
results. We solved the moving boundary problem with topology change under the situation of acoustics of fluidstructure interaction, and reproduced the pitch change in the opening and closing the tone holes of the recorder
like woodwind instrument model.

Keywords: Flue instruments, Tone hole, Moving boundary conditions, Numerical simulation

1

Introduction

The transient phenomena around tone holes of woodwind musical instrument have been studied as a long standing problems in the field of musical acoustics[1, 2, 3, 4, 5, 6, 7, 8]. The main difficulty in studying the functionof tone holes comes from the highly complicated interaction around compressible fluid dynamics (non-linear dynamics), aerodynamic sound as a linear wave motion and a moving boundary problem with topology change.
In order to achieve a rigorous study on it, we had developed an compressible LES solver resolving themoving boundary problem[9]. Then we had studied the function of a tone hole by using a 2D model of asmall recorder-like-instrument with a tone hole[10]. In that study, the pad above a tone hole is set to a fixed
position, that is, the distance between the pad and the tone hole is taken at fixed values, 0, 0.5, 1, 2, 3, 5,
10, 20mm, and the change of pitch depending on the position of the pad is observed. The results is consistent
with Keefe’s experimental results. In the next study, we had developed numerical models in order to manipulate
moving mesh without topology change[11].
In this paper, we give a short review of recent results about this series of study. We reproduce numericallythe compressible fluid behavior around two tone holes of woodwind instruments by using compressible LES.
We focus on the situation that the tone holes are opened and closed with moving pads above the tone holes,
which is regarded as a moving boundary problem with topology change. In order to achieve seamless numericalcalculation of moving boundary problem with topology change, we develop compressible LES solver and amethod to move and remap the mesh structure. The numerical results of our models are consistent with Keefe’s
experimental results, and error rates of pressure between mesh remapping with/without topology change are in
the same order ∼ 10−9 .

2MODEL AND NUMERICAL METHOD

We have developed a two dimensional “recorder” like model based on a model which is a 1/4 scaled 2D
analog of Keefe’s experimental model (see Figure.1 upper), which has two tone holes placed at the center ofthe resonance tube and the point of 1/4 length from the end of the resonance tube. The pipe length L, pipe
diameter H, hight of tone holes d, diameter of tone holes b, embouchure distance D, embouchure inlet diameterh, edge angle θ , are set to L = 138.5mm, H = 10mm, d = 1.5mm, b = 6.6mm, D = 5mm, h = 1mm, θ = π/7rad,respectively. To reproduce the opening and closing the tone holes, the pads are moved continuously.

Figure 1. (upper) Dimensions of the model. (lower) typical mesh structure. “dgap” is the distance between thepad and the top of the tone hole.
We employ a compressible Large Eddy Simulation (LES) with the one-equation sub-grid-scale (SGS) model[12].
In practice, we have developed a extended compressible LES solver “rhoPimpleMyDyMFoam” which incluldes
a function of “DynamicMesh” for the numerical calculation of moving mesh. This solver “rhoPimpleMyDyMFoam” is based on “rhoPimpleFoam” in the open source software, OpenFOAM ver.5. The pressure and temperature at the rest are taken as p0 = 105 Pa and T0 = 300K, respectively. The smallest mesh size around the tone
holes is ∆x = 0.17mm and the time step of the numerical integration is ∆t = 1.0 × 10−7 s. The number of meshcells is 243,528. To excite an acoustic wave in the resonance tube at a given frequency, the jet velocity “U” at
the embouchure is taken as |U| = 20m/s in the steady state.
To numerically solves the moving boundary problem with topology change, we use a combination of “DynamicMesh” and “mapFields”, which controls the pads movement and the mesh manipulation. In practice, we
configure moving velocity of mesh in the configuration file “pointMotionU” and “cellMotionU”. In order torelieve the distortion of mesh structure, we use “mapFields” every interval of 0.2mm for the pad movement.
We configure this mesh structure remapping in the configuration file “mapFieldsDict”.
The actual pad movement is shown in Figure 3. The quantity “dgap” as a function of time must be continuously differentiable in order to avoid shock waves. And furthermore, at the point (dgap= 0mm) where thetopological change of the mesh-boundary arises, the velocity of the pad as a function of time must be also
continuously differentiable. The maximum velocity of the pad is set to 0.1m/s.
To ensure that the number of meshes between the tone hole and the pad does not change significantly when
the pad is moved, default mesh structures for every 0.2mm from 0mm to 1mm are constructed. If the dgap isdgap=1mm

dgap=0.8mm

dgap=0.6

dgap=0.4mm

dgap=0.2mm

closedFigure 2. Mesh structures around the pad and tone hole. These are the default “dgap” mesh structures, respectively. The distance between the pad and the top of the tone hole is 1 to 0mm.

Figure 3. Time evolution of the pad movement. (a) the velocity, (b) “dgap”: the distance between the pad andthe top of tone hole

between 1mm and 0.8mm, a default “dgap” mesh of 1mm is used. When the dgap narrows to 0.8mm, switch to
the 0.8mm default “dgap” mesh. From a dgap of 0.8mm to 0.6mm, use a 0.8mm mesh, The process is repeatedin the same manner until it is completely closed at 0 mm.
Table 1. Mesh dimensions and corresponding time of them
RegionTime[s](1)0∼0.053(2)

0.053∼0.055(3)

0.055∼0.057(4)

0.057∼0.059(5)

0.059∼0.06198(6)

0.06198∼0.07

Time of mapping mesh[s]
0.053
0.055
0.057
0.059
0.06198

Default “dgap”
1mm
0.8mm
0.6mm
0.4mm
0.2mm
0mm

Table 1 shows the mesh dimensions and the time of mesh structure remapping.

3

NUMERICAL RESULT

3.1 Resonance frequencies of typical four conditions
Our numerical model has two tone holes. Thus, the model has four typical conditions by the combination
states of the tone holes, which are open and/or closed. That is, the combination states of tone holes “left-right”
can be set to “open-open”(Fig.4 (1)), “open-close”(Fig.4 (2)), “close-open”(Fig.4 (3)), “close-close”(Fig.4 (4)).
Since, the resonance frequency is depend on the effective pipe length, these four conditions make different
resonance frequencies each other. The shorter the effective resonance pipe length, the higher the resonancefrequency, thus the resonance frequency increases in the order “close-close”, “close-open”, “open-close”, “openopen”. Numerical results of the power spectra of the pressure oscillations are shown in Figure 5.
As shown in Figure 5, the resonance frequencies of “close-close”, “close-open”, “open-close”, “open-open”
are 1097Hz, 1317Hz, 1427Hz, 1586Hz, respectively. Thus our numerical model of a recorder with two tone
holes was successfully reproduces the resonance frequencies according to the state of the tone holes.
3.2 Transition of the resonance frequency
Two tone holes are simultaneously closed from 1mm to 0mm by applying a constant jet velocity “U” of 20m/s
at the embouchure. The default “dgap” mesh is switched at the times shown in Table 1. Figure 6 (left) shows
the time evolution of the pressure distribution. The coloring corresponds to the default “dgap” meshes. The
amplitude suddenly increases from dgap=0.2mm in Region 6. This is due to the fact that the jet velocity “U”
of 20 m/s is too large for a pipe length of 138.5 mm.
Figure 6 (right) shows the results for resonance frequency. As with the left graph, the regions are colorcoded. It can be seen that the resonance frequency decreases from Region 6 to 1. Furthermore, the resonancefrequency is the same from Regin 2 to 4, and then decreases abruptly at Region 5. This indicates that theresonance frequency does not decrease continuously in synchronization with the decrease in the distance between
the tone hole and the pad, but rather decreases at a certain fixed distance. This phenomenon has been provenin the Keef’s experiments, and we thought that the same phenomenon might have occurred in the numericalanalysis.
However, in the condition that this pads move in succession, the period of time in each Region is as short
as 0.002 sec. Thus the frequency resolution of the Fourier transform is rough. Therefore, it is necessary
to determine the resonance frequency at each region with high accuracy. The resonance frequencies at each
“dgap” were determined by numerical calculations for 0.1 sec at appropriate values of “dgap” from 1 mm to 0mm.

(1)“Open-Open”(2)“Open-Close”(3)“Close-Open”(4)“Close-Close”

Figure 4. Mesh structures: (1) the both tone holes are open, (2) the right side tone hole is closed and the left
side tone hole is open, (3) the left side tone hole is closed and the right side tone hole is open, (4) the bothtone holes are closed. Effective pipe lengths are shortened in the order (1), (2), (3), and (4). The distance
between the open pad and the top of the tone hole is set to 1mm.

1000

Open-Open

100

100

10

10

pressure [pa]pressure [pa]

1000

1

0.1

0.01

Open-Close

1

0.1

0

500

1000

1500
2000
2500
frequency [Hz]

3000

3500

0.01

4000

0

“Open-Open”: 1586Hz

1500
2000
2500
frequency [Hz]1000

Close-Open

100

100

10

10

1

0.1

0.01

1000

3000

3500

4000

“Open-Close”: 1427Hz

pressure [pa]pressure [pa]

1000

500

Close-Close

1

0.1

0

500

1000

1500
2000
2500
frequency [Hz]

3000

3500

0.01

4000

0

“Close-Open”: 1317Hz

500

1000

1500
2000
2500
frequency [Hz]

3000

3500

4000

“Close-Close”: 1097Hz

Figure 5. Power spectra of the pressure oscillations.

1000

Region 1
Region 2
Region 3
Region 4
Region 5
Region 6

100400Region 1
Region 2
Region 3
Region 4
Region 5
Region 6

100

pressure [pa]Pressure [Pa]100200100000

10

1

99800
0.1
99600

0

0.01

0.02

0.03
0.04
Time [sec]

0.05

0.06

0.07

0.01
500

1000

1500

2000
2500
frequency [Hz]

3000

3500

4000

Figure 6. The distance between the tone holes and the pads are varied continuously from 1 mm to 0 mm, (left)time evolution of pressure p and (right) resonance frequency transition. In the range of “Region” from 2 to 4mm, the resonance frequency is almost no change.

1000

1000

dgap = 0.8mm

10

1

0.1

0.01

10

1

0.1

0

500

1000

1500 2000 2500
Frequency [Hz]

3000

3500

0.01

4000

0

500

3000

3500

0.01

4000

1000

1500 2000 2500
Frequency [Hz]

3000

3500

10

1

0.01

4000

0

500

1000

1500 2000 2500
Frequency [Hz]

3000

3500

0.1

500

1000

1500 2000 2500
Frequency [Hz]3000

0.075mm: 1244Hz

1

0

500

3500

4000

1500 2000 2500
Frequency [Hz]

3000

3500

4000

dgap = 0.0mm

100

10

1

0.01

1000

1000

dgap = 0.05mm

0.1

0

4000

0.1mm: 1269Hz

Pressure [pa]Pressure [pa]

1

3500

10

0.01

4000

100

10

3000

0.1

1000

100

1500 2000 2500
Frequency [Hz]dgap = 0.1mm

0.15mm: 1293Hz

dgap = 0.075mm

1000

100

0.2mm: 1403Hz
1000

500

1000

dgap = 0.15mm

0.1

500

0

0.3mm: 1462Hz

Pressure [pa]Pressure [pa]Pressure [pa]1

0.1

Pressure [pa]1500 2000 2500
Frequency [Hz]

100

10

0.01

1000

1000

dgap = 0.2mm

01

0.4mm: 1463Hz

100

0.01

10

0.1

0.8mm: 1475Hz
1000

dgap = 0.3mm

100

Pressure [pa]100

Pressure [pa]Pressure [pa]

100

1000

dgap = 0.4mm

10

1

0.1

0

500

1000

1500 2000 2500
Frequency [Hz]

3000

3500

0.05mm: 1086Hz

4000

0.01

0

500

1000

1500 2000 2500
Frequency [Hz]

3000

3500

4000

0.0mm: 1085Hz

Figure 7. Power spectra of pressure p. In the range of “dgap” conditions from 0.8 mm to 0.3 mm, the resonancefrequency is almost no change. These results are consistent with Keef’s experimental results.

In order to find out from which position the resonance frequency changed to increase, we prepared several
conditions with the distance between the tone hole and the pad ranging from 0.8 mm to 0 mm, fixed theconditions at each position, performed the analysis, and observed the change in the resonance frequency. The
series of resonance frequencies at each point is shown in Figures 7. In the range of “dgap” conditions from 0.8
mm to 0.3 mm, the resonance frequency is almost no change. However, it begins to decrease from 0.2mm to0.1mm of the “dgap”. Although there is a difference in this frequency change, the resonance frequency value
continues to decrease up to 0.05mm. From this result, it is found that the resonance frequency decreased after
0.2mm of the distance between the sound hole and the pad. These results are quantitatively consitent withKeef’s experimental results.
3.3 Error estimations of mesh manipulation
Table 2. Errors of pressure at the “Center” sampling point
Mesh manipulation
decrease mesh (1mm → 0.8mm)

topology change (Open → Close)

Default mesh dimension

Pressure [Pa]1mm(Before)

100089.0219

0.8mm(After)

100089.0338

0.2mm(Before)

99953.51932

0mm(After)

99953.51892

Error rate
1.0 × 10−8

4.0 × 10−9

Table 2 shows errors of pressure at the “Center” sampling point. In the case of “increase mesh (0.8mm →1mm)”, error is bigger than the case of “decrease mesh (1mm → 0.8mm)”. We assume that this increasing oferror rate is caused by a higher order complement of physical quantities when they are mapped from on small
mesh onto large one.
Other error rates are in the almost same order ∼ 10−8 . Thus, the function of moving boundary calculation
with topology change in our numerical model is ensured.

4

CONCLUSIONSIn this paper, we study the numerical reproducibility of the compressible fluid behavior around two tone holesof woodwind instruments by using compressible LES. In particular, we focus on the situation that the tone holes
are opened and closed with moving the pads above the tone holes, which is regarded as a moving boundary
problem with topology change.
Our numerical model is a two dimensional “recorder” like model based on a model which is a 2D analog of
1/4 scaled Keefe’s experimental model. For seamless numerical calculation of moving boundary problem with
topology change, we have also developed our own compressible LES solver “rhoPimpleMyDyMFoam” on an
open source software OpenFOAM ver.5. This solver is applied to our numerical tone holes model and ensured
that the functions of mesh manipulation as moving boundary problems with/without topology change work very
well. Error rates of pressure at the mesh manipulation is also consistent with this reproducibility. From theview point of numerical reproducibility, the most important result is as follows. When the position of the pads
are continuously changed in the order of "open - close", the transitions of resonance frequency is observedin the range of “dgap” value between 0.2 and 0 mm. These results are quantitatively consistent with Keefe’s
experimental results.
It is necessary to check that the same results can be reproduced for a 3D model, and can be also reproduced
the pitch change in the opening and closing the tone holes of more practical situation. For example, by opening
and closing multiple tone holes independently of each other, the conditions of the numerical calculation can bebrought closer to the fingering of actual instrument performances. This allows to discuss the situation as fluid
sound around the tone holes and the pitch change during actual instrumental performance.

ACKNOWLEDGEMENTS
The present work was supported by JST CREST Grant Number JPMJCR1501, japan, Grant-in-Aid for Scientific
Research (C) Nos. 16K05477 and 19K03655 from the Japan Society for the Promotion of Science (JSPS) and“Joint Usage/Research Center for Interdisciplinary Large-Scale Information Infrastructures” and “High Performance Computing Infrastructure” in Japan (Project IDs: jh180007-MDH, jh190010-MDH, and jh220001). Partof the work was carried out under the Collaborative Research Project of the Institute of Fluid Science, Tohoku
University.

REFERENCES[1] N. H. Fletcher and T. D. Rossing, “The Physics of Musical Instruments,” Springer-Verlag, New York, 2ndedition (1998).
[2] B. Fabre, J. Gilbert, A. Hirschberg and X. Pelorson, “Aeroacoustics of Musical Instruments”, Annu. Rev.
Fluid Mech. 44 1-25 (2011).
[3] D. Keefe, “Theory of the single woodwind tone hole”, J.Acoust.Soc.Am. 72, 676–687 (1982).
[4] D. Keefe, “Experiments on the single woodwind tone hole”, J.Acoust.Soc.Am. 72, 688–699 (1982).
[5] D. Keefe, “Woodwind air column models”, J.Acoust.Soc.Am. 88, 35–51 (1990).
[6] C.J. Nederveen, J.K.M. Jansen, R.R. van Hassel, “Corrections for woodwind tone-hole calculations”, Acustica · acta acustica, 84, 957–966 (1998).
[7] V. Dubos, J. Kergomard, A. Khettabi, J.-P. Dalmont, D.H. Keefe, C.J. Nederveen, “Theory of sound propagation in a duct with a branched tube using modal decomposition”, Acustica · acta acustica, 85, 153–169(1999).
[8] J.-P. Dalmont, C. J. Nederveen, V. Dubos, S. Ollivier, V. Mśerette, E. teSligte, “Experimental determinationof the equivalent circuit of an open side hole: linear and non linear behaviour”, Acta Acustica united with
Acustica, 88, 567–575 (2002).
[9] Y. Ito, T. Kobayashi, K. Takahashi, T. Takami, A. Nishida, M. Aoyagi, “Reproduction of Transitions among
Notes on an Air-reed Musical Instrument with Compressible LES Combined with Moving Boundary Technique”, in Proceedings of Open Source CFD International Conference 2011, Volume Session V, A 1 (OSCIC2011) (2011)[10] T. Iwasaki, T. Kobayashi, K. Takahashi, T. Takami, A. Nishida, M. Aoyagi, “Numerical study on thefunction of tone holes of a recorder like instrument from the viewpoint of the aerodynamic sound theory”,
Proc. Mtgs. Acoust. 19, 035024 (2013)[11] T. Kobayashi, H. Matsuda, T. Takami, K. Takahashi, “Numerical study for the function of moving pad ontone hole acoustics”, The Journal of the Acoustical Society of America 140(4):3037-3037, (2016)[12] C. Wagner, T. Hüttl, and P. Sagaut, eds. Large-Eddy Simulation for Acoustics, (Cambridge Univ. Press, New
York, 2007).

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022 in Gyeongju, Korea

ABS-0437

A framework for the analysis of bowing actions withincreased realisticness
Montserrat PÀMIES-VILÀ 1; Anna SCHEIBLAUER1;
Alexander MAYER1; Vasileios CHATZIIOANNOU1
1

Department of Music Acoustics - Wiener Klangstil (IWK), mdw - University of Music andPerforming Arts Vienna, Austria

ABSTRACTThe action of the bow on the string is one of the key aspects of the player-instrument interaction in bowedstring instruments. For the analysis of this interaction, it is common to use custom-built artificial excitation
mechanisms, where the human arm is substituted by a mechanical setup. Such artificial means allow
recreating the excitation mechanism repeatedly. In this study, a robotic arm with 6 degrees of freedom is used
to achieve highly precise motion of a cello bow. Preliminary tests have achieved repeatable reproduction ofplucked and bowed string sounds using linear trajectories of a plectrum or a bow. In order to take the nuancesof the player actions into account, a more realistic bow movement is required. To that aim, the robot might
be instructed with bowing trajectories obtained from a real playing situation with expert cellists. Thesebowing trajectories are recorded using optical motion capture. The motion capture system consists of infrared
cameras that track the position of a set of reflective markers attached to the bow, musician, and instrument.
The combination of motion-capture data with a fine control of the robotic arm creates a framework for theaccurate analysis of the player-instrument interaction in bowed string instruments.
Keywords: Bowed strings, Robotic arm, Cello gesture, Playing technique

1. INTRODUCTION

In music acoustics, artificial playing machines are frequently used to evaluate musical instruments
under controlled laboratory conditions. In bowed-string instruments, most machines have been
dedicated to explore the sound production in steady state [1, 2] and, only lately, some studies included
the analysis of articulation and transitory phenomena [3, 4]. Since the early experiments by Raman[1], bowing devices have been complemented with measurements of several relevant variables, suchas bow force and velocity [3, 4, 5]. As an example, Schoonderwaldt [3] and colleagues used a
computer-controlled linear bowing machine to bow on a monochord in order to investigate howbowing manipulations affect the violin sound.
In the discipline of performance science, motion capture devices are frequently employed to record
body movements so as to analyze the musicians’ gestures during musical performance [6]. Opticalmotion capture uses infrared cameras to track the position of reflecting markers attached to musicians’
body parts and instruments. To better understand and incorporate the player actions in the analysis of
bowed-string instruments, the body motions have been measured and taken into account in few studies
considering violin and cello playing [7, 8, 9], particularly those regarding the motion of the bowingarm. Such studies made use of motion capture systems to locate the position of the human body parts
and the position of the bow and instrument during playing.
Current research at the Department of Music Acoustics (IWK) seeks at providing a framework forthe analysis of the acoustics of bowed-string instruments that considers repeatable and realistic
bowing patterns. To that aim, we first use state-of-the-art motion capture technology to track the
motion of the bow during real performance with experienced cellists. This data is then used to instructa robotic arm that holds the bow. In a first test-case, linear bowing patterns that resemble the player
actions are used to instruct the robot. This paper shows ongoing research towards achieving
1pamies-vila@mdw.ac.atartificially-articulated bowing strategies in which the motion of the bow is as close as possible to thereal-playing situation.

2. METHODS
2.1 Playing tests with cellists
Six advanced cello students were invited to the motion-capture laboratory of the University ofMusic and Performing Arts Vienna and were asked to play some exercises on an instrumented celloand bow while wearing reflective markers (on their body, on the bow and on the cello), as seen in
Figure 1 (left). These tests served at gathering information on their playing style and posture [10] andat collecting 3D data regarding the motion of the bow. Besides motion-capture (frame rate of 240 Hz,
OptiTrack infra-red cameras), the sound was recorded with a microphone in front of the cello as wellas with a piezoelectric-sensor on the bridge (frame rate of 41000 Hz). Signals were synchronizedusing a clapperboard. Video recordings are also available to support the analysis of the 3D data.

Figure 1: Images of the real-playing and the artificial-playing conditions showing the position of thereflective markers and the marker-set as seen in the motion-capture software Motive.
2.2 Robot control
The robotic arm used in this study is the model UR5e by Universal Robots, which has a weight of
20.6 kg, a reach of 850 mm and it can hold objects up to 5 kg. The bow is attached to the holdingjoint of the robot using 3D printed parts. The robot tool-position is given by its 3D position in space(with respect to the origin at its base) and the inclination of the held object is given in axis-angle
representation. These are three degrees of freedom for the position and three degrees of freedom forthe orientation. The robotic arm can be controlled via a user-friendly environment that allows for aset of custom-made instructions. Alternatively, the tool-position of the robot or the robot joint angles
can be driven externally via Real-Time Data Exchange (RTDE) [11].
Using the custom-made instructions approach, one can create motions that may be repeatedaccording to several requirements. As such, this study presents a test-case where linear bowing
patterns including velocity and acceleration changes were tested, while accurate increases in bow-tostring distance were introduced, increasing the bowing force on the string.
2.3 Experimental procedure
Inspired by some of the exercises that players performed, we designed a bowing pattern using
linear trajectories, which aims at testing the capability of the robotic arm in such bowing tasks (taking
player num. 5 as the reference). The robot was instructed with linear motions along the longitudinal
axis of the bow (being held at the last joint of the robot). This axis was set to be perpendicular to theD string, the only string used. Following the axis perpendicular to the string and to the holding joint,
increments of 0.2 mm where applied and the bowing pattern was repeated 12 times. This means thatthe bowing pattern was repeated with increments of bow force on the string. Accelerations from 2
m/s 2 to 15 m/s 2 , velocities from 0.15 m/s to 0.45 m/s and trajectories from 0.09 m to 0.5 m were tested.
These tests were carried out in the same room, with the same bow, cello, sensors and reflective markers

as for the tests with real players. All 3D motion-capture data were low-pass filtered at 6 Hz using athird-order Butterworth filter, which is commonly used when considering slow motions (bow strokes
are longer than 1 second) to significantly reduce measurement noise and it is necessary to computethe kinematics of each marker (velocity and acceleration). The motion-capture data showed in thisstudy corresponds to the movement of one marker along the x-axis, which is an axis perpendicular tothe strings where the movement of the bow during playing on the D string mainly takes place. The
cello was held immobile in the same position for both cases (Figure 1), with its longitudinal axis
perpendicular to the x-axis.

3. RESULTS

Two examples of the performed bowing patterns have been chosen, containing four bow strokes
starting at the frog. Figure 2 shows position (x), velocity (v), acceleration (a) of one marker mounted
on the bow, in front of the index finger (for players) or in front of the holding part of the robotic arm.
On the left plots, the portion of the bow used is constant and every stroke is performed at a similar
velocity, using the whole bow (50 cm). On the right, the players were instructed to play frompianissimo to fortissimo along four bow strokes. The robot was accordingly instructed to play four
strokes using a trajectory of 20, 30, 40 and 50 cm, and every stroke at a higher velocity (from 0.17m/s to 0.41 m/s), yet the bow-to-string distance, i.e., the force of the bow on the string was kept thesame between strokes (7 th tested bow-force increment).

Figure 2: Position (x), velocity (v), acceleration (a) of one marker on the bow, together with the signal of
the piezoelectric sensor on the bridge, comparing one player (orange) and the robotic arm (blue). On theleft, whole bow strokes, on the right, ascending velocity pattern.
In the whole-bow example (left) one can observe the regularity of the bowing patter for both player
and robotic arm along the x-axis. Regarding velocity, players keep gently increasing velocity during
the long note, indicating a preparation for the bow-change [12]. The acceleration peaks reached at thebow change are set at 2 m/s 2 for the robot, while for the player slightly higher values are observed.
In the velocity-increasing exercise (right), the acceleration peaks are very similar in both player
and robot, and are higher than for the whole-bow exercise, achieving values up to 10 m/s2. With highacceleration, an additional challenge seems to be the bouncing of the bow held on the robot (seechanges in amplitude in the piezoelectric signal). One can also observe that players might smoothenor shorten the note release, and achieve quieter note attacks at pianissimo.
Figure 3 shows the superposition of signals obtained at the 12 tested bow forces. These wereobtained by adjusting the distance between the bow and the D string, with increments of Δz = 0.2 mm
on the z-axis, an axis normal to the plane defined by the bow-hair. Doing so, at every tested condition,
the wood of the bow moves slighly towards the string, thus the hair of the bow excerts a higher force
on the string. The upper plot shows the position of one marker of the bow along the x-axis (as in
Figure 2), the middle plot shows the signal of the piezoelectric sensor for all 12 repetitions, and thebottom plot shows the root-mean-square envelope (rms) of these signals. The bowing pattern begins
with the frog on the string. The increase in amplitude in the low bow-force plots shows a difference
when playing next to the frog or playing next to the string. This indicates a difference in bow force
along the bow, which was not directly imposed in these tests. The difference in amplitude disapears
at higher bow force, where the sounds are steady after the attack transient (around 0.4 seconds long).

Figure 3: Four bow strokes at 12 bow-to-string distances considering increases of Δz = 0.2 mm (high bow
force in darker color, low bow force in lighter color). The upper plot shows the position along the x-axis of
one marker on the bow, next to the holding joint at the frog of the bow. The signal of the piezoelectric
sensor on the bridge and its rms-envelope are shown with superposition of all tested conditions.

4. DISCUSSION AND PERSPECTIVES

The presented preliminary results show the capability and potential of the proposed framework forthe analysis of bowed-string instruments under controlled laboratory conditions. The custom-madeinstructions approach allows for rapid testing of repetitive motions. The trajectories of the bow were
mimicked using linear bow motion, and some of the challenges of the setup were encountered. Highacceleration results in bouncing of the bow, showing a lack of damping or bowing pressure. Higher
bowing pressures and lower accelerations show more steady sounds. For long notes using the wholebow, as in Figure 3, particularly at high bow force, the setup achieved more steady sounds.
After 1.2 mm increase of bow position pressing on the string, the security limit for the applied
force was met. This limit was sufficient to play from soft to loud sounds at the several tested velocities
and accelerations. Yet the given security limit might be adapted if higher bow forces are required.
With the control of the bow motion in the real-time approach, which allows driving the robot sample-

by-sample, playing nuances as the ones observed in the data of the cellists may be included in theartificial playing setup. The presented methodology, which is for now purely kinematic, might beenhanced if the dynamics of the bow-string interaction are also considered.

ACKNOWLEDGEMENTS

The authors acknowledge the cellists who took part in the experiments, as well as Sara D’Amario
for her assistance regarding the motion capture system. This research was funded in whole, or in part,by the Austrian Science Fund (FWF) [T1295-G, P34852-N].

REFERENCES

1. Raman CV. On the mechanical theory of the vibrations of bowed strings and of musical instruments ofthe violin family, with experimental verification of the results, part I, Indian Association for the
Cultivation of Science, 15:243–276, 1918.
2. Cronhjort A. A computer-controlled bowing machine (mums), Speech, Music and Hearing Quarterly
Progress and Status Report, 33(2-3):61–66, 1992.
3. Schoonderwaldt E. The violinist’s sound palette: spectral centroid, pitch flattening and anomalous low
frequencies, Acta Acustica united with Acustica, 95(5):901–914, 2009.
4. Galluzzo P, Woodhouse J. High-performance bowing machine tests of bowed-string transients, Acta
Acustica united with Acustica, 100(1):139–153, 2014.
5. Mores R, Precise cello bowing pendulum, Proc. 3rd Vienna Talk on Music Acoustics, page 106, 2015.
6. Verrel J, Pologe S, Manselle W, Lindenberger U, Woollacott M. Exploiting biomechanical degrees of
freedom for fast and accurate changes in movement direction: coordination underlying quick bow
reversals during continuous cello bowing. Frontiers in Human Neuroscience. Apr 26;7:157, 2013.
7. Van Der Linden J, Schoonderwaldt E, Bird J, Johnson R. Musicjacket - combining motion capture andvibrotactile feedback to teach violin bowing, IEEE Transactions on Instrumentation and Measurement,
60(1):104–113, 2010.
8. Schoonderwaldt E, Demoucron M, Extraction of bowing parameters from violin performance combining
motion capture and sensors, The Journal of the Acoustical Society of America, 126(5):2695–2708, 2009.
9. Rozé J, Aramaki M, Kronland-Martinet R, Ystad S. Cellists’ sound quality is shaped by their primary
postural behavior. Scientific reports. Aug 17;10(1):1-7, 2020.
10. Scheiblauer A, Mayer A, Pàmies-Vilà M. Investigating the cello position, bow motion and cellist postureusing motion capture. Proc. of 4th Vienna Talk on Music Acoustics, 2022.
11. Mayer A, Pàmies-Vilà M, Chatziioannou V. The Universal Robots Real-Time Data Exchange (RTDE)
and LabVIEW. IWK Tech Report 1, University of Music and Performing Arts Vienna, 2022.
12. Mantel G. Cello technique: principles and forms of movement. Indiana University Press; 1995.

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022, in Gyeongju, Korea
Extended abstractABS-0673

A Comparison of Player-Dízi Performance Parameters across FourMembers of Dízi Family
Hsien Han NG; Jer-Ming CHEN 1
Singapore University of Technology & Design, Singapore

ABSTRACTThe Dízi is a traditional Chinese flute made from bamboo with characteristic bright timbre arising from nonlinear interactions due to a membrane stretched over a membrane-hole. Being keyless, the Dízi is alsoavailable in different octave and keys, resulting in a large family with numerous representative members. To
understand how different Dízi family members compare in terms of player-instrument interaction, we survey
the performance parameters of four different Dízi members (two Dízi each of “High” C, G, D & “Low” G
keys, played by the same expert performer) by quantifying how pitch and SPL (‘loudness’) vary to according
to four musical dynamic markings, when sounded across each instrument’s standard range. We also identify
regions of the parameter space that is easy or difficult to control (and at what expense: pitch deviation or
loudness control?), revealing the expected consistency (and variability) of the Dízi’s performance as playedby an expert. The results suggest that certain performance features may scale with the particular Dízi’ssize/octave, design and construction.
Keywords: Performance Parameters, Transverse Flute, Player-Instrument Interaction

1. INTRODUCTION

The Dízi is a traditional Chinese flute with characteristic bright timbre arising from non-linear
interactions due to a membrane stretched over a membrane-hole. For traditional and cultural reasons[1], it retains a simple design and construction: a hollow bamboo tube (stopped at one end) with six
finger-holes (no keys) and is played in a transverse manner like the western flute
Its construction differs from the western flute, however, featuring a keyless bamboo body
integrated with embouchure-, membrane- and finger-holes, such that its precise geometric dimensions
are not necessarily standardized (the bamboo stock is a ‘found object’ and remains largely ‘unworked’
for traditional reasons). Being keyless, the Dízi is thus available in different octave and keys, resulting
in a large family with numerous representative members, presenting considerable practical challenge
to makers, performers, composers, teachers and students.
Consequently, we investigate how output performance parameters of pitch (‘deviation’) and soundpressure level (SPL, ‘loudness’) vary across four different Dízi members (two Dízi each of “High” C,
G, D & “Low” G keys, played by the same expert) when performing across each instrument’s standard
range at four musical dynamic markings (pp, mp, f, ff), thereby ‘mapping out’ gross similarities anddifferences of these four Dízi family members, and expectations of scalability (or lack thereof). Whileprevious studies have investigated the effect of dynamic markings on sound pressure output producedby Chinese instruments [2-5], they do not report the relationship between loudness and pitch.

2. METHODOLOGY
Two sets of four Dízi family members (“High” C, G, D & “Low” G) were chosen for this study;
the names refer to the reference tonic (‘Do’) sounding respectively C6, G5, D5 and G4 when playedusing the standard ‘Do’ fingering xxx ooo. All Dízi used were concert-grade instruments, made bydifferent reputable masters and thus we may consider them idealized representations of that family
member (this was also anecdotally confirmed by several expert players available to the authors). An
expert Dízi player was engaged to play these eight instruments, and the player is a professional
1

jerming_chen@sutd.edu.sg

member of a traditional musical ensemble, frequent concert soloist, winner of international
performance competitions and has over twenty years of performance and teaching experience – again
we may thus consider him to be an idealized representation of a typical expert Dízi player.
To understand the relationship between each Dízi’s SPL (‘loudness’) and pitch output, four
dynamic level where used: pp, mp, f and ff (where pp here means “as soft as possible” and ff means“as loud as possible” musically). The player was instructed to deliver the full standard tessitura of the
Dízi (two octaves plus a major second; Wang, 2014) by performing four sets repeated ascending anddescending diatonic scales, holding each note least three seconds. The player was asked to play eachnote with respect to equal temperament (an electronic tuner provided, A4 = 442Hz) while delivering
the assigned musical dynamic level; we do this because pitch and loudness is determined not only by
the Dízi, but rather arises from complex interactions in the player-instrument system, especially since
the air jet depends on the player’s embouchure and breath control [6, 7]: e.g. distance from the lips tothe sharp edge, shape of the lips/embouchure and speed of the jet. Such parameters contribute to thetimbre, loudness and fine pitch control in wind instruments [8, 9], and so we wish to investigate howdifficult (or easy) it is for an expert to deliver ‘on-pitch’ while assigned a particular musical dynamic
level (whilst maintaining musicality).
Audio measurements were collected in a recording studio treated for noise isolation and absorption,with the player located 1 m from the microphone (Zoom H5 Recorder) co-axial with the player’s face.
The prevailing sound pressure level (SPL) is calibrated against a constant excitation source of known
SPL sited at the same position as the player with respect to the microphone. Pitch and SPL of eachnote played (segment of stable pitch identified by ear) was extracted using Adobe Audition CC 2018,
and its pitch and SPL then extracted to a resolution of 1 cent and 0.1 decibel respectively.

3. RESULTS and DISCUSSION

For the purposes of this extended abstract, we limit ourselves only to the nominal overall behaviorof the four Dízi members studied: accordingly we collated the pitch deviation (cents) and SPL (dB)of all notes played and plotted it as Figure 1, with each of the 8 Dízi represented as a separate curve(same color for each family member), and each instrument’s “curve” of 4 data points representing,from left to right respectively, pp, mp, f and ff.

Figure 1: Pitch Deviation vs SPL for the four Dízi members (“High” C, G, D & “Low” G keys, 2 Dízi each)studied, collated for four dynamic levels pp, mf, f and ff (points going left to right); Orange shaded region
represents nominal boundaries of the mean ± standard deviation for these points.

The standard deviation for each data point is bounded by the ‘convex’ orange shaded area andindicates nominally the effective performance parameter space (pitch deviation vs SPL) of the four
Dízi members: the typical loudness achieved across all Dízi members studied range from 72–89 dB
SPL (each Dízi member’s SPL range is fairly consistent, ~9-10 dB for, regardless of size) whileaccompanying pitch deviation varied from 0 to +24 cents.
In general, increasing musical dynamic level (increasing SPL) attracted a ‘penalty’ of increasing
pitch deviation (‘sharper’) – the largest ‘pitch penalty’ arises going from f to ff (steepest slope). In
terms of practical performance, this may suggest it is musically ‘risky’ to deliver ff if pitch is an
important consideration; hence if we exclude ff data point, we note only a modest pitch deviation of<10 cents for all Dízi studied.
We also note that High C, G and D share an overlapping parameter space (higher SPL, ranging
from ~80-90 dB) than the Low G (lower SPL, from ~72-84 dB, with somewhat greater pitch deviation).
This is not surprising, as High C, G and D Dízis often play a “soprano” role as soloist and handling
main melodic motifs which cut through the orchestra, while Low G plays “alto” role in accompanimentand harmonic function. The ~4-5 dB difference in gross SPL levels may also provide some insight interms of dynamic balance constraints often faced when Low G Dízi has to play against ‘soprano’
instruments. Among the three ‘soprano’ instruments, however, we note that in fact it is the G Dízi
which is loudest (most “dynamically efficient”), indicating that size (key) of the instrument does notnecessarily scale monotonically with loudness; G Dízi also seems to enjoy the least “pitch penalty”
compared with the other family members.
In regard to each family member, the two representations (Dízi from different masters) show
remarkable consistency in terms differences in SPL range (<1.8 dB) and pitch deviation range (<3cents). (Granted, the “pitch-centre” of both instruments may not be in agreement, as that depends onthe maker’s preference of pitch reference; something even an expert player may not entirely accountfor.) This observation also implies that Dizi members (of the same key) are rather more similar thanDizi members across different keys, for performance parameters such as pitch and SPL (timbre is not
explored in this study).
These insights drawn may inform how performers, composers, and arrangers may balance practicalconsiderations and priorities when they need to choose the most optimal Dízi member to play on. Of
course, pitch and loudness are only very simple performance parameters to consider, and we have notyet explored timbre, ease of playing (avoiding cross-fingering or half-holing), note range required,balance and blending, musical context etc., and can be the focus of future studies.

4. CONCLUSION

In this extended abstract, we summarized and report the overall pitch deviation and sound pressurelevel of how four Dízi family members (two concert-grade examples of each members) behave whenan expert player is asked to perform the instrument’s standard tessitura at four dynamic levels – theoperating performance parameter space is thus identified and outlined. When playing from soft (pp)to very loud (ff), SPL rises as expected, but pitch deviation also rises (sharpens). We identified that
the SPL rise from f to ff may be accompanied with an unfavorably greater pitch deviation (‘pitch
penalty’ effect) which is more severe that other dynamic levels. We also identified that the parameter
space of Dízi family members do not scale simply with size (key), but instead the soprano instruments
overlap and behave somewhat distinctly from the alto instrument, which show an overall 4-5dB lower
sound level. These insights to the performance parameter spaces of Dízi family members will be usefulfor both performers and composers.

REFERENCES

1. Wang, C., Chow, J. Y., & Wong, S. S. (2020). The Teng Guide to the Chinese Orchestra. World Scientific
Publishing Co. Pte. Ltd and The Teng Ensemble Ltd.
2. Zhao, Y. Z., Wu, S. X., Qiu, J. Z., Wu, L. L., & Huang, H. (2008). Sound power level measurement ofSheng, a Chinese wind instrument. The Journal of the Acoustical Society of America, 123(5), 3240–3240.
https://doi.org/10.1121/1.2933489
3. Zhao, Y. Z., Wu, S. X., Qiu, J. Z., Huang, H., & Wu, L. L. (2010). Sound intensity radiated by gaohu.
Science China Technological Sciences, 53(7), 1792–1797. https://doi.org/10.1007/s11431-010-4002-8
4. Li, N., Zhao, Y., Wu, S., Huang, H., & Wu, L. (2012). Sound power level measurement of Chinese bowedstringed instrument-gaoyinbanhu. The Journal of the Acoustical Society of America, 131(4), 3219–3219.
https://doi.org/10.1121/1.4708005
5. Qiu, J., Wu, S., Zhao, Y., Huang, H., & Wu, L. (2021). Sound power characteristics of Chinese traditional wind instrument Qudi. International Journal of Architecture, Arts and Applications, 7(1), 8.
https://doi.org/10.11648/j.ijaaa.20210701.12

6. Wolfe, J., Fletcher, N. H., & Smith, J. (2015). The interactions between wind instruments and their players.
Acta Acustica United with Acustica, 101(2), 211-223. https://doi.org/10.3813/AAA.918820
7. De la Cuadra, P., Fabre, B., Montgermont, N., & Chafe, C. (2005). Analysis of Flute Control parameters:
A comparison between a novice and an Experienced Flautist. Acta Acustica United with Acustica, 94(5),740-749. https://doi.org/10.3813/AAA.918091
8. Fuks, L. & Sundberg, J. (1996) “Blowing Pressures in Reed Woodwind Instruments”, Department for
Speech, Music and Hearing. Quarterly Progress and Status Reports (TMH-QPSR), 37(3), 41–56.
9. Almeida, A., George, D., Smith, J., & Wolfe, J. (2013). The clarinet: How blowing pressure, Lip Force,lip position and Reed “hardness” affect pitch, sound level, and spectrum. The Journal of the AcousticalSociety of America, 134(3), 2247–2255. https://doi.org/10.1121/1.4816538

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022, in Gyeongju, Korea
Extended abstractABS-0859

Performance Parameter Survey of the Chinese ShengSamuel GOH, Christopher CLARKE, Da Yang TAN; Vincent TAN; Jer-Ming CHEN 1
Singapore University of Technology & Design, Singapore

ABSTRACTThe Sheng is a Chinese free-reed instrument (mouth organ) with documentation dating back to 1100 BCE,
and has extensive related counterparts in other East Asian cultures, e.g. Saenghwang in Korea, Shō in Japanand Khene in Laos. In this study, we survey the full range of a contemporary “alto” Sheng for its performanceparameters in terms of loudness vs pitch variation: across its 3 octave chromatic range, for 3 dynamic levels(pp, mf and ff), and for blow vs draw gestures. Additionally, we identify notes showing differences due toplaying range, blow vs draw and dynamic levels. Lastly, we relate the output performance parameters interms of the player’s input of blowing pressure and how it changes with respect to musical dynamic, output
sound level and pitch deviation. By characterizing these multidimensional output parameters, we provide forthe first time, a systematic description of the Sheng’s performance parameter space, thereby facilitating
further documentation, improvement and propagation of the instrument’s design, performance and pedagogy.
Keywords: Performance Parameter, Free-Reed

1. INTRODUCTION

The Sheng is a Chinese free-reed instrument (mouth organ) with documentation dating back to1100 BCE, and has extensive related counterparts in other East Asian cultures, e.g. Saenghwang in
Korea, Shō in Japan and Khene in Laos. As a mouth organ it consists of a series of pipe resonators
coupled, with a free-reed coupled to each pipe, and the reeds share a common pressure reservoir(“wind chamber”) which may be driven by both inhaling (draw) and exhaling (blow).
The acoustics of free-reed oscillators have been studied [1-3] and their mechanisms in interaction
with the corresponding pipe resonators are now somewhat well-understood. However, investigationsof the Sheng as a system of free-reed and pipe resonators, sharing a common pressure reservoir andhomogenous design, have not been previously studied under performance contexts.
In this study, we survey the full range of a contemporary “alto” Sheng for its performanceparameters in terms of loudness vs pitch variation: across its 3 octave chromatic range, for 3 dynamic
levels (pp, mf and ff), and for blow vs draw gestures. We accordingly relate the output performanceparameters in terms of the player’s input of blowing pressure and how it changes with respect tomusical dynamic, output sound level and pitch deviation. This complements earlier surveys of other
Chinese musical instruments [4-6].

2. METHODOLOGY

A concert-grade “alto” (Zhongyin) Sheng was chosen: it plays 3-octaves chromatically from C3 to
B5, and is tuned nominally to equal-temperament (A4 = 442 Hz). As a concert-grade instrument, we
thus consider this instrument to be an idealized representations of a typical “alto” (Zhongyin) Sheng.
An expert Sheng player was engaged to play the instrument, and the player is a professional memberof a traditional musical ensemble, frequent concert soloist, winner of international performancecompetitions and has over thirty years of performance and teaching experience – we may thus
similarly consider him to be an idealized representation of a typical Sheng expert player.
To understand the relationship between the Sheng’s relative SPL (‘loudness’) and pitch across its
full tessitura, the player was asked to play C3 to B5 chromatically (36 notes), repeating each pitch for
4 sets of draw vs blow gestures, sounded at three dynamic levels: pp, mf and ff (where pp means “assoft as possible” and ff means “as loud as possible” musically, while mf was “moderately loud”),

1

jerming_chen@sutd.edu.sg

holding each note at least three seconds. In many wind instruments, pitch and sound pressure level(“loudness”) are inherently interrelated [7-10] so we wish to understand how these performanceparameters interact on the Sheng – a free-reed instrument – and investigate how difficult (or easy) it
is for an expert to deliver ‘on-pitch’ while assigned a particular musical dynamic level whilst
maintaining musicality.
Audio measurements were collected in a recording studio treated for noise isolation and absorption,with the player-instrument facing the microphone (Røde NT5) 1 m away. The prevailing soundpressure level (SPL) is calibrated against a constant excitation source of known SPL sited at the same
position as the player with respect to the microphone. Pitch and relative sound pressure of each note
played (segment of stable pitch identified by ear) was extracted using Adobe Audition CC 2018 andextracted to a resolution of 1 cent and 0.1 dB respectively.

3. RESULTS & DISCUSSION
3.1 Dynamic Levels and Pitch Deviation
The pitch deviation for all sounded notes (C3 to B5 chromatically, repeating each pitch for 4 setsof draw vs blow gestures, played at three dynamic levels: pp, mf and ff) are summarized in Figure 1.

Figure 1. Pitch Deviation (cents) vs Chromatic Notes sounded from C3 to B5 on the alto Sheng, forthree dynamic levels (pp, mf, ff) and for four repetitions of draw vs blow gestures.
Of the three octaves surveyed, most of the notes observed – regardless of dynamic level – in theupper octaves have a sounding range of pitches deviating by about Δ10 cents, while broader ranges(Δ20 cents) can be seen in the lowest octave. Some notes seem to suffer from poorer pitch stability,e.g. C#3, G3, D4, E5, where the pitch range approach Δ40 cents.
In terms of ‘tonal’ center, the second octave seems generally sharper, centering around +10 cents,
in contrast to the first and third octaves which center around 0 cents.
For wind instruments in general, playing at louder musical dynamics is usually associated withslightly raised pitch, however for the Sheng this dynamic-pitch bias is not generally observed here: infact, the reverse can be seen, where softer dynamic results in pitch sharpening (e.g. C#3, G3, F4, B4,
C5, C#5, F5, B5, etc). At the same time, louder dynamic results in pitch sharpening can still be seen
in some instances (e.g. C3, E3, F3, F#3, D4, D#4, A#4, C#5, etc). Thus, as both effects are observed
and well-spread across the instrument’s tessitura, it is possible any dynamic-pitch bias is idiosyncratic
for that particular reed-pipe configuration, rather than a function of the instrument maker’s deliberate
design.

In terms of pitch differences arising from blow (blue) vs draw (red) gestures, there are no clear
trends observed, except for the note A4, and D4, where a loud draw is consistently sharper (~10 cents)
than a loud blow.
In the second and third octaves, it is also interesting to observe modest local pitch deviations that
repeat with octave-level correspondences, for example between D4-G4 and D5-G5, there is a slightrise in pitch centered around E and F. Although the Sheng is expected to nominally tune for equaltemperament [11], there may be some evidence of a non-equal temperament being favored here.
3.2 Dynamic Levels and Sound Pressure Level
Figure 2 summarizes the sound pressure levels for all notes (C3 to B5 chromatically, repeatingeach pitch for 4 sets of draw vs blow gestures) sounded at three dynamic levels: pp, mf and ff.

Figure 2. Relative Sound Pressure (dB) vs Chromatic Notes sounded from C3 to B5 on the alto
Sheng, for three dynamic levels (pp, mf, ff) and for four repetitions of draw vs blow gestures. ‘Max’and ‘Min’ refer to the maximum positive pressure and minimum negative (AC) pressure signal.
In general, we note a clear separation of sound pressure levels according to the musical dynamic
delivered – this is not unexpected, as musical dynamic and perceived loudness are inherently
correlated. However, it is satisfying to see that the player is able to achieve fairly discrete dynamic
regions without much overlap (first octave is the exception, where particularly C3-D3 have extensive
overlap between ff and mf). The second octave plays the loudest while the third octave is the softest,
suggesting that this alto sheng is designed to be most efficient for notes in the middle of the
instrument’s tessitura.
Each note on the Sheng has a dynamic range of ~20-25 dB; within that range, each dynamic level(pp, mf, ff) has up to ~7 dB of variability, indicating the extent to which an expert player has dynamic
control.
In terms of blow vs draw gestures, no clear relationships are observed – neither blow nor draw
have a systematic influence on sound pressure and in many instances, they can be comparably loud.
Furthermore, different notes show different responses with regards to the direct of draw vs blow:“positive bias” (where ‘max’ > ‘min’ pressure magnitude signal, e.g. D#3, F4, G#5 ff), “negative bias”(where ‘max’ < ‘min’ pressure magnitude signal, e.g. D4, D5) or “zero bias” (where ‘max’ ≈ ‘min’pressure magnitude signal, e.g. F#3, F#4, B4).

4. CONCLUSION

In this extended abstract, we surveyed and report the pitch deviation and relative sound pressurelevel of a typical concert-grade “alto” (Zhongyin) Sheng when an expert player performed the
instrument’s standard tessitura at three dynamic levels – the operating performance parameter spaceof the instrument is thus identified and outlined.
Unlike other wind instruments, the Sheng – being a free-reed instrument – does not necessarily
show sharpening pitch deviation with increasing dynamic level; instead, the reverse can be observed.
Nonetheless, we observe that the middle and upper octave of the Sheng has generally very good pitch
consistency independent of dynamic level. No systematic difference in pitch was also observed withregard to draw vs blow gestures.
In terms of sound pressure level, we observe the Sheng to be most efficient at the middle of its
tessitura. A typical Sheng each note has a dynamic range of ~20-25 dB and an expert player can deliver
each dynamic level to ~7 dB. Again, no systematic difference between draw and blow gestures areobserved, in terms of dynamic levels. Further, we note three types of ‘pressure bias’ responses are
possible, regardless of direction of draw/blow.
These survey of the performance parameter space of the Sheng will be useful for both performers,
instrument makers and composers.

REFERENCES

1. Cottingham, J. P. (2002). The Asian free-reed mouth organs. Journal of the Acoustical Society of America,
112(5), 2365-2365.
2. Kuang, W., Leistner, P., Angster, J., Dubovski, Z., Miklos, A., & Yang, J. (2014). Simultaneous detection
and analysis of reed vibration and sound of sheng pipes. In Proceedings of the 21st International Congress
on Sound and Vibration (pp. 4783-4790).
3. Cottingham, J. (2011). Acoustics of free-reed instruments. Physics Today, 64(3), 44-48.
4. Zhao, Y. Z., Wu, S. X., Qiu, J. Z., Huang, H., & Wu, L. L. (2010). Sound intensity radiated by gaohu.
Science China Technological Sciences, 53(7), 1792–1797. https://doi.org/10.1007/s11431-010-4002-8
5. Li, N., Zhao, Y., Wu, S., Huang, H., & Wu, L. (2012). Sound power level measurement of Chinese bowedstringed instrument-gaoyinbanhu. The Journal of the Acoustical Society of America, 131(4), 3219–3219.
https://doi.org/10.1121/1.4708005
6. Qiu, J., Wu, S., Zhao, Y., Huang, H., & Wu, L. (2021). Sound power characteristics of Chinese traditional wind instrument Qudi. International Journal of Architecture, Arts and Applications, 7(1), 8.
https://doi.org/10.11648/j.ijaaa.20210701.12
7. Wolfe, J., Fletcher, N. H., & Smith, J. (2015). The interactions between wind instruments and their players.
Acta Acustica United with Acustica, 101(2), 211-223. https://doi.org/10.3813/AAA.918820
8. De la Cuadra, P., Fabre, B., Montgermont, N., & Chafe, C. (2005). Analysis of Flute Control parameters:
A comparison between a novice and an Experienced Flautist. Acta Acustica United with Acustica, 94(5),740-749. https://doi.org/10.3813/AAA.918091
9. Fuks, L. & Sundberg, J. (1996) “Blowing Pressures in Reed Woodwind Instruments”, Department for
Speech, Music and Hearing. Quarterly Progress and Status Reports (TMH-QPSR), 37(3), 41–56.
10. Almeida, A., George, D., Smith, J., & Wolfe, J. (2013). The clarinet: How blowing pressure, Lip Force,lip position and Reed “hardness” affect pitch, sound level, and spectrum. The Journal of the AcousticalSociety of America, 134(3), 2247–2255. https://doi.org/10.1121/1.4816538
11. Wang, C., Chow, J. Y., & Wong, S. S. (2020). The Teng Guide to the Chinese Orchestra. World Scientific
Publishing Co. Pte. Ltd and The Teng Ensemble Ltd.

ABS-0929

Bowing virtual strings with realistic haptic feedback
Marius-George Onofrei(1) , Federico Fontana(2) , Silvin Willemsen(3) ,Stefania Serafin(4)(1) Multisensory Experience Lab, CREATE, Aalborg University, Denmark, mariusgono@gmail.com(2) Human Computer Interaction Lab, University of Udine, Italy, federico.fontana@uniud.it(3) Multisensory Experience Lab, CREATE, Aalborg University, Denmark, sil@create.aau.dk(4) Multisensory Experience Lab, CREATE, Aalborg University, Denmark, sts@create.aau.dk

ABSTRACT
We present a music interface implementing a bowed string. The bow is realised using a commercially available haptic device, consisting of a stylus attached to a robotic arm. While playing the virtual strings with thestylus reproducing the bow, users feel both the elastic force from the strings and the friction resulting from theinteraction with their surfaces. The audio-haptic feedback is obtained by a physical model: four stiff strings
are simulated using a finite difference time domain method, modelled as 1-Delements in the virtual 3-D space.
The bow is simply modelled as a rigid cylinder that can move free in this space, and interact with the strings.
Finally, the frictional interaction between such elements is modelled by a nonlinear friction model capable ofreproducing the characteristic stick-slip phenomenon observed during string bowing. Moreover, the model canbe dynamically controlled in one parameter so as to become more sticky or slippery. By turning on and off
the frictional feedback, users can appreciate the significance of this interaction. A real-time visualisation of thebowed strings complements the audio-haptic display.

Keywords: Musical interface, Digital music instrument, Haptic feedback, Bowed string model, Finite
difference

1

INTRODUCTION

In traditional musical performance, there exists a physical coupling between the musician and their mechanicalinstrument. This follows, as highlighted by Luciani et al. [1], from the energetic exchange between the human
instrumental gestures and the sound being produced. According to O’Modharin and Gillespie [2], this energetic
flow starts from the mechanical energy injected into the instrument by the performer, whose aim is to produce
acoustic energy via the instrument’s resonance. Some of this injected energy is stored in the instrument itself,
which is to be reflected back to the performer at some point, e.g. in the form of haptic feedback. They conclude
that this back and forth coupling of energy informs “the idea that an instrument becomes the extension of aplayer’s body”, a feeling that is shared by many musicians according to Nijs et al. [3].
In contrast to this energetic feedback loop interaction, the performance of digital musical instruments (DMIs)
typically has a unidirectional energy path, namely from the musician towards the instrument. Such instruments
are defined according to Miranda and Wanderley [4] as devices split between two independent units: a control
surface and a sound generation unit, linked together by mapping strategies. However, this independence does
not have to be the norm and it follows perhaps due to an aim toward generality in terms of possible sounds fora given control surface—at least for the case of software sound generating units, e.g. illustrated by the ubiquityof the MIDI keyboard as a possible controller for almost any software instrument.
DMIs are increasingly becoming a larger part of music production. In the case of pop music, the backboneof a music market sitting at an all time peak [5], synthesizers and samplers provide most if not all of the sounds
on many recordings, according to Warner [6]. The accessibility of such instruments as software applications
gives rise to large communities of learners and creators who share, play and teach music on web platforms
further redefining notions of music making [7] and reinforcing the position of DMIs as part of the general

musical canon. Therefore, one could argue that there should be a greater emphasis on DMIs in the field ofmusician-instrument interaction.
As part of a more general trend of haptic based applications in multimedia [8], audio-haptic musical applications based on interactions with virtual strings have shown a growing interest in the sound and music computing
community. Willemsen et al. [9] used an older version of the Touch haptic device used in the current project,
to control a physical model of a tromba marina—a bowed single-string instrument, in a virtual reality (VR)environment. The interaction included the elastic response of the string (through the collision between VR objects) but no frictional feedback during bowing was included. Keeping in the realm of VR, Fontana et al. [10]as well as Passalenti et al. [11] investigated the effect of haptic feedback with respect to plucked guitar string
simulations. Concerning bowed strings, Sinclair et al. [12] included frictional force feedback in their digital
waveguide based model, but however, it resulted from a different model than the one used for auditory feedback. Other custom-made haptic interfaces, including bowing, have been designed by researchers at ACROE in
Grenoble to control physical models built from systems of interconnected mass-springs [13, 14, 15]. A project
presenting a user study comparing the experience of rubbing virtual mass-spring-dampers by means of different
control strategies, including the use of a 3D Systems haptic device, was carried out by Onofrei et al. [16]. The
aim of the different controls was to mimic with increasing accuracy the gestural action of rubbing, culminatingwith the presence of frictional haptic feedback resulting from the actual simulation of the frictional interaction,i.e. the haptic friction force was the same as the force which excited the sound synthesis model. The music
interface appearing in this paper is rooted on the design just described.
In this paper we present an implementation of a real-time physically-modeled music application simulating
the bowing of strings, controlled via an interface which allows not only for intuitive instrumental gestures but
also for realistic mechanical feedback as a result of the physical simulation. This is achieved by making use of
the features of a 3D Systems Touch haptic device: in particular, by mapping the virtual bow-string interaction
point on the tip of its stylus and by rendering both the elasticity and the frictional force of the strings, as
resulting from the interaction of the bow with their surfaces. The strings are placed in a 3-D virtual space in a
trapezoidal cross section, similar to how they would be placed on the neck of a violin. This allows for the bow
to interact either with individual strings but also with pairs of strings at a time, as they lie on the same plane.
Additionally, the displacement of the elements in the 3-D space is visualized at runtime, complementing theaudio-haptic interaction. The setup aims at preserving a consistent link between the control and the synthesizedsound, as suggested by Cadoz [17], with the energetic feedback loop of the instrumental gesture together withthe haptic feedback giving rise to subtle nuances in the timbre and the dynamics of the sound. This link isgreatly facilitated by the use of physical modelling synthesis, as all the different modalities—sonic, tactile andvisual—result from the solution of the same mechanical system.
The rest of this paper is structured as follows: Section 2 presents the physical model behind the audio-haptic
synthesis, Section 3 describes the control interface and introduces the haptic device, then Section 4 presents thereal-time application illustrating the graphical user interface (GUI) and details the mapping between the app(particularly the physical model behind it) and the haptic device. A discussion follows in Section 5 focused on
the playability of the app, while the final Section 6 concludes the paper and introduces possible future work.

2

PHYSICAL MODEL

This section describes the physical model used in this project and provides some considerations for its discretisation.
Consider a damped stiff string with a circular cross-section of length L (in m) defined for time t ≥ 0 (in s)
and space x ∈ D (in m) with domain D = [0, L]. One can describe its transverse displacement by state variableu = u(x,t) (in m), and after adding a bow excitation, its dynamics can be described by the following partial
differential equation (PDE) [18, 19]∂t2 u = c2 ∂x2 u − κ 2 ∂x4 u − 2σ0 ∂t u + 2σ1 ∂t ∂x2 u − δ (x − xB )fB
Φ(vrel )ρA(1)

where ∂t and ∂x describe
p a derivative with respect to time and space respectively. The parameters are as
follows: wave speed c = T /ρA (in m/s) with tension T (in N), material density ρ (in kg/m3 ), cross-sectional

parea A = πr2 (in m2 ), radius r (in m), stiffness coefficient κ = EI/ρA (in m2 /s), Young’s modulus E (in Pa),
area moment of inertia I = πr4 /4, frequency-independent loss coefficient σ0 (in s−1 ) and frequency-dependent
loss coefficient σ1 (in m2 /s). The final term in Eq. (1) describes the bow frictional force excitation, Ff r (in N),
using the following friction model [19]:√2(2)
Φ(vrel ) = 2avrel e−avrel +1/2 ,characterized by the friction parameter a (in s2 /m2 ). Equation (2) is nonlinearly dependent on the relative
velocity between the string—at externally supplied bow location xB = xB (t) ∈ D (in m)—and the bow:vrel = vrel (t) = ∂t u(xB ,t) − vB ,(3)

where vB = vB (t) is the externally supplied bow velocity (in m/s). The bow is located along the string using
the spatial Dirac delta function δ (x − xB ). Finally, the externally supplied bow force is fB = fB (t) (in N). In
this work, the boundary conditions of Eq. (1) are chosen to be simply supported such thatu = ∂x2 u = 0,

at x = 0, L.

(4)

2.1 Discretisation
This work uses finite-difference time-domain (FDTD) methods to approximate the equations above. These methods discretize the continuous system into a grid in space and time according to x = lh with spatial index l andgrid spacing h (in m), and t = nk with temporal index n and time step k = 1/ fs (in s) and sample rate fs (inHz). The state variable u(x,t) then becomes a grid function unl which describes a grid point with spatial index
l at temporal index n. A full discretisation of the bowed stiff string will not be given here for brevity, but iswell covered in the literature (see e.g. [19]). A full derivation of the scheme used in this work can be foundin [20, Sec. 8.4]. Instead, several considerations for discrete implementation will be given here. Notice that thevarious parameters receive a superscript n to indicate that they are time-varying.
For approximating first-order temporal derivatives there are various options, including the forward, backward
and centred differences [19]. The latter can be proven to be second-order accurate and is defined as

1 n+1
u
− uln−1 .
2k l
In this work, this approximation is used to discretise Equation (3) yielding
δt· unl =vnrel = Il (xBn )δt· unl − vnB

(5)

(6)

where Il (xBn ) is an interpolation operator retrieving the state of the discrete string at location xBn , and is definedas [19]−αi (αi − 1)(αi − 2)/6,l = li − 1,





(α
−
1)(α
+
1)(α
−
2)/2,
l = li ,
i
i
 i
(7)
Il (xi ) = −αi (αi + 1)(αi − 2)/2,l = li + 1,



α
(α
+
1)(α
−
1)/6,
l
=
l
+
2,

i i
i
i


0,
otherwise,with li = floor(xi /h) and αi = xi /h − li . As vnrel is used in a nonlinear function (see Equation (2)), the scheme is
now implicitly dependent on un+1
. In this work, the iterative Newton-Raphson method is used at every samplinglstep to solve the nonlinear bow function with a maximum of 99 iterations per sample.
The output of the discrete system can be retrieved by selecting a grid point and ‘listening’ to this over time.
As the displacement of a medium is related to sound pressure through a temporal derivative [21], the output isretrieved as follows:out[n] = Il (xo )δt· unl(8)where xo = 0.7L is the output location (in m).
Finally, the spatial Dirac delta function in Eq. (1) is discretised using a cubic spreading operator Jl (xB ) =Il (xB )/h making changes in the bow location smooth.

ubp [m]4#10 -4

0.8
0

0.6

-2
-4
0.5

0.505

0.51

0.515

0.52

0.525

0.53

0.535

0.54

0

-0.5

-1
0.5

0.4

)(vrel ) [-]vrel [m/s]

1

2

0.505

0.51

0.515

0.52

0.525

0.53

0.535

0.2
0
-0.2

0.54

-0.4

Ff r [N]

0.2

-0.6

0.1
0

-0.8
-0.1
-0.2
0.5

0.505

0.51

0.515

0.52

0.525

t [s]
(a)

0.53

0.535

0.54

-1
-0.6

-0.4

-0.2

0

0.2

0.4

0.6

vrel [m/s](b)

Figure 1. (a) Simulation results for the bowing of a stiff string tuned to 196 Hz. Top: String displacement
at bowing position, ubp (in m). Middle: Relative velocity between the string and the bow at bowing position,
vrel in in (m2 /s2 ). Bottom: Resulting frictional force at bowing excitation point, Ff r n in (N). (b) Shape of thedimensionless nonlinear friction characteristic, Φ(vrel ) using the friction parameter a = 80 (in s2 /m2 )2.2 Prototype model results
A prototype implementation of the physical model was first carried out in an offline setting using Matlab1 . This
allowed for more flexibility with respect to investigating the results and calibrating the physical parameters, with
their choice based on [20].
Of notable interest was the behavior of the friction model and its capability of producing the stick-slip
behavior typical of frictional interaction, which gives rise to the experimentally observed Helmholtz motion of
bowed strings [22].
Figure 1a shows the displacement of the string at the bowing position, u(xB ), with xB = 0.25L, together withthe relative velocity between the bow and the string vrel as well as the resulting frictional force resulting fromthe bowing simulation of a steel string tuned to a fundamental frequency of 196 Hz, at a sampling frequency
fS of 44100 Hz. The tuning is achieved by setting the relevant string tension T in the model. Other physicalparameters of the string are: L = 0.5, ρ = 7850, r = 0.0005, E = 2 · 1011 , typical to a steel violin string.
Furthermore, the damping parameters are set to zero and the friction parameter is taken as a = 80, resulting
in the shape of the nonlinear friction characteristic Φ(vrel ) shown in Figure 1b. The string is bowed with aconstant bow velocity vB = 0.2 and bow force fB = 0.15. Units for these parameters were introduced in theprevious section. The stick-slip behavior can be seen when looking at vrel , with the value hovering over zerofor the most part of the periodic motion, indicating that the bow and the string are stuck together, followed by
an abrupt slip in which the magnitude of vrel greatly increases. This results in a triangular wave shape in thedisplacement of the string. Additionally, during the slipping phase, the resulting frictional force can be observed
to be zero, as there is essentially a loss of contact between the bow and the string, i.e. no interaction.

3

CONTROL INTERFACE

To control the physical model system described in Section 2, a suitable interface needs to be found. The
requirements of this interface are: 1) to enable flexibility in terms of instrumental gesture motion, and 2) toallow for the corresponding instantaneous haptic feedback.
1 https://www.mathworks.com/

Figure 2. 3D Systems Touch haptic device.
3.1 Haptic Device
We found that the Touch professional haptic device manufactured by 3D Systems [23] is an ideal candidate forthis task. It is a 6-degree of freedom haptic system equipped with silent internal motors which can provide
force feedback in 3-degrees of freedom. The device is illustrated in Figure 2 together with its local coordinate
system (CSYS) and the pivot joints B1, B2 and B3 which enable the translation in x, y, z directions of thegimbal joint (equivalent to pivot joint B2) and the 3 possible rotations of the stylus pen. This flexibility of
motion in the 3-D space allows for replicating the gestural motion of bowing. In fact, one can hold the stylus
in a similar fashion as they would hold an actual bow. Furthermore, the pivot joints A1, A2 and A3 provide
force feedback, as opposed to the B joints which are only for rotational degrees of freedom. The combinationof the force feedback in the A joints is enough to produce unique haptic response at the gimbal position ineach of its three translation directions.
The maximum workspace dimensions of the device, resulting from its mechanical limits are [-210, 210] (inmm) in x direction (width), [-110, 205] (in mm) in the y direction (height) and [-85,130] (in mm) in z direction(depth). These represent the bounds of the haptic domain, in which the position of the gimbal joint will always
lie. This positional information as well as the orientation of the stylus can be accessed from the device using
the OpenHaptics API [24]. Another functionality of interest part of the API, relevant to the mapping strategy ofthe device to the physical model used for synthesis—described in more detail in the next section—is that haptic
force feedback can be set at the location of the gimbal joint. This is achieved by the use of callback functions
inside a high-priority scheduler thread. For the current application, the refresh rate of these actions is set to 1
kHz, by adjusting the number of times the scheduler ticks its callbacks every second.

4

REAL-TIME APPLICATIONThe real-time application was developed in C++ using the JUCE framework2 . It is open-source and available at[25] and a demo video can be found at [26].
A total of four simply supported damped stiff strings, modelled as per the details given in Section 2, areplaced in a virtual 3-D space in a trapezoidal cross-section. By adjusting the tension parameter T the strings are
tuned to the fundamental frequencies of the strings of a violin, i.e., 196.0, 293.6, 440.0 and 659.2 Hz, whichcorrespond to the musical notes: G3, D4, A4 and E5 respectively. The approach for tuning as well as typicalparameters for such strings can be found in [20]. The strings can be excited by a perfectly rigid bow, with theirinteraction described by the friction model given in Equation (2). The CSYS of this virtual space is identicalto the one of the Touch haptic device. Both the strings and the bow are modelled as infinitely thin lines.
2 https://www.juce.com/

Figure 3. Snapshot from the demo video [26] of the digital music instrument showing both the software application window as well as the control via the haptic device.
4.1 Graphical user interface (GUI)A snapshot of the application during use together with the its external control, taken from the demo video, canbe seen in Figure 3. The window is divided into 3 areas of interest. On the right, a top view of the strings
can be seen, i.e. the strings are projected onto the xz horizontal plane. The long transparent orange rectangle
is the bow, whose color is grey when not in contact with any of the strings, e.g. when hovering above themin the 3-D virtual space. Its opacity is directly proportional to the externally supplied bow force, fB , meaningthat when the bow exerts more pressure onto the strings the transparency of its color is reduced, until becoming
fully opaque when the force becomes largest. A copy of this same bow and strings can be seen on the top
left part of the GUI window, viewed as projected in the frontal plane xy. The maximum displacement of thestrings in this view is illustrated as red ellipses placed underneath the static cyan ones, which represent the "atrest" position. Finally, in the bottom left part of the application window, a number of knobs controlling various
parameters in the physical model can be found as well as a button which turns on or off the frictional haptic
feedback in the control interaction. Going from left to right the knobs control the: damping amount, which
is a combination of the frequency in-dependent damping, σ0 ∈ [0.1, 10], and the frequency dependent damping,σ1 ∈ [0.0001, 0.015], introduced in Section 2. The choice of combining them is for ease of use for users who
are unfamiliar with the details of the physical model underlying the simulation. Their ranges are chosen suchthat the control of the knob gives either a very large release of the sound or almost none at all. The second
knob controls the friction parameter, a ∈ [0.01, 15000], mapped logarithmically such that the friction interaction
between the bow and the strings smoothly goes from "slippery" to "sticky". Lastly, a dimensionless global
volume gain knob is included, controlling the master volume of the app.
Both the strings and the bow elements are depicted in the GUI as having some spatial volume: a width andheight in addition to their segment length, forming for example the cross-section of the strings in the upper-left
illustration. This is not the case for the virtual 3-D space of the physical model, where they are in fact infinitely
thin (1-Delements), thus simplifying their interactions.
4.2 Haptic mapping
As previously mentioned, the stylus of the Touch device is an excellent control interface for a virtual bow.
The position of the gimbal joint is mapped to the center of the bow in the virtual physical model space. Itsorientation is extracted from the interface of the program with OpenHaptics API as the rotation angles of thestylus relative to the haptic CSYS, shown in Figure 2. For ease of control, the orientation of the bow in thephysical model is fixed to be parallel with the frontal plane, i.e. xy, meanwhile allowed to rotate with respectto the horizontal plane xz. This means the bow is always perpendicular to the strings in the horizontal plane
projection. Moving the stylus in the haptic space is equivalent to moving the bow in the physical model space.
In the haptic feedback callback function code, the shortest distance between the bow and each string is
calculated at every haptic frame as the shortest line/segment perpendicular to both elements. The exact end
point coordinates of this segment can also be found, keeping track of which point lies on the string and which
one lies on the bow. Knowing this, the distance can be calculated relative to a direction vector going fromthe point on the string towards the point on the bow, therefore, when the distance is negative we know the
bow is pushing onto the string. An elastic spring force equal to some heuristically chosen spring stiffness
times the magnitude of this distance is then sent to the gimbal joint point, with an orientation given by the
direction vector. This is how the user can feel each of the strings in the virtual space. When the bow is incontact with multiple strings, the reaction forces from all of them are summed up and sent as haptic feedback.
Through linear mapping, this force is directly proportional to the externally supplied bow force in the physicalmodel, fB , with a maximum value fB = 2, hence the force with which the user presses onto the haptic strings
gives the first audio synthesis model input. The second input, the externally supplied bow velocity vB (in m/s),
is similarly retrieved from the gestural motion of the user. It is again a linear mapping to some reasonablephysical values: between -0.4 and 0.4 m/s, see [27], of the change in position of the gimbal joint in the 3-D
haptic space between subsequent scheduler ticks. Essentially it is retrieved from the velocity of the stylus.
During the interaction of the bow with any of the strings, the frictional force resulting from the physicalmodel is sent as haptic feedback to the gimbal joint, split in x, y, z components given by the opposite direction of motion of the bow. As for the case of the elastic feedback, frictional forces resulting from multiple
synchronous interactions are summed up.

5

DISCUSSION

The DMI as prsened in the previous sections, i.e. the software application as sound generation unit and the
Touch device as interface, is capable of producing "good" sounds without much practice. This is most likely
due to the mapping choice of the gestural motion of the user to the input parameters of the friction model: thebow force fB and the bow velocity vB , which are limited in ranges that are relevant for the current physicalmodel. Additionally, the haptic feedback, helps restrain the user from applying large bow forces that result inraucous motion of the string, corresponding to scratchy sound. This is achieved both by the elastic feedback of
the strings, as well as the fact that the magnitude of the frictional force—and the haptic feedback it produces,increases with applied bow force. With some practice, the user can achieve also a "whistling" type of sound,resulting from multiple slipping of the bow, with two or more slipping phases per fundamental period [27].
What does take some practice is the ability to easily change from bowing individual strings to pairs ofstrings. With this, more interesting sounds can be achieved especially by modulating the applied bow force
individually for different strings. The ability to bow the strings at a fixed location along their length also
requires some rehearsing, as the user needs to get familiar with the mechanical response of the haptic device,
independent of the haptic feedback. This reinforces the necessity to use the cubic interpolation operator given
in Equation 7, used to place the bow on the string, in order to avoid clicks which would otherwise result fromsudden jumps in the excitation location. The real-time visual feedback shown in the GUI helps the user better
evaluate their movements in the 3-D virtual space.
Also, once the user becomes more accustomed to the control, variations in the attack and timbre of thesound can be achieved effortlessly. The modulation of the "damping amount" knob can be used to change therelease of the sound in real-time and the "friction amount" knob can be used to increase the "stickiness" of theinteraction and achieve a more plucked type sound.

6

CONCLUSION AND FUTURE WORKThis paper presents the implementation of a bowed strings musical interface. The audio-haptic synthesis isachieved by physical modelling, with the transverse displacement of a string being described by a PDE encompassing the string’s physical properties and an external excitation. This excitation is provided by the contact
with a perfectly rigid bow, and is described by a nonlinear frictional model, capable of exhibiting physicallyconsistent behavior observed in bowed string experiments, particularly the stick-slip interaction between the twoelements. The numerical simulation of this system is carried out using FDTD methods and is implemented in areal-time software application written in C++. This app is then controlled via an external haptic device, the 3D
Systems Touch, which allows for the inclusion of the haptic feedback resulting from the simulation. The actual
frictional force which is used to excite the strings, resulting from the physical model, is felt by the user whenbowing the strings. Both the audio feedback and the frictional feedback are directly linked to the instrumental
gesture, allowing for a more personal experience with the instrument as opposed to typical DMIs whose control
is often disconnected from the underlying audio synthesis model.
Future work can involve including an additional control with the hand which is not holding the stylus of thehaptic device. This control could be for instance a haptic glove with which the user could press the different
strings against a virtual neck, changing the notes being played or slightly damping the strings and introducing
harmonics. A sound-board could be added to the strings, simulating the body of the violin, or perhaps theoutput sound could be convolved in real-time with the impulse response of the body of a real violin, using
partitioned convolution algorithms. Also, spring connections—potentially nonlinear—could be added between
various strings, thus creating a more unphysical instrument which could be played in a physical manner. Alsoof great interest may be developing cheaper alternative haptic devices for the control, as the price of the Touchcurrently limits the accessibility of the current instrument.

REFERENCES[1] Luciani A, Florens JL, Couroussé D, Castet J. Ergotic Sounds: A New Way to Improve Playability, Believability and Presence of Virtual Musical Instruments. Journal of New Music Research. 2009
Sep;38(3):309–23.
[2] Papetti S, Saitis C, editors. Musical Haptics [Internet]. Cham: Springer International Publishing; 2018 [cited 2022 Jul 15]. (Springer Series on Touch and Haptic Systems). Available from:http://link.springer.com/10.1007/978-3-319-58316-7[3] Nijs L, Lesaffre M, Leman M. The musical instrument as a natural extension of the musician. In: 5th
Conference of Interdisciplinary Musicology (Paris: LAM-Institut jean Le Rond d’Alembert), 132–133.
[4] Miranda ER, Wanderley MM. New Digital Musical Instruments: Control And Interaction Beyond the Keyboard. In: Computer Music and Digital Audio Series. 2006.
[5] International Federation of the Phonographic Industry (IFPI) Global Report 2022; Available from:https://www.ifpi.org/wp-content/uploads/2022/04/IFPI_Global_Music_Report_2022-State_of_the_Industry.pdf[6] Warner T. Pop Music Technology and Creativity Trevor Horn and the Digital Revolution. Routledge Publishing, 2017, https://doi.org/10.4324/9781351218504[7] Gouzouasis P, Bakan D. The future of music making and music education in a transformative digital world.
2011;2(2):22. UNESCO Observatory Refereed E-Journal, Multi-Disiplinary Research in the Arts. 2.
[8] Magnenat-Thalmann N, Bonanni U. Haptics in Virtual Reality and Multimedia. In: IEEE Multimedia. 2006
Jul;13(3):6–11.
[9] S. Willemsen, R. Paisa, and S. Serafin, “Resurrecting the tromba marina: A bowed virtual reality instrument using haptic feedback and accurate physical modelling,” in Proceedings of the 17th Sound and Music
Computing Conference, 2020.
[10] Fontana F, Paisa R, Ranon R, Serafin S. Multisensory Plucked Instrument Modeling in Unity3-D: From
Keytar to Accurate String Prototyping. Applied Sciences. 2020 Feb 21;10(4):1452.
[11] Passalenti A, Paisa R, Nilsson NC, Andersson NS, Fontana F, Nordahl R, et al. No Strings Attached: Forceand Vibrotactile Feedback in a Virtual Guitar Simulation. In: 2019 IEEE Conference on Virtual Reality and3-D User Interfaces (VR) [Internet]. Osaka, Japan: IEEE; 2019 [cited 2022 Jul 15]. p. 1116–7. Available
from: https://ieeexplore.ieee.org/document/8798168/[12] Sinclair S. Audio-haptic interaction with the digital waveguide bowed string. In: Proceedings of the 2009
International Computer Music Conference, ICMC 2009, 2012.
[13] Florens JL, Cadoz C. Modèles et simulation en temps réel de corde frottée. In: Le Journal de Physique
Colloques, vol. 51, no. C2, pp. C2–873–C2–876, 1990.
[14] Luciani A, Florens J, Castagne N. From Action to Sound: A Challenging Perspective for Haptics. In:
First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment andTeleoperator Systems [Internet]. Pisa, Italy: IEEE; 2005 [cited 2022 Jul 15]. p. 592–5. Available from:http://ieeexplore.ieee.org/document/1407022/[15] Leonard J, Villeneuve J. Design and Implementation of Real-Time Physically-Based Virtual Musical Instruments: A Balancing Act. In: Sound and Music Computing Conference 2020 Jun 17[16] Onofrei MG, Fontana F, Serafin S. Rubbing a Physics Based Synthesis Model: From Mouse Control to
Frictional Haptic Feedback. In: Proceedings of the 19th Sound and Music Computing Conference, 2022.
[17] Cadoz C. Instrumental gestures and musical composition. In: ICMC, 1988.
[18] Bensa J, Bilbao S, Kronland-Martinet R, Smith JO. The simulation of piano string vibration: From physicalmodels to Finite Difference Schemes and Digital Waveguides. J. Acoust. Soc. Am., vol. 114, no. 2, pp.
1095-1107, 2003.
[19] Bilbao S. Numerical Sound Synthesis, John Wiley & Sons, 2009.
[20] Willemsen S. The emulated ensemble: Realtime simulation of musical instruments using finitedifferencetime-domain methods. Ph.D. dissertation, Aalborg University, 2021.
[21] Serway RA, Jewett JW. Physics for scientists and engineers. 6th ed. Belmont, CA: Thomson-Brooks/Cole;2004. 1 p.
[22] Helmholtz HLF. On the sensations of tone. Dover Publications, Inc. New York, 1954. Original edition:
Die Lehre von den Tonempfindungen als physiologische Grundlage für die Theorie der Musik. Vieweg undSohn, Braunschweig, 1863.
[23] 3D Systems, Inc. 3D Systems Touch Haptic Device. accessed July 15,
https://www.3dsystems.com/haptics-devices/touch.

2022,

available at[24] 3D Systems, Inc. 3D Systems OpenHaptics Toolkit Version 3.5.0 API Reference Guide. available athttps://s3.amazonaws.com/dl.3dsystems.com/binaries/Sensable/OH/3.5/OpenHaptics_Toolkit_API_Reference_Guide.pdf[25] Onofrei MG. Github Project: Bowed_Strings_TouchCtrl. accessed July 15,
https://github.com/mariusono/Bowed_String_TouchCtrl

2022,

available at[26] Onofrei MG. Bowed virtual strings with realistic haptic feedback - demo. accessed July 15, 2022, available
at https://youtu.be/uPHZMpQ5Z0k[27] Schoonderwaldt S. Mechanics and acoustics of violin bowing. Ph.D. Dissertation, KTH Royal Institute ofTechnology, Stockholm, 2009.

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022, in Gyeongju, Korea
Extended abstractABS-0968

Presenting a Compendium of Dízi Impedance Spectra
Hsien Han NG; Da Yang TAN; Teck Seng NG; Jer-Ming CHEN 1
Singapore University of Technology & Design, Singapore

ABSTRACTThe Dízi is a popular Chinese transverse flute with a long-documented history, featuring a distinctive
membrane hole (‘mokong’), a bamboo body and typically six keyless fingerholes. Despite its construction
from raw bamboo stock with minimal shaping and geometric manipulation, skilled masters are able toconsistently create concert-grade instruments. We measured one such Dízi’s acoustic impedance spectra forall standard (and non-standard) fingerings to characterize the instrument’s acoustic response over its tessitura.
Consequently, we describe how instrument resonances vary with fingering choice (standard vs alternatives),
the effect of the membrane on resonance, and offer practical considerations with regards to performance
practice, instrument design, conservation and transmission of the Dízi.
Keywords: Acoustic Impedance, Transverse Flute

1. INTRODUCTION

The Dízi is a popular Chinese transverse flute with a long-documented history, featuring a
distinctive membrane hole (‘mokong’), a bamboo body and typically six keyless fingerholes (the Dízi
uses a seven-note diatonic scale and relies on “movable Do” solfège notation).
For traditional and cultural reasons [1], it retains a simple design and is constructed from raw
bamboo stock with minimal shaping and geometric manipulation. Because of the nature of bamboo(an organic material), the maker is limited by the “found” state of the bamboo blank available tohim/her: its internal and external dimensions, taper, uniformity, surface conditions, stiffness anddensity all vary one from the other. Consequently, skilled masters must rely on experience (priorknowledge, trial-and-error and ‘intuition’) to approach every bamboo blank uniquely to shape it(within construction/design parameters and geometric/material constraints) such that they canconsistently create concert-grade instruments that perform and interact with the player with consistent
acoustic characteristics (e.g. conforms to standard playing range and fingerings, similar embouchureand breath control responses) and reliable manner (delivers consistent playability, etc.).
To understand the dizi’s linear acoustics, its geometric design and performance implications, wemeasure a Dízi’s acoustic impedance spectra for all standard (and non-standard) fingerings tocharacterize the instrument’s acoustic response over its tessitura. Consequently, we describe how
instrument resonances vary with fingering choice (standard vs alternatives), the effect of themembrane on resonance, and offer practical considerations with regards to performance practice,
instrument design, conservation and transmission of the Dízi.
The impedance spectra measured for these fingerings constitute a database [2,3] of Dízi’s acoustic
impedance and sound spectra (here: https://acoustics.sutd.edu.sg/dizi-impedance/), and complements
an earlier database of impedance spectra presented on western classical and modern flutes [4,5].

2. METHODOLOGY

For this study, a typical concert-grade Dízi in the key of G (note: being keyless, the Dízi presents
as a family of instruments made in different keys) made by a Singaporean master Dízi-maker (coauthor T.S. Ng) was chosen. To measure the Dízi’s input acoustic impedance Z(f), the threemicrophone two-calibration (3M2C) technique [6] is used to achieve high precision over a wide
dynamic range. An impedance measurement head of 7.8 mm internal diameter coupled the Dízi at theembouchure hole [7] in accordance with previous studies on western classical flutes [4,5].
1

jerming_chen@sutd.edu.sg

The fingerings measured in this study were “played” on the setup by one of the authors, anexperienced Dízi performer. A total of 35 fingerings (both standard and alternative – Table 1) for 24
semi-chromatic notes D5 to G7 (i.e. typical playing range) were investigated on the G-key Dízi, bothwith the membrane attached and without (the membrane hole is sealed over using stiff adhesive tape).
The impedance measurements were completed in a single 3-hour session, such that ambient conditions
and measurement calibration are not expected to vary. To complement the impedance measurement ofeach fingering studied, an accompanying audio recording was made using that fingering on the Dízi
measured, played by the same Dízi performer, and the resulting pitch extracted for its fundamental
frequency using Adobe Audition. Importantly, the player was asked to perform each note to produce
the most ‘natural’ pitch elicited by the Dízi “as is”, without particular pitch adjustments toaccommodate any particular temperament preference in question.
Table 1. Fingering chart for standard and alternative fingerings measured on the G-Key Dizi; ● –closed tone hole, ○ – open tone hole, both ◒ & ◐ indicate the hole is partially covered (half-hole).
Musical Note

Standard Fingering

D5

●●●

●●●

E5

●●●

●●○

F5

●●●

●◒○

F#5

●●●

●○○

G5

●●●

○○○

A5

●●○

○○○

A#5

●◓○

○○○

●○●

●●●

B5

●○○

●●●

●○○

○○○

C6

○●●

○○○

○●●

○●●

C#6

○○○

○○○

D6

●●●

●●●

○●●

●●●

E6

●●●

●●○

F6

●●●

●◒○

F#6

●●●

●○○

G6

●●●

○○○

A6

●●○

○○○

A#6

●○●

○○○

B6

●○○

○○○

●○○

●●●

C7

○●○

●●●

◐○○

○○○ (Alt1)◓◓○○○○ (Alt2)○○○○○○ (Alt1)○○○●●○ (Alt2)

C#7

○●●

●○○

Alternative Fingering

D7

●●●

●●●

○●●

●●●

E7

●●○

●●○

●●○

○●○

F#7

●○●

●○●

G7

●○●

○○○

3. RESULTS & DISCUSSION

For this extended abstract, we selectively focus on the impedance spectra measured for fingerings
associated with three notes as an example: D5/D6 and A#5 (half-hole vs cross-fingering).
Figure 1 shows Z(f) measured for D5 and D6 (‘Sol’ in G key) fingering. Since the Dízi is largely
cylindrical, its impedance spectrum for this fingering (all holes closed) is nominally equivalent to asimple open cylinder, hence we observe regularly spaced resonances (minima) to arise (oscillations
are sustained at |ZMin | because the embouchure hole opens to the atmosphere, so the air jet excites
resonances with large air flow at [low] acoustic pressure [4]). To select either D5 or D6 (circled in
Figure 1), the musician adjusts the speed, separation, length and shape of the air jet, as well as
adjusting the extent of covering the embouchure hole [8,9]. For this fingering, the first two resonances
are often played, although the third and fourth resonances here (A6 and D7 respectively) are also
occasionally used for reasons of pitch preference, ease of playing/fingering and tone colour.

Figure 1. Z(f) measured for a G-key Dízi for the fingering sounding the notes D5 and D6, with
operating resonances (|Z Min |) 589.5 Hz (D5+6 cents) and 1205 Hz (D6+44 cents) respectively (circled).

Figure 2. Z(f) comparing between half-hole (red) vs cross-fingering (blue) both sounding the noteA#5. The four red curves are ‘snapshots’ reflecting the range of continuous pitches available to theplayer at various degrees of half-holing, showing pitch deviation of Δ53 cents here.

Being keyless, the Dízi allows non-binary finger control over tone hole closure: many subtle modesof partial closure and control are possible (not just open/close). Figure 2 compares half-hole fingeringand cross-fingering, for sounding the note A#5. The first resonance (~940 Hz, circled) associated withcross-fingering is flatter in pitch when compared to the half-hole fingering. Figure 2 further shows a
representative range of resonance frequencies made available by adjusting the degree of half-holeclosure, thereby allowing continuous pitch deviation (‘glissando’) of Δ53 cents (a quarter-tone) to be
performed; wider deviations are possible but at the expense of limited timbre and dynamic control.
Thus, half-holing can be a useful technique to elicit pitch-bending effect. Another practicalconsideration: half-hole fingering is sometimes preferred as it requires only engaging one finger (goodfor fast passages), whereas cross-fingering requires careful coordination across fingers and hands. The
trade-off, however, is that fine finger control is required to ensure precise control of pitch during halfholing and consistency over each repetition, whereas cross-fingering delivers consistent pitch.

4. CONCLUSION

35 standard and alternative Dízi fingerings representing the instrument’s tessitura were measured
for their acoustic impedance spectra on a concert-grade Dízi, allowing insights on how fingerings
determine the operating resonances available on the Dízi and show how each fingering can support
several resonances and corresponding sounding pitches; conversely, a number of sounding pitches canbe generated using several fingering alternatives. Together, these insights contribute to an appreciationof how expert Dízi players navigate a complex ‘map’ of fingering possibilities, performanceparameters (ease of sounding and pitch control) and musical requirements. The compendium of Dízi
impedance spectra is available here https://acoustics.sutd.edu.sg/dizi-impedance/ alongside the soundspectra, allowing acousticians, Dízi performers and makers a systematic visualization of the linear
acoustics of the Dízi and insight into how fingerings and dizi design translate to acoustic response
and its implications on ease of playability, pitch, and practical performance considerations.

ACKNOWLEDGEMENTS

The impedance measurements were conducted at the Acoustics Lab, School of Physics, UNSW,
Sydney Australia. The authors are grateful to John Smith and Joe Wolfe for hosting us. The acoustic
impedance measurement hardware and software were developed by Paul Dickens and Noel Hanna.

REFERENCES

1. Wang, C., Chow, J. Y., & Wong, S. S. (2020). The Teng guide to the Chinese orchestra. Singapore:
World Scientific Publishing Co. Pte. Ltd and The Teng Ensemble Ltd.
2. Ng, H.H., Tan, Y.X., Ng, T.S., & Chen, J.-M. (2022). “Dizi Fingerings, Impedance and Audio”.
Retrieved August 22, 2022, from https://acoustics.sutd.edu.sg/dizi-impedance/
3. Ng, H.H., Tan, Y.X., Ng, T.S., & Chen, J.-M. (2022). “Dízi Fingering Dataset: Acoustic Impedance& Audio Spectra, with Sound Files”. Zenodo. https://doi.org/10.5281/zenodo.6333932
4. Wolfe, J., Smith, J., Tann, J. and Fletcher, N.H. (2001) “Acoustic impedance of classical and modern
flutes” Journal of Sound and Vibration , 243, 127-144.
5. Wolfe, J., Smith, J., Tann, J. and Fletcher, N.H. (2001) “Acoustics of classical and modern flutes: acompendium of impedance spectra, sound spectra, sounds and fingerings”, Retrieved August 22, 2022,
from https://newt.phys.unsw.edu.au/music/flute/ .
6. Dickens, P., Smith, J. & Wolfe, J. (2007). Improved precision measurements of acoustic impedance spectrausing resonance-free calibration loads and controlled error distribution, J. Acoust. Soc. America, 121,
1471-1481.
7. Balamurali BT, Da Yang Tan, Teck Seng Ng* & Jer-Ming Chen (2018), “Fabrication of Chinese Transverse
Flute, Dízi: an acoustic impedance analysis”, WESPAC 2018, New Delhi
8. De la Cuadra, P., Fabre, B., Montgermont, N., & Chafe, C. (2005). Analysis of Flute Control parameters:
A comparison between a novice and an Experienced Flautist. Acta Acustica United with Acustica, 94(5),740-749. https://doi.org/10.3813/AAA.918091
9. Wolfe, J., Fletcher, N. H., & Smith, J. (2015). The interactions between wind instruments and their players.
Acta Acustica United with Acustica, 101(2), 211-223. https://doi.org/10.3813/AAA.918820

PROCEEDINGS of the24th International Congress on Acoustics
October 24 to 28, 2022, in Gyeongju, Korea
Extended abstractABS-0969

Violinist, violin, bow: What can we hear and recognize?
Claudia FRITZ1; Romain VIALA2; Victor SALVADOR-CASTRILLO1
1

Institut Jean le Rond d'Alembert, Sorbonne University, France
2

Pôle Innovation, ITEMM, France

ABSTRACTThe Science and violin making workshop organized every year by researcher Claudia Fritz and violin makerPaul Noulet in France gathers researchers, instrument makers and musicians to make progress in our
understanding of the bowed string instruments. One of the daily sessions consists in critical listening, to train
our ears, learn how to express and share what we hear, and search for correlations with construction
parameters or study various modifications on the instrument, the bow or the playing technique.
It is generally accepted that the sound of a bowed string instrument depends on three parameters: the player,
the instrument and the bow. During the 2021 workshop, formal listening tests were conducted to explore
what a group of trained listeners is able to hear and recognize when one of these three parameters varies.
Results challenge common ideas among violinists about the sound quality associated to the bow. A large
difficulty of comparing instruments when listening was observed, as the musician keeps adapting to the
instrument and only few cues are available to the listeners to decouple the player’s actions and the instrument
behavior with respect to their influence on the resulting sound.
Keywords: Violin, Bow, Perception

1. INTRODUCTION
Many listening tests have been recently designed in violin acoustics to better understand how
listeners evaluate violins as well as explore the influence of various parameters (the age (1), the model(2), the soundpost (3), the thickness of the plate (4), the effect of playing (5), etc.) on the perceived
sound. While being live or via recordings, they all involved at least one player, playing many times
the same short musical excerpt on various instruments. These studies involved a lot of discussions
and pilot tests to design experimental conditions and protocols which could counterbalance theinfluence, on the quality evaluations of the instruments by the listeners, of the player, thereproducibility of the player, the bow that was used, the chosen excerpt, the order of the violins, thetypes of comparisons, … But to which extent? This study aims at investigating to which extent an
instrument has a sound quality per se, independently of the player and the bow, by exploring what canbe heard and discriminated within these experimental conditions (beyond the qualitative evaluations
for which a large disagreement between listeners has been shown by these studies).
We took advantage of the gathering of violin and bow makers and scientis ts during a European
workshop ‘Science and lutherie’ organized yearly by the first author and violin maker Paul Noulet.
Critical listening sessions are part of the workshop daily routine, which thus make this group of people
a relatively well trained audience. These sessions have many goals: ear training, negotiating somecommon vocabulary, comparing instruments and testing the influence of various modifications which
are done during the workshop in order to correlate them with measured acoustical differen ces. We
used these sessions during the 2021 workshop to design a series of listening tests to investigate what,among the three factors that are the player, the bow and the violin, can be recognized and discriminate d.

1claudia.fritz@sorbonne-universite.fr

2. LISTENING TESTS
2.1 General design
All the listening tests were designed as N successive trials, during which one or two violins wereplayed by one or two musicians, behind an acoustically transparent screen, on stage of a 340-seatauditorium. For each test, the participants (between 11 and 12 violin/bow makers and violin
acousticians) had to answer the same yes-no question. The probability of a correct answer being 50%
for each trial, the distribution of answers for a given participant follows a binomial law with p=1/2.
Based on how this law varies with N, we considered that N=8 was an interesting number as theprobability of making one or no mistake is below the usual 5% chance threshold. Therefore, we canrule out that a participant did the test by chance if they made no more than one mistake.
The downside of such design is the repetition. Both the audience and the players can get tired overthe length of the test. This tiredness can lead the players to be less reproducible. But at the same time,
the repetition can make them adapt more and more to obtain the sound that they want, thus smoothing
the differences. This will be discussed more later.
2.2 Violin recognition
A first series of tests was conducted to investigate whether a trained audience could reliably
distinguish between two violins and could recognize specific characteristics of an instrument acrossplayers and trials.
Tests 1a et 1b
For each trial, two violins were played successively. They were chosen randomly among a groupof 8 violins. The participants had to tell whether they were two different violins or the same one played
twice. In test 1.a, the two violins were played by two professional orchestra players, using their own
bow. In test 1.b, they were played by only one of these two players, the same one for all trials. Aftera few pre-tests, the chosen musical excerpt consisted in only 9 notes, a G Major diatonic scale on the
G string with 5 notes up and 4 notes down, to keep the length of the test reasonable and, so we thought,
to reduce the difficulty with a short excerpt, which is not too mus ical.
Test 2a and 2bHere, only one violin is played per trial, which shortens the test by more than half and thus reduce s
the tiredness and the problems of reproducibility. The violin played was selected randomly among
two violins, labelled A and B, which were chosen collectively before the test as being the mostdifferent among the 8 that were available. So for each trial, participants had to tell whether it was
violin A or not (i.e. violin B). The violins were played by a semi-professional violinist, who played
an ascending and descending A Major scale (covering the register of the violin) followed by the
opening of the Tchaikovsky concerto. We thought that the test being reduced, we could afford hearing
a bit more and thus have more cues to make our judgment on. In test 2a, the player played with adifferent bow (selected randomly among a pool of 10 bows) for each trial to limit the adaptation of
the player while in test 2b, the player always used his own bow, to reduce the source of variation.
2.3 Bow recognition
Talking about the famous recording by Janine Jansen of 12 Stradivari for which she chose carefully
which bow to use among her two bows for each violin, Pauline Harding wrote: “Her two bows had adramatic effect. It was interesting how everything changed.” (6) This resonates with beliefs in thecommunity of violinists and bow / violin makers who claim that changing bow can completely change
the sound of an instrument, and even that a bow has a sound per se. We thus designed two tests tostudy whether an audience could reliably recognise a bow. As pre-tests made us realise that it may bemuch harder than expected based on the beliefs, we decided to use two extreme bows: a violin bow
made by a renown bow maker and and a middle-range viola bow (about 10g heavier).
Tests 3a and 3b:
For each trial, a violin was played with either the violin or the viola bow, by the same semi professional violinist as in test 2. The musical material included a Ab Major scale covering the registerto the violin, with one bow stroke per note on the way up and with an alternating of six rapid andstaccato up and down bows (to illustrate the attacks allowed by the bow) on the way down followed
by a very short excerpt from a Bach sonata. Similarly to what was done in test 2, test 3a was donewith a different violin for each trial (chosen randomly among the pool of available violins) while test
3b was done on the player’s own violin.

3. RESULTS AND DISCUSSION
The number of participants among the total number of participants who did each test at better thanchance level is reported in table 1.
Table 1: Proportion of participants who succeeded better than 5% chance level, i.e. who made no more than
one mistake for each test.
Test
Numberofparticipants

who

1a

1b

2a

2b

3a

3b

1/11

1/11

5/11

5/12

3/12

2/12

succeeded better than chance level
Table 1 shows that for each test, only a very limited number of participants were able to succeed
at better than chance level. The results of test 1.a seem to indicate that the same violin played bydifferent players can have very different sound qualities which makes it difficult to tell whether it is
the same violin or two different ones. Test 1.b’s results show that it is not easier when it is the sameplayer. This seems to indicate that the way a violinist play can have as much influence on the timbre
than playing different violins. It is difficult for a violinist to play in a strictly reproducible way along
the course of the test because of tiredness but more importantly because of adaptation. A good player
is always trying to achieve “his/her” sound and so will modify his/her technique to get the soundhe/she wants. Therefore, the differences decrease along the test and so it becomes harder for theaudience to tell apart the violins. We thought that this adaptation could be limited by playing less theviolins, as in test 2 (only one violin per trial instead of 2) and by playing them with different bows(2a) rather than one bow (2b). This seems to be the case, though not to a large extent, with still fewer
than half of the participants being able to succeed. The protocol with only one violin in each trial
allows a shorter test, but at the same time, may be more difficult because participants need toremember which violin is violin A. If a participant loses track at some point during the experiment,
he may well be wrong for the rest of the test. This could be possibly solved in future tests by providing
participants with the correct answer after each trial.
The last two tests, about bow recognition, have not been able to show any “bow sound” with only
very few people having been able to distinguish two very different bows at better than chance level.
Though players claim that the bow can have a large influence on the sound, one can expect, from amechanical point of view, that this influence will be rather on temporal aspects (like the precision of
the attacks or the articulation between notes for instance) rather than on the timbre of the
instrument (7). Therefore, one may expect a smaller influence on the resulting sound from the bow
than from the violin, and it is thus not surprising that participants could not recognize bows if they
were not able to recognize violins.

4. CONCLUSION
These series of listening tests are a first attempt at exploring, beyond any qualitative judgement,
what can possibly be recognized in a reliable way by a trained audience when violins are played insimilar experimental conditions as previous published studies on violins sound quality, involving therepetition of rather short excerpts on various violins. The results raise interesting questions about the
timbre of an instrument: can a violin have some intrinsic sound qualities that can be recognized acrossplayers and bows and if yes, how can we access/perceive them in a reliable way? Further work andmore listening tests are definitely needed. They could be of significant importance in the current
context about the shortage of the Pernambuco wood (traditionally used for bows), which is still used,
despite more and more CITES regulations attempts (8), since makers and musicians claim it gives a“better sound”.

ACKNOWLEDGEMENTS
We would like to thank the “Villefavard team”, the dedicated group of violin makers who took partin all the discussions and pre-tests to design the protocols and then participated in the tests.

REFERENCES
1. Fritz C, Curtin J, Poitevineau J, Tao F-C. Listener evaluations of new and Old Italian violins. Proc. Nat.
Ac. Sci. of the USA, 2017; 114 (21): 5395-5400.
2. Fritz C, Fraisse V, Dubois D. The ObieAlto project: Looking for correlations between perceptual
properties and constructional data. Proc. Intern. Symposium on Musical Acoustics, Sep 2019; Detmold,
Germany.
3. Fu L, Fritz C, Scavone G. Perception of violin soundpost tightness through playing and listening tests. J
Acoust Soc Am. Acoustical Society of America, 2021; 150 (1):540-550.
4. Fritz C, Stoppani G, Igartua U, Jardón Rico R, Arroitajauregi A. The Bilbao project: How violin makers
match backs and tops to produce particular sorts of violins. Proc. Intern. Symposium on Musical Acoustics,
Sep 2019; Detmold, Germany.
5. Inta R, Smith J, Wolfe J. Measurement of the effect on violins of ageing and playing. Acoust. Australia,
2005; 33:25-29.
6. Harding P. A spiritual connection. The Strad, Oct 2021, p. 26-36.
7. Ablitzer F. Influence des paramètres mécaniques et géométriques sur le comportement statique del’archet de violon en situation de jeu. PhD Dissertation, Le Mans Université, 2011.
8. CITES. Consideration of Proposals for Amendment of Appendices I and II. Nineteenth Meeting of theConference of the Parties, 2022. https://cites.org/sites/default/files/eng/cop/19/prop/as_received/EPaubrasilia_echinata.pdf