Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

Suiview: A Web-based Application that Enables Users toPractice Wind Instrument PerformanceMisato Watanabe1 , Yosuke Onoue1 , Aiko Uemura1 , and Tetsuro Kitahara1
Nihon University, Tokyo, Japanchmi19013@g.nihon-u.ac.jp, {onoue.yousuke, uemura.aiko, kitahara.tetsurou}@nihon-u.ac.jp

Abstract. This paper presents a web-based application that enables users tocheck the stability of the pitches, intensities, and timbres of the sounds they play.
Amateur musicians have opportunities to play wind instruments, at a brass-bandclub at school. To make sounds with the stable pitches, intensities, and timbres,
players have to carefully control the shapes of their mouth and lips, the strengthof the breath, and their vibration. But this is difficult for most amateur musicians,
who rely on expert players to check whether they are appropriate and advise
them how to improve them. To solve this problem, we have been developing a
web-based application to enable amateur musicians to check whether the pitches,
intensities, and timbres of their sounds are stable without help from an expert
player (https://suiview.vdslab.jp/). In this paper, we describe its basic system design, the current implementation, and preliminary results of its trial use.
Keywords: Wind instrument, Musical practice, Stability, Web application

1Introduction

Wind instruments are popular among amateur musicians. They are indispensable inbrass-band clubs at junior high school and/or high school, and many people enjoy playing a wind instrument as a hobby. However, playing a wind instrument is not easy. To
produce sounds with stable pitches, intensities, and timbres, players have to carefully
control the shapes of their mouth and lips, the strength of the breath, and their vibration.
One problem in learning a wind instrument is a lack of appropriate instructors. In
the case of the above-mentioned brass-band clubs at school, the responsible teacher at
the club might not be a wind instrument expert. At such clubs, it is often common for
novice-level players to teach freshman players. Also, there are fewer music schools thatteach wind instruments than the piano.
Wind instrument performances have been investigated from different points of view
such as acoustic, psychological, and physiological ones. Brown [1] investigated acoustic features for automatic identification of woodwind instrument sounds. Hirano et al.
[3] analyzed muscular activity and related skin movement during French horn performances. Micheal [5] examined the effects of self-listening and self-evaluation in thecontext of woodwind and/or brass practice by junior high school instrumentalists, andfound that self-evaluation was important for improving the instrument.
This research was supported by JSPS Kakenhi Nos. JP-19K12288 and JP-20K19947.

5

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

More recently, there have been attempts to develop systems that allow users to easily
understand how their performances are good from visual feedback or computational assessment. Pati et al. [7] applied deep neural networks to automatic assessment of student
musical performances. Giraldo et al. [2] developed a system that analyzes sound qualityof violin performances and provides visual feedback to users in real time. Knight et al.
[4] developed a visual feedback system of musical ensemble focusing on phrase articulation and dynamics. Morishita et al. [6] developed a system that gives novice practitioners (especially children) visual feedback of acoustic features in long-tone training ofwind instruments. These systems have been aiming at a goal close to ours, but most ofthem are not designed to enable anyone to easily check his/her performances on his/her
smartphone and/or tablet.
In this paper, we present a web-based application for practicing playing wind instruments by themselves. The important is to give users objective feedback. Because
its target users are novice players, we consider that sounds should be stable, in otherwords, sounds should keep a close pitch, intensity, and timbre from the beginning to theend. Our app. analyzes the pitch, intensity, and timbre of sounds recorded on the app,evaluates their stability, and gives visual feedback to the user. It also provides a functionthat enables the user’s teacher to give comments to the recorded sounds.

2Basic Design and Functions

Our app aims to provide wind instrument practicers with useful information about thesounds performed by them. For novice-level players, as discussed in the Introduction,acquiring skills for sounding stably is important. Therefore, one of the important functions of our app. is therefore to visualize the stability of the acoustic characteristics (i.e.,pitches, intensities, and timbres) of the sounds performed by the user.
Recognizing how well the user is incrementally improving such stability day by
day is also important. Therefore, we implement a function for visualizing recordingby-recording variations in the stability of the pitches, intensities, and timbres as well as
visualizing the acoustic characteristics of each recording.
Also, we implement a teacher-to-student comment function. Although objective visualization is useful for novice players, subjective evaluation and comments by theirteacher is also important. By linking a teacher-mode user to student-mode users, theteacher-mode users can listen to the recordings of the linked student-mode users andgive them his/her evaluations and comments.
2.1

RecordingOnce the user opens and logs into our app., he/she can select what to play from a long
tone, a scale, and an arpeggio (Fig. 1). The scores displayed are shown in Fig. 2. After
selecting one from these three scores, the user starts recording his/her performance witha sampling rate of 48 kHz (Fig. 3). Recorded sounds are automatically stored on ourweb server with some metadata such as the user ID, and the recording date.

6

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

(a) Score for a long tone(b) Score for a scale(c) Score for an arpeggio
Fig. 2. Three scores currently supported by our app.

Fig. 1. Screen for selecting what to play

Fig. 4. Example of analysis results (A: self assessment, B: stability scores, C: chart)

Fig. 3. Screen for recording a sound7

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021(b) Intensity(a) Pitch(c) Timbre(d) Intensities for multiple sounds

Fig. 5. Examples of visualization of acoustic features of recorded sounds

2.2

Visualizing the acoustic characteristics

Once a recording is stored on the webserver, its acoustic analysis starts. The fundamental frequency (F0), amplitude, and spectral roll-off are extracted with a 512-pointshift from the recorded sound. We use Librosa (https://librosa.org/) for extracting thesefeatures. Next, these features are plotted on the screen, as shown in Fig. 5 (a) to (c).
Features for multiple sounds can be plotted on the same screen, as shown in Fig. 5 (d).
2.3Visualizing the recording-by-recording variations in the stability

The stability of the pitch (F0), intensity (amplitude), and timbre (spectral roll-off) is
calculated for each recording. The stability is defined based on the temporal standard
deviation of each feature. Let σF0 , σAmp , σSp represent the temporal standard deviations for the F0, amplitude, and spectral roll-off, respectively. Then, their stability si(i ∈ {F0, Amp, Sp}) is defined as si = 100 exp(−σi /ai ), where ai are pre-definedconstants (aF0 = 4, aAmp = 70, aSp = 1500). Thus si has a value between 0 to 100.
The stability is visualized in two ways to enable the user to check the stability for
multiple recordings at a glance (Fig. 6). One is a stacked bar chart that represents thestability of each of the pitch, intensity, and timbre (Fig. 6 (a)). The other is a line chartthat represents overall stability scores (Fig. 6 (b)).
2.4

Teacher-to-student comment

Logging in with the teacher mode, the user can listen to sounds recorded by the linked
student-mode users and check the visualization of their acoustic features and stability
scores. Also, using the teacher-mode, the user can write comments. The comments areautomatically sent to the corresponding student-mode user.

8Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021(b) Line chart (for total stability score)(a) Stacked bar chart (for each stability)

Fig. 6. Examples of visualization of Recording-by-recording stability variations

Fig. 7. Screen for the teacher mode3

Trial UseThree participants used our app for a preliminary evaluation of the effectiveness of theapp. Out of the three participants, one (P 1) was an active player with an intermediateto-advanced level while the other two (P 2 and P 3) were novices, though they had
experience in playing instruments in the past.
Logging in with the student mode, the participants played a long tone, a scale, andan arpeggio on the clarinet several times and recorded them on our app. They saw thevisualization of their sounds made by our app. and were asked to answer the following
questions on a four-level scale (4: agree, 1: disagree):
Q1 Do you think this app helps you produce stable sounds?
Q2 Did you get useful information from the visualization?
Q3 Are the stability scores close enough to your own impression?
The results, listed in Table 1, imply that the participants comparatively highly evaluated our app. In fact, the two novice-level participants gave us comments such as:– By listening alone, it was difficult to find what to improve to produce stable sounds.

9Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

Table 1. Results of the preliminary questionnaire (1 to 4)[Q1] Do you think this app helps you produce stable sounds?
[Q2] Did you get useful information from the visualization?
[Q3] Are the stability scores close enough to your own impression?

P1P2P3
3 4 3
2 4 4
2 4 4

– With graphical visualization , novice-level players could find what to improve.
– Line charts were easy to grasp which were good and which were not.
On the other hand, one participant answered that he/she could not understand what
each graph means. More intuitive visualization should be explored. We also received anopinion that they wanted to see the analysis for sounds given by professional players.

4Conclusion

In this paper, we presented a web-based application that enables users to recognize thestability of wind instrument sounds played by them by visualizing their acoustic features and stability scores. Once the user records his/her wind instrument sounds on theapp, their acoustic features including the pitches, intensities, and timbres are analyzed
as well as their stability is evaluated. Three participants in a preliminary experimentgave us comments that the visualization was useful to produce stable sounds.
Although we focused on the stability of pitches, intensities, and timbres, more complex expressions such as detailed dynamics would be important for more advancedplayers. We will extend the app to support such advanced level players’ practice as well
as systematic evaluation of our app.

References
1. Brown, J.C., Houix, O., McAdams, S.: Feature dependence in the automatic identification ofmusical woodwind instruments. The Journal of the Acoustical Society of America 109(3),1064–1072 (2001)
2. Giraldo, S., Ramirez, R., Waddell, G., Williamon, A.: A real-time feedback learning tool tovisualize sound quality in violin performances. In: Proc. of MML 2017. pp. 19–24 (2017)
3. Hirano, T., Kudo, K., Ohtsuki, T., Kinoshita, H.: Orofacial muscular activity and related skin
movement during the preparatory and sustained phases of tone production on the french horn.
Motor Control 17(3), 256–272 (2013)
4. Knight, T., Bouillot, N., Cooperstock, J.R.: Visualization feedback for musical ensemble practice: A case study on phrase articulation and dynamics. In: Proc. VDA 2012 (2012)
5. Michael, P.H.: The effects of modeling, self-evaluation, and self-listening on junior highinstrumentalists’ music performance and practice attitude. SAGE journals 49(4), 307–322(2001)
6. Morishita, T., Oguchi, H., Kunimune, H., Kirihara, A., Honma, Y.: Development of a support system for beginners to practice long-tones in school wind instruments. Kyoiku Jissen
Kenkyu, Shunshu University Higher Education System Center (2018), (in Japanase)
7. Pati, K.A., Gururani, S., Lerch, A.: Assessment of student music performances using deepneural networks. Applied Sciences 8(4) (2018)10