# Automatic detection of speech inspiration using SVM and decision trees

Younes El Belghiti

Laboratory for research in computer science and telecommunications
Faculty of Sciences, Mohammed V University

Rabat, Morocco

younes.elbelghiti@um5r.ac.ma

Khalid Daoudi

Geostat team
INRIA Bordeaux Sud-Ouest

Bordeaux, France

khalid.daoudi@inria.fr

Khalid Minaoui

Laboratory for research in computer science and telecommunications
Faculty of Sciences, Mohammed V University

Rabat, Morocco

khalid.minaoui@fsr.um5.ac.ma

Alexis Deighton MacIntyre

MRC Cognition and Brain Sciences Unit
University of Cambridge

Cambridge, UK

alexisdeighton.macintyre@mrc-cbu.cam.ac.uk

**Abstract**—Respiration is one of the most important subsystems in speech production. The detection and analysis of speech breathing patterns has numerous benefits in speech analysis, synthesis, recognition and in the development of voice biomarkers. Despite the close relation between speech and respiration, acoustic analysis of speech breathing is not well explored. In this paper, we propose a method for the automatic detection of inspiratory events from speech signals. This method is based on learning SVM and decision tree models using mel frequency cepstral coefficients (MFCC). We carried out an evaluation using a state-of-the-art database. The results showed, in particular, that fusion of the two models achieves an F1-score of 0.80, which significantly outperforms recently reported results.

***Index Terms***—Speech breathing, SVM, Decision trees, Machine learning, MFCC.

# I. INTRODUCTION

Speech is an important means of communication between humans and is a complex physiological process involving a production-perception loop [1, 2, 3]. One of the main components in speech production is the respiratory mechanisms which act as a trigger and a regulator of speech [4]. These mechanisms are central in shaping rhythm, prosody, articulation and other aspects of speech [5, 6, 7].

In their pioneering work, Hoit and Hixon explored how age, gender, and body type impact speech breathing [8, 9]. The dynamics of the chest and of the airflow during speech were first discussed in [10, 11, 12]. Analysis of the variations in lung volumes during reading was carried out in [13], showing the influence of speech intensity and linguistic factors. Other works discussed the differences in breathing patterns between read and spontaneous speech, highlighting cognitive load effects [14, 15, 16]. It has also been shown that speech breathing differs significantly from quiet breathing, with speech breathing being characterized by shorter inhalations and longer exhalations to accommodate speech flow [4, 17].

Early devices to record the acoustics, airflow, and air pressure during speech production or in clinical settings were the pneumotachograph and the Rothenberg mask, a sealed face mask placed over the nose and mouth [18, 19, 20]. Nowdays, inductance plethysmography is the most commonly used technique [21]. It consists in wearing two flexible belts which measure the changes in thoracic and abdominal circumferences during inhalation and exhalation. This yields a respiratory signal that can be analyzed along with the speech signal recorded in parallel. This technique is, however, time-consuming, not cost-effective, and requires difficult calibration steps. From this perspective, the inference of respiratory activity from the acoustics of the speech signal alone is an appealing approach, as it would allow for easy, cost-effective and large-scale studies and applications.

Surprisingly, there exist, however, only few studies of the acoustics of respiratory patterns during speech. Recently, deep learning techniques have been used to predict the respiratory signal from the speech signal [22, 23, 24, 25, 26]. Another approach is to automatically detect the inspiratory events during speech. This task requires less demanding because it comes back to a classification task of speech events, rather than inferring the entire respiratory signal (with a regression model for instance). Still, this approach can be very useful in studying speech breathing dynamics, for example, to assess how often a patient needs to breathe during conversational speech in clinical settings. Our contribution in this paper falls within this framework. To the best of our knowledge, only the following studies have been reported in this setting. The first one used a template-matching approach, based on MFFC, to detect the beginning and the end of inspiration segments [27]. In [28, 29, 30], an analysis of some acoustic-phonetic features of inspirations was reported, such as intensity, duration and spectral properties. In [31], the work of [27] served as a basis to combine an SVM classifier with the cepstrogram matrix to identify the inspiration segments. Later, the same authors
added a voice activity detector to improve the performance
and to reduce the execution time of their method [32].

In this paper, using standard machine learning models, we present a new approach to detect inspiration events which is based on the material introduced in [33]. In the latter, an effective algorithm to detect inspiration activity from respiratory signals was proposed. Here, we extend that work to speech signals.

The paper is organized as follows. Section 2 presents the dataset and our methodology for feature extraction and classification of inspiratory events. The results are shown and discussed in section 3. Section 4 presents our conclusion and perspectives for future work.

## II. MATERIALS AND METHODS

In this section, we first present the speech data used in the study and then describe the methodology we adopted. The block diagram of the full procedure is shown in Figure 1.

<figure><img src="image_1.png" /><figcaption>Fig. 1. Bock-diagram of the methodology: pre-processing, feature extraction and machine learning</figcaption></figure>

### A. Speech data

We used a state-of-the-art dataset, the UCL-SBM database [33, 34], which consists in parallel recordings of speech and respiratory signals captured in a controlled office environment. The speech tasks include read passages, automated dialogue, poetry recitations, resting-state and spontaneous monologues. The recordings were captured via an AKG C555L microphone, supplemented by dual piezoelectric belts which sense inhalation/exhalation activity. The speech and respiratory signals are sampled at 40 kHz. Participants produced an average of five minutes of spontaneous discourse, typically reflecting on personal anecdotes or experiences related to life in London. The database is structured into multiple channels, ensuring a comprehensive framework for the detailed examination of the diverse speech types and their associated respiratory patterns [34]. In this paper, we use a subset of the database which was

considered in a previous study. It consists in of read speech
uttered by 10 English speakers (3 male and 7 female, ages
25-50)[33].

### B. Manual annotation

The first step of our processing methodology was to generate ground truth data to evaluate of the performance of our inspiration detector. To do so, we manually annotated the dataset using Audacity [35], a commonly used software for speech editing and processing. The process of manual annotation was executed in a two-stage manner. Initially, the audio tracks were attentively listened by trained annotators to identify segments that potentially contain inspirational events. The latter are characterized by audible cues that correspond to the intake of breath during speech. The minimal duration of an inspiration was set to 45ms. Once a potential inspiratory event was detected by auditory analysis, a visual inspection of the mel-spectrogram was then carried out. This process allowed the differentiation between true inspiration events and similar-sounding phonetic components which are not inhalations. An illustration of the annotation technique is shown in Figure 2. This methodical approach of cross-referencing auditory observations with visual spectrograms ensures a high level of precision in identifying and labeling inspiration events. Thus, it guarantees that the models are trained and tested on accurately labeled data, thereby improving the overall reliability of the study's findings.

<figure><img src="image_2.png" /><figcaption>Fig. 2. Example of a inspiration segment obtained by manual annotation.</figcaption></figure>

### C. Feature extraction

We first performed a pre-processing step in order to improve the acoustic quality of the recordings. We applied a 0.5Hz high-pass filter to remove low-frequency noise. Then, we applied an amplification function (Normalization) in order to boost the energy of the low-amplitude segments, thus allowing a better distinction between inspiration and non-inspiration segments. Feature extraction was carried out using the MFCC which are the most widely features in speech analysis, segmentation, coding, recognition, and
other fields of speech processing [36]. We considered 15-dimensional MFCC vectors computed using a frame size of 45ms and frame shift of 10ms. A Hanning window was applied to each frame. The heatmaps in Figure 3 illustrates a comparison between the MFCC distribution of inspiratory and non-inspiratory segments. It shows that inspiration events present a homogeneous distribution, characteristic of breath sounds, while non-inspiration presents an heterogeneous pattern, mirroring the complex structure of speech. This illustration shows the adequacy of MFCC as features for the discrimination between inspiration and speech.

<figure><img src="image_3.png" /><figcaption>Fig. 3. Comparative Heatmaps of MFCC computed on an inspiration (left) and a speech (right) segment.</figcaption></figure>

## D. Classification

We first used 2 standard classification models, the Support Vector Machines (SVM) with an RBF kernel [37], Decision Trees (D-tree) [37]. These are known to be complementary and suited in unbalanced-data classification tasks. Our data is indeed unbalanced as the amount of annotated inspiration frames constitutes about 88.5% of the total amount of frames. As classically done [38], training (resp. testing) was performed on randomly selected 80% (resp. the remaining 20%) of the data. This process was repeated 100 times and the classification scores averaged over the 100 trials. We considered precision, recall, and F1-score as the evaluation metrics. We do not report accuracy as this metric can be biased when data is unbalanced, whilst the F1-score is more robust and commonly used in such scenarios.

Second, in order to take advantage from the complementarity between SVM and D-tree, we considered a fusion of the 2 models. Each base model was trained separately, then a Logistic Regression (LR) model was trained on their combined predictions. That is, an unseen instance was first evaluated by the base models, then their predictions were considered as input into the LR model for the final decision. This approach merges the strengths of SVMs and Decision Trees, aiming to reduce bias and variance while enhancing generalization capabilities. The pipeline of the fusion process is illustrated in Figure 4

## III. RESULTS

The classification scores are reported in Table I. The SVM model achieved a precision of 0.84, indicating a high level of accuracy in identifying non-inspiratory segments within the speech signal. Its recall rate was 0.73, reflecting the model's

<figure><img src="image_4.png" /><figcaption>Fig. 4. Pipeline of the fusion process</figcaption></figure>

<table><caption>TABLE I
CLASSIFICATION SCORES OF SVM, D-TREE & FUSION</caption><thead><tr><td>Model</td><td>Precision</td><td>Recall</td><td>F1 Score</td></tr></thead><tbody><tr><td>SVM</td><td><strong>0.84</strong></td><td>0.73</td><td>0.77</td></tr><tr><td>D-tree</td><td>0.77</td><td>0.78</td><td>0.77</td></tr><tr><td>SVM + D-tree</td><td>0.81</td><td><strong>0.79</strong></td><td><strong>0.80</strong></td></tr></tbody></table>

capacity to capture a substantial proportion of inspiratory segments, albeit with some room for improvement.

In comparison, the Decision Tree model yielded a lower precision than SVM of 0.77, which reflected a lower performance in recognizing non-inspiratory segments. On the other hand, the D-tree model's recall was higher at 0.78, suggesting a higher performance in identifying inspiratory segments.

Both models yielded an F1 Score of 0.77. This score, representing the harmonic mean of precision and recall, is particularly informative as it provides a metric that balances precision and recall, offering a holistic view of the model's performance. The identical F1 scores suggest that both SVM and D-tree models are globally equally effective.

The results suggested that the choice between SVM and D-tree models for inspiratory event detection should be informed by the specific application requirements. In settings where avoiding false positives is paramount, the SVM model's higher precision would be advantageous. In contrast, if capturing as many true inspiratory events as possible is critical, the D-tree's higher recall would constitute a better choice.

On the other hand, the fusion model yielded a notable improvement in precision compared to the decision tree, though it did not surpass the SVM performance. When considering recall, the fusion model outperformed both SVM and the decision tree. More importantly, the fusion model achieved a significant improvement in the F1 score, reaching 0.80, with a good balance between precision and recall.
As compared to state-of-the-art, our method significantly outperformed the one of [32] on (read) news reports, even though we did not use voice activity detection. Moreover, [32] evaluated the speech breathing rate which is calculated as the ratio between the number of detected inspirations and the time between the first and last breath. Such an evaluation is in favor of higher metric scores, compared to the evaluation of the performance of a pure binary inspiration/non-inspiration classification.

## IV. CONCLUSION AND PERSPECTIVES

We presented a preliminary study on the automatic detection of inspiratory events from speech, using classical acoustic features and standard classification models. We manually annotated a subset of the state-of-the-art UCL-SBM database to obtain reliable ground-truth data. The results show that both SVM and decision trees, fed by MFCC, can yield relatively good and complementary performances, as compared to recently reported works. These interesting results are encouraging towards continuing investigations in the development of more accurate and robust speech breathing detection techniques, a field which is poorly studied by the community despite its relevance in many areas of speech processing and voice biomarkers.

In particular, future research in inspiratory event detection could leverage deep learning models to enhance pattern recognition capabilities beyond classical pipeline methodologies, as the one presented in this paper. Exploring a broader range of acoustic features and employing advanced data processing, such as noise reduction and data augmentation, will be key in improving model performance. Additionally, expanding the training datasets to include diverse environmental and demographic data will increase the robustness of the models. Such advancements will be instrumental in developing more accurate and generalizable systems for real-world applications, in healthcare settings for instance. This is the purpose of our ongoing research.

## REFERENCES

[1] Gunnar Fant. *Acoustic theory of speech production*. Mouton, The Hague, 1960.

[2] John Laver. *Principles of phonetics*. Cambridge university press, 1994.

[3] Willem JM Levelt. Relations between speech production and speech perception: Some behavioral and neurological observations. *Language, brain and cognitive development: Essays in honor of Jacques Mehler*, pages 241–56, 2001.

[4] Thomas J Hixon. Respiratory function in speech and song. *College-Hill Press*, 1987.

[5] Alexis Deighton MacIntyre. *The analysis of breathing and rhythm in speech*. Ph.D. thesis, University College London (UK), 2022.

[6] Raphael Johannes Werner. *The phonetics of speech breathing: pauses, physiology, acoustics, and perception*. Ph.D. thesis, Saarland University (Germany), 2023.

[7] Susanne Fuchs and Amélie Rochet-Capellan. The respiratory foundations of spoken language. *Annual Review of Linguistics*, 7:13–30, 2021.

[8] Jeannette D Hoit and Thomas J Hixon. Age and speech breathing. *Journal of Speech, Language, and Hearing Research*, 30(3):351–366, 1987.

[9] Jeannette D Hoit, Thomas J Hixon, Mary Ellen Altman, and Wayne J Morgan. Speech breathing in women. *Journal of Speech, Language, and Hearing Research*, 32(2):353–365, 1989.

[10] Thomas J Hixon, Jere Mead, and Michael D Goldman. Dynamics of the chest wall during speech production: Function of the thorax, rib cage, diaphragm, and abdomen. *Journal of speech and hearing research*, 19(2):297–356, 1976.

[11] Dennis H Klatt, KN Stevens, and J Mead. Studies of articulatory activity and airflow during speech. *Annals of the New York Academy of Sciences*, 155(1):42–55, 1968.

[12] B Conrad and Paul Schoenle. Speech and respiration. *Archiv für Psychiatrie und Nervenkrankheiten*, 226:251–268, 1979.

[13] Alison L Winkworth, Pamela J Davis, Elizabeth Ellis, and Roger D Adams. Variability and consistency in speech breathing during reading: Lung volumes, speech intensity, and linguistic factors. *Journal of Speech, Language, and Hearing Research*, 37(3):535–556, 1994.

[14] Alan Henderson, Frieda Goldman-Eisler, and Andrew Skarbek. Temporal patterns of cognitive activity and breath control in speech. *Language and Speech*, 8(4):236–242, 1965.

[15] Heather L Mitchell, Jeannette D Hoit, and Peter J Watson. Cognitive-linguistic demands and speech breathing. *Journal of Speech, Language, and Hearing Research*, 39(1):93–104, 1996.

[16] Yu-Tsai Wang, Jordan R Green, Ignatius SB Nip, Ray D Kent, and Jane Finley Kent. Breath group analysis for reading and spontaneous speech in healthy adults. *Folia Phoniatrica et Logopaedica*, 62(6):297–302, 2010.

[17] Marcin Włodarczak and Mattias Heldner. Respiratory constraints in verbal and non-verbal communication. *Frontiers in psychology*, 8:708, 2017.

[18] Martin Rothenberg. Measurement of airflow in speech. *Journal of Speech and Hearing Research*, 20(1):155–176, 1977.

[19] Christopher Dromey, Lorraine Olson Ramig, and Antonia B Johnson. Phonatory and articulatory changes associated with increased vocal intensity in parkinson disease: A case study. *Journal of Speech, Language, and Hearing Research*, 38(4):751–764, 1995.

[20] Laura L Koenig. Laryngeal factors in voiceless consonant production in men, women, and 5-year-olds. *Journal of Speech, Language, and Hearing Research*, 43(5):1211–1228, 2000.

[21] Victoria S McKenna and Jessica E Huber. The accuracy of respiratory calibration methods for estimating lung volume during speech breathing: A comparison of
four methods across three adult cohorts. *Journal of Speech, Language, and Hearing Research*, 62(8):2632–2644, 2019.

[22] Alexis Deighton MacIntyre, Georgios Rizos, Anton Batliner, Alice Baird, Shahin Amiriparian, Antonia Hamilton, and Björn W Schuller. Deep attentive end-to-end continuous breath sensing from speech. 2020.

[23] Aki Härmä, Ulf Grossekathöfer, Okke Ouweltjes, and Venkata Srikanth Nallanthighal. Sensing of inspiration events from speech: comparison of deep learning and linguistic methods. *arXiv preprint arXiv:2305.11683*, 2023.

[24] Venkata Srikanth Nallanthighal, Zohreh Mostaani, Aki Härmä, Helmer Strik, and Mathew Magimai-Doss. Deep learning architectures for estimating breathing signal and respiratory parameters from speech recordings. *Neural Networks*, 141:211–224, 2021.

[25] Venkata Srikanth Nallanthighal and H Strik. Deep sensing of breathing signal during conversational speech. 2019.

[26] Venkata Srikanth Nallanthighal, Aki Härmä, and Helmer Strik. Speech breathing estimation using deep learning methods. In *ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, pages 1140–1144. IEEE, 2020.

[27] Dima Ruinskiy and Yizhar Lavner. An effective algorithm for automatic detection and exact demarcation of breath sounds in speech and song signals. *IEEE transactions on audio, speech, and language processing*, 15(3):838–850, 2007.

[28] Jürgen Trouvain, Bernd Möbius, and Raphael Werner. On acoustic features of inhalation noises in read and spontaneous speech. In *1st International Seminar on the Foundations of Speech: BREATHING, PAUSING, AND VOICE*, 2019.

[29] Jürgen Trouvain, Raphael Werner, and Bernd Möbius. An acoustic analysis of inbreath noises in read and spontaneous speech. In *Proceedings of Speech Prosody*, pages 789–793, 2020.

[30] Raphael Werner, Susanne Fuchs, Jürgen Trouvain, and Bernd Möbius. Inhalations in speech: Acoustic and physiological characteristics. In *Interspeech*, pages 3186–3190, 2021.

[31] Aurobinda Routray et al. Automatic measurement of speech breathing rate. In *2019 27th European Signal Processing Conference (EUSIPCO)*, pages 1–5. IEEE, 2019.

[32] Mohamed Ismail Yasar Arafath K and Aurobinda Routray. Automatic detection of breath using voice activity detection and svm classifier with application on news reports. In *INTERSPEECH*, pages 609–613, 2019.

[33] Alexis Deighton MacIntyre and Raphael Werner. An automatic method for speech breathing annotation. 2023.

[34] Björn W Schuller, Anton Batliner, Christian Bergler, Eva-Maria Messner, Antonia Hamilton, Shahin Amiriparian, Alice Baird, Georgios Rizos, Maximilian Schmitt, Lukas Stappen, et al. The interspeech 2020 computational paralinguistics challenge: Elderly emotion, breathing & masks. 2020.

[35] Audacity Team. Audacity, 2023.

[36] Lawrence Rabiner and Ronald Schafer. *Theory and applications of digital speech processing*. Prentice Hall Press, 2010.

[37] Aurélien Géron. *Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow*. ” O’Reilly Media, Inc.”, 2022.

[38] Ron Kohavi et al. A study of cross-validation and bootstrap for accuracy estimation and model selection. In *Ijcai*, volume 14, pages 1137–1145. Montreal, Canada, 1995.