# Frame-Based Phone Classification Using EMG Signals +
Footnote †: This paper is an extended version of our paper published in Proceedings of IberSPEECH 2022. [PERSON]  
1  
[PERSON]  
1  
[PERSON]  
2  
[PERSON]  
2  
[PERSON]  
1  
HiTZ Basque Center for Language Technology, University of the Basque Country, [PERSON], 1, 48013 Bilbao, Spain; ederdelblancoftheu.eus (E.A.B. ); [EMAIL_ADDRESS] (E.N. ); [EMAIL_ADDRESS] (IJ.H. ); [EMAIL_ADDRESS] (X.d.Z)1  
Footnote 1: footnotemark:  
Footnote 2: footnotemark:  
Footnote 3: foot  
SSIs aim to convert non-acoustic biosignals into text or acoustic speech [12; 13]. Biosignals refer to the product of chemical, electrical, physical, and biological processes taking place during speech production, such as neural activity, articulator motor control, muscle activity, articulatory gestures, the vibration of the vocal folds, and pulmonary activity. Technologies to capture these biosignals include vocal tract imaging [14], magnetic tracing [15], electroencephalogram [16], and EMG [17; 18]. The conversion from these silent biosignals to audible speech can be done directly--using some machine-learning algorithms that model the relationship between the feature vectors extracted from the biosignals and the acoustic signals [5; 19]--or indirectly--by first producing the related text [20; 21; 22] and then using a text-to-speech (TTS) model to generate synthetic speech. The non-acoustic biosignals that are used in this work are EMG signals or, more specifically, surface (i.e., non-invasive) EMG [23]. Electromyography is a technique used to measure and record the electrical activity of muscles. When a muscle is active, it produces an electrical signal, called an action potential, that can be detected by an electrode placed on the skin over the muscle. Since for this study we are interested in speech, we target muscles in the face and the neck. In order to develop an EMG-to-speech SSI, a large database of EMG and speech data is required. The main idea is to obtain a model trained on large amounts of parallel EMG and speech data. To ensure the generalization capabilities of the models, it is important to use a diverse and representative dataset for training. However, the process of acquiring the data is complex and presents a number of difficulties. Two prominent challenges in the development of these interfaces are the dependency of the trained models on the session (session dependency) and on the speaker (speaker dependency). Session dependency arises from the variations observed in the obtained EMG signals when electrodes are positioned differently on the subject's face. Speaker dependency is due to differences in the way of speaking from person to person. Additionally, an important issue arises from inadequate adhesion of the electrodes to the skin, leading to the detachment of electrodes over time and the generation of noisy signals. As a consequence, long sessions are difficult to carry out, thus limiting the amount of data available per session. EMG signals have been previously used to perform phone classification [24; 25], syllable identification [9], word recognition [11; 26], continuous speech recognition [27], speaker recognition [28; 29], and direct speech generation [22; 30; 31; 32]. In this study, we perform a set of phone classification experiments using data from different speakers and sessions. Classifying phones offers a straightforward means of gaining valuable insights into the information conveyed by each muscle involved in the speech production process, making it an advantageous task for studying a setup performance [33]. This work is an extension of the study presented in [34], which describes a set of experiments designed to validate the acquisition setup of the newly developed ReSSInt-EMG database. Using data from nine recording sessions, in our previous study, we compared the performance of the new database with that of a comparable subset extracted from the well-known EMG-UKA Trial Corpus [4].