振兴语言能力：双峰静默交互系统
Authors: Guo Yusen , Huang SI , Song Zixuan , Li Qiliang , Gou Guangyang 
Published: December 2024

Vocal disorders due to laryngeal cancer, cerebral palsy, autism, autism and other complicated diseases makes rehabilitation very difficult. Most patients are unable to communicate reliably or easily using their natural voice. Studies have shown that most patients have laryngeal motor capabilities. The laryngeal vibrations as well as muscle activity related to vocal system contain rich speech information. Hence, by monitoring and deciphering laryngeal movement data, we can facilitate silent communication, offering potential relief to individuals coping with vocal disorders, enabling them to “articulate their thoughts and reclaim their voices”.

The signals of silent interaction systems stem from laryngeal electromyographic signals or vibration signals, resulting in a single dimension of sensed signals, limiting voice recognition, and challenges in sensor adherence to the skin. Therefore, there is a need to develop a wearable, non-invasive silent interaction system to facilitate seamless communication for individuals grappling with vocal impairments. In this paper, we introduce a bimodal voiceless interaction system which aims to achieve the monolithic integration of myoelectric and pressure sensors using micro-nano processing techniques. Through the correlation of laryngeal myoelectric signal and the vibration mechanics signal with speech signal, and the integration of signal acquisition, transmission, machine learning algorithms and speech recognition, our aim is to achieve highly precise sound signal output. This initiative endeavors to surpass the constraints of current voiceless interaction interfaces, addressing issues such as signal perception limitations, low sound recognition rates, and single-dimensional voice recognition constraints. The specific work of this paper includes:

(1) Explore the sensing mechanism of bimodal sound signals

We construct a quantized flexible piezoelectric pressure sensor model and an electrode-skin electrical model to investigate impedance measurement mechanisms and their performance with diverse electrode materials. Furthermore, we create a macroscopic model of the gel muscle electrode in the larynx to clarify the interplay among various sensing parameters, including sensitivity, motion artifacts, frequency response, and impedance.

(2) Prepare high-performance hydrogel materials

We explore the preparation process of PVA hydrogel material, concentrating on designing doping formulations, enhancing biocompatibility modification, improving adhesion and conductivity, and implementing other performance improvements. This includes modifying hydrogels using both natural polymer materials and synthetic polymers. We introduce a preparation method of throat PVA hydrogel muscle electrode and produce gel electrode with enhanced adhesion and conductivity.

(3) Design and fabricate bimodal sensors

We utilize a PVDF piezoelectric film as the sensing unit to capture laryngeal kinematic mechanical signals, while exploring processing and encapsulation techniques for PVDF pressure sensors. For laryngeal electromyographic signals, we utilize PVA hydrogel as the sensing unit, investigating the preparation and encapsulation process for multi-channel laryngeal muscle electrode arrays. Finally, we devise a range of integration and encapsulation methods for bisensory devices, introducing wearable bimodal sensors distinguished by their comfort, convenience and reliability.

(4) Construct bimodal silent interaction system and verify speech recognition function

Utilizing the bimodal signals generated by the pressure sensor and the laryngeal EMG sensor, we engineer a high-quality sensing signal transfer and readout circuit. We investigate noise reduction and filtering algorithms for the bimodal signals, and design models for signal enhancement, feature extraction, fusion, and recognition. Subsequently, we construct the bimodal silent interaction system and verify its functionality and reliability.

Figure 1