# Deep Learning for Neuromuscular Control of Vocal Source for Voice Production
###### Abstract  
A computational neuromuscular control system that generates lung pressure and three intrinsic laryngeal muscle activations (cricothyroid, thyroarytenoid, and lateral cricoarytenoid) to control the vocal source was developed. In the current study, _LeTalker_, a biophysical computational model of the vocal system was used as the physical plant. In the _LeTalker_, a three-mass vocal fold model was used to simulate self-sustained vocal fold oscillation. A constant \(/a/\) vowel was used for the vocal tract shape. The trachea was modeled after MRI measurements. The neuromuscular control system generates control parameters to achieve four acoustic targets (fundamental frequency, sound pressure level, normalized spectral centroid, and signal-to-noise ratio) and four somatosensory targets (vocal fold length, and longitudinal fiber stress in the three vocal fold layers). The deep-learning-based control system comprises one acoustic feedforward controller and two feedback (acoustic and somatosensory) controllers. Fifty thousand steady speech signals were generated using the _LeTalker_ for training the control system. The results demonstrated that the control system was able to generate the lung pressure and the three muscle activations such that the four acoustic and four somatosensory targets were reached with high accuracy. After training, the motor command corrections from the feedback controllers were minimal compared to the feedforward controller except for thyroarytenoid muscle activation. nonlinear control systems; artificial neural networks; voice production; speech acoustics; TensorFlow +
Footnote †: journal: Applied  
Elaborating on the neural control of the vocal system, in the human brain, the spatial and temporal auditory information is converted to perceptual information by the peripheral and central auditory nuclei [5; 6]. More accurately, the physical properties of the acoustic pressures, i.e., fundamental frequency, sound pressure level, harmonic and noise content are transformed to corresponding auditory perceptual properties, i.e., pitch, loudness, brightness, and roughness. The planning for producing a sound with desired characteristics and the required laryngeal muscle activations are performed in cortical regions of the brain. These direct pathways can be called feedforward pathways. The auditory feedback of the produced phonation integrates with the planned information to generate auditory error maps. These maps are transformed to motor information in the motor cortex. Similarly, the proprioceptive and somatosensory information from the larynx is fed back to the motor areas via the somatosensory cortex and cerebellum [7; 8]. The somatosensory error maps are also transformed into corrective motor commands in the motor cortex. The combined motor commands from the direct and feedback pathways in the motor cortex, gated by the basal ganglia, are projected on to the phonatory neurons that directly control laryngeal muscles. A detailed map of the brain regions involved in planning of auditory and somatosensory targets, feedback and descending pathways can be found in [PERSON] (2002) [9]. In humans, the learning of vocal communication is dependent on the feedback of auditory and somatosensory information during development. As speech motor planning is honed, auditory feedforward control dominates production [2]. Auditory feedback, however, is required to maintain the quality and precision of speech throughout adulthood [5; 6]. Evidence based on compensation to perturbation in auditory feedback and jaw movement indicate a key role for somatosensory feedback in both the learning and maintenance of speech [7]. The auditory and somatosensory feedback is essential to understand if the desired outcome was achieved through the feedforward control, and if not, to generate corrective motor commands such that the intended outcome is attained [10]. The majority of approaches to vocal source simulation to date have been open-loop. Physical parameters are specified, and acoustic output is produced.