Neural models of speech production and speech acquisition

Neural models of sensorimotor control of speech production are rare. One current approach is the model given by Guenther (2006 and Guenther et al. 2006). Following the ideas of Guenther a comprehensive neural model of speech production using self-organizing neural networks can be given (Kröger et al. 2006a and 2006b). Both approaches simulate early processes of speech acquisition (babbling and imitation) as occur by toddlers during their first (and second) year of lifetime. The Kröger model is capable of generating acoustic speech signals and sensory feedback signals by using a high quality 3-dimensional articulatory-acoustic speech synthesizer as a front-end device. A mental syllabary forms the central layer within this model. The mental syllabary comprises a heap of neural SOM layers (phonetic map) which can be interpreted as a system of mirror neurons co-activating phonemic, sensory, and motor states of a syllable under production (feed-forward control). Feedback control is modeled by comparing the current sensory feedback state produced by the articulatory-acoustic model with the prestored sensory state, activated during feed-forward control via the mental syllabary. Both models can be integrated easily as a phonetic module within a more general linguistic model of speech production. Introduction A model of speech production can be subdivided into a control module – comprising feed-forward and feedback control – and an articulatory-acoustic module, i.e. into a controller and a controlled system or plant. While a lot of knowledge has been collected concerning the plant over the last decades – i.e. concerning static articulatory data and time-dependent kinematic articulatory data and concerning the articulatory-acoustic modeling – much less knowledge is availabe concerning the neural control of speech production (cp. Guenther 2006 and Guenther et al. 2006). The goal of our current work is to develop a comprehensive neural model of speech production which closely reflects as many aspects as possible of the natural human speech production process including selforganization of cortical structures occurring during speech acquisition. The articulatory-acoustic module and motor and sensory parameters Our three-dimensional speech synthesizer (i.e. articulatory-acoustic vocal tract model) (Birkholz et al. 2006) is controlled by a set of 10 articulatory parameters (Tab. 1). This set of parameters represents quasi-static articulatory states of all model articulators i.e. lips, tongue, jaw, velum, and larynx (Fig. 1). 10 neurons of the joint coordinate motor map (Fig. 1) directly control the articulatory (or low-level motor) state. The acoustic model is driven by the vocal tract area function calculated from the geometrical data of the articulatory model for each articulatory state. The acoustic model is capable of generating vocal tract transfer functions as well as the acoustic speech signal. Table 1: List of articulatory parameters, i.e. joint coordinate motor parameters ABBR. NAME OF ARTICULATORY PARAMETER JAA lower jaw angle TBA tongue body angle TBL tongue body horizontal location TTA tongue tip angle TTL tongue tip horizontal location LIH relative lip height LIP lip protrusion VEH velum height HLH hyoid horizontal location HLV hyoid vertical location Relative lip height means: lip height relative to jaw. Figure 1: Articulatory parameters (for abbreviations see Tab. 1) and geometrical grid-representation of the 3D model. Somatosensory preprocessing comprises proprioceptive and tactile preprocessing. Proprioceptive preprocessing is accomplished by extracting the location of 7 flesh points relative to the cranial coordinate system (Fig. 2 and Tab. 2). This sensory information is directly used as a high-level motor representation, namely as the spatial coordinate motor parameters or tract variables within the spatial coordinate motor map. Tactile preprocessing is represented in our approach by extraction of the contact area between (i) lower and upper lip and between (ii) tongue and hard palate, soft palate, and pharyngeal wall (Fig. 2 and Tab. 3). The first 6 tactile parameters in Tab. 2 indicate the contact area at vocal tract walls while the last 3 parameters indicate the contact area at the movable articulators. Auditory preprocessing is represented in our approach by extraction of bark-scaled formant values F1, F2, and F3 from the vocal tract transfer function (Fig. 2).