To illustrate the potential of multilayer neural networks for adaptive interfaces, we used a VPL DataGlove connected to a DECtalk speech synthesizer via ve neural networks to implement a hand-gesture to speech system. Using minor variations of the standard back-propagation learning procedure, the complex mapping of hand movements to speech is learned using data obtained from a single \speaker" in a simple training phase. With a 203 gesture-to-word vocabulary, the wrong word is produced less than 1% of the time, and no word is produced about 5% of the time. Adaptive control of the speaking rate and word stress is also available. The training times and nal performance speed are improved by using small, separate networks for each naturally dened subtask. The system demonstrates that neural networks can be used to develop the complex mappings required in a high bandwidth interface that adapts to the individual user.
Fels et al. REF apply the MLP model to the "Data-Glove" to produce a hand-gesture-to-speech system.
Glove-Talk: A neural network interface between a data-glove and a speech synthesizer.
IEEE transactions on neural networks
IEEE transactions on neural networks
Computer Science Medicine