But adding visual information made the syllables easy to distinguish because of the difference in lip closure. In unimodal presentations, acoustic recognition is more accurate than visual among hearing subjects who have not been trained in lipreading. These perceptual-confusion matrices reveal complementarity of the auditory and visual components of speech. Not only do audible and visible speech provide two independent sources of information, but each also provides strong information where the other is weak. No fundamental theory describes the evolution of complementarity in bimodal speech.
Complementarity confers two important research benefits.
First, it increases speechreading's value as a system in which to probe the general properties of sensory integration in human beings. The bimodal effects are most pronounced when both modes are fallible for some features, but one mode is accurate while the other is not. Second, complementarity is especially welcome in automated speechreading systems because the visual information improves recognition of those utterances most difficult for current acoustic recognition systems to discriminate. The results of experiments producing confusion matrices help determine exactly how sensory integration takes place, or whether it takes place at all. The two sources may not be integrated for distinguishing a particular feature.