In such machines, the designer must specify the method for integrating visual and acoustic information. Despite the difference in our disciplines, we have found a satisfying convergence of results that has led us to the identification of a law of human information processing as well as to the creation of a new class of accurate automatic speech-recognition systems. The image of a talker's face provides not only a window into her emotions and motivations, but also important information used in understanding speech. People with normal hearing and sight use both modalities to understand speech, although we are generally not aware of the visual component. For example, when you have difficulty understanding speech in a very noisy room, you will do better by watching the speaker's face more closely.
Some people claim they can understand the dialogue on TV better with their glasses on.
Research suggests that, whenever possible, people use and integrate both sight and sound for speech recognition. The contributions of sight and sound become more obvious when people must use only one modality. Harry McGurk, a senior developmental psychologist at the University of Surrey in England, and his research assistant John MacDonald were studying how infants perceive speech during different periods of development. For example, they placed a videotape of a mother talking in one location while the sound of her voice played in another. This audio-visual illusion has become known as the McGurk effect or McGurk illusion.