For late integration we must determining the value of the weight alpha applied to the audio and visual vectors. This is the best possible result obtained by searching the full range of values for alpha. Noise compensation Compare spectral subtraction and matched models results. Using matched models gives better performance than audio-visual using uncompensated audio. Noise compensated audio-visual Visual information is added to the compensated audio results.
Visual information is still useful even when noise compensation is used.
Summary These results show that using visual speech information is beneficial even if one uses acoustic noise compensation techniques. The main problems with MSA are that it is not scale invariant and cannot track the mouth. Both of these methods have been implemented in realtime or near realtime on an SGI O2 workstation. We can easily run MSA at 30fps without using all of the processor time. An ASM fitter takes all processor resources and it limited by that to an average of 15fps.