Recent improvements in tracking and feature extraction mean that speaker-dependent lip-reading of continuous speech using a medium size vocabulary (around 1000 words) is realistic. However, the recognition of previously unseen speakers has been found to be a very challenging task, because of the large variation in lip-shapes across speakers and the lack of large, tracked databases of visual features, which are very expensive to produce. By adapting a technique that is established in speech recognition but has not previously been used in lip-reading, we show that error-rates for speakerindependent lip-reading can be very significantly reduced. Furthermore, we show that error-rates can be even further reduced by the additional use of Deep Neural Networks (DNN). We also find that there is no need to map phonemes to visemes for context-dependent visual speech transcription.
REF proposes a speaker-adaptive training procedure to achieve speaker independent lipreading system with the use of deep neural networks.
Improved speaker independent lip reading using speaker adaptive training and deep neural networks
2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Computer Science