# Diverse Pose Lip-Reading Framework
## 2 Literature Review  
The current works in lip-reading are based on frontal views/poses of face. The available frontal view lip-readings are speaker dependent [7] as well as speaker independent [8; 9; 10; 11; 12; 13; 14; 15; 16]. The speaker dependent systems are trained by individuals who use the system. On the other hand, the speaker independent systems are trained to respond to a word regardless of who speaks. Apart from these works, other works performed lip-reading for isolated speech recognition where a short pause is needed between words and for continuous speech recognition where no such pauses are needed. In [7; 11; 12; 17], the hidden Markov model (HMM) has been used and to adopt certain techniques to make a speaker independent lip-reading system and also tried to improve recognition rate in the continuous visual speech. These works performed well for isolated word recognition and speaker-dependent frontal view lips-reading but did not achieve high results in continuous visual speech. In addition, it has achieved a lower accuracy rate of a speaker independent lip-reading system. Lip-reading based on deep neural networks (DNN) and Recurrent Neural Networks (RNN) are introduced in [8; 9; 10; 13; 14; 18; 19; 20]. These approaches performed well on isolated word recognition, but their performance was low over continuous speech. The continuous visual speech for a lip-reading system is problematic due to extensive co-articulation between words as a result homophenes (Homovisemes) produced [7; 8; 9; 14]. It has been observed that the Convolutional Neural Networks model was unable to classify more accurately on some datasets due to the insufficient data and low-quality videos [8; 14]. CNN needs large datasets with high definition videos. Lip-reading was carried out in different pose angles that are known as multiple view lip-reading. Multiple views were the focus of attention in high quality videos but relatively less works appeared [13; 21; 22] due to its poor performance in low quality videos. These studies performed sentence recognition in multiple views. These studies performed well on high quality videos and show better results but did not perform well on low quality videos. Machine based lip-reading is affected by certain factors such as video resolution, multiple people speaking at the same time and their poses. The most important factor for the MLR system is that it requires high-resolution videos. Another factor is multiple people talking at the same time which creates confusion as who is speaking which word and it causes misrecognition of words. This problem is addressed in [8; 13] by implementing a synchronization network (SyncNet). The SyncNet learned joint embedding between audio/sound and lip-movements to detect the active speaker. The pose is another main factor where different poses affect the MLR system. Most of the former studies presented frontal pose lip-reading [7; 8; 10; 11; 12; 13; 14; 17; 18; 23; 24] due to the unavailability of data but few works demonstrated lip reading on multiple views [13; 21; 22] and showed good results on high quality videos. Compared to the frontal view lip-reading, the performance of multiple-view lip-reading was lower because the complete mouth region was often not clearly visible in most cases. However, these studies only work well for high quality videos. It is not essential that high-quality cameras are used in each domain because mobile phone camera, CCTV cameras and webcam recorders generate low quality videos with diverse poses. A face characterization approach exploiting dynamic appearance and time-dependent local features during speech utterance in the Internet-of-Things (IoT) environment is proposed in [25]. The Lombard effect has been introduced in [23]. It is speaking style with increasing speech in noise which is understandable to listener to make robust lip-reading systems. However, it is used more specifically in noisy conditions.