# Spatiotemporal Feature Enhancement for Lip-Reading: A Survey
## 1 Introduction  
Lip-reading, as a specialized visual language recognition system, attempts to interpret the semantic information of spoken content by analyzing the motion patterns of a speaker's lip movement captured in image sequences. Lip-reading relies solely on the sequences of lip movements from silent visual data different from acoustic signals in speech recognition and audio-visual speech recognition, which challenges the recognition precision of diverse deep learning models in intricate tasks. Recent research has highlighted the pivotal role of spatiotemporal feature enhancement in improving recognition performance [1, 2], such that significant progress has been driven in lip-reading methods and model architectures. Figure 1 illustrates the fundamental venation and development orientation of spatiotemporal feature enhancement techniques in the field of lip-reading.  
In recent years, deep learning-based approaches to lip-reading have garnered significant attention, leading to a surge of review papers that synthesize existing work. These reviews typically focus on aspects such as applications [3], datasets [3, 4, 5], front-end feature extraction networks [4, 6], back end classifiers [4], model architectures [5, 6], and audio-visual analysis [7, 8]. However, there is a notable gap in the literature regarding a comprehensive survey on spatiotemporal feature enhancement. To address this gap, this paper offers a thorough review of recent advancements in lip-reading research and outlines the classificationof lip-reading methods, with a particular emphasis on spatiotemporal feature enhancement techniques. Additionally, we analyze the challenges faced in enhancing spatiotemporal features and discuss potential future directions for research.  
In this survey, we conduct a comprehensive survey of more than 110 research papers focused on lip-reading. Our objective is to facilitate researchers in gaining a rapid and systematic understanding of the pertinent methods and technologies in this domain. The information from the survey is presented in Figure 1. To collect the published materials related to the research area, the following steps are followed to find corresponding keywords: First, search for literature with terms such as \"lip reading\", \"lip read\", \"audio-visual speech recognition\", \"speech recognition\", \"visual speech analysis\", \"lip extraction\" in their titles or keywords. Next, search for alternative spellings of all critical terms. Then use Boolean operators AND and OR to connect different main clauses and spellings. It is noted that the literature originates from IEEE, Elsevier, Google Scholar, MDPI, Springer, and others to guarantee that the above search terms cover the most relevant work.  
The subsequent sections are organized as follows: In Section 2, we classify various lip-reading methods and provide an in-depth analysis of four distinct lip-reading frameworks. Section 3 offers a review of the spatiotemporal feature enhancement techniques employed in lip-reading. In Section 4, we analyze the challenges associated with spatiotemporal feature enhancement. Section 5 explores potential future directions for the development of spatiotemporal feature enhancement techniques. Finally, Section 6 presents the conclusions drawn from this study. A taxonomy of spatiotemporal feature enhancement methods is illustrated in Figure 2.  
Figure 1: Development of spatiotemporal feature enhancement techniques for lip-reading. The color of the circle indicates the type of technology, and the size of the circle represents the quantity of literature.  
Figure 2: Taxonomy of spatiotemporal feature enhancement methods.