# Spatiotemporal Feature Enhancement for Lip-Reading: A Survey
## 3 Spatiotemporal Feature Enhancement Methods
### Spatiotemporal Feature Enhancement Based on Spatial Features
#### 3.1.3 Spatial Feature Enhancement Based on Lightweight Networks  
Generally, the high computational demands and stringent performance requirements of lip-reading models have severely constrained their deployment on terminal and mobile devices, thus limiting the practical applicability of lip-reading methods. As a result, reducing the parameter count and computational complexity of recognition models has become a focal point of research. In this regard, feature enhancement techniques based on lightweight neural networks, such as MobileNet, EfficientNet, GhostNet, and ShuffleNet, have gained considerable attention as effective solutions for addressing these challenges. Figure 7 shows the schematic diagram of the spatial feature enhancement method based on lightweight networks. Figure 7: Spatial feature enhancement methods based on lightweight networks. (**a**) The architecture of Ref. [77]. (**b**) The architecture of Ref. [78]. (**c**) The architecture of Ref. [79]. (**d**) The architecture of Ref. [80]. (**e**) The architecture of Ref. [81]. (**f**) The architecture of Ref. [82]. (1) MobileNet  
MobileNet employs depthwise separable convolution, which decomposes standard convolution into depthwise and pointwise convolutions. The method effectively reduces the parameter count and computational complexity of the network, thereby lowering the hardware requirements. In 2019, [PERSON] et al. [77] developed a lightweight lip-reading network using MobileNet, which extracts spatiotemporal features from each image sequence and then applies LSTM for temporal modeling. The network's simplicity enabled lip-reading on a Raspberry Pi, with its success largely attributed to MobileNet's use of depthwise separable convolutions, which enhance computational efficiency and reduce model complexity. (2) EfficientNet  
EfficientNet, as a lightweight network model, incorporates depthwise separable convolutions and Squeeze-and-Excitation (SE) blocks. The role of depthwise separable convolution is to improve computational efficiency by reducing both computational complexity and parameter count. The SE block learns inter-channel relationships and adaptively adjusts channel weights, enabling the network to emphasize important features and thereby enhancing overall performance. In 2022, MobiLipNet, as a visual speech recognition model based on EfficientNetV2 is proposed, in which the first layer was replaced with 3D convolution. MobiLipNet uses channel width and depth multipliers for scaling to obtain multiple models with different performances. Meanwhile, inverted Residual Bottleneck (MBConv) and Fused-MBConv blocks were introduced to jointly scale the model's depth, width, and resolution. MobiLipNet provides multiple EfficientNetV2 models with varying levels of accuracy and efficiency [83]. LipSyncNet is also a lightweight lip-reading model that employs 3D CNN and EfficientNetB0 to extract temporal and spatial features, enhancing the balance across the network's width, depth, and resolution [78]. The model introduces a composite scale factor to uniformly adjust the network's dimensions, optimizing image classification performance and efficiency with disproportionately increasing computational demands. (3) GhostNet  
GhostNet is a lightweight deep learning framework based on Ghost modules, which generate phantom features through simple operations such as 1 \(\times\) 1 convolution. These features retain high information content while being computationally efficient, achieving a balance between model performance and computational cost. Additionally, GhostNet incorporates group convolution and depthwise separable convolution to further reduce computational complexity. By maintaining high accuracy and reducing FLOPs, GhostNet enables efficient real-time inference on mobile devices. In 2023, [PERSON] et al. [79] proposed Efficient-GhostNet, an optimized version of GhostNet for lip-reading.