# Spatiotemporal Feature Enhancement for Lip-Reading: A Survey
## 3 Spatiotemporal Feature Enhancement Methods
### Spatiotemporal Feature Enhancement Based on Spatial Features
#### 3.1.2 Spatial Feature Enhancement Based on DenseNet  
DenseNet, characterized by its dense connectivity pattern, facilitates the construction of feature extraction and enhancement networks, preserving information flow and gradient propagation, which simplifies the training process. With a deeper network structure compared to ResNet, DenseNet excels at extracting more complex spatial features. Figure 6 shows the schematic diagram of the spatial feature enhancement method based on DenseNet.  
In virtue of this better characteristic of DenseNet, many efforts have been made to construct novel architectural structure for simulating lip movement, such as LCANet [73], MouthNet [74], and the work proposed by [PERSON] et al. [75], and [PERSON] et al. [76]  
LCA-Net [73] integrates 3D convolution with fast networks in the front-end design, forcing seamless information flow across layers. Their fast network, equipped with transformation gates \(t\) and carry gates \(1-t\), allows the deep neural network to directly propagate partial input information to the output layer. By adaptively combining local features detected by individual CNN filters, the fast network encodes richer semantic features,preserving signal strength and mitigating the vanishing gradient problem, although some redundancy in spatial convolutions remains.  
The work proposed by [PERSON] et al., aimed at effectively enhancing the model's ability to extract spatiotemporal features, utilized multiple DenseNet embedded by 3D convolution to capture both spatial and temporal features [75]. This approach encourages DenseNet to process not only the spatial characteristics of individual frames but also dynamic temporal patterns, thereby strengthening the temporal feature representation capabilities at the front end.  
To extract more refined image features based on DenseNet, MouthNet is constructed by stacking DenseBlocks and TransBlocks, where each output in a DenseBlock is concatenated with the outputs of all preceding layers, performing feature reuse and dense convolution [74]. Each DenseBlock is paired with a TransBlock for sampling and dimensionality reduction, thereby achieving dense feature connections and efficient reuse. The results indicated the superior performance on the Oulu-VS2, GRID, and proprietary datasets.  
In the work proposed by [PERSON] et al. [76], 3D convolution was employed to construct Transition layers and DenseBlocks, stacking them multiple times to form a feature enhancement network. In addition, to construct diverse spatiotemporal information, they aggregated features from multiple CNNs at different scales and depths. Specifically, they utilized the idea of Dropout to concatenate the outputs of DenseNet through Dropout free, Dropout, and Spatial Dropout, respectively, to generate richer visual features containing multi-scale and multi-depth spatiotemporal information. The comparative experiment demonstrated robust generalization performance across eight different noisy environments.  
Figure 6: Spatial feature enhancement methods based on DenseNet. (**a**) The architecture of Ref. [73]. (**b**) The architecture of Ref. [75]. (**c**) The architecture of Ref. [76].