# Gaze Estimation Based on a Multi-Stream Adaptive Feature Fusion Network
## 3 Proposed Method
### Fusion Stream  
Previous studies have demonstrated the importance of facial images in gaze estimation tasks. However, directly extracting features from facial images does not always provide accurate guidance for gaze estimation and may even introduce misleading information. Given the high correlation between eye images and facial images, this study proposes leveraging eye features to guide the effective extraction of facial features, thereby enhancing their representation in gaze estimation. Considering the limitations of high-dimensional eye features in terms of interpretability and guidance, as well as the potential overfitting issues caused by a single high-dimensional feature stream, we introduce a dedicated new stream in addition to the eye image stream to further optimize feature extraction and fusion. This stream focuses on extracting low-dimensional features from both eye and facial images and employs the JointCBAM to fuse these features. This fusion strategy aims to leverage eye features to better guide the extraction of critical facial features relevant to the gaze estimation task. With this design, the fusion stream effectively integrates information from both eye and facial regions, enhancing feature representation quality and improving the model's generalization ability. As shown in Figure 3, the structural design of the fusion stream aims to optimize the feature extraction process, ensuring higher accuracy and robustness in the gaze estimation task. During the binocular feature extraction stage, this study adopts an independent input approach for left and right eye images and applies 3 \(\times\) 3 convolution kernels to extract key visual features. Subsequently, data normalization techniques are employed to process the convolutional outputs, aiming to stabilize the training process and enhance the model's generalization ability. Additionally, the ReLU activation function is selected to introduce non-linearity, further strengthening feature representation. To mitigate potential bias errors caused by convolutional layer parameter deviations while preserving image texture information, a 2 \(\times\) 2 max pooling layer is incorporated. This design helps reduce the spatial dimensions of feature maps while maintaining sensitivity to fine details. Moreover, inspired by the design principles of Residual Networks (ResNet) [38], we construct residual modules to enhance the representation of original features within the network. Within the residual module, a 7 \(\times\) 7 convolution kernel is specifically employed, contrasting with the 3 \(\times\) 3 convolution kernels used in the main network. The primary purpose of this design is to expand the receptive field, allowing the network to capture a broader range of contextual information, thereby providing richer feature representations. Through this comprehensive network structure, the feature extraction process not only precisely captures local details of eye images but also effectively integrates more contextual information using larger convolutional kernels and residual connections. Before fusing facial and eye features, considering the differences in acquisition conditions between facial and eye images in the dataset, this study designs an AWAM. As shown in Figure 4, this mechanism consists of three gating mechanisms, which adjust the features processed by the gating operations using a weight matrix initialized to 1. During training, the gating mechanism's weights and the initially set weight matrix are adaptively optimized through backpropagation, enabling the adaptive adjustment of weights for the three feature types. Through network training, this mechanism dynamically adjusts the weight distribution between eye and facial features, allowing the model to adapt to different image characteristics and improve the overall accuracy of gaze estimation. Figure 4: Adaptive Weight Adjustment Mechanism. Figure 3: Fusion Stream Architecture and Joint CBAM Feature Fusion Module Diagram.