# Gaze Estimation Based on a Multi-Stream Adaptive Feature Fusion Network
## 3 Proposed Method
### Fusion Stream
After passing through the shallow feature extraction network, the extracted facial and eye features are fed into the gating mechanism. The gating mechanism utilizes the sigmoid function to obtain the self-attention weight of each feature. These weights are then applied to the original input features through matrix multiplication to achieve adaptive self-weight adjustment. The specific function for computing the self-gating weights is as follows:  
\[\omega=\sigma(x)=\frac{1}{1+e^{-x}} \tag{1}\]  
where \(x\) is the input feature map, and \(\sigma(x)\) is the sigmoid function applied to the input \(x\), generating a weight \(\omega\) that is between 0 and 1. After calculating the self-gating, the self-gated weights are applied to the input features. The specific function is as follows:  
\[\hat{x}=x\cdot\sigma(x) \tag{2}\]  
where \(\sigma(x)\) is the sigmoid function; \(x\) is the input feature, and \(\hat{x}\) is the weighted output. In addition, to further enhance the model's generalization capability and prevent overfitting of the weight parameters, we introduced an L2 regularization term into the loss function. L2 regularization effectively controls the model's complexity by imposing a penalty on the model parameters, thereby improving the model's performance on unseen data. The total cost function is as follows:  
\[L_{total}=L_{MES}+\lambda\cdot L \tag{3}\]  
Here, \(\lambda\) is the regularization parameter, initialized to 0.0001, and we use the default parameter value. \(L_{MES}\) represents the mean squared error loss; \(L\) is the regularization term, calculated as follows:  
\[L=\sum_{i}\omega_{i}^{2} \tag{4}\]  
In this context, \(\omega\) represents the model parameters. After the AWAM, we concatenate the eye and face features along the dimension and feed the concatenated features into the JointCBAM for more effective feature fusion. The traditional CBAM module evaluates attention weights on the channel dimension first, then adjusts the weights on the spatial dimension through concatenation. However, the improved JointCBAM module proposed in this study processes the input features in parallel by dividing them into two parallel branches, which handle channel and spatial attention separately. Finally, the attention information from both branches is interacted with to achieve more efficient feature fusion. As shown in Figure 5, in the compression part, the input features are divided into two branches, with max pooling and average pooling used to obtain channel descriptors and spatial descriptors, respectively. In the interaction part, the channel and spatial descriptors are concatenated and processed by a multi-layer perceptron (MLP) to model the correlation between global spatial and global channel features. After sufficient interaction between the global spatial and channel features, they are split and restored to their original dimensions. The weights are calculated using a sigmoid function and applied to the input features. Finally, the weighted features are fused through element-wise summation. Compared to the traditional CBAM module, the JointCBAM module not only avoids separately analyzing the spatial and channel modeling process but also effectively merges channel attention and spatial attention information to achieve better space-channel interaction and generate more comprehensive features. To validate the effectiveness of our improved module, we conducted related comparative experiments in Section 4. The JointCBAM channel attention module is shown in Figure 5. The input feature map \(F\) undergoes global average pooling and global max pooling along the spatial dimension, and after passing through a multi-layer perceptron, the results are summed to obtain the channel feature map.