# MFEM: Multi-View Feature Enhanced Attention Model for Image Captioning
## 3 Method
### Teacher Language Model  
As depicted in Figure 1b, the teacher model is the Transformer-based language model, incorporating the multi-view feature enhanced attention. The features \(F\) are sent to the encoder. After being processed by multiple encoding layers, the contextual features \(F_{c}\) are obtained. These contextual features are passed to each decoding layer to facilitate the development of image descriptions. During the decoding stage, the multi-view feature enhanced attention not only captures the dependencies within the partially generated sequence but also further extracts important details from the \(F_{c}\). Figure 1: Overview of the proposed MFEAM framework. (**a**) The full architecture includes a visual encoder, a student model, and a teacher model. The teacher model guides the student model via knowledge distillation and model averaging from both visual and textual views. (**b**) Detailed internal structure of the teacher model, where the Multi-View Feature Enhanced Attention (MFE) module is embedded in both encoder and decoder. The key idea of the MFE is to refine attention weights from two complementary views--the textual view and the visual view--improving the semantic representation capability across modalities of the model. Specifically, the self-enhanced module strengthens attention weights by analyzing their distribution, encouraging the model to focus more on semantically informative tokens. In parallel, the sliding window module applies a convolutional kernel over the visual attention map, enhancing the model's spatial awareness and its ability to capture local structure. The outputs from both views are then fused and used to reweight the value vectors, resulting in more expressive multi-modal representations. The overall architecture is illustrated in Figure 2. Firstly, we compute this attention weight \(att_{i}\) using the input vectors \(q_{i}\) and \(k_{i}\), as defined by the following equation:  
\[att_{i}=Softmax\left(\frac{q_{i}k_{i}^{T}}{\sqrt{d_{k}}}\right) \tag{3}\]  
Compared to the self-attention mechanism, which performs calculations between \(att_{i}\) and \(v_{i}\), the multi-view feature enhanced attention separately processes the attention weights from two views-visual features \(att_{i}^{l}\) and textual features \(att_{i}^{T}\). For the attention weights of textual features \(att_{i}^{T}\), we enhance the textual features by the self-enhanced module. \[\alpha=1-\max\left(att_{i}^{T}\right)+\min\left(att_{i}^{T}\right) \tag{4}\]  
\[EA\_att_{i}^{T}=\left((e^{\alpha}+1)\cdot\gamma\right)\cdot att_{i}^{T} \tag{5}\]  
In Equation (4), \(\alpha\) reflects the flatness of the attention distribution. When \(\max\left(att_{i}^{T}\right)\) and \(\min\left(att_{i}^{T}\right)\) are close, \(\alpha\) becomes larger, indicating that the model lacks a strong semantic focus. A higher \(\alpha\) increases the overall magnitude of the attention to encourage more discriminative attention weights. In Equation (5) we use \(\gamma\) as a weight parameter to prevent excessive enhancement of \(att_{i}^{T}\). Over-enhancing \(att_{i}^{T}\) can lead to biased understanding of features by the model, resulting in an unstable training process. This adaptive mechanism enables more stable and effective modeling of textual focus in ambiguous or diffuse semantic contexts. For the attention weights of visual features \(att_{i}^{l}\), we enhance the visual features by the sliding window module. In contrast to the self-enhanced module, the sliding window  
Figure 2: Architecture of the Multi-View Feature Enhanced Attention (MFE). MFE consists of the self-enhanced module and the sliding window module, which enhance attention weights from the textual and visual views. Visual and textual features are processed through separate channels for processing, enhancing the relationship between visual and textual features. module employs a convolution-based approach, where the convolutional kernel slides over the attention weight matrix to capture localized features and enhance the spatial awareness of the model.