# MFEM: Multi-View Feature Enhanced Attention Model for Image Captioning
## 1 Introduction  
Images and text serve as two important mediums for conveying information. Images encompass more extensive information, but they are more challenging to interpret. On the other hand, text conveys information in a more straightforward manner. Early researches utilize Convolutional Neural Network (CNN) [1, 2, 3, 4] and Recurrent Neural Network (RNN) [1, 2, 4, 5, 6]. RNN facilitates the transmission of information over different time steps, efficiently capturing temporal dependencies within the data. Nevertheless, RNN encounter difficulties such as gradient vanishing and explosion, which significantly limit the capacity to capture long-distance dependencies, especially when processing lengthy sequences. The Long Short-Term Memory (LSTM) network [7, 8, 9, 10] represents a substantial advancement over RNN. The advent of the Transformer has rectified the computational inefficiency of LSTM in handling lengthy sequences, facilitating parallel processing of the input sequence and enhancing training speed. In addition, vision-language pre-trained models have dramatically enhanced the capability of cross-modal applications, such as Contrastive Language-Image Pretraining (CLIP) [11], by learning cross-modal alignment representations on large-scale image-text pairs. Nowadays, image captioning methods have significantly advanced with the adoption of Transformer-based models [12, 13, 14, 15], whilethe visual feature extraction phase is swiftly advancing, transitioning to multimodal architectures [11; 16; 17; 18] trained on large-scale datasets. Despite advances in architectures and structural networks, most contemporary image captioning methods utilize a single language model to generate captions. Recently, the structure of two language models [13; 19] has been employed to learn visual representations through knowledge distillation and model averaging [20; 21; 22]. Meanwhile, many image captioning datasets [23; 24; 25; 26; 27] contain large amounts of images and text, resulting in highly complex and diverse input features. As a result, Transformer-based image captioning models are capable of effectively extracting visual and textual features, but the characteristics of the input result in suboptimal performance in accurately expressing the entangled features of vision and text. To address this challenge, we introduce the multi-view feature enhanced attention to strengthen the representation of entangled features of vision and text. Through the integration of the Mean Teacher Learning approach, the teacher model can attain a superior comprehension of important features, facilitating more effective parameter updates for the student model. In this paper, we propose a knowledge distillation framework named MFEAM, which is built upon a multi-view feature enhanced attention. MFEAM adopts an encoder-decoder architecture as its foundational framework. The CLIP model functions as the visual encoder, while the language decoder comprises the teacher model and the student model. The teacher model employs the multi-view feature enhanced attention to interpret important features from both visual and textual views. Through knowledge distillation and model averaging, the teacher model can gradually follow the state of the student model through time. In addition, inspired by the Guided-Embedding Network [28], we divide the decoder of the student model into two groups to separately process instance and relationship features, which helps prevent performance degradation caused by overemphasizing features. We formulate and evaluate strategies that implement this interaction paradigm during training phase. Experimental results demonstrate that MFEAM attains competitive performance on the MSCOCO when trained without leveraging external data. 1. We present a novel architecture called MFEAM, which incorporates the multi-view feature enhanced attention.