# A Survey of Full-Cycle Cross-Modal Retrieval: From a Representation Learning Perspective
## 2 Overview of Cross-Modal Retrieval
The attention mechanism creates long-distance feature connections, allowing the tracker to focus on important information while extracting a wealth of semantic information. The combination of self-attention and guided attention is known as co-attention. A distillation attention framework [54] is a dual-encoder model that achieves faster inference speeds than a standard fusion encoder thanks to its deep interaction module. In this study, dual-encoder training is guided by fusion encoder instructor information in the annotation, and the proposed knowledge distillation consists of two stages pre-training distillation and fine-tuning distillation, ultimately outperforming other approaches. The use of meshed memory allows the encoder to operate at multiple levels, learning both low-level and high-level relationships. X-linear attention, developed by [PERSON] et al. [55], enables high-order feature interactions, while bi-linear fusion technology improves content interpretation in cross-modal information by capturing second-order interactions between input types using spatial and channel bi-linear attention distributions. Stacked cross-attention is widely used by many researchers to maximize the investigation of vision language features. Previously, the stack cross-attention network, named SCAN [56], has become a new benchmark for calculating similarity on all potential pairs and not only areas in images and words in sentences. The authors believe sentence descriptions are weakly explainable, implying that words in phrases cannot suit particular positions. As a result, considering the possible connections between visual regions and words, fine-grained interactions between vision and language must be documented. They infer the similarity of entire images and phrases by mapping word and picture areas to a shared embedding space. SCAN uses Faster-RCNN to substitute CNN in DNN and extracts 36 border features with the border indicating the position of candidate regions. Then, the layers of average pooling and fully connection encode outputs are used. In addition, the revolutionary Bi-GRU assesses the relative value of each word in sentences. Bottom-up attention is employed to forecast and encode visual areas to features, and it maps words to sentence context characteristics. Stacking cross-attention infers similarity between visuals and phrases. Several studies have been derived from the SCAN model, including CAMP [57], IMPRAM [52], MMCA [58], METER [59], and SMAN [60]. For example, CAMP differs from SCAN in that SCAN only interacts in discrete subspaces, whereas CAMP maps image text in the same subspace. CAMP involves specular highlights in vision and salient phrases in language, alternately considers information from the two modalities, and filters out irrelevant information to find fine-grained features for cross-modal matching. CAMP comprises two modules: an aggregate module and a gated fusion module. IMRAM iteratively matches with recurrent attention memory in image text retrieval. Previous studies typically focused on examining all semantic units to ensure uniform alignment, but the complexity of alternative meanings can make it difficult for key semantics to be reflected. To address this issue, iterative matching with repeated attention memory was introduced in IMRAM, which compares the corresponding information between the text and image in several phases to progressively investigate the corresponding fine-grained connections. Given that individuals advance in the retrieval process, a memory distillation unit based on SCAN is introduced to alignment via multi-step iterations. In IMRAM, the image input is for the Faster-RCNN extraction of regional features, followed by a complete connection layer to map each regional feature to a dimensional space. Text input is encoded using bi-GRU to obtain vector representations of each word in training.