# A Survey of Full-Cycle Cross-Modal Retrieval: From a Representation Learning Perspective
## 3 Fine-Grained Deep Learning Methodologies
### Cross-Modal Interaction
Unlike vision language alignment, the reconstruction focuses more on global information. DSPE [63] learns from image text embedding to solve matching problems. The optimization of the loss function aims to improve the distribution of features in high-dimensional space, resulting in a more effective clustering effect. MASLN [70] proposes a solution to the issue of classes being unable to traverse instances. The proposed solution involves the use of a reconstruction sub-network, which rebuilds each modal dataset using conditional autoencoders. The sub-network leverages information from the input to the output, while minimizing discrepancies in the distribution. Additionally, MASLN introduces an adversarial sub-network to develop semantic representations. The referenced study [71] investigated neural networks for embedding and similarity calculations. The embedding network learns a latent embedding space with a new neighborhood restriction and a maximum margin ranking error. The authors improved neighborhood sampling to produce tiny batches compared to ordinary triplet sampling. The similarity network uses an elementwise product and applies regression loss training to forecast the similarity score directly. A significant number of trials indicate that this network can accurately locate phrases. The problem of visual and textual retrieval has been reformulated as a text and vision conversion task in recent research [72]. To address this task, a cycle-consistent network was proposed by the authors. In another related study [73], the attention mechanism was enhanced by incorporating a scene graph structure. Specifically, the sentence reconstruction network creates a scene graph from the objects, attributes, and relations extracted by the detection network. The resulting graph is then processed by a graph convolution network to generate a word vector, which is subsequently fed into a pre-trained dictionary shared by the encoder decoder model. This approach results in more natural and human-like visual descriptions in the generated corpus. The study on reconstruction overcomes the constraint of embedding spaces. The reconstruction approach employs a deep autoencoder to minimize heterogeneity and improve the semantic discrimination capacity. Furthermore, compared to cross-modal alignments, cross-modal reconstruction has fewer dataset requirements and a lower annotation cost, making it suited for small- and medium-sized datasets. Vision language embedding. Joint embedding may develop superior feature discrimination by integrating global and local information as semantic feature embedding. The study of DSCMR [74] presented a supervised learning structure to retain semantic distinction and modal invariance. It creates two sub-networks with weight-sharing restrictions. The authors reduced discrimination losses in labels and common representation space, increasing the significance of the learned common representation. The learning strategy of DSCMR can fully unitize paired labels and classification information, successfully learning the typical representation of heterogeneous data. PCME [75] matches one image to numerous titles or one title corresponding to multiple images. The authors believe that most existing models' deterministic functions are insufficient to capture a one-to-many correspondence. A federated representation space PCME paradigm maps one-to-many relationships. It uses probabilistic maps and does not need the precise formulation of many-to-many matches. The uncertain estimation enables PCME to evaluate retrieval difficulty and failure probability, i.e., the auxiliary interpretability aspect. The probabilistic model learns from a more decadent embedding space, where set relations are also beneficial, whereas only similarity relations are helpful in precise spaces. Probability mapping is a supplement to the accurate retrieval system. ViSTA [76] presents a transformer framework for learning an aggregated visual representation by directly encoding patches and scene embedding.