# Deep Vision Multimodal Learning: Methodology, Benchmark, and Trend
## 1 Introduction  
Humans perceive objects in various ways, such as sight, hearing, and touch. \"Modality\" is defined as a signal that an object is represented in a certain way. Common unique modality information includes text, image, video, and sound. Modality information from multiple channels can be more widely collected and recorded in modern networked society. For example, recording a short video with subtitles and posting it conveys information that includes multiple modalities such as video, text, and sound. \"Multimodality\" refers to modal information for the same object. As humans can use multimodal information to help make judgments, machines can also perform representation learning and its downstream tasks through input with multimodal information. Recent work has shown that deep learning is widely used in the field of computer vision, with great success in, for example, image classification, semantic segmentation, object detection, and other vision tasks. Meanwhile, large-scale pretrained models based on deep learning are well developed for natural language processing tasks. Some researchers have found that in some real-world tasks, better performance is achieved by using information from both visual and language modalities. Deep learning models that use multimodal information tend to be better than those that use a unimodality. [PERSON] et al. [1] has demonstrated that multimodal learning yields more accurate estimates of latent space representations. Several surveys have been published on the topic of multimodal learning [2]. The first to thoroughly propose this concept was a widely accepted taxonomy for multimodal machine learning by [3] as representation, translation, alignment, fusion, and co-learning. This taxonomy allows researchers to better grasp the state and future trends in multimodallearning. Among them, multimodal representation learning is the focus of this paper. Representation learning for unimodality has been a long-standing topic in deep learning. Essentially, it is the process of converting input signals into features usable by the model for machine learning [4]. [PERSON] et al. [5] provide a comprehensive analysis of works on deep multimodal learning from three perspectives: learning multimodal representations, fusing multimodal signals at various levels, and multimodal applications. [PERSON] et al. [6] highlight the critical issues of newly developed technologies, such as the encoder-decoder model, generative adversarial networks, and attention mechanism in a multimodal representation learning perspective. [PERSON] et al. [7] discuss multimodal problem formulation, methods, existing datasets, and metrics, including the results obtained with corresponding state-of-the-art methods. However, the existing review articles focus on the analysis and classification of abstract theories and are not comprehensive enough to discuss the applications and existing methods, especially the new implementations in recent years. Unlike previous reviews, this paper focuses on the visual-language multimodal learning techniques that have become popular in recent years, presenting recent progress in three main parts of learning architectures, learning paradigms, and multimodal data analysis. Meanwhile, we collect larger scale multimodal tasks and benchmarks, as well as state-of-the-art (SOTA) models and metrics corresponding to each dataset. To reflect currency, innovation, and diversity, the vast majority of methods mentioned in this article are from articles included in major deep learning conferences after 2019 and have a large impact. This review focuses on specific technologies rather than comprehensive taxonomy. We hope this article will be helpful to both novice and veteran researchers in the multimodal field. Generally, the major contributions of our work can be summarized as follows:  
* In terms of **methodology**, we propose a rational classification, including architectures, paradigms, and issues. We pay particular attention to recent works in this field.