Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Prediction (2022)

Abstract: Modern Review Helpfulness Prediction systems are dependent upon multiple modalities, typically texts and images. Unfortunately, those contemporary approaches pay scarce attention to polish representations of cross-modal relations and tend to suffer from inferior optimization. This might cause harm to model’s predictions in numerous cases. To overcome the aforementioned issues, we propose Multi-modal Contrastive Learning for Multimodal Review Helpfulness Prediction (MRHP) problem, concentrating on mutual information between input modalities to explicitly elaborate cross-modal relations. In addition, we introduce Adaptive Weighting scheme for our contrastive learning approach in order to increase flexibility in optimization. Lastly, we propose Multimodal Interaction module to address the unalignment nature of multimodal data, thereby assisting the model in producing more reasonable multimodal representations. Experimental results show that our method outperforms prior baselines and achieves state-of-the-art results on two publicly available benchmark datasets for MRHP problem.

## introduction current e-commerce sites such as amazon, ebay, etc., construct review platforms to collect user feedback concerning their products. these platforms play a fundamental role in online transactions since they help future consumers collect useful reviews which assist them in deciding whether to make the purchase or not. unfortunately, nowadays the number of user-generated reviews is overwhelming, raising doubts related to the relevance and veracity of reviews. therefore, there is a need to verify the quality of reviews before publishing them to prospective customers. as a result, this inspires a recent surge of interest targeting the review helpfulness prediction (rhp) problem. ## model architecture in this section we delineate the overall architecture of our mrhp model. particular modules of our system are depicted in figure 1 . ## conclusion in this paper, we propose methods to polish representation learning for the multimodal review helpfulness prediction task. in particular, we aim to advance cross-modal relation representations by learning mutual information through contrastive learning. in order to further enhance our framework, we propose an adaptive weighting strategy to encourage flexibility in optimization. moreover, we integrate a cross-modal interaction module to loose the model's reliance on unalignment nature among modalities, continuing to refine multimodal representations. our framework is able to outperform prior baselines and achieve state-of-the-art results on the mrhp problem.