SANCL : Multimodal Review Helpfulness Prediction with Selective Attention and Natural Contrastive Learning (2022)

Abstract: With the boom of e-commerce, Multimodal Review Helpfulness Prediction (MRHP) that identifies the helpfulness score of multimodal product reviews has become a research hotspot. Previous work on this task focuses on attention-based modality fusion, information integration, and relation modeling, which primarily exposes the following drawbacks: 1) the model may fail to capture the really essential information due to its indiscriminate attention formulation; 2) lack appropriate modeling methods that takes full advantage of correlation among provided data. In this paper, we propose SANCL: Selective Attention and Natural Contrastive Learning for MRHP. SANCL adopts a probe-based strategy to enforce high attention weights on the regions of greater significance. It also constructs a contrastive learning framework based on natural matching properties in the dataset. Experimental results on two benchmark datasets with three categories show that SANCL achieves state-of-the-art baseline performance with lower memory consumption.

## introduction it is unbelievable to witness an e-commerce boom that has transpired over the past decades @xcite . in the virtual bazaar, countless deals are made between mutually invisible sellers and customers from time to time, under the administrator's supervision. for customers, it may be their biggest headache to determine whether they should pay for a good when being overwhelmed by tempting advertisements, as they know quite a little information about a product in face of the seller's meticulous promotions without any external references. in this situation, reviews in e-shops that can provide justification information, are thus of great value to customers. however, the quality of reviews under a certain product page can be disparate-many customers are willing to leave informative feedback on the product, while many others arbitrarily write a few words and even paste irrelevant messages in their comments. therefore, from the perspective of online shopping platforms, they would be welcome and attractive to customers if they provide a service that can intelligently filter and place the most helpful reviews at the top position. the task in the machine learning field to solve this problem is review helpfulness prediction (rhp) @xcite . with the thriving of multimodal learning @xcite research and the handy accessibility of multimodal data in this internet era, the latest progress incorporated image (vision modality) information into the review helpfulness prediction (rhp) @xcite as @xcite . although previous work attained excellent results in mrhp, there are still some drawbacks. first, the attention mechanism in these works for representation learning follows the most basic setting-it directly computes out the attention scores based on the representation vectors of tokens or sentences, without any further intervention on the obtained weights @xcite . generally, the amount of task-related information in each sentence in a given piece of review may vary greatly-since customers usually casually write these reviews and may insert some meaningless words, such as emotional appreciation or complaints that can not benefit the viewers. we observed that due to dataset characteristics in the mrhp task, there are cues to help locate those key sentences in the review text. therefore, we proposed a probe-based selective attention mechanism to employ them for better attention results. secondly, it has been revealed that the correlation, e.g., the similarity of feature vectors, among multimodal and multi-field data is an essential factor for task modeling @xcite . nevertheless, existing studies @xcite simply quantified them by similarity metrics, such as cosine value or negative euclidean distance, and conduct the main tasks based on these similarity scores as features. though gained appreciative results, we believe they can be better utilized through the contrastive learning scheme to refine the learned representations, which enables the output layer to make more accurate predictions. in this paper, we propose a novel framework, sancl, which incorporates these two basic ideas. we first generate a special "probe" mask that highlights the key sentences from the product and review text. thereafter, these masks are inserted into the computation attention modules to help focus more on task-related sentences. then we construct a contrastive learning framework, which harnesses the internal correlations within multimodal data and the fundamental contrastive predictive coding (cpc) model @xcite , to learn better multimodal representations for the main task. the framework is composed of two feature spaces, dubbed domains. each domain takes a specific combination of projected representations as input, according to their relation types through semantic analysis. by minimizing the auxiliary contrastive loss, the multimodal and multi-domain representations can be refined with the inherent relations. our contribution can be summarized as follows: ## related work in this section, we briefly recap some relevant work in the field of review helpfulness prediction and multimodal contrastive learning. review helpfulness prediction customer reviews play an important role in helping customers investigate products before determining whether to purchase @xcite @xcite @xcite . support vector machine (svm) was first employed to automatically judge the review helpfulness @xcite @xcite . later, linear regression @xcite , extended tensor factorization @xcite , and probabilistic matrix factorization models @xcite have been applied to integrate complicated constraints into the learning process. with the development of deep learning, deep neural networks @xcite @xcite have been utilized to model the sophisticated elements in this task. recently, @xcite proposed a graph neural network to capture the intrinsic relationship between the products and their reviews. however, most existing studies only focus on the text of reviews, neglecting the images that usually exist in online reviews. this paper takes advantage of the images and proposes a novel contrastive learning framework with a selective attention mechanism to learn expressive multimodal features. ## method in this section, we first introduce the problem definition of multimodal review helpfulness prediction (mrhp). then we elaborate on the model architecture and processing pipeline of our method. ## experimental settings this section presents the datasets, baseline models, and metrics used and compared in our experiments. ## results and analysis in this section, we will compare our approach with several advanced baselines and explore how it improves the multimodal helpfulness prediction task. ## conclusion we propose a novel framework, sancl, for the task of multimodal review helpfulness prediction (mrhp) in this paper. we first present a selective attention mechanism, which purposefully aggregates information from these crucial sentences in the review text by generating the probe mask that exerts re-normalization on the attention weights and pooling stage. we then build up a multidomain natural contrastive learning framework in our model. it exploits the natural relations among the data from different fields and modalities in the dataset to enhance the model's capacity for multimodal representation learning. results of comprehensive experiments and analyses demonstrate the superiority of our model over the comparable baselines and the efficacy of the novel components.