Gradient-Boosted Decision Tree for Listwise Context Model in Multimodal Review Helpfulness Prediction
Year: 2023

Abstract:
Multimodal Review Helpfulness Prediction (MRHP) aims to rank product reviews based on predicted helpfulness scores and has been widely applied in e-commerce via presenting customers with useful reviews. Previous studies commonly employ fully-connected neural networks (FCNNs) as the final score predictor and pairwise loss as the training objective. However, FCNNs have been shown to perform inefficient splitting for review features, making the model difficult to clearly differentiate helpful from unhelpful reviews. Furthermore, pairwise objective, which works on review pairs, may not completely capture the MRHP goal to produce the ranking for the entire review list, and possibly induces low generalization during testing. To address these issues, we propose a listwise attention network that clearly captures the MRHP ranking context and a listwise optimization objective that enhances model generalization. We further propose gradient-boosted decision tree as the score predictor to efficaciously partition product reviews’ representations. Extensive experiments demonstrate that our method achieves state-of-the-art results and polished generalization performance on two large-scale MRHP benchmark datasets.

heading: 1 Introduction
text: E-commerce platforms, such as Amazon and Lazada, have achieved steady development. These platforms generally provide purchasers’ reviews to supply justification information for new consumers and help them make decisions. Nevertheless, the quality and usefulness of reviews can vary hugely: some are helpful with coherent and informative content while others unhelpful with trivial or irrelevant information. Due to this, the Multimodal Review Helpfulness Prediction (MRHP) task is proposed. It ranks the reviews by predicting their helpfulness scores based on the textual and visual
∗Corresponding Author
modality of products and reviews, because helpful reviews should comprise not only precise and informative textual material, but also consistent images with text content (Liu et al., 2021; Nguyen et al., 2022). This can help consumers find helpful reviews instead of unhelpful ones, resulting in more appealing E-commerce platforms.
In MRHP, multimodal reviews naturally form ranking partitions based on user votings, where each partition exhibits distinct helpfulness feature level (Ma et al., 2021). As such, the MRHP score regressor’s function is to assign scores to indicate the partition for hidden features of product reviews. However, current MRHP approaches employ fullyconnected neural networks (FCNNs), which cannot fulfill the partition objective. In particular, FCNNs are ineffective in feature scaling and transformation, thus being inadept at feature space splitting and failing to work efficiently in ranking problems that involve ranking partitions (Beutel et al., 2018; Qin et al., 2021). An illustration would be in Figure 1, where the helpfulness scores predicted by FCNNs do not lucidly separate helpful and unhelpful reviews. Severely, some unhelpful reviews possess logits that can even stay in the range of helpful ones, bringing about fallacious ranking.
In addition to incompetent model architectures, existing MRHP frameworks also employ suboptimal loss function: they are mostly trained on a pairwise loss to learn review preferences, which unfortunately mismatches the listwise nature of review ordering prediction. Firstly, the mistmatch might empirically give rise to inefficient ranking performance (Pasumarthi et al., 2019; Pobrotyn and Białobrzeski, 2021). Second, pairwise traning loss considers all pairs of review as equivalent. In consequence, the loss cannot differentiate a pair of useful and not useful reviews from a pair of moderately useful and not useful ones, which results in a model that distinguishes poorly between useful and moderately useful reviews.
1670
To address these issues, we first propose a Gradient-Boosted Decision Tree (GBDT) as the helpfulness score regressor to utilize both its huge capacity of partitioning feature space (Leboeuf et al., 2020) and differentiability compared with standard decision trees for end-to-end training. We achieve the partition capability with the split (internal) nodes of the tree implemented with non-linear single perceptron, to route review features to the specific subspace in a soft manner.
Furthermore, we develop a theoretical analysis to demonstrate that pairwise training indeed has lower model generalization than listwise approach. We proceed to propose a novel listwise training objective for the proposed MRHP architecture. We also equip our architecture with a listwise attention network that models the interaction among the reviews to capture the listwise context for the MRHP ranking task.
In sum, our contributions are four-fold:
• We propose a novel gradient-boosted decision tree score predictor for multimodal review helpfulness prediction (MRHP) to partition product review features and properly infer helpfulness score distribution.
• We propose a novel listwise attention module for the MRHP architecture that conforms to the listwise context of the MRHP task by relating reviews in the list.
• We perform theoretical study with the motivation of ameliorating the model generalization error, and accordingly propose a novel MRHP training objective which satisfies our aim.
• We conducted comprehensive experiments on two benchmark datasets and found that our approach significantly outperforms both textonly and multimodal baselines, and accomplishes state-of-the-art results for MRHP.

heading: 2 Background
text: In this section, we recall the Multimodal Review Helpfulness Prediction (MRHP) problem. Then, we introduce theoretical preliminaries which form the basis of our formal analysis of the ranking losses for the MRHP problem in the next section.

heading: 2.1 Problem Definition
text: Following (Liu et al., 2021; Han et al., 2022; Nguyen et al., 2022), we formulate MRHP as a ranking task. In detail, we consider an instance Xi to consist of a product item pi, composed of product description T pi and images Ipi , and its respective review list Ri = {ri,1, ri,2, . . . , ri,|Ri|}. Each review ri,j carries user-generated text T ri,j , images Iri,j , and an integer scalar label yi,j ∈ {0, 1, . . . , S} denoting the helpfulness score of review ri,j . The ground-truth result associated with Xi is the descending order determined by the helpfulness score list Yi = {yi,1, yi,2, . . . , yi,|Ri|}. The MRHP task is to generate helpfulness scores which match the groundtruth ranking order, formulated as follows:
si,j = f(pi, ri,j), (1)
where f represents the helpfulness prediction model taking ⟨pi, ri,j⟩ as the input.

heading: 2.2 Analysis of Generalization Error
text: The analysis involves the problem of learning a deep θ-parameterized model fθ : X → Y that maps the input space X to output space Y and a stochastic learning algorithm A to solve the optimization problem as follows:
fθ ∗ = argmin
fθ E(x,y)∼P
[ l(fθ; (x,y)) ] , (2)
where P denotes the distribution of (x,y), l the loss function on the basis of the difference between ŷ = fθ(x) and y, and Rtrue(fθ) = E(x,y)∼P [ l(fθ; (x,y)) ] is dubbed as the true risk.
Since P is unknown, Rtrue is alternatively solved through optimizing a surrogate empirical risk Remp(f θ D) = 1 N ∑N i=1 l(f
θ; (xi,yi)), where D = {(xi,yi)}Ni=1 denotes a training dataset drawn from P that fθD is trained upon.
Because the aim of deep neural model training is to produce a model fθ that provides a small gap between the performance over D, i.e. Remp(fθD), and over any unseen test set from P, i.e. Rtrue(fθD), the analysis defines the main focus to be the generalization error E(fθD) = Rtrue(f θ D)−Remp(fθD), the objective to be achieving a tight bound of E(fθD), and subsequently the foundation regarding the loss function’s Lipschitzness as:
Definition 1. (Lipschitzness). A loss function l(ŷ,y) is γ-Lipschitz with respect to ŷ if for γ ≥ 0,∀u,v ∈ RK , we have:
|l(u,y)− l(v,y)| ≤ γ|u− v|, (3)
where | · | denotes the l1-norm, K the dimension of the output ŷ.
Given the foundation, we have the connection between the properties of loss functions and the generalization error:
Theorem 1. Consider a loss function that 0 ≤ l(ŷ,y) ≤ L that is convex and γ-Lipschitz with respect to ŷ. Suppose the stochastic learning algorithm A is executed for T iterations, with an annealing rate λt to solve problem (2). Then, the following generalization error bound holds with probability at least 1− δ (Akbari et al., 2021):
E(fθD) = Rtrue(f θ D)−Remp(fθD) ≤ L
√ log(2/δ)
2N +
2γ2 T∑
t=1
λt
( 2 √ log(2/δ)
T +
√ 2 log(2/δ)
N +
1
N
) .
(4)
Theorem (1) implies that by establishing a loss function L with smaller values of γ and L, we can burnish the model generalization performance.

heading: 3 Methodology
text: In this section, we elaborate on our proposed architecture, listwise attention network, tree-based helpfulness regressor, and listwise ranking loss along with its comparison against the pairwise one from the theoretical perspective. The overall architecture is illustrated in Figure 2.

heading: 3.1 Multimodal Encoding
text: Our model receives product description T pi , product images Ipi , review text T ri,j , and review images Iri,j as input. We perform the encoding procedure for those inputs as follows. Textual Encoding. For both product text T pi and review text T ri,j , we index their sequences of words into the word embeddings and forward to the respective LSTM layer to yield token-wise representations:
Hpi = LSTMp(Wemb(T pi)), (5) Hri,j = LSTMr(Wemb(T ri,j )), (6)
where Hpi ∈ Rlpi×d,Hri,j ∈ Rlri,j×d, lpi and lri,j denote sequence lengths of the product and review text, respectively, d the hidden dimension. Visual Encoding. We adapt a pre-trained Faster RCNN to extract ROI features of m objects {epit }mt=1 and {eri,jt }mt=1 for product and review images, respectively. We then feed those object features into the self-attention module to obtain visual representations as:
Vpi = SelfAttn({epit }mt=1), (7) Vri,j = SelfAttn({eri,jt }mt=1), (8)
where Vpi ,Vri,j ∈ Rm×d, and d denotes the hidden size.

heading: 3.2 Coherence Reasoning
text: We then learn intra-modal, inter-modal, and intraentity coherence among product-review elements. Intra-modal Coherence. There are two types of intra-modal coherence relations: (1) product text - review text and (2) product image - review image. Initially, we designate self-attention modules to capture the intra-modal interaction as:
HintraMi,j = SelfAttn([H pi ,Hri,j ]), (9) VintraMi,j = SelfAttn([V pi ,Vri,j ]). (10)
Then, intra-modal interaction features are passed to a CNN, then condensed into hidden vectors via pooling layer:
zintraMi,j = Pool(CNN([H intraM i,j ,V intraM i,j ])), (11)
where [·] denotes the concatenation operator. Inter-modal Coherence. The inter-modal coherence comprises two relation types: (1) product text (pt) - review image (ri) and (2) product image (pi) -
review text (rt). Similar to the intra-modal coherence, we first perform cross-modal correlation by leveraging the self-attention mechanism:
H pt-ri i,j = SelfAttn([H pi ,Vri,j ]), (12) H pi-rt i,j = SelfAttn([V pi ,Hri,j ]). (13)
Thereafter, we pool the above features and concatenate the pooled vectors to attain the inter-modal vector:
z pt-ri i,j = Pool(H pt-ri i,j ), (14) z pi-rt i,j = Pool(H pi-rt i,j ), (15)
zinterMi,j = [ z pt-ri i,j , z pi-rt i,j ] . (16)
Intra-entity Coherence. Analogous to the intermodal coherence, we also conduct self-attention and pooling computation, but on the (1) product text (pt) - product image (pi) and (2) review text (rt) - review image (ri) as follows:
H pt-pi i = SelfAttn([H pi ,Vpi ]), (17) Hrt-rii,j = SelfAttn([H ri,j ,Vri,j ]), (18)
z pt-pi i = Pool(H pt-pi i ), (19)
zrt-rii,j = Pool(H rt-ri i,j ), (20)
zintraRi,j = [ z pt-pi i , z rt-ri i,j ] . (21)
Eventually, the concatenation of the intra-modal, inter-modal, and intra-entity vectors becomes the result of the coherence reasoning phase:
zi,j = [ zintraMi,j , z interM i,j , z intraR i,j ] . (22)

heading: 3.3 Listwise Attention Network
text: In our proposed listwise attention network, we encode list-contextualized representations to consider relative relationship among reviews. We
achieve this by utilizing self-attention mechanism to relate list-independent product reviews’ features {zi,1, zi,2, . . . , zi,|Ri|} as follows:
{zlisti,j}|Ri|j=1 = SelfAttn({zi,j} |Ri| j=1), (23)
where Ri denotes the review list associated with product pi.

heading: 3.4 Gradient-boosted Decision Tree for Helpfulness Estimation
text: In this section, we delineate our gradient-boosted decision tree to predict helpfulness scores that efficaciously partition review features. Tree Structure. We construct a dtree-depth binary decision tree composed of internal nodes N (|N | = 2dtree−1 − 1) and leaf nodes L (|L| = 2dtree−1). Our overall tree structure is depicted in Figure 2. Score Prediction. Receiving the list-attended vectors {zlisti }Ni=1, our decision tree performs soft partitioning through probabilistic routing for those vectors to their target leaf nodes. In such manner, each internal node n calculates the routing decision probability as:
pleftn = σ(Linear(z list)), (24)
prightn = 1− pleftn , (25)
where pleftn and p right n denote the likelihood of directing the vector to the left sub-tree and right sub-tree, respectively. Thereupon, the probability of reaching leaf node l is formulated as follows:
µl = ∏
n∈P(l) (pleftn ) 1 ln · (prightn )1 rn , (26)
where 1ln denotes the indicator function of whether leaf node l belongs to the left sub-tree of the
internal node n, equivalently for 1rn , and P(l) the node sequence path to leaf l. For example, in Figure 2, the routing probability to leaf 6 is µ6 = p right 1 p left 3 p right 6 .
For the score inference at leaf node l, we employ a linear layer for calculation as follows:
sl,i,j = Linearl(zlisti,j ). (27)
where sl,i,j denotes the helpfulness score generated at leaf node l. Lastly, due to the probabilistic routing approach, the final helpfulness score fi,j is the average of the leaf node scores weighted by the probabilities of reaching the leaves:
fi,j = f(pi, ri,j) = ∑
l∈L sl,i,j · µl . (28)

heading: 3.5 Listwise Ranking Objective
text: Since MRHP task aims to produce helpfulness order for a list of reviews, we propose to follow a listwise approach to compare the predicted helpfulness scores with the groundtruth.
Initially, we convert two lists of prediction scores {fi,j}|Ri|j=1 and groundtruth labels {yi,j} |Ri| j=1 into two probability distributions.
f ′i,j = exp(fi,j)
|Ri|∑ t=1 exp(fi,t)
, y′i,j = exp(yi,j)
|Ri|∑ t=1 exp(yi,t)
. (29)
Subsequently, we conduct theoretical derivation and arrive in interesting properties of the listwise computation. Theoretical Derivation. Our derivation demonstrates that discrimination computation of both listwise and pairwise functions (Liu et al., 2021; Han et al., 2022; Nguyen et al., 2022) satisfy the preconditions in Theorem (1). Lemma 1. Given listwise discrimination function on the total training set as Llist = − |P |∑ i=1 |Ri|∑ j=1 y′i,j log(f ′ i,j), where P denotes the product set, then Llist is convex and γlist-Lipschitz with respect to f ′i,j . Lemma 2. Given pairwise discrimination function on the total training set as Lpair = |P |∑ i=1 [ −fi,r+ + fi,r− + α
]+, where r+, r− denote two random indices in Ri and yi,r+ > yi,r− , and α = max
1≤j≤|Ri| (yi,j) − min 1≤j≤|Ri| (yi,j), then
Lpair is convex and γpair-Lipschitz with respect to fi,r+ , fi,r− .
Based upon the above theoretical basis, we investigate the connection between Llist and Lpair. Theorem 2. Let Llist and Lpair are γlist-Lipschitz and γpair-Lipschitz, respectively. Then, the following inequality holds:
γlist ≤ γpair. (30) Theorem 3. Let 0 ≤ Llist ≤ Llist and 0 ≤ Lpair ≤ Lpair. Then, the following inequality holds:
Llist ≤ Lpair. (31) We combine Theorem (1), (2), and (3), to achieve the following result.
Theorem 4. Consider two models f listD and f
pair D under common settings trained to minimize Llist and Lpair, respectively, on dataset D = {pi, {ri,j}|Ri|j=1} |P | i=1. Then, we have the following inequality:
E(f listD ) ≤ E(fpairD ), (32)
where E(fD) = Rtrue(fD)−Remp(fD). As in Theorem (4), models optimized by listwise function achieve a tighter bound on the generalization error than the ones with the pairwise function, thus upholding better generalization performance. We provide proofs of all the lemmas and theorems in Appendix A. Indeed, empirical results in Section 4.6 also verify our theorems.
With such foundation, we propose to utilize listwise discrimination as the objective loss function to train our MRHP model:
Llist = − |P |∑
i=1
|Ri|∑
j=1
y′i,j log(f ′ i,j). (33)

heading: 4 Experiments

heading: 4.1 Datasets
text: For evaluation, we conduct experiments on two large-scale MRHP benchmark datasets: LazadaMRHP and Amazon-MRHP. We present the dataset statistics in Appendix B. Amazon-MRHP (Liu et al., 2021) includes crawled product and review content from Amazon.com, the international e-commerce brand, between 2016 and 2018. All of the product and review texts are expressed in English. Lazada-MRHP (Liu et al., 2021) comprises product information and user-generated reviews from Lazada.com, a popular e-commerce platform in
Southeast Asia. Both product and review texts are written in Indonesian.
Both datasets are composed of 3 categories: (1) Clothing, Shoes & Jewelry (Clothing), (2) Electronics (Electronics), and (3) Home & Kitchen (Home). We divide the helpfulness votes of the reviews into 5 partitions, i.e. [1, 2), [2, 4), [4, 8), [8, 16), and [16,∞), corresponding to 5 helpfulness scores, i.e. yi,j ∈ {0, 1, 2, 3, 4}.

heading: 4.2 Implementation Details
text: For input texts, we leverage pretrained word embeddings with fastText embedding (Bojanowski et al., 2017) and 300-dimensional GloVe word vectors (Pennington et al., 2014) for Lazada-MRHP and Amazon-MRHP datasets, respectively. Each embedded word sequence is passed into an 1-layer LSTM whose hidden dimension is 128. For input images, we extract their ROI features of 2048 dimensions and encode them into 128-dimensional vectors. Our gradient-boosted decision tree score predictor respectively exhibits a depth of 3 and 5 in Lazada-MRHP and Amazon-MRHP datasets, which are determined on the validation performance. We adopt Adam optimizer, whose batch size is 32 and learning rate 1e−3, to train our entire architecture in the end-to-end fashion.

heading: 4.3 Baselines
text: We compare our approach with an encyclopedic list of baselines:
• BiMPM (Wang et al., 2017): a ranking model that uses 2 BiLSTM layers to encode input sentences.
• EG-CNN (Chen et al., 2018): a RHP baseline which leverages character-level representations and domain discriminator to improve cross-domain RHP performance.
• Conv-KNRM (Dai et al., 2018): a CNNbased system which uses kernel pooling on multi-level n-gram encodings to produce ranking scores.
• PRH-Net (Fan et al., 2019): a RHP baseline that receives product metadata and raw review text as input.
• SSE-Cross (Abavisani et al., 2020): a crossmodal attention-based approach to filter nonsalient elements in both visual and textual input components.
• DR-Net (Xu et al., 2020): a combined model of decomposition and relation networks to learn cross-modal association.
• MCR (Liu et al., 2021): an MRHP model that infers helpfulness scores based on crossmodal attention-based encodings.
• SANCL (Han et al., 2022): a baseline which extracts salient multimodal entries via probebased attention and applies contrastive learning to refine cross-modal representations.
• Contrastive-MCR (Nguyen et al., 2022): an MRHP approach utilizing adaptive contrastive strategy to enhance cross-modal representations and performance optimization.

heading: 4.4 Main Results
text: Inspired by previous works (Liu et al., 2021; Han et al., 2022; Nguyen et al., 2022), we report Mean Average Precision (MAP) and Normalized Discounted Cumulative Gain (NDCG@N), where N = 3 and N = 5. We include the performance of baseline models and our approach in Table 1 and 2.
On Amazon dataset, we consistently outperform prior methods of both textual and multimodal settings. Particularly, our architecture improves over Contrastive-MCR on MAP of 15.2 points in Clothing, NDCG@3 of 20.4 points in Electronics, and NDCG@5 of 21.0 points in Home subset. Furthermore, we accomplish a gain in MAP of 2.2 points in Clothing over PRH-Net, NDCG@3 of 16.4 points in Electronics and NDCG@5 of 11.8 points in Home category over Conv-KNRM baseline, where PRH-Net and Conv-KNRM are the best prior text-only baselines.
For Lazada dataset, which is in Indonesian, we outperform Contrastive-MCR with a significant margin of MAP of 10.4 points in Home, NDCG@5 of 11.6 points in Electronics, and NDCG@3 of 12.4 points in Clothing domain. The text-only variant of our model also gains a considerable improvement of 4.7 points of NDCG@5 in Clothing, 5.0 points of MAP in Electronics over PRH-Net, and 1.4 points of NDCG@3 in Home over ConvKNRM model.
These outcomes demonstrate that our method is able to produce more sensible helpfulness scores to polish the review ranking process, not only being efficacious in English but also generalizing to other language as well. Over and above, it is worth pointing out in Lazada-Electronics, the textual setting of our approach even achieves higher helpfulness
prediction capacity than the state-of-the-art multimodal baseline, i.e. the Contrastive-MCR model.

heading: 4.5 Ablation Study
text: To verify the impact of our proposed (1) Gradientboosted decision tree regressor, (2) Listwise ranking loss, and (3) Listwise attention network, we conduct ablation experiments on the Home category of the Amazon and Lazada datasets. GBDT Regressor. In this ablation, we substitute our tree-based score predictor with various FCNNs score regressor. Specifically, we describe each substitution with a sequence of dimensions in its
fully-connected layers, and each hidden layer is furnished with a Tanh activation function.
As shown in Table 3, FCNN-based score regressors considerably hurt the MRHP performance, with a decline of NDCG@3 of 16.7 points, and MAP of 6.9 points in the Amazon and Lazada datasets, respectively. One potential explanation is that without the decision tree predictor, the model lacks the partitioning ability to segregate the features of helpful and non-helpful reviews. Listwise Ranking Loss. As can be observed in Table 3, replacing listwise objective with the pairwise one degrades the MRHP performance substantially, with a drop of NDCG@3 of 11.8 scores in Amazon, and NDCG@5 of 7.3 scores in Lazada dataset. Based on Theorem 4 and Table 4, we postulate that removing listwise training objective impairs model generalization, revealed in the degraded MRHP testing performance. Listwise Attention Network (LAN). We proceed to ablate our proposed listwise attention module and re-execute the model training. Results in Table 3 betray that inserting listwise attention brings about performance upgrade with 16.9 and 9.1 points of MAP in Amazon-MRHP and Lazada-
MRHP, respectively. We can attribute the improvement to the advantage of listwise attention, i.e. supplying the MRHP model with the context among product reviews to assist the model into inferring the reviews’ ranking positions more precisely.

heading: 4.6 Analysis of Generalization Error
text: Figures 3 and 4 illustrate the approximation of the generalization error Ê(fθD) = Rval(f θ D) − Rtrain(f θ D) of the model after every epoch, where Rval and Rtrain indicate the average loss values of the trained model fθD on the validation and training sets, respectively. Procedurally, due to different scale of the loss values, we normalize them to the range [0, 1]. The plots demonstrate that generalization errors of our MRHP model trained with the listwise ranking loss are constantly lower than those obtained by pairwise loss training, thus exhibiting better generalization performance. Additionally, as further shown in Table 4, fθ, listD incurs a smaller training-testing performance discrepancy △MAP = |MAPtraining − MAPtesting| than fθ, pairD , along with Figures 3 and 4 empirically substantiating our Theorem (4).

heading: 4.7 Case Study
text: In Figure 1, we present helpfulness prediction results predicted by our proposed MRHP model and Contrastive-MCR (Nguyen et al., 2022), the previous best baseline. While our model is capable of producing helpfulness scores that evidently sepa-
rate helpful with unhelpful product reviews, scores generated by Contrastive-MCR do mingle them. Hypothetically, our method could partition product reviews according to their encoded helpfulness features to obtain inherent separation. We provide more detailed analysis of the partitioning capability of our model and individual produced helpfulness scores in Appendix D and E.

heading: 5 Related Work
text: For real-world applications, existing methods are oriented towards extracting hidden features from input samples (Kim et al., 2006; Krishnamoorthy, 2015; Liu et al., 2017; Chen et al., 2018; Nguyen et al., 2021). Modern approaches have gradually taken into account additional and useful modalities, for instance meta-data (Tuan et al., 2016; Fan et al., 2019; Qu et al., 2020), images (Liu et al., 2021; Han et al., 2022), etc. They also depend on hand-crafted features, such as argument-based (Liu et al., 2017), lexical (Krishnamoorthy, 2015; Luu et al., 2015), and semantic features (Yang et al., 2015; Luu et al., 2016; Nguyen and Luu, 2022) to utilize automatic deep representation learning to train the helpfulness predictor. Some also utilize unsupervised learning techniques to polish the learned representations of input samples (Wu et al., 2020, 2023a; Nguyen and Luu, 2021; Wu et al., 2022, 2023b).
Despite performance upgrade, deep neural approaches for multimodal RHP (MRHP) problem, have been shown to still be inadept at modeling partitioned and ranking data (Qin et al., 2021), which is the crucial characteristic of MRHP reviews (Ma et al., 2021). In this work, we seek to address those issues for the MRHP system with our proposed tree-based helpfulness predictor and listwise architectural framework.

heading: 6 Conclusion
text: In this paper, for the MRHP task, we introduce a novel framework to take advantage of the partitioned structure of product review inputs and the ranking nature of the problem. Regarding the partitioned preference, we propose a gradientboosted decision tree to route review features towards proper helpfulness subtrees managed by decision nodes. For the ranking nature, we propose listwise attention network and listwise training objective to capture review list-contextualized context. Comprehensive analysis provides both theoretical and empirical grounding of our approach in terms of model generalization. Experiments on two largescale MRHP datasets showcase the state-of-the-art performance of our proposed framework.
Limitations
Firstly, from the technical perspective, we have advocated the advantages of our proposed listwise loss for the MRHP task in terms of generalization capacity. Nevertheless, there are other various listwise discrimination functions that may prove beneficial for the MRHP model training, for example NeuralNDCG (Pobrotyn and Białobrzeski, 2021), ListMLE (Xia et al., 2008), etc. Moreover, despite the novelty of our proposed gradient-boosted tree in partitioning product reviews into helpful and unhelpful groups, our method does not employ prior contrastive representation learning, whose objective is also to segregate helpful and unhelpful input reviews. The contrastive technique might discriminate reviews of distinctive helpfulness features to bring further performance gain to multimodal review helpfulness prediction. At the moment, we leave the exploration of different listwise discrimination functions and contrastive learning as our prospective future research direction.
Secondly, our study can be extended to other problems which involve ranking operations. For instance, in recommendation, there is a need to rank the items according to their appropriateness to present to the customers in a rational order. Our gradient-boosted decision tree could divide items into corresponding partitions in order for us to recommend products to the customer from the highly appropriate partition to the less appropriate one. Therefore, we will discover the applicability of our proposed architecture in such promising problem domain in our future work.

heading: Acknowledgements
text: This work was supported by Alibaba Innovative Research (AIR) programme with research grant AN-GC-2021-005.

heading: A Proofs
text: Lemma 1. Given listwise loss on the total training set as Llist = − |P |∑ i=1 |Ri|∑ j=1 y′i,j log(f ′ i,j), where P denotes the product set, then Llist is convex and γlist-Lipschitz with respect to f ′i,j . Proof. Taking the second derivative of Equation (33), we have
∇2f ′i,jL list =
|P |∑
i=1
|Ri|∑
j=1
y′i,j (f ′i,j) 2 > 0, (34)
proving the convexity of Llist. The Lipschitz property of Llist can be derived from such property of the logarithm function, which states that
| log(u)− log(v)| = ∣∣∣log(1 + u
v − 1) ∣∣∣ ≤ ∣∣∣u v − 1 ∣∣∣ = ∣∣∣∣ 1 v (u− v) ∣∣∣∣ ≤ γ|u− v|, (35)
where the first inequality stems from log(1 + x) ≤ x ∀x > −1 and γ is chosen s.t. |v| ≥ 1γ . Let x = ui,jyi,j , z = vi,j yi,j . Applying the above result for Llist, we obtain
|log(ui,j)− log(vi,j)| = ∣∣∣∣log ( ui,j yi,j ) − log ( vi,j yi,j )∣∣∣∣ ≤ γ ∣∣∣∣ ui,j yi,j − vi,j yi,j ∣∣∣∣ , (36)
Multiplying both sides by yi,j , and integrating the summation on all inequalities for i ∈ {1, 2, . . . , |P |} and j ∈ {1, 2, . . . , |Ri|}, we achieve
|P |∑
i=1
|Ri|∑
j=1
|yi,j log(ui,j)− yi,j log(vi,j)| ≤ γ |P |∑
i=1
|Ri|∑
j=1
|ui,j − vi,j | . (37)
Utimately, we obtain: |Llist(u,y)− Llist(v,y)| ≤ γlist|u− v|, (38)
Where γlist = γ. This proves the γlist-Lipschitz property of Llist. Lemma 2. Given pairwise loss on the total training set as Lpair = |P |∑ i=1 [ −fi,r+ + fi,r− + α
]+, where r+, r− denote two random indices in Ri and yi,r+ > yi,r− , and α = max
1≤j≤|Ri| (yi,j)− min 1≤j≤|Ri| (yi,j), then
Lpair is convex and γpair-Lipschitz with respect to fi,r+ , fi,r− . Proof. Let hpairi (⟨fi,r+ , fi,r−⟩),yi) = [−fi,r+ + fi,r− + α]+, ui = ⟨fi,u+ , fi,u−⟩, vi = ⟨fi,rv+ , fi,rv− ⟩ be two inputs of hpairi . For θ ∈ [0, 1], we have
h pair i (θui + (1− θ)vi,yi) = h pair i (θ⟨fi,u+ , fi,u−⟩+ (1− θ)⟨fi,v+ , fi,v−⟩,yi)
= h pair i (⟨θfi,u+ + (1− θ)fi,v+ , θfi,u− + (1− θ)fi,v−⟩,yi) = [ −(θfi,u+ + (1− θ)fi,v+) + (θfi,u− + (1− θ)fi,v−) + α ]+ = [ θ(−fi,u+ + fi,u− + α) + (1− θ)(−fi,v+ + fi,v− + α) ]+ ≤ θ[−fi,u+ + fi,u− + α]+ + (1− θ)[−fi,v+ + fi,v− + α]+ = θh pair i (ui,yi) + (1− θ)h pair i (v,yi).
(39)
Employing summation of the inequality on all i ∈ {1, 2, . . . , |P |}, we have
Lpair(θu+(1−θ)v,y) ≤ θ |P |∑
i=1
h pair i (ui,yi)+(1−θ)
|P |∑
i=1
h pair i (vi,yi) = θLpair(u,y)+(1−θ)Lpair(v,y),
(40)
which proves the convexity of Lpair. Regarding the Lipschitz property, we first show that hpairi holds the property:
|hpairi (ui,yi)−h pair i (vi,yi)| = [ (−u+i + u−i + α)− (−v+i + v−i + α) ]+ = [ −u+i + u−i − v−i + u−i ]+ . (41) Note that ymin ≤ u+i , u−i , v+i , v−i ≤ ymax, since we take the non-negative values in (41). Thus,
|hpairi (ui,yi)− h pair i (vi,yi)| ≤ 2(ymax − ymin). (42)
Similarly, applying the aforementioned observation, we have:
|ui − vi| = ∣∣u+i − v+i ∣∣+ ∣∣u−i − v−i ∣∣ ≥ 2(ymax − ymin). (43)
Combining (42) and (43) leads to:
|hpairi (ui,yi)− h pair i (vi,yi)| ≤ γpair|ui − vi|, (44)
such that γpair ≥ 1. Adopting the summation of (44) on all i ∈ {1, 2, . . . , |P |}, we obtain:
|Lpair(u,y)− Lpair(v,y)| = ∣∣∣∣∣∣ |P |∑
i=1
h pair i (ui,yi)−
|P |∑
i=1
h pair i (vi,yi) ∣∣∣∣∣∣ ≤ γpair |P |∑
i=1
|ui − vi| = γpair|u− v|.
(45) The Lipschitz property of Lpair follows result (45). Theorem 2. Let Llist and Lpair are γlist-Lipschitz and γpair-Lipschitz, respectively. Then, the following inequality holds:
γlist ≤ γpair. (46)
Proof. In order to prove Theorem (2), we first need to find the formulation of γlist and γpair. We leverage the following lemma:
Lemma 3. A function L is γ-Lipschitz, if γ satisfies the following condition (Akbari et al., 2021):
γ = sup fi,j
∣∣L′i,j(fi,j) ∣∣ . (47)
With the foundation in mind, we take the derivative of Llisti,j and L pair i,j :
(Llisti,j (fi,j))′ =  y ′ i,j log |Ri|∑ t=1 exp(fi,t)
exp(fi,j)
  ′ = y′i,j  
exp(fi,j) |Ri|∑ t=1 exp(fi,t) − 1
  = −y ′ i,j   |Ri|∑ k=1,k ̸=j exp(fi,k)
|Ri|∑ t=1 exp(fi,t)
  ,
(48)
(Lpairi,j (fi,j))′ = ±1. (49)
(48) and (49) imply that ∣∣∣ [ Llisti,j (fi,j) ]′∣∣∣ ≤ y′i,j ≤ 1 = ∣∣∣∣ [ Lpairi,j (fi,j) ]′∣∣∣∣ . (50)
Combining equation (50) and Lemma (3), we obtain γlist ≤ γpair. ■ Theorem 3. Let 0 ≤ Llist ≤ Llist and 0 ≤ Lpair ≤ Lpair. Then, the following inequality holds:
Llist ≤ Lpair. (51)
Proof. Adoption of Jensen’s inequality on Llist gives:
Llist = − |P |∑
i=1
|Ri|∑
j=1
y′i,j log f ′ i,j (52)
=
|P |∑
i=1
|Ri|∑
j=1
y′i,j log
|Ri|∑ t=1 exp(fi,t)
exp(fi,j) (53)
=
|P |∑
i=1
|Ri|∑
j=1
y′i,j
 log |Ri|∑
t=1
exp fi,t − fi,j   (54)
=
|P |∑
i=1
|Ri|∑
j=1
y′i,j
 log   1 |Ri| |Ri|∑
t=1
exp(fi,t)  − fi,j + log |Ri|   (55)
≤ |P |∑
i=1
|Ri|∑
j=1
y′i,j   1 |Ri| |Ri|∑
t=1
fi,t − fi,j + log |Ri|   (56)
≤ |P |∑
i=1
|Ri|∑
j=1
y′i,j(f max − fmin + log |Ri|) (57)
= |P |(fmax − fmin) + |P | log |Ri|, (58)
where fmin ≤ fi,j ≤ fmax, ∀i, j. Now, such bounds of fi,j on Lpair yields:
Lpair = |P |∑
i=1
[ −fi,r+ + fi,r− + α ]+ ≤ |P |(fmax − fmin) + |P |(ymax − ymin), (59)
where ymax = max 1≤i≤|P | max 1≤j≤|Ri| (yi,j), ymin = min 1≤i≤|P | min 1≤j≤|Ri| (yi,j). Note that Table 5 reveals that max |Ri| ≤ 2043. Therefore, log |Ri| ≤ 3.31, whereas ymax − ymin = 4, giving rise to the conclusion log |Ri| ≤ ymax − ymin. Therefore, Llist ≤ Lpair, (60) which concludes the proof of Theorem (3). Theorem 4. Consider two models f listD and f pair D learned under common settings utilizing listwise and pairwise ranking losses, respectively, on dataset D = {pi, {ri,j}|Ri|j=1} |P | i=1. Then, we have the following inequality: E(f listD ) ≤ E(fpairD ). (61)
where E(fD) = Rtrue(fD)−Remp(fD). The inequality immediately follows from Theorems (1), (2) and (3). From Theorems (1) and (2), because T and N are constant, the second term of Llist is always smaller than that of Lpair. From Theorems (1) and (3), we realize that Llist ≤ Lpair, thus proving the smaller value of the first term of Llist.

heading: B Dataset Statistics
text: In this section, we provide dataset statistics of the Amazon and Lazada datasets on the MRHP task. All of the numerical details are included in Table 5.

heading: C Generalization Errors of the Models trained with Listwise and Pairwise Ranking Losses
text: In this Appendix, we illustrate the empirical evolution of generalization errors of pairwise-trained and listwise-trained models on the remaining categories of the Amazon-MRHP and Lazada-MRHP datasets. The discovered characteristics regarding generalization in Figures 5 and 6 agree with those in Section 4.6, corroborating the intensified generalizability of our proposed listwise ranking loss.

heading: D Analysis of Partitioning Function of Gradient-Boosted Decision Tree
text: We examine the partitioning operation of our proposed gradient-boosted decision tree for the multimodal review helpfulness prediction. In particular, we inspect the µ = [µ1, µ2, . . . , µ|L|] probabilities, which route review features to the target leaf nodes in a soft approach. Our procedure is to gather µ at the leaf nodes for all reviews, estimate their mean value with respect to each leaf, then plot the results on Clothing and Home of the Amazon and Lazada datasets, respectively, in Figures 7, 8, 9, 10, and 11.
From the figures, we can observe our proposed gradient-boosted decision tree’s behavior of assigning high routing probabilities {µi}|L|i=1 to different partitions of leaf nodes, with the partitions varying according to the helpfulness scale of the product reviews. In consequence, we can claim that our GBDT divides the product reviews into corresponding partitions to their helpfulness degrees, thus advocating the partitioned preference of the input reviews.
Mean µi values on 2-rating reviews of Lazada-Clothing dataset
Mean µi values on 3-rating reviews of Lazada-Clothing dataset

heading: E Examples of Product and Review Samples
text: We articulate product and review samples in Figure 1, comprising their textual and visual content, with the helpfulness scores generated by Contrastive-MCR (Nguyen et al., 2022), whose score predictor is FCNN-based, and our GBDT-based model.
E.1 Product B00005MG3K Libbey Imperial 16-Piece Tumbler and Rocks Glass Set
E.2 Product B00Q82T3XE Dasein Frame Tote Top Handle Handbags Designer Satchel Leather Briefcase Shoulder Bags Purses
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Section 7
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.
3 A3. Do the abstract and introduction summarize the paper’s main claims? Abstract and Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4
3 B1. Did you cite the creators of artifacts you used? Section 4
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Appendix B
C 3 Did you run computational experiments? Section 4
C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Not applicable. Left blank.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Section 4
C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Not applicable. Left blank.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Not applicable. Left blank.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: At the moment, we leave the exploration of different listwise discrimination functions and contrastive learning as our prospective future research direction. Secondly, our study can be extended to other problems which involve ranking operations. For instance, in recommendation, there is a need to rank the items according to their appropriateness to present to the customers in a rational order. Our gradient-boosted decision tree could divide items into corresponding partitions in order for us to recommend products to the customer from the highly appropriate partition to the less appropriate one. Therefore, we will discover the applicability of our proposed architecture in such promising problem domain in our future work