Venue: NAACL 2024

# Personalized Review Recommendation based on Implicit dimension mining

Bei Xu1,2,+ Yifan Xu1,++\({}^{*}\)

\({}^{1}\)School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing 210023, China

\({}^{2}\)Jiangsu Key Laboratory of Big Data Security & Intelligent Processing, Nanjing 210023, China xubei@njupt.edu.cn xyf-0209@foxmail.com

###### Abstract

Users usually browse product reviews before buying products from e-commerce websites. Lots of e-commerce websites can recommend reviews. However, existing research on review recommendation mainly focuses on the general usefulness of reviews and ignores personalized and implicit requirements. To address the issue, we propose a Large language model driven Personalized Review Recommendation model based on Implicit dimension mining (_PRR-L_). The model mines implicit dimensions from reviews and requirements, and encodes them in the form of "text + dimension". The experiments show that our model significantly outperforms other state-of-the-art textual models on the Amazon-MRHP dataset, with some of the metrics outperforming the state-of-the-art multimodal models. And we prove that encoding _"text + dimension"_ is better than encoding _"text"_ and _"dimension"_ separately in review recommendation.

## 1 Introduction

Online product reviews are referential because they reflect the experience of past users. Some studies [54] have shown the impact of online reviews on new users' purchase intention. Therefore, recommending useful reviews is helpful for users as well as e-commerce websites.

Current review recommendation techniques focus on review helpfulness prediction, in which a key step is to extract features from reviews and user requirements. Most features are extracted from the textual content [67], which mainly includes: lexical, textual, readability, and others [81, 19, 20]. Other features include non-textual content [18, 17], product-related factors [16, 17], and reviewer-related factors [15, 14, 18]. Previous review recommendation methods take the product attributes or user preferences that directly appear in reviews as features [16], such as appearance, size, price, or components of products. However, some implicit features are ignored. For example, in the review of a computer: _"My game runs very smoothly"_, _"performance"_ is implicit because _"performance"_ does not appear in the review. And a requirement _"I want to buy a computer to run my 3D game"_ also implicitly indicate a request for performance.

Semantic enhancement is an approach to enhance semantic information of data. Related studies mainly use knowledge graphs or external knowledge to extend input or enrich knowledge facts [13, 12, 14]. But current semantic enhancement methods are hard to enhance reviews because reviews are often unprofessional and casual. They are also hard to mine the implicit features from requirements because of the lack of context.

We propose a Large language model driven Personalized Review Recommendation based on Implicit dimension mining (_PRR-Li_). The model only uses textual content of reviews and

requirements. The implicit dimensions of reviews and requirements are mined by using a large language model (_LLM_). We design prompts to guide the _LLM_ to rewrite review text while keeping the original meaning, and then mine the implicit dimensions in reviews. At the same time, implicit dimensions are also mined from requirements. Finally, _PRR-LI_ encodes enhanced reviews and requirements together by combined encoding. The experiments show that our model significantly outperforms other state-of-the-art text-only models, and some of the metrics exceed nearly 10% or are close to the performance of state-of-the-art multimodal models.

## 2 Review Dimension

We define review dimension as any entity or attribute expressed by a review that can reflect an explicit or implicit requirement. We classify the dimensions as explicit or implicit depending on whether the dimensions are directly mentioned in the review. Let \(R\) represent a review, the dimension \(D\) of \(R\) is denoted as {\(d_{i}\),\(d_{2}\),...,\(d_{n}\)}. If \(R\) literally contains \(d_{i}\), \(d_{i}\) is an explicit dimension of \(R\). If \(R\) does not literally contain \(d_{i}\), \(d_{i}\) is an implicit dimension of \(R\). For example, "_gift_" is an explicit dimension in the review "_The packaging is perfect for a gift_". In the reviews "_The phone is easy to hold in one hand_" and "_This monitor is too big for my desk_", "_size_" does not appear directly, but is implied in the reviews. So "_size_" is an implicit dimension.

## 3 Model

The framework of _PRR-LI_ is shown in Figure 1. The model takes reviews as input, acquires explicit and implicit entities by _LLM_, then inputs the reviews and the entities into the _LLM_ again to obtain the rewritten reviews, and finally uses the tool He and Choi [5] to tokenize the rewritten reviews and preserve words with parts of speech1\(n\), \(nz\), \(nx\) as review dimensions. The acquired review dimensions include both explicit and implicit dimensions expressed in the original reviews. We use the _API_ version of the basic _LLM_, ChatGLM-Pro, with temperature and top_p set to 0.9 and 0.7 respectively. Then, the requirement and the acquired review dimensions are fed into the _LLM_ to find the dimensions that meet the requirements. The prompts are shown in Table 1.

Footnote 1: [https://hanlp.hankcs.com/docs/annotations/pos/pku.html](https://hanlp.hankcs.com/docs/annotations/pos/pku.html)

We design a text combined encoding module based on _MSE-Base_. _MSE-Base_-_Text_dimension_ is a version of _MSE-base_ after fine-tuning. The data "_review_" and "_review dimension_" are combined and then input into the module to be transformed into enhanced review embedding. The data "_requirement_" and "_requirement dimension_" are combined and input into the module to be

Figure 1: Framework of _PRR-LI_



transformed into enhanced requirement embedding.

Then we use cosine distance to calculate the semantic similarity between enhanced review embedding and enhanced requirement embedding. The model recommends the Top-N reviews in descending order.

## 4 Experiments

### Dataset

We compare our model with others on the benchmark dataset _Amazon-MRHP_(Ni et al., 2019; Liu et al., 2021), which contains 87,492 reviews for clothing, 79,570 reviews for electronics, and 111,193 reviews for home. Under the premise of not violating relevant laws and regulations, as well as the website's robot exclusion protocol, we built a dataset JDDataset from the JingDong website for other experiments. JDDataset is available at [https://www.modelscope.cn/datasets/JerryO/JDData](https://www.modelscope.cn/datasets/JerryO/JDData) asset. It contains 437,646 reviews, of which 90,000 were used for training, 2,000 for validation, and 880 for testing.

### Experimental setups

We use the v2.1 native version of HanLP (He and Choi, 2021). The stop words contain both Chinese and English. The Adam optimizer is chosen for fine-tuning, batch_size is 16, the learning rate is 5e\({}^{*}\), weight_decay is 1e\({}^{*}\), and epoch is 4.

We use the metrics commonly used in the recommendation: (1) Recall@N, denoted as _R@N_; (2) MAP@N, denoted as _M@N_; (3) NDCG@N Jarvelin and Kekalainen, 2017), denoted as _N@N_.

We compare our model with two types of state-of-the-art review recommendation models. One is the models that only use textual content: BiMPN Wang et al. (2017), EG-CNN Chen et al. (2018), Conv-KNRM Dai et al. (2018), and PRHNet Fan et al. (2019). The other is the multimodal models: SSE-Cross Abavisani et al. (2020), D&R Net Xu et al. (2020), and MCR Liu et al. (2021).

_PRR-LI_FT_ is a version of _PRR-LI_ after fine-tuning. The two models are text-only models.

### Results on _Amazon-MRHP_

We conduct comparative experiments on the benchmark dataset _Amazon-MRHP_. The results are shown in Table 2. _PRR-LI_FT_ and _PRR-LI_

\begin{table}
\begin{tabular}{l l c} \hline \hline
**Name** & \multicolumn{2}{c}{**Prompt templates**} \\ \hline Entity & _NER_ Task: You need to perform fine-grained entity recognition on the text of a user’s review of \\ recognize & product. Please perform fine-grained entity recognition on the following reviews:n\{content\} \\ \hline Text rewrite & Text rewriting task, you need to rewrite the text of the user’s review of the \\  & product:n\{entity\}\textbackslash{}nPlease rewrite the following reviews in conjunction with the entity recognition \\  & results, and output the rewritten text without any other explanatory notes:n\{content\} \\ \hline Check & \{_content\}\textbackslash{}nIf there is any direct or indirect reference to \textless{}_{dimension\}\textgreater{}_ in the text above, please \\ dimension & answer \textless{}yes\textgreater{} or \textasci{}no\textgreater{} No further explanation is required. \\ \hline User & I will give you a paragraph of text describing the user’s requirements and a dimension word and ask you to judge whether the user is likely to be interested in this dimension.nPlease make a judgement on the following, if the user is likely to be interested, answer ’yes’, otherwise answer  ‘no’, do not add any other irrelevant explanatory notes.nText:n\{content\}\textbackslash{}nWords:n\{dimension\} \\ \hline \end{tabular}
\end{table}
Table 1: The prompt templates.

\begin{table}
\begin{tabular}{p{11.4pt} p{11.4pt} p{11.4pt} p{11.4pt} p{11.4pt} p{11.4pt} p{11.4pt} p{11.4pt} p{11.4pt} p{11.4pt}} \hline \hline \multirow{2}{*}{**Type**} & \multirow{2}{*}{**Method**} & \multicolumn{4}{c}{**Clothing**} & \multicolumn{4}{c}{**Electronics**} & \multicolumn{2}{c}{**Home**} \\  & & _M@S_ & _N@S_ & _N@S_ & _M@S_ & _N@S_ & _N@S_ & _M@S_ & _N@S_ \\ \hline \multirow{4}{*}{Text-only} & _BiMPN_ & 57.7 & 41.8 & 46.0 & 52.3 & 40.5 & 44.1 & 56.6 & 43.6 & 47.6 \\  & _EG-CNN_ & 56.4 & 40.6 & 44.7 & 51.5 & 39.4 & 42.1 & 55.3 & 42.4 & 46.7 \\  & _Conv-KNRM_ & 57.2 & 41.2 & 45.6 & 52.6 & 40.5 & 44.2 & 57.4 & 44.5 & 48.4 \\  & _PRHNet_ & 58.3 & 42.2 & 46.5 & 52.4 & 40.1 & 43.9 & 57.1 & 44.3 & 48.1 \\ \hline \multirow{4}{*}{Multimodal} & _SSE-Cross_ & 65 & 56 & 59.1 & 53.7 & 43.8 & 47.2 & 60.8 & 51 & 54 \\  & _D\&R Net_ & 65.2 & 56.1 & 59.2 & 53.9 & 44.2 & 47.5 & 61.2 & 51.8 & 54.6 \\  & _MCR_ & 67 & **58.1** & 61.1 & 56 & **56.5** & 49.7 & 63.2 & **54.2** & 57.3 \\ \hline Ours & _PRR-LI_ & 62.7 & 44.4 & 54.2 & 59.6 & 44.1 & 53.1 & **66.6** & 46.3 & **57.9** \\  & _PRR-LI FT_ & **71.1** & 51.5 & **62.1** & **68.8** & 54 & **61.2** & 64.6 & 50.1 & 57.1 \\ \hline \end{tabular}
\end{table}
Table 2: Results on the Amazon-MRHP dataset.

 


* [37] significantly outperform the text-only models. After fine-tuning, _PRR-LI_FT_ continues to improve significantly on most metrics because _PRR-LI_FT_ can encode the type of data "text + dimension" better than _PRR-LI_ And _PRR-LI_ FT is better than the multimodal models on _MAP@5_. The performance of _PRR-LI_ and _PRR-LI_ FT is not as good as the multimodal models in _N@3_ and _N@5_ for home data, while the performance of _PRR-LI_ and _PRR-LI_FT_ is close to the multimodal models for clothing data. One reason is that the images of home and clothing products help reflect the requirements more visually. For electronics data, _PRR-LI_ and _PRR-LI_FT_ outperform the multimodal model by almost 10% in both _MAP@5_ and _N@5_. One reason is that the images of electronic products do not reflect the requirements as much as the images of home and clothing.

### Ablation experiment

Figure 2 shows that adding different parts of _PRR-LI_ can effectively optimize recommendation. The dataset is JDDataset. The performance decreases significantly without _rewrite_, _review dimension_, or _require dimension_. And _rewrite with NER_ is better than _rewrite_.

We further test other _LLM_s' abilities to rewrite with NER as shown in Table 4. "_Rewrite_" and "_NER_rewrite_" respectively means rewrite text without and with _NER_. The values are average _proffer_. _Proffer_ reflects the implicit dimension mining effect, and refers to the proportion of acquired dimensions to the total dimensions as shown in equation 1,

\[{}^{\textit{diff}}=\frac{id}{id+ed} \tag{1}\]

Where _id_ is the number of implicit dimensions and _ed_ is the number of explicit dimensions. We can see that some _LLM_s are not suitable for rewriting with _NER_.

### Experiments on encoding models

We test other encoding models in _PRR-LI_ on JDDataset as shown in Figure 3. "_dimension_" refers to vectorizing the text using the dimensions of the review. _MSE-base_ and _text2vec-bge-large_ series are from [https://huggingface.co](https://huggingface.co). We can see that the _M3E-base-TextDimension_ reaches the best. The results on "_dimension_" show that ignoring the text content weakens the ranking and the recall.

### Experiments on the encoding method

We test separated encoding, which encodes text and dimension separately, and combined encoding,

\begin{table}
\begin{tabular}{l c c} \hline  & _LLM_s & Rewrite & NER\_rewrite \\ \hline ChatGLM2-6B v1.0.12 & 35.5 & **37.1** \\ Qwen-7B-Chat v1.1.5 & **40.7** & 34.7 \\ Baichuan2-7B-Chat v1.0.4 & **39.7** & 31.9 \\ interlm-chat-7b v1.0.1 & **13.3** & 3.5 \\ Llama2-Chinese-7b-Chat-ms v1.0.0 & 20.3 & **23.8** \\ ChatGLM-Pro & 29.2 & **33.6** \\ \hline \end{tabular}
\end{table}
Table 4: Rewrite with NER. The _LLM_s with parameters 6b and 7b are from [https://www.modelscope.cn](https://www.modelscope.cn).

Figure 3: Results on encoding models

Figure 2: Ablation experiment



which encodes text and dimension in the form of "text + dimension". Table 3 shows that the combined encoding achieves better results on both _M3E_ models, and _M3E-base_ can handle the type of "text + dimension" data better after fine-tuning.

## 5 Conclusion

_PRR-LI_ and the fine-tuned version, _PRR-LI__FT_, significantly outperform the text-only review recommendation models, and even outperform the multimodal models in some metrics. This reveals that review text may contain a wealth of implicitly semantic information that has yet to be fully exploited. Furthermore, the results on electronics are better than those on clothing and home products. This suggests that review text can reveal more information about objects that lack intuitive visual information, compared to objects that possess abundant visual representations.

We also demonstrate that, in review recommendation, encoding "_text + dimension_" together is better than encoding "_text_" and "_dimension_" separately. It indicates that "_text + dimension_" may serve as a more effective input for language models compared to plain text.

In conclusion, our model offers a method to extract implicit dimension from review text and integrate them with the text itself. Our model has the potential to be utilized in other applications that involve processing the semantics of short text.

## References

* A. Abavisani, L. Wu, S. Hu, J. Tetreault, and A. Jaimes (2020)Multimodal categorization of crisis events in social media. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 14667-14677, Seattle, WA, USA, pp. 14667-14698. Cited by: SS1.
* M. Allahbakhsh, A. Ignjatovic, H. R. Motabari-Nezhad, and B. Benatalhal (2015)Robust evaluation of products and reviewers in social rating systems.. World Wide Web18 (1), pp. 73-109. External Links: Document, ISSN 1078-1228, Link Cited by: SS1.
* S. Bhatt, A. Sheth, V. Shalin, and J. Zhao (2020)Knowledge graph semantic enhancement of Input Data for Improving AI. IEEE Internet Computing24 (2), pp. 66-72. External Links: Document, ISSN 1078-1228, Link Cited by: SS1.
* C. Chen, Y. Yang, J. Zhou, X. Li, and F. Bao (2018)Cross-domain review helpfulness prediction based on convolutional neural networks with auxiliary domain discriminators. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), New Orleans, Louisiana, pp. 602-607. External Links: ISBN 978-3-319-142-1, Link Cited by: SS1.
* Z. Dai, C. Xiong, J. Callan, and Z. Liu (2018)Convolutional neural networks for soft-matching n-grams in ad-hoc search. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, New York, NY, USA, pp. 126-134. External Links: ISBN 978-3-540-3213-X, Link Cited by: SS1.
* M. Fan, C. Feng, L. Guo, M. Sun, and P. Li (2019)Product-aware helpfulness prediction of online reviews. In The World Wide Web Conference, New York, NY, USA, pp. 2715-2721. External Links: ISBN 978-3-540-3213-X, Link Cited by: SS1.
* A. Ghose and P. G. Ipeirotis (2011)Estimating the helpfulness and economic impact of product reviews: mining text and reviewer characteristics. IEEE Transactions on Knowledge and Data Engineering23 (10), pp. 1498-1512. External Links: Document, ISSN 1078-1228, Link Cited by: SS1.
* H. He and J. D. Choi (2021)The stem cell hypothesis: dilemma behind multi-task learning with transformer encoders. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, pp. 555-557. External Links: Document, ISSN 1078-1228, Link Cited by: SS1.
* H. Hong, D. Xu, G. Alan Wang, and W. Fan (2017)Understanding the determinants of online review helpfulness: a meta-analytic investigation. Decision Support Systems102, pp. 1-11. External Links: Document, ISSN 1078-1228, Link Cited by: SS1.
* N. Hu, N. Sian Koh, and Srinivas K. Reddy (2014)Ratings lead you to the product, reviews help you clinch it? The mediating role of online review sentiments on product sales. Decision Support Systems57, pp. 42-53.

 

* Jarvelin and Kekalainen (2017) Kalervo Jarvelin and Jaana Kekalainen. 2017. IR evaluation methods for retrieving highly relevant documents. _ACM SIGIR Forum_, 51(2):243-250.
* Korfiatis et al. (2012) Nikolaos Korfiatis, Elena Garcia-Bariocanal, and Salvador Sanchez-Alonso. 2012. Evaluating content quality and helpfulness of online product reviews: The interplay of review helpfulness vs. review content. _Electronic Commerce Research and Applications_, 11(3):205-217.
* Krishnamoorthy (2015) Srikumar Krishnamoorthy. 2015. Linguistic features for review helpfulness prediction. _Expert Systems with Applications_, 42(7):3751-3759.
* Lee et al. (2018) Pei-Ju Lee, Ya-Han Hu, and Kuan-Ting Lu. 2018. Assessing the helpfulness of online hotel reviews: A classification-based approach. _Telematics and Informatics_, 35(2):436-445.
* Lee and Choeh (2014) Sangjae Lee and Joon Yeon Choeh. 2014. Predicting the helpfulness of online reviews using multilayer perceptron neural networks. _Expert Systems with Applications_, 41(6):3041-3046.
* Liu et al. (2005) Bing Liu, Minqing Hu, and Junsheng Cheng. 2005. Opinion observer: analyzing and comparing opinions on the Web. In _Proceedings of the 14th International Conference on World Wide Web_, pages 342-351, New York, NY, USA. Association for Computing Machinery.
* Liu et al. (2021) Junhao Liu, Zhen Hai, Min Yang, and Lidong Bing. 2021. Multi-perspective Coherent Reasoning for Helpfulness Prediction of Multimodal Reviews. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 5927-5936, Online. Association for Computational Linguistics.
* Lyu et al. (2023) Ziyu Lyu, Yue Wu, Junjie Lai, Min Yang, Chengming Li, and Wei Zhou. 2023. Knowledge Enhanced Graph Neural Networks for Explainable Recommendation. _IEEE Transactions on Knowledge and Data Engineering_, 35(5):4954-4968.
* Malik and Hussain (2018) Msi Malik and Ayyaz Hussain. 2018. An analysis of review content and reviewer variables that contribute to review helpfulness. _Information Processing & Management_, 54(1):88-104.
* Ni et al. (2019) Jiammo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 188-197, Hong Kong, China. Association for Computational Linguistics.
* Qazi et al. (2016) Aika Qazi, Karim Bux Shah Syed, Ram Gopal Raj, Erik Cambria, Muhammad Tahir, and Daniyal Alghazzawi. 2016. A concept-level approach to the analysis of online review helpfulness. _Computers in Human Behavior_, 58:75-81.
* Saumya et al. (2023) Sunil Saumya, Pradeep Kumar Roy, and Jyoti Prakash Singh. 2023. Review helpfulness prediction on e-commerce websites: A comprehensive survey. _Engineering Applications of Artificial Intelligence_, 126:107075.
* Ventre and Kolbe (2020) Ivan Ventre and Diana Kolbe. 2020. The Impact of Perceived Usefulness of Online Reviews, Trust and Perceived Risk on Online Purchase Intention in Emerging Markets: A Mexican Perspective. _Journal of International Consumer Marketing_, 32(4):287-299.
* Wang et al. (2017) Zhiguo Wang, Wael Hamza, and Radu Florian. 2017. Bilateral Multi-Perspective Matching for Natural Language Sentences. In _Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence_, pages 4144-4150, Melbourne, Australia. International Joint Conferences on Artificial Intelligence Organization.
* Xu et al. (2020) Nan Xu, Zhixiong Zeng, and Wenji Mao. 2020. Reasoning with Multimodal Sarcastic Tweets via Modeling Cross-Modality Contrast and Semantic Association. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreatult, editors, _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 3777-3786, Online. Association for Computational Linguistics.
* Zhang et al. (2019) Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. ERNIE: Enhanced Language Representation with Informative Entities. In Anna Korhonen, David Traum, and Lluis Marquez, editors, _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 1441-1451, Florence, Italy. Association for Computational Linguistics.