Distantly Supervised Aspect Clustering And Naming For E -Commerce Reviews (2022)

Abstract: Product aspect extraction from reviews is a critical task for e-commerce services to understand customer preferences and pain points. While aspect phrases extraction and sentiment analysis have received a lot of attention, clustering of aspect phrases and assigning human readable names to clusters in e-commerce reviews is an extremely important and challenging problem due to the scale of the reviews that makes human review infeasible. In this paper, we propose fully automated methods for clustering aspect words and generating human readable names for the clusters without any manually labeled data. We train transformer based sentence embeddings that are aware of unique e-commerce language characteristics (eg. incomplete sentences, spelling and grammar errors, vernacular etc.). We also train transformer based sequence to sequence models to generate human readable aspect names from clusters. Both the models are trained using heuristic based distant supervision. Additionally, the models are used to improve each other. Extensive empirical testing showed that the clustering model improves the Silhouette Score by 64% when compared to the state-of-the-art baseline and the aspect naming model achieves a high ROUGE-L score of 0.79.

## introduction the aspect mining based insights and its polarity extraction from reviews is a critical task for e-commerce services that enables seller to understand fine-grained customer preferences and improve product offerings. extracting important keywords and analyzing their sentiment is a very well studied area. however, the sheer scale of e-commerce services poses important novel challenges. firstly, review phrases/keywords about * work done while author was at india machine learning, amazon the same aspect category need to be grouped together, since each product may have thousands of reviews and there are millions of products. such aggregation will enable downstream individual aspect analysis by sellers. secondly, each review phrases/keyword group needs to be assigned an interpretable aspect name to enable easy analysis. finally, both steps have to be done without human annotations, as human review at e-commerce scale is infeasible. note that, in this paper, we would refer to the terms, "phrase", "review phrase" and "snippet" interchangeably to denote subsets of a review text, obtained by splitting a multi-context review into smaller sentences of single context. for example, if review text is "the headphone has a good sound quality but not so good bass quality. it is useful for playing music while working out." then the corresponding review phrases would be "the headphone has a good sound quality", "not so good bass quality" and "it is useful for playing music while working out." we have used some syntactic/lexical rules for context splitting. for unsupervised aspect grouping, extant methods use clustering @xcite (eg. kmeans) and topic modeling @xcite (eg. lda) approaches. lda based topic models assume the words are independently generated given the topic and consequently can't leverage the full context of the review sentences. k-means based techniques can overcome the drawback by using contextual embeddings typically generated by transformer based models @xcite . however, these general purpose transformer language models fail to capture the nuances of e-commerce reviews' language characteristics, such as code mixed sentences including vernacular, incomplete sentence formation, spelling errors. consequently, these models fail to generalize to e-commerce domain.another ma-jor drawback of the lda/k-means based methods is that these techniques are not able to generate a human interpretable name for the aspects (topics/clusters). in this paper, we propose a practical framework for grouping aspect phrases from reviews into clusters and generate meaningful aspect names for the clusters at scale without any human labeled data. specifically, the contributions of this paper are as follows: to arrive at aspects, we first cluster the phrases by clustering the phrase embeddings generated by the state-of-the-art general purpose semantic matching sbert model @xcite . we fine-tune the transformer based conditional natural language generation (nlg) model t5 @xcite for aspect name generation that is distantly supervised using a heuristic tf-idf distance based algorithm using the above clustering. finally, to improve the aspect clustering, we train a transformer on the reviews corpus using masked language model (mlm) and subsequently fine-tune it siamese style using the pairwise triplet loss. the training data (relevant and irrelevant pairs of phrases) for triplet loss is generated using a novel distant supervision strategy that leverages the earlier clustering output and the name generation model outputs. consequently, the learned text embeddings are very robust to nuances of the e-commerce reviews domain. we empirically evaluate our framework at scale on reviews from a popular e-commerce service. the distantly supervised semantic embedding based clustering model is able to improve silhouette score by 64% over a baseline technique using a state-of-the-art general purpose semantic embedding model. our distantly supervised aspect name generation model is able to improve the rouge-l score by 16%. ## related works aspect phrase extraction from text corpus is a widely researched topic @xcite @xcite @xcite @xcite . in this paper we explore two tasks after aspect phrase extraction, (1) aspect grouping into clusters, and (2) aspect name generation, that are specifically important to the e-commerce reviews domain due to its large scale and lack of annotation requiring unsupervised techniques. aspect grouping is done typically by clustering/topic modeling approaches once the aspect phrases have been extracted. topic modeling approaches include lda, plsa, nmf based aspect extraction @xcite @xcite @xcite . a number of clustering approaches have also been explored @xcite @xcite @xcite . one limitation of extant topic modeling/clustering approaches is that these techniques fail to leverage the semantic context of the entire text while clustering. recently, pre-trained models capable of capturing contextual representations have been developed @xcite . however, vanilla pre-trained embeddings doesn't lead to coherent groupings of aspects as the e-commerce review language is significantly different from general english/web text on which these embeddings models are pre-trained. in this paper, we propose a transformer language embedding model that captures the semantics of ecommerce reviews, thereby leading to robust clustering. note that our generated embeddings may be used with any existing clustering techniques to improve their quality. ## proposed solution the proposed framework for aspect grouping and naming has two main components: (i) phrase clustering, and (ii) aspect name generation. our goal is to develop a phrase embedding model that captures the nuances of e-commerce reviews, and a conditional nlg model that generates meaningful names for the aspects without any manually labeled data. to achieve this, we propose a novel distant supervision scheme that uses the two components to improve the other along with some heuristic based automated supervision. note that table we get the review phrases extracted by the existing pipeline at a popular e-commerce service. figure 1 shows an overview of the proposed framework and ## conclusion in this paper we presented a practical aspect clustering and naming framework for e-commerce reviews. our models leverage distant supervision thereby avoiding the need of manually labeled data. extensive evaluations show improvement in clustering by 64% and naming by 16%. survey results in appendix show that the approach generates more interpretable aspects when compared to an existing e-commerce baseline. we hypothesize that our novel distant supervision paradigm is generalizable across domains and in future we wish to explore the application of our novel distant supervision scheme to other domains. we also plan to explore principled approaches to handle multi-context phrases (phrase talking about multiple aspects) without needing manual annotations. (2) is the supplementary information helping in understanding the aspect better? the results of the survey is summarized in