Identifying Hijacked Reviews (2021)

Abstract: Fake reviews and review manipulation are growing problems on online marketplaces globally. Review Hijacking is a new review manipulation tactic in which unethical sellers “hijack” an existing product page (usually one with many positive reviews), then update the product details like title, photo, and description with those of an entirely different product. With the earlier reviews still attached, the new item appears well-reviewed. So far, little knowledge about hijacked reviews has resulted in little academic research and an absence of labeled data. Hence, this paper proposes a three-part study: (i) we propose a framework to generate synthetically labeled data for review hijacking by swapping products and reviews; (ii) then, we evaluate the potential of both a Siamese LSTM network and BERT sequence pair classifier to distinguish legitimate reviews from hijacked ones using this data; and (iii) we then deploy the best performing model on a collection of 31K products (with 6.5 M reviews) in the original data, where we find 100s of previously unknown examples of review hijacking.

## introduction reviews are an essential component of many online marketplaces, helping new consumers assess product quality, legitimacy, and reliability. recent surveys indicate that an overwhelming majority of people read reviews @xcite . indeed, 79% of people overall and 91% of people ages 18-34 trust online reviews as much as personal recommendations @xcite . naturally, reviews have become a target of manipulation, misuse, and abuse @xcite . in this paper, we focus on the problem of review hijacking, a relatively new attack vector and one that has received little, if any, research attention. review hijacking is a fraud technique wherein a blackhat seller "hijacks" a product page that typically has already accumulated many positive reviews and then replaces the hijacked product with a different product (typically one without any positive reviews). the sellers then reap the ratings "halo" from consumers who assume the new product is highly rated. this review hijacking (also referred to as review reuse or bait-and-switch reviews) provides the sellers with a shortcut to many undeserved positive reviews. figure 1 : an example of review hijacking on amazon (may 7, 2021) figure 2 : hijacked reviews associated with the hair removal product in figure 1 an example is shown in figure 1 which we discovered in the first week of may 2021. this hair removal product has 4,069 reviews with an average rating of close to five stars. on inspection of the reviews (see figure 2 ), we find many that refer to other products like dishwasher cleaners and diapers. also, these reviews are for verified purchases which can provide added weight to the ostensible veracity of the reviews. we have identified at least three different methods that blackhat sellers adopt to conduct review hijacking depending on the particular e-commerce platform. a seller can incrementally change aspects of their own product (like the title, photo, description), resulting in an entirely new product, though still associated with the original reviews. alternatively, a seller can add his product as a product variation of some other product to aggregate reviews from the former product. one can also merge reviews from some other products to their own by changing country or using some other platformspecific loopholes. while review hijacking has been recognized in the press and social media as a growing problem, e.g., @xcite @xcite @xcite , there has been no structured research to date on identifying review hijacking. we attribute this to several key challenges: hence, this paper proposes an initial investigation into the potential of identifying review hijacking. we conduct a three-part study. due to the challenges of finding high-quality examples of review hijacking, we first propose a framework to generate synthetic examples of review hijacking by swapping products and reviews. we do so both at the inter-category level (where presumably it should be easier to determine if a review is associated with a product) and at the intra-category level (where product similarity within the category may make this more challenging). over this synthetic dataset, we evaluate the potential of both a twin lstm network and bert sequence pair classifier to distinguish legitimate reviews from hijacked ones. based on the encouraging results from this experiment, we then deploy the bert sequence pair classifier algorithm on a real collection of 31k products (with 6.5 m reviews). by averaging the review scores from the classifier for each product, we find that products with an average review score (or suspiciousness score) > 0.5 have 99.95% of the listings containing unrelated or hijacked reviews. these findings suggest the promise of large-scale detection of review hijacking in the wild. ## related work the manipulation of reviews and review platforms has been widely studied, e.g., @xcite @xcite @xcite , though there is little research literature on the problem of review hijacking. here, we highlight several efforts related to the methods proposed in this paper. higgins et al. developed models for an essay rating system to detect bad-faith essays by comparing the essay titles to the essay text to determine whether the title and text were in agreement through the use of word similarity @xcite . a similar idea motivates our approach that compares product titles/descriptions with review text. louis and higgins continued this line of research to determine whether a particular essay was related to the essay prompt or question by expanding short prompts and spell correcting the texts @xcite . rei and cummins extended this work and combined various sentence similarity measures like tf-idf and word2vec embeddings with moderate improvement over higgins' work @xcite . apart from the essay space, ryu et al. investigated the detection of out-of-domain sentences @xcite . they proposed a neural sentence embedding method representing sentences in a low-dimensional continuous vector space that emphasizes aspects in-domain and out-of-domain for a given scenario. in another direction, fake news detection and clickbait detection can be viewed as related tasks. for example, hanselowski et al.used a bilstm model with attention to determine if the headline of a news article agrees, disagrees, or is unrelated to the text as part of a fake news challenge @xcite . ## generating synthetic examples of review hijacking in our preliminary investigation, we examined hundreds of reviews from the amazon dataset provided by mcauley @xcite . the dataset contains 233.1 million reviews from may 1996 to october 2018, with reviews and product information including title, description, etc. however, we find very few examples of review hijacking. hence, we concluded that hiring crowd labelers or subject matter experts to label product-review pairs as hijacked or not hijacked might not be fruitful. instead, we propose a method to generate synthetic examples for studying the potential of models to identify hijacked reviews. ## identifying synthetic examples given these synthetic datasets of hijacked reviews, can we detect them? in this section, we report on experiments with two approaches: one based on a twin lstm and one based on bert sentence pair classification. we shuffled the product-review pairs and split them into training, validation and test set in ratio 70:10:20 for both of the datasets. the actual number of reviews in each set depends on the swapping categories and is discussed in section 3.2 we train on the train set, tune models on the validation set, and have reported results on the test set. ## detecting hijacked reviews in-the-wild even though encouraging, these results are on synthetic data, and the data itself may contain noisy labels. hence, we next turn to the task of uncovfigure 7: roc curve for bert seq. pair classifier run on intra-category data (jaccard distance) figure 8: roc curve for twin lstm network run on inter-category data figure 9: roc curve for bert seq. pair classifier run on inter-category data