The Reliability of Online Review Helpfulness

ABSTRACTMany online reviews have a helpfulness rating, and such ratings are being widely used by online shoppers for shopping research. Researchers also use them as a review quality benchmark. However, there is scant research about the reliability of such ratings. This paper explores the reliability of helpfulness ratings and their resistance to manipulations. We found that the existing helpfulness ratings for most helpful reviews are inflated and significantly higher than ratings we collected from a random population due to online shopper self-selection behavior. We also found existing helpfulness ratings for most helpful favorable reviews have an anchoring effect on subsequent votes, thus could be potentially manipulated to boost sales. In contrast, ratings for most helpful critical reviews have a counter-anchoring effect due to risk aversion, thus could backfire if manipulated. Implications and future research are discussed.Keywords: Online review; Helpfulness; Amazon.com; User generated content; B2C ecommerce1. IntroductionUser-generated online reviews (online reviews hereafter) are becoming an essential component of B2C ecommerce. Online reviews mainly serve two functions in electronic commerce. One is to help online shoppers evaluate products and services before making purchase decisions [Park, Lee et al. 2007]. The other is informant [Clemons, Gao et al. 2006], which allows consumers to become familiar with a product or service even though they do not have an immediate intent to purchase [Chen and Xie 2008]. Online reviews can offer important value to customers [Mudambi and Schuff 2010]. Empirical studies done in the past ten years report that popular reviews have strong influences not only on commodities [Zhu and Zhang 2010] and new products [Cui, Lui et al. 2012] but also on services [Ye, Law et al. 2011]. Amazon.com revolutionized many online review features to enhance consumers' shopping experiences. For example, once a new online review has been posted and read by a registered shopper, the shopper can vote on its helpfulness by simply clicking the Yes or No button under the review content. The aggregated number of Yes vote and total votes a review received are then updated and displayed at the top of the review content as an indicator of helpfulness.Based on the aggregated helpfulness votes a review receives, amazon.com could use its proprietary computer algorithms to automatically rank and sort out those most helpful reviews and feature them at the top of the review section. This simple feature seems very helpful when the number of reviews keeps increasing and consumers feel difficult to go through even a small percentage of them. Shoppers could then spend their limited product-research time on the most helpful ones to avoid information overload [Maes 1994]. Gradually, because of the market share and influence of amazon.com, this voting-for-helpfulness feature was not only being adopted by many other online retailers, but also being utilized by many researchers as a de facto review quality standard [Mudambi and Schuff 2010, Ghose and Ipeirotis 2011, Korfiatis, Garcia-Bariocanal et al. 2012]. Helpfulness votes are important to help consumers on product research and making purchase decisions. It also serves as a benchmark for academic studies on online review. So understanding their reliability and resistance to manipulation could help us better utilize this feature. Several studies already identified bias in online reviews [Li and Hitt 2008, Kapoor and Piramuthu 2009, Cui, Lui et al. 2012, Purnawirawan, Dens et al. 2012]. The voting for helpfulness of reviews may also suffer from similar or related biases. We need to explore the impact of such bias on the helpfulness rating outcomes.In addition to bias, there are increasing concerns about the review manipulation by interested parties like product manufacturers. Ordinary consumers may not be aware of online review manipulation but such practices were observed by both small businesses owners, like those listed on Yelp. …