Designing Ranking Systems for Hotels on Travel Search Engines by Mining User-Generated and Crowd-Sourced Content 1

User-Generated Content (UGC) is changing the way consumers shop for goods. Based on a unique dataset of hotel reservations over a 3-month period from Travelocity.com, we estimate the demand for hotels using a random coefficient structural model. We obtain user-generated data from three sources: (i) text of hotel reviews from two well-known travel search engines, Travelocity.com and Tripadvisor.com, (ii) social geo-tags from Geonames.org identifying the different location-based attributes of hotels, and (iii) user-contributed opinions on the most important hotel characteristics by accessing a wide consumer demographic using Amazon Mechanical Turk. These data sources are merged with satellite images of the different hotel locations to create one comprehensive dataset summarizing the location and service characteristics of the hotels in our sample. We use text analysis techniques to measure the quality of available reviews. Using these analyses, we quantify how the extent of subjectivity, readability, complexity and other stylistic features of user-generated reviews are associated with hotel room sales. Finally, based on the hybrid model we estimate the weight that consumers place on different location and service-related features of hotels. We extend the basic model to examine interaction effects between travel purpose, price, and hotel characteristics. Business travelers are the least price sensitive while tourists are the most price sensitive. Business travelers have the highest marginal valuation for hotels located closer to a highway and having easy access to public transportation. In contrast, romance travelers have the highest marginal valuation for hotels located closer to a beach and those with a high service rating. As the ultimate goal of this research, we use the generated estimates from our model in order to build a better ranking system for hotel search. Specifically, we leverage the econometric analysis and compute the average utility gain that a consumer gets by staying in a particular hotel. We propose to rank the hotels, in response to a search query, using this utility gain, which is a measure of ―value‖ that a consumer gets from this transaction. By doing so, one can provide customers with the ―best-value" hotels early on, thereby improving the quality of online hotel search compared to existing systems. Several field experiments in six major cities (New York, Los Angeles, San Francisco, Orlando, New Orleans, and Salt Lake City), using 15,600 ranking comparisons from Amazon Mechanical Turk, suggest that our ranking system is superior to existing systems. 1 We thank Susan Athey, Peter Fader, Francois Moreau, Aviv Nevo, Minjae Song, Daniel Spulber, and Hal Varian for extremely helpful comments that have significantly improved the paper. We also thank participants at NBER IT Economics & Productivity Workshop, 2010 Workshop on Digital Business Models, 2010 Marketing Science Conference, Searle Research Symposium on the Economics and Law of Internet Search at NorthWestern University, Customer Insights Conference at Yale University, 2010 SCECR conference, 2009 Workshop on Information Technology and Systems (WITS), 2009 Workshop on Economics and Information Systems and seminar participants at Temple University, University of Minnesota for helpful comments. Anindya Ghose and Panos Ipeirotis acknowledge the financial support from National Science Foundation CAREER Awards IIS-0643847 and IIS0643846, respectively. Support was also provided through a MSI-Wharton Interactive Media Grant (WIMI) and a Microsoft Virtual Earth Award. The authors thank Travelocity for providing the data and Uthaman Palaniappan for research assistance.