Quantifying the Effects of COVID -19 on Restaurant Reviews (2021)

Abstract: The COVID-19 pandemic has implications beyond physical health, affecting society and economies. Government efforts to slow down the spread of the virus have had a severe impact on many businesses, including restaurants. Mandatory policies such as restaurant closures, bans on social gatherings, and social distancing restrictions have affected restaurant operations as well as customer preferences (e.g., prompting a demand of stricter hygiene standards). As of now, however, it is not clear how and to what extent the pandemic has affected restaurant reviews, an analysis of which could potentially inform policies for addressing this ongoing situation. In this work, we present our efforts to understand the effects of COVID-19 on restaurant reviews, with a focus on Yelp reviews produced during the pandemic for New York City and Los Angeles County restaurants. Overall, we make the following contributions. First, we assemble a dataset of 600 reviews with manual annotations of fine-grained COVID-19 aspects related to restaurants (e.g., hygiene practices, service changes, sympathy and support for local businesses). Second, we address COVID-19 aspect detection using supervised classifiers, weakly-supervised approaches based on keywords, and unsupervised topic modeling approaches, and experimentally show that classifiers based on pre-trained BERT representations achieve the best performance (F1=0.79). Third, we analyze the number and evolution of COVID-related aspects over time and show that the resulting time series have substantial correlation (Spearman’s \rho =0.84) with critical statistics related to the COVID-19 pandemic, including the number of new COVID-19 cases. To our knowledge, this is the first work analyzing the effects of COVID-19 on Yelp restaurant reviews and could potentially inform policies by public health departments, for example, to cover resource utilization.

## introduction the outbreak of the sars-cov-2 virus in december of 2019 and its evolution to the covid-19 pandemic have had many devastating consequences in society. restaurants have been among the hardesthit businesses during the pandemic. 1 yelp data (as of september 2020) shows that out of the 32,109 restaurant closures in the u.s., 61% have been permanent, and a greater impact is observed in local businesses in larger metropolitan areas, such as new york city and los angeles county, on which we focus in this paper. restaurants operate under great uncertainty during this ongoing situation and, therefore, it is critical to understand how the pandemic has affected public attitude towards restaurants. the disruption in daily routines as well as fear and anxiety due to the pandemic have been shown to affect eating habits @xcite . the pandemic may have also affected customers' preferences, such as changes in cuisine types, or higher expectations of hygiene and social distancing practices followed by restaurants. in this paper, we present our efforts to understand the effects of covid-19 on restaurant reviews. reviewers provide ratings and free-form text to express their opinions and experiences about restaurants and we argue that the pandemic has affected such reviews. as an example, figure 1 shows a yelp review discussing the hygiene practices of a restaurant, including a mention of "coronavirus" and associated concerns. to understand more broadly the effect of the pandemic on restaurant reviews, we analyze 3 million yelp reviews published before and during the pandemic, for restaurants in two large metropolitan areas, namely, new york city and los angeles county. we measure changes in user activity, ratings, and restaurant type preferences using the corresponding metadata, and quantify changes in written text using relevant extraction and classification techniques. overall, we make the following contributions. ## related work the natural language processing community has been increasingly pushing efforts towards the better understanding and management of the pandemic. valuable insight can be extracted from text data, including the covid-19 scientific literature @xcite , and web search data @xcite . below, we review related work on the analysis of online user-generated reviews and posts on social media. social media reflects public attitudes during the pandemic @xcite . existing work on sentiment or emotion analysis has considered twitter (drias and drias, 2020; nemes and kiss, 2020; @xcite , reddit @xcite , weibo @xcite , and other platforms @xcite . for example, @xcite analyzed how the pandemic has influenced the online behavior of reddit users and found an increase in posts expressing mental health concerns, including anxiety and concerns for health and family. beyond sentiment analysis, existing work has considered deep learning techniques for the identification of informative ber 31, 2020. tweets that contain information relevant to the pandemic @xcite @xcite . to our knowledge, our work is the first that analyzes the effects of covid-19 on restaurant reviews. to perform this analysis, we extract fine-grained covid-19 aspects related to restaurants (e.g., hygiene practices, sympathy and support, social distancing, etc.). other work has studied nutrition during the pandemic by conducting surveys @xcite or by analyzing twitter @xcite . @xcite observe a shift from mentions of healthy to unhealthy foods. @xcite propose a framework for action to maintain optimal nutrition during the pandemic. as part of our work, we show trends in restaurant preferences, such as increased interest in fast food restaurants compared to traditional american-food restaurants. while prior work demonstrates changes in public attitude and nutrition during the pandemic, it is not clear how and to what extent restaurant reviews have changed during the pandemic. yelp has introduced special covid-19 review guidelines, and subsequently removed more than 4,000 reviews that violated those guidelines. 3 our work demonstrates that many aspects of the review content and metadata have changed during the pandemic. ## data we now describe our procedure for yelp data collection (section 3.1) and covid-19 aspect annotation (section 3.2). out of the 600 reviews, 81 reviews were annotated with more than one aspect. ## methodology we now describe the techniques that we apply to the 3.1 million yelp reviews from section 3 for covid-19 aspect analysis (section 4.1) and time series analysis (section 4.2), leveraging the labeled reviews of section 3.2. ## findings we use the methodology from section 4 to address various questions on the 3.1 million-review ## conclusions and future work we presented our effort to understand the effects of covid-19 on restaurant reviews. we created a dataset with fine-grained covid-19 aspect annotations, evaluated fully-and weakly-supervised techniques for covid aspect detection, and showed that bert-based classifiers outperform bag-ofwords classifiers. we observed changes in restaurant reviews (e.g., increased discussions of hygiene practices and messages of solidarity), and showed that they correlate with critical covid-19 statistics. we found a shift of ratings towards extreme values (1 and 5 stars) and shifts of user activity towards specific types of cuisines. our insights could potentially be interesting for restaurant owners, customers, and public health officials. in future work, we plan to expand the regional coverage of our analysis to reveal distinct patterns across cities. it would also be interesting to improve aspect-based sentiment analysis approaches @xcite by considering the new aspects explored in this work.