Classification and Geotemporal Analysis of Quality-of-Life Issues in Tenant Reviews (2021)

Abstract: Online tenant reviews of multifamily residential properties present a unique source of information for commercial real estate investing and research. Real estate professionals frequently read tenant reviews to uncover property-related issues that are otherwise difficult to detect, a process that is both biased and time-consuming. Using this as motivation, we asked whether a text classification-based approach can automate the detection of four carefully defined, major quality-of-life issues: severe crime, noise nuisance, pest burden, and parking difficulties. We aggregate 5.5 million tenant reviews from five sources and use two-stage crowdsourced labeling on 0.1% of the data to produce high-quality labels for subsequent text classification. Following fine-tuning of pretrained language models on millions of reviews, we train a multi-label reviews classifier that achieves a mean AUROC of 0.965 on these labels. We next use the model to reveal temporal and spatial patterns among tens of thousands of multifamily properties. Collectively, these results highlight the feasibility of automated analysis of housing trends and investment opportunities using tenant-perspective data.

## introduction the use of artificial intelligence in commercial real estate investing has grown given the availability of new data modalities. motivated by the potential for new insights and improving investment decisions in the large real estate market, recent efforts have used cellular network data @xcite , satellite images @xcite , building permits @xcite , interior and exterior photos for luxury estimation and automated appraisal @xcite , and construction of new retail stores for predicting future rent growth @xcite , among others. however, one mostly untapped, yet highly informative, data source, is online tenant reviews. online tenant reviews of the properties in which tenants reside present a unique source of information in the multifamily domain due to their distinctive, tenant-perspective view @xcite . in recent years, the popularity of such reviews has grown such that there are now millions of newly generated reviews annually, with some properties garnering hundreds and even thousands of reviews over time. nonetheless, as they are rarely constrained to a specific format and can drastically vary in length and linguistic style, classifying reviews for detection of quality-of-life issues is a challenging task. text classification refers to the process of categorizing textual data into a set of defined classes. classical approaches to text classification rely on feature extraction techniques such as n-grams, bagof-words, and tf-idf, a potential dimensionality reduction step, followed by learning a classification model such as logistic regression, naive bayes, support vector machines, latent dirichlet allocation, and nearest-neighbours algorithms @xcite . more recently, deep-learning-based language models that are trained using contextualized word representations have been used to achieve state-of-the-art results on a wide range of natural language benchmarks and datasets, including text classification @xcite @xcite @xcite . deep-learning language models generally require large training data, use up to billions of parameters, and are costly to train. fortunately, language models pretrained on large corpora such as wikipedia or common crawl can be adapted to perform tasks in diverse domains, very effectively and with little labeled data @xcite . the above process is referred to as fine-tuning or transfer learning and entails modifying the parameters of the pretrained model to adapt to the statistical properties of the new corpus. fine-tuning figure 1 : study workflow. 5.5m reviews were collected, of which a small subset were manually labeled via crowdsourcing using multiple labelers per review. a larger set of reviews was used for language model fine-tuning, and the full set was used for uncovering domain-specific insights. prior nlp-based efforts on online reviews have used both classical @xcite and deep learning-learned representations @xcite to extract sentiment polarity and/or classify reviews @xcite . one popular group of methods, known as aspect-based sentiment analysis (absa), attempts to combine these two tasks by evaluating sentiment polarity with respect to specific aspects @xcite . one notable example in the real-estate domain performed a local analysis of 7,673 neighborhoodlevel reviews in new york city using absa and topic modeling @xcite . a commonality across many review classification efforts is that the review classes are generally broadly defined. however, carefully-tuned class definitions are often of high value to practitioners. for such cases, an approach that goes beyond coarse-grained classification may be beneficial. in this paper, we analyze a dataset of nearly 5.5 million tenant reviews from multiple online sources, covering tens of thousands of multifamily properties in the us. after analyzing the textual characteristics of this unique corpus, we describe an iterative crowdsourcing-based approach to ensure accurate labeling of a random sample of re-views for multiple, non-mutually exclusive classes. we then show how, using state-of-the-art nlp techniques, we label millions of reviews using a model that was trained on a few thousand annotated samples, and that the labeled corpus provides important insights on spatiotemporal trends affecting the real estate market (fig 1 ). ## corpus the data used in this study consisted of 5,468,037 online tenant reviews gathered from five different sources, covering approximately 96,134 different us multifamily properties foot_0 and spanning 21 years from 2000 -2020 ( the data for each review consisted of the review body text and metadata containing the date and the specific property associated with the review. the distribution of reviews per property was highly skewed as was the distribution of words per review (fig 2a and 2b ). the majority of the reviews (66%) were from recent years (2015-2020), consis- the reviews varied significantly in their sentiment and linguistic style. while the majority of the reviews were positive -"the [property name] staff are great and the residents are nice. it is a quiet and safe place to live", some expressed anger and frustration with the property, its surroundings, or its management -"this place is horrible i would not alow my dogs to live their, drugs being sold and apartments getting robbed stay away from these people". we randomly sampled 500 reviews and 500 wikipedia articles of similar lengths to measure the statistical discrepancy between the reviews corpus and a more general corpus such as wikipedia. correspondingly, we obtained 1000 document embeddings using fasttext @xcite , for which we computed the pairwise euclidean distance matrix between embeddings (fig 2e ). the block-diagonal structure of the resulting dissimilarity matrix implied that the model representations of reviews were clustered compared to random articles, reflecting their statistical and linguistic idiosyncrasies. this suggested the importance of fine-tuning a pretrained language model to the reviews corpus -see section 4. ## data labeling we labeled 0.1% (5,500) of the reviews in order to train models that can detect four detrimental quality-of-life issues. if accurate, these models may enable domain-specific analysis of the entire corpus, especially when paired with property-level geographical and temporal metadata. ## discussion in this study, we applied nlp-techniques to investigate a unique dataset of millions of online tenant reviews. we demonstrated that tenant reviews have idiosyncratic textual and statistical properties, differentiating them from other commonly used textual datasets. we further presented a resource-effective multi-labeling approach, and showed that using a limited set of high quality labels can achieve excellent results in a previously little studied domain. finally, we illustrated that nlp-based scores are informative, as verified by domain-specific validations, and can be used to study financial, demographic, geographical and temporal trends in a quantitative way. our work is in line with prior observations that with a relatively small number of labels, fine-tuned language models can be trained to accurately predict human annotations in novel corpora @xcite . although we focused on four key labels of interest, we expect this approach will generalize to other informative labels such as maintenance issues, management-related concerns, and renovation needs. additionally, while our analysis bears similarity to aspect-based sentiment analysis @xcite , the class definitions used are more precise. for example, a review that mentions a single event of a pest sighting in a property might demonstrate a negative sentiment towards pests, but is not necessarily indicative of a recurrent problem in the property as we defined in labeling instructions. domain-specific validation serves as an orthogonal means for validating model usefulness. encouragingly, model predictions often correlated with prior domain knowledge: crime and pest issues were higher in lower grade properties, all four labels improved in newer properties, and cities with higher crime rates had a higher amount of crimerelated reviews. these serve as secondary validations that strengthens our conviction in the value of model predictions. our results reveal differences between crime and pests issues versus parking and noise issues in relation to external, non-review data. whether this is an artifact, for instance due to the latter two being sparser labels, or whether it is a true real estate phenomenon warrants further investigation. one potential explanation may be variation in tenant base. for example, tenants in grade a properties may be more sensitive to noise and parking issues, and thus lower noise levels may receive increased mention. construction-wise, the evolution of building standards may be associated with the differences in pest, noise, and parking issue mentions in newer buildings. finally, demographic changes may also be linked to the strong reduction in crime mentions with newer year builds. one concern when analyzing online reviews is the potential presence of fake or solicited reviews. non-authentic reviews can bias the average score of a given property, in turn compromising the accuracy of downstream inferences. while online review sites have made large efforts to ensure review authenticity, there is nonetheless a risk. initial results indicate that nlp-based analysis might help in identifying these reviews @xcite ; applying this to our dataset and investigating the sensitivity of the results to such preprocessing is a potentially exciting future direction. ## conclusion the use of ai and non-traditional data in commercial real estate is expected to have far-reaching implications. our work contributes to this broader scope by highlighting how online tenant reviews, which have become ubiquitous, can uncover valuable insights that support both real estate investment decisions and research into local and nationwide housing trends.