Airbnb Rental Listings Dataset Mining
By: Sarang Gupta
Tags: Airbnb, Towards Data Science, R Programming, Data Visualization, Data Science

Exploring this at a granular level reveals that the demand is lowest in January and increases until October, when it begins to falls until the end of the year.

Is there a seasonality in the prices of rentals? Let us look at the daily average prices of the listings across the years.

The average prices across listings tends to increase as one progresses along the year and spikes in December. The pattern is similar to that of the number of reviews/demand except in the months of November and December, where the number of reviews (indicative of demand) starts to fall.

We can also see two sets of points on the charts which depicts that average prices on certain days were higher compared to the other days. Following, I will plot a box plot of average prices by day of the week to understand this phenomenon.

As we can see, Fridays and Saturdays are more expensive compared to the other days of the weeks, perhaps due to higher demand for lodging.

I will end the analysis of the section by studying how the occupancy looks like for the next year. Using data from the table calendar , I will find out the percentage occupancy for the next year i.e. as of November 3, 2018 (the date when the data was collected), what percentage of listings have already been booked. I was not able to obtain the past data on the occupancy, hence I could not study what the actual occupancy rates looked like.

It can be inferred that January tends to be the quietest, and the occupancy rate increases as we progress through the year. This ties up with the results from the analysis of the number of reviews (indicative of the demand) that shows an increasing trend across the year.

Getting into the Customer’s Head : Analysing Customer Reviews

The dataset provides us with a ton of data, but nothing as insightful and close to the customer as their reviews/feedback. If mined properly, they can tell us a lot about the customer mindset, their expectations and how well those were met. For the final result to make sense, the review text data requires a lot of cleaning — eg. the words need to be stemmed, commas-fullstops-percentages etc need to be removed, common English words and stop words need to be removed etc. There are more than a million reviews, so I take a random sample of this data, in this case ~30k reviews.

An analysis of the word cloud shows interesting trends; Location seems to be key, since the words “neighbourhood”, “location”, “area” are featured prominently in the word cloud. Transport options like “subway”, “walk” also find frequent mention. Airbnbs are short term rentals, yet people seem to lay stress on the comfort aspect of their stay, words like “kitchen” tell us that many folks would rather cook than eat out. Availability of “Restaurants” close by find mention too. Bathrooms and Beds, as expected can be clear deal breakers if not in top condition. The word “Host” finds a lot of mention; indicating the important role that hosts play in shaping the Airbnb experience.

Word vectors provide an effective way to figure out the closest words to particular search terms. Using the review data, I constructed a vector space to build a word cloud of similar words to derive interesting insights.

The first word cloud is for the word “uncomfortable”. Words similar to “uncomfortable” are usually those that occur in conjunction with it frequently, i.e reasons for the discomfort. The word cloud shows just that — notice words like “cramped”, “crowded”,“small”, “stuffy” and “cluttered” indicating that lack of space is one of the most common complaints. “Hot”, “damp” and “cold” are some of the common temperature issues. “Dusty”, “dirty” and “unclean” surroundings will prompt people to write negative feedback. Many feel “nervous”,“unsafe” and “stressful”; clearly a red flag for future tenants.

Similarly, querying by the keyword “comfortable”, we hope to see things that led to a positive experience. Prominently featured are words like “quiet”, “walkable”, “clean”, “spotless”, etc, again demonstrating the importance of environment, location and cleanliness. Helpful “hosts” and “communication” lead to a comfortable. Cleanliness of linens and the bed size leave a decisive impression.

Check out the Shiny App that serves as an extension to the Word-Vector analysis of the customer reviews. You can query the vector to find “similar” words and build a custom word-cloud. Enter any valid query word and set the “max words” in the word cloud. A valid query word would be one that is present in the corpus, else it wouldn’t be part of the word vector.

Below is a screenshot, you can try the actual app out through the link: https://ankitpeshin.shinyapps.io/wordcloud_generator/ (allow it some time to load).

Hosts with Super Powers! : Analysing what it takes to become a Superhost

Airbnb awards the title of “Superhost” to a small fraction of its dependable hosts. This is designed as an incentive program that is a win-win for both the host, Airbnb, and their customers. The superhost gets more business in the form of higher bookings, the customer gets improved service and Airbnb gets happy satisfied customers.

But what does it take to be a Superhost? Airbnb’s site has a set of requirements that must be fulfilled in order to become one. Maintaining a review rate above 50%, a response rate above 90%, etc. Our findings, while mostly in line with the Airbnb’s guidelines, also show some interesting outliers.While most super-hosts are in the high-rating:high-response-rate region, we can also see a few hosts with response rates less than 75% (which violates the 90%+ criteria set by Airbnb). This is a very small fraction of the hosts. In terms of Ratings, almost all hosts are rated 80% and above.

With that being said, most Airbnb hosts lie in the high-rating:high-response region, but only a small fraction get to be super hosts. So clearly, becoming a Superhost takes a lot more than high ratings & response rates.