Meet CRISP-DM: An approach to answer all your questionsssss
By: Sumit Kumar
Tags: Data Science, Technology, Machine Learning, Exploratory Data Analysis, Airbnb

I just finished the first term of Udacity’s Data Scientist Nanodegree and started my second term few days back.While the first term was all about applying machine learning concepts to solve business problems,the second term focuses more on data engineering,experiment design and communicating your findings to the stakeholders.

The first lesson of the second term asks the students to analyse a data set and communicate their findings in a blog post. So,here I am…………….writing my first ever blog.I always wanted to write a blog but hardly had I ever thought that I would be writing a blog on Data Science and Machine Learning.

“The art of communication is the language of leadership-James Humes ”

In this blog,I will be explaining how to analyze a dataset using the CRISP-DM process.I will use Seattle’s Airbnb data set for this purpose. The CRISP-DM process is as follows:

Business Understanding

Data Understanding

Data Preparation

Modeling

Evaluation

Deployment

Business Understanding

A‌i‌r‌b‌n‌b‌,‌ ‌I‌n‌c‌.‌ operates a global online marketplace and hospitality service accessible via its websites and mobile apps. Members can use the service to arrange or offer lodging, primarily home stays, or tourism experiences. The company does not own any of the real estate listings, nor does it host events; it acts as a broker, receiving commissions from every booking

Since,this company receives commission for each booking that is carried on between the customers and the host.So,if the company has to increase its revenue,it has to increase the number of bookings for the listings on their marketplace.Now, the number of bookings will depend on two main KPIs:

Price

Review ratings

Taking these two main KPIs into consideration,I decided to answer the below mentioned three questions:

1) How does the price of Seattle Airbnb listing vary throughout the year?When are the prices the highest and where are they lowest? 2) Which neighborhoods have got the best average ratings from the reviewers?What are the most important factors influencing the ratings of an Airbnb listing? 3) What are the most important factors influencing the price of an Airbnb listing?

Data Understanding

After going through different datsets ,I finally decided to analyse Seattle’s Airbnb open data available on Kaggle .This data set consists of three files:

calendar: including listing id,the price and availability of a listing on a given day listings:including full listing descriptions and their review scores reviews:including the id of each reviewers and their comments

Data Preparation

The data in these input files was not clean and had a lot of missing values.So,for this purpose,I created few functions to clean the data and treat the missing values so that the data is ready for the analysis to be properly carried out.

Modeling,Evaluation and Deployment

After I had cleaned the data and it was ready for modelling, I applied linear regression,Decision Tree Regressor and Random Forest Regressor to find out the important features influencing the review ratings and the price of the listings.I used Mean Squared Error(MSE) as the evaluation metric to evaluate the performance of all these three models.

Using the above process,I was able to find the answers to my questions .Here is the result of my analysis:

Listings Price Fluctuation

When we analysed the price of the listings,we figured out that the prices of the listings were maximum from July to September and were minimum from January to March which is understood as July to September is the best time to visit Seattle because of the nice weather and therefore the prices are highest during that time because of the high demand.

Price fluctuation throughout the year

Also,let us check how the prices vary throughout the week.

Price fluctuation throughout the week

From the above visualization,we can see that the price is maximum during Fridays and Saturdays which is expected as people tend to go out more during weekends than on weekdays.

Factors influencing Listings’ Ratings

Before figuring out what features influence the ratings of the listings,I wanted to know which neighborhoods had the best average ratings for their listings.

Ratings by Neighborhood

Ratings by Neighborhood

From the above visualization,we can see that Arbor Heights has the best average ratings followed by Riverview and Southeast Magnolia.

Now,let us try to figure out the most important factors influencing the ratings..

For this purpose,I applied linear regression and Decision Tree Regressor on the listings data set after applying all the necessary transformations on the input features like converting the string column price into float,encoding the categorical variables into dummy variables,treating the missing values etc…etc….

When I compared the Mean Squared Error of both these models,the MSE for linear regression was 6.1015317019418545 whereas that of Decision tree regressor was 18.388801796094899.So,I went ahead and used Linear Regression to find out the most important features impacting the ratings

Features influencing Ratings

So,we can see that the features influencing the listings ratings are:

host_is_superhost :If a host is a superhost,then it receives high ratings host_response_rate:If the response rate of a host is high,then the listings rating is high bedrooms:If the number of bedrooms in a listing is higher,higher is the rating beds:The ratings of a listing is also influenced by the number of beds as well.More is the number of beds,higher is the rating. Amenities: More is the number of amenities provided in a listing,better is its rating.

Factors influencing Listings’ Price:

For this purpose,I applied three machine learning models on the transformed data set: Linear Regression,Decision Tree Regressor and Random Forest Regressor. When these three models were evaluated on the basis of the Mean Squared Error,Linear Regression performed the best.

When we trying creating a histogram for the 30 most important features,this is what we got.

So,from the above histogram,we can clearly say that price of a listing depends on its neighborhood,size(number of bedrooms,number of bathrooms),type of the property i.e. if it is a bungalow,the price is higher than the other property and on whether the host is a superhost or not.

Conclusion:

On the basis of my analysis,below are my findings:

The prices of the listings are maximum from July to September and are minimum from January to March which is understood as July to September is the best time to visit Seattle because of the nice weather and therefore the prices are highest during that time because of the high demand. The ratings of a listing is influenced by whether a host is a superhost or not,the host’s response rate,the size of the listings,the number of amenities provided in the listing and also on its price. Price of a listing depends on its neighborhood,size(number of bedrooms,number of bathrooms),type of the property i.e. if it is a bungalow,the price is higher than the other property and on whether the host is a superhost or not.

For full analysis of this work,please find my code here. Also,feel free to leave any comment or feedback to help me improve my blog.