Feature Selection and Regression On the Airbnb Berlin Dataset | by Somesh Kumar Bhattacharya | Towards Data Science

The multidisciplinary field of data science can extract deep insights and knowledge from structured and unstructured data. It unifies the concepts of statistics, data analytics and machine learning and other related concepts to understand and analyze the actual phenomena with data. Machine learning (ML) equips computers to learn and interpret without being explicitly programmed. ML methods are used widely in finance, healthcare, materials science, transportation, oil and gas etc.
In this study we undertake the analysis of Airbnb dataset from the Berlin area [1] and predict the prices of Airbnb using the regression analysis. We have used the random forest regression [2] model available in the “scikit-learn” [3] library of the python-based open source data analytics platform and the plots were created using the Matplotlib.
The shape of the Airbnb dataset is (22552, 96). Many of the 96 features are non-numeric features and many features have missing values. Some of the non-numeric features were converted to numeric ones using the Label Encoder. The missing values were filled using the forward filling method and the certain features were cleaned from the ‘$’, ‘,’ etc. using the strip method. Additionally, we created a new feature ‘distance’ of the Airbnb from the center of Berlin using the longitude and latitude of the Airbnb. This new feature was then appended to the original dataset. Once the data processing is done, we used ‘feature selection’ available in the scikit-learn to select the important features. We calculate χ2 between each feature and the target and select the desired number of features with best χ2 scores. The best 25 features and the χ2 score are tabulated below in Table 1.
Table 1. Best 25 features of the dataset and their corresponding χ2 scores.   Features                     χ2 scoresmaximum_nights                 2.152729e+10minimum_nights                 8.856912e+05security_deposit               7.716334e+05number_of_reviews              8.341015e+04square_feet                    2.982368e+04property_type                  2.222414e+04calculated_host_listings_count 1.072021e+04neighbourhood_cleansed         1.071895e+04accommodates                   7.771539e+03room_type                      4.234147e+03neighbourhood                  3.254091e+03guests_included                3.061216e+03bedrooms                       2.380380e+03cancellation_policy            1.444670e+03distance                       9.580979e+02host_is_superhost              6.349556e+02neighbourhood_group_cleansed   5.957898e+02instant_bookable               3.502228e+02bathrooms                      3.290317e+02review_scores_rating           2.151681e+02is_location_exact              9.053878e+01review_scores_cleanliness      5.194306e+01review_scores_accuracy         2.410470e+01bed_type                       7.055763e+00longitude                      1.016177e-01
Next, only the top five features were selected. This subset was split into training and test data. The ‘price’ was set as the target feature. The training data subset was used for fitting the random forest while the test subset was used for the prediction. This was repeated by using different subsets where top ten, top fifteen, top twenty and top twenty-five features were included. For each case we calculated the value which is defined as
adjusted R^2 = 1-R^2(N-1)/(N-M-1)
where N is the number of points in the dataset and M is the number of independent feature(s) while is the measure of the goodness of fit. increases with the number of features to the model. As always increases and never decreases, it can appear to be a better fit with the more terms that is added to the model. In regression analysis, it can be tempting to add more variables to the data which can be misleading. Some of those variables may be significant, but one can’t be sure that significance is just by chance. The compensate for this by that penalizing for those extra variables. The corresponding plot is shown in Figure 1.
Figure 1. Plot of the number of features and the corresponding adjusted R2
Exploratory data analysis (EDA) via graphs is definitely useful and important. However, for large datasets it becomes extremely difficult to adopt EDA as there are a large number of plots to analyze. Thus, features selection via the SelectKBest method of scikit learn provides a well defined way for feature selection as demonstrated here.