aurele1/nyc_airbnb_price1

Project Video Walkthrough

Click here to watch my 4–6 minute project walkthrough

NYC Airbnb – Price Prediction & Classification

This repository contains the code and trained models for a data science project based on the New York Airbnb Bookings dataset (Kaggle).

The goal of the project is to:

• Predict the nightly price of an Airbnb listing in New York City (regression).
• Classify listings into three price levels (low / medium / high) based on their characteristics (classification).
• Explore the data with EDA, feature engineering (including clustering-based features), and model evaluation.

Dataset

This project uses the New York Airbnb Bookings dataset from Kaggle – New York Airbnb Bookings (Kaggle). The dataset contains 48,895 listings and 16 columns.
Each row represents an Airbnb listing in New York City, including the nightly price in USD, the main area of the city (neighbourhood_group), the specific neighbourhood name, and the room type (entire home/apartment, private room, or shared room).
Additional fields describe the listing’s activity and availability, such as minimum nights required, number of reviews, reviews per month, availability over 365 days, and the listing’s latitude and longitude coordinates.

Targets

The project includes both a regression and a classification task:

• Regression target – price: nightly price in USD (numeric).
• Classification target – 3-level price category (low / medium / high), created using quantiles on the training set.

Exploratory Data Analysis & Preprocessing

I started with brief descriptive statistics to understand the distribution and scale of the main variables.
As part of the initial exploration, the data was lightly cleaned: handling missing values (for example, filling reviews_per_month with 0 for listings with no reviews), converting date columns to datetime, filtering unrealistic outliers in price (very expensive listings above 1000 USD per night) and extreme values in minimum_nights (stays longer than 365 nights).
Text columns that are not useful for numeric modeling (such as name and host_name) were dropped to keep the feature space focused.

Below are some of the main visual insights from the exploratory analysis:

1. Outliers and price distribution

Figure 1 – Distribution of nightly price before filtering, showing a long right tail with very expensive listings.

Figure 2 – Boxplot of nightly price before filtering, highlighting many high-price outliers.

2. Research questions

Price by room type:

Figure 3 – Nightly prices by room type. Entire homes/apartments tend to be the most expensive, while shared rooms are the cheapest.

Average price by neighbourhood group:

Figure 4 – Price differences across neighbourhood groups. Listings in Manhattan are the most expensive on average, followed by Brooklyn.

Price vs. number of reviews:

Figure 5 – Relationship between number of reviews and price. There is no strong linear pattern, and highly reviewed listings are spread across different price levels.

3. Correlations

Figure 6 – Correlation matrix of the main numerical features. Correlations with price are relatively weak compared to other relationships in the data (e.g. between reviews per month and number of reviews).

Overall, the EDA suggests that neighbourhood group and room type are the strongest drivers of price, with entire homes and listings in Manhattan being the most expensive on average.
In contrast, popularity measures such as the number of reviews show a much weaker and less linear relationship with price, indicating that they are not sufficient on their own to predict the nightly rate.

Baseline regression model

As a starting point, I trained a simple linear regression model to predict the nightly price using only the original numerical features (without feature engineering).

• The data was split into train and test sets.
• The baseline model achieved roughly:
  • MAE: ~71 USD
  • RMSE: ~106 USD
  • R²: ~0.11

A scatter plot of true vs. predicted prices shows that the model systematically underestimates very expensive listings and overestimates cheap ones, which is consistent with the low R².
This baseline serves as a reference point for the later, more advanced models.

Figure X – Baseline linear regression model: predicted vs. true price, showing strong errors for very cheap and very expensive listings.

Feature Engineering & Clustering

To improve the models, several new features were engineered:

• reviews_per_year – computed from reviews_per_month to capture yearly popularity.
• Converted categorical features (neighbourhood_group, room_type, location_cluster) into numeric one-hot encoded features for modeling.
• Cleaning and preparation of numerical features for the different models.

In addition, a KMeans clustering step was applied on the geographical coordinates (latitude, longitude) to create a new location-based feature:

• location_cluster – a cluster ID (5 clusters) that groups listings into spatial segments within the city.

Location-based clustering visualization

Figure X – PCA view of the KMeans location clusters. Each colour represents one cluster of listings in NYC; this cluster ID is later used as an additional categorical feature (location_cluster).

Improved regression models

After feature engineering, I trained three regression models on the enriched feature set:

• Linear Regression (with engineered features)
• Random Forest Regressor
• K-Nearest Neighbors Regressor (KNN)

All models were evaluated on a held-out test set using MAE, RMSE and R².
Compared to the baseline linear regression (R² ≈ 0.11), the models with feature engineering achieved a substantial improvement:

• Linear Regression with engineered features: MAE ≈ 55 USD, RMSE ≈ 91 USD, R² ≈ 0.36
• Random Forest Regressor: MAE ≈ 49 USD, RMSE ≈ 83 USD, R² ≈ 0.45
• KNN Regressor (k = 10): weaker performance than the Random Forest and slightly worse than the linear model.

The Random Forest Regressor provides the best trade-off between bias and variance and clearly outperforms both the baseline and the other improved models.

Most important features for price prediction

Figure X – Top features for the linear regression model (top) and the Random Forest regressor (bottom). Location-related features (latitude, longitude and location clusters), room type and availability have the strongest impact on predicted nightly price.

Winning regression model

The final winning regression model is a Random Forest Regressor with 200 trees and a fixed random seed for reproducibility.
On the test set it achieves approximately:

• MAE ≈ 49 USD
• RMSE ≈ 83 USD
• R² ≈ 0.45

The model is trained on the full feature-engineered dataset and saved in this repository as:

• winning_model.pkl – serialized Random Forest regressor for nightly price prediction.

Price level classification

In addition to predicting the exact nightly price, the project also defines a classification task: predicting the price level of a listing (low / medium / high).

To create the classification labels, the continuous price variable is transformed into three classes using the 33rd and 66th percentiles of the training set only:

• class 0 – low price
• class 1 – medium price
• class 2 – high price

This quantile-based binning produces three roughly balanced classes, which is helpful for training classification models: in both the train and test sets each price class contains roughly one third of the samples (around 32–35% per class).
From a business perspective, the most important group is the high-price class (class 2), so recall and F1-score for this class are especially important: misclassifying a high-price listing as low/medium is more costly than mistakenly flagging a cheaper listing as “high-price”.

Classification models

For the price-level classification task (low / medium / high), I trained three different models on the feature-engineered dataset:

• Multinomial Logistic Regression
• Random Forest Classifier
• K-Nearest Neighbors (KNN) Classifier

All models were evaluated on the test set using accuracy, macro F1 and weighted F1, as well as confusion matrices to inspect the types of mistakes. Because the classes are reasonably balanced, accuracy is informative, but F1 scores are used to better compare performance across classes.

On the test set, the models achieved approximately:

• Logistic Regression: accuracy ≈ 0.66, macro F1 ≈ 0.65
• Random Forest Classifier: accuracy ≈ 0.70, macro F1 ≈ 0.70
• KNN (k = 10): accuracy ≈ 0.49, macro F1 ≈ 0.49

The confusion matrices show that the medium-price class (class 1) is the most challenging and is often confused with both low and high prices. Among the three models, the Random Forest provides the best overall separation between classes and the strongest performance across all metrics.

Confusion matrix – winning classifier

Figure X – Confusion matrix for the Random Forest classifier on the test set.

Winning classifier & hyperparameter tuning (bonus)

The final chosen classification model is a Random Forest Classifier, which provides the best balance between accuracy and F1-score across the three price levels. In addition to the default model, a small GridSearchCV was run over a few combinations of hyperparameters (such as the number of trees and maximum depth) to slightly improve performance and check the robustness of the results. The tuned Random Forest classifier is saved in this repository as:

• winning_classifier.pkl – serialized Random Forest classifier for predicting the price level (low / medium / high).

Files

• data1.csv – dataset used in the notebook
• Another_copy_of_Assignment_2_Classification,_Regression,_Clustering,_Evaluation.ipynb – full analysis (EDA, feature engineering, regression and classification)
• winning_model.pkl – best regression model (Random Forest)
• winning_classifier.pkl – best classification model (Random Forest)

The models and notebook are also available on Hugging Face:
aurele1/nyc_airbnb_price1

Summary

This project explored Airbnb listings in New York City and built both regression and classification models to predict nightly prices. Feature engineering (especially location-based clustering, room type and neighbourhood information) significantly improved performance over a simple baseline. A Random Forest Regressor was selected as the final price prediction model, and a Random Forest Classifier as the final model for predicting price levels (low / medium / high). The full workflow — from EDA and preprocessing to modeling and evaluation — is documented in the accompanying notebook.

The project shows that:

• Location and room type are among the strongest drivers of price in NYC Airbnb listings.
• Simple feature engineering (reviews per year, location clusters, one-hot encoding) significantly improves model performance.
• Tree-based models (Random Forest) work well for both regression and classification on this dataset.