YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

Flight Price Prediction — Regression, Classification & Clustering

Author: Shira Bash

project walkthrough presentation: Loom Video

Project Overview

This project aims to analyze and predict flight prices using multiple machine learning approaches.
The assignment includes:

Exploratory Data Analysis (EDA)
Regression modeling (baseline → advanced)
Feature Engineering
Clustering (K-Means + PCA visualization)
Transforming regression into classification
Training & evaluating classification models

Project Goal

The goal is to understand what drives flight prices, build effective predictive models, and communicate insights clearly.

Part 1 - Dataset and Overview

The dataset contains detailed information about commercial flights, including airline, departure/arrival times, number of stops, duration, and price. After cleaning (removing duplicates, dropping an unnamed column, and removing outliers in two columns using IQR), the final dataset shape is:

298,128 rows
24 columns

Main columns used:

airline, source_city, departure_time, stops, arrival_time
destination_city, class, duration, days_left
Target (Regression): price

Part 2 - EDA (Exploratory Data Analysis)

Goals:

Understand price distribution
Explore relationships between price and flight features
Identify key factors influencing price

Key steps performed:

Visualizing distributions (price, duration, stops, class)
Checking correlations
Detecting and removing outliers

Placeholders for EDA Graphs:

Price Distribution

Conclusion: Ticket prices are highly right-skewed: most tickets are relatively cheap, with a small number of very expensive flights forming a long right tail of the distribution.

Price by Stops

Conclusion: Flights with one or more stops tend to be much more expensive than non-stop flights, and 1-stop itineraries are the priciest on average – showing that the number of stops is strongly related to ticket price.

Price by Duration

Conclusion: Longer flights generally have higher ticket prices, but the scatter is large, so duration alone cannot explain price; other features such as class, airline and number of stops also play an important role.

Research Questions

1. How does the average ticket price vary by month? Conclusion: The average ticket price drops noticeably from February to March. This suggests a seasonal pattern where flights in March tend to be cheaper, possibly due to lower demand or post-holiday travel trends.

2. How do ticket prices vary across travel class and number of stops? Conclusion: Business-class tickets are consistently more expensive than Economy across all stop categories. Prices also increase sharply with additional stops, especially for Business class. This indicates a strong interaction between travel class and number of stops in determining ticket price.

3. What is the relationship between flight duration and ticket price for non-stop flights? Conclusion: There is a positive trend—longer non-stop flights tend to have higher ticket prices. However, the large scatter shows that duration alone cannot fully explain price variation, meaning additional factors like class, route, and airline also significantly influence ticket cost.

4. How is the price distribution different between Economy and Business class? Conclusion: Economy tickets are heavily concentrated in the low-price range, while Business-class tickets appear at much higher prices with a wider spread. This clear separation shows that travel class is one of the strongest predictors of ticket price.

These insights were later connected to model performance and feature importance.

Part 3 - Regression Models

Baseline Model — Linear Regression

Using cleaned + encoded baseline features (11 features).

Results:

Train set:

MAE ≈ 4,493
RMSE ≈ 6,880
R² ≈ 0.91

Test set:

MAE ≈ 4,470
RMSE ≈ 6,880
R² ≈ 0.91

The baseline model performs strongly, showing that the engineered features are meaningful.

Baseline Regression Plot

Conclusion: The baseline linear regression captures the general upward trend between true and predicted prices, but many points deviate from the ideal diagonal line. This indicates that the model struggles with non-linear patterns and underfits in several price ranges, especially for very cheap and very expensive tickets.

Part 4 - Feature Engineering

To improve model performance, I added several engineered features on top of the baseline numeric features.

Added Features

duration_hours – flight duration converted from minutes to hours
is_long_flight – indicator for long flights (duration ≥ 600 minutes)
is_non_stop – indicator for non-stop flights (stops == 0)
is_night_arrival – indicator for night arrivals

K-Means Features

Using K-Means (k = 4), I created:

cluster_id – assigned cluster label
cluster_distance – distance to cluster centroid
PCA 2D visualization of clusters

These features help the models capture non-linear patterns and interactions in the data.

Part 5 - Improved Regression Models + Part 6 - Winning Model

Models:

Random Forest Regressor
Gradient Boosting Regressor

Results summary:

Model	MAE	RMSE	R²
Random Forest	best	best	highest
Gradient Boosting	slightly worse	slightly worse	lower

Random Forest was the winning regression model.

Feature Importance Plot

Conclusion: The Random Forest model identifies class as by far the most important feature for predicting ticket prices. Other features such as duration_in_min, route_index, and airline_index contribute much less but still show meaningful influence. This confirms that travel class is the strongest driver of ticket pricing in the dataset.

Part 7 - Regression-to-Classification

In addition to predicting the exact ticket price, I also reframed the problem as a 3-class classification task.
The goal here is to classify each flight into a price level (cheap / medium / expensive) based on the numeric target price.

Creating the `price_class` target

To create the classification labels I transformed the continuous price variable into three classes using quantile binning on the training set only:

Class 0 – cheap tickets – bottom 33% of prices
Class 1 – medium tickets – middle 33%
Class 2 – expensive tickets – top 33%

This strategy has two advantages:

It matches an intuitive business interpretation of cheap / medium / expensive tickets.
Using quantiles keeps the classes relatively balanced, so the classifier does not focus on one dominant class.

The resulting discrete labels are stored in a new target column called price_class, which is used throughout the classification part.

Class balance

After creating the labels, I checked the class distribution in both the train and test sets.
Each of the three classes (0 – cheap, 1 – medium, 2 – expensive) contains roughly one third of the samples in both splits (≈ 32–34% per class).
This means the classes are reasonably balanced and:

Overall Accuracy is a meaningful metric.
I also track Macro F1 score to make sure the model performs well across all classes, not just one.

Part 8 - Classification Models

Using the engineered features from the regression part, I trained several classification models to predict the new target price_class (0 = cheap, 1 = medium, 2 = expensive).

Business perspective: Precision vs Recall

From a user perspective, the most important group is the cheap tickets (class 0):

I prefer the model to catch as many truly cheap flights as possible, even if some non-cheap flights are mistakenly labeled as cheap.
Therefore, for the cheap class, Recall is more important than Precision.

In terms of error types:

A False Negative (a truly cheap flight classified as not cheap) is more critical than a False Positive (a not-cheap flight classified as cheap), because the user will never see a good deal that actually exists.

Trained classification models

I trained three different classifiers on the same feature matrix (X_train_fe, X_test_fe) and the new target price_class:

Logistic Regression (multiclass baseline) – a simple linear baseline for multi-class classification.
Random Forest Classifier – an ensemble of decision trees that can capture non-linear relationships.
Gradient Boosting Classifier – another tree-based ensemble that focuses on correcting previous errors and often gives strong performance.

All models were evaluated on the same train/test split using:

classification_report (precision, recall, F1, support) for train and test sets
Confusion matrix on the test set
Overall Accuracy and Macro F1 on the test set

Evaluation of classification models

On the test set, the models achieved the following results:

Model	Accuracy (test)	Macro F1 (test)
Logistic Regression	0.794	0.795
Gradient Boosting	0.869	0.869
Random Forest	0.953	0.952

Because the classes are balanced, these metrics can be compared fairly across models.

Types of mistakes (based on confusion matrices)

Logistic Regression
- Makes many mistakes between class 0 (cheap) and class 1 (medium).
- Also confuses class 2 (expensive) with class 1, so expensive tickets are often predicted as medium.
- The confusion matrix is more “spread out”, which matches the lower accuracy/F1.
Gradient Boosting
- Performs better than Logistic Regression but worse than Random Forest.
- Still shows noticeable confusion between class 1 and 2.
- The diagonal in the confusion matrix is strong, but off-diagonal cells are larger than in Random Forest.
Random Forest
- The confusion matrix is strongly concentrated on the diagonal.
- Only a small number of tickets are misclassified, mostly between neighbouring classes (0 vs 1, 1 vs 2).
- Very few cheap tickets are predicted as expensive (0 → 2) or the opposite (2 → 0).
- This model captures the non-linear relationships in the data much better than Logistic Regression.

Winning classifier

The Random Forest Classifier is the winning model for the classification task because:

It has the highest Accuracy and Macro F1 on the test set.
It makes relatively few “severe” mistakes between cheap and expensive tickets.

Files Included

Loom Video - project walkthrough presentation
Freshly_cleaned.csv - dataset used for the analysis
Assignment#2-Classification,Regression,Clustering,Evaluation-Shira Bash.ipynb – full Python notebook
README.md – summary of results, questions, and insights
flight_price_random_forest_classifier.pkl - 1 pickle model for classification
flight_price_random_forest.pkl - 1 pickle model for regression

Final Notes

This project demonstrates the full ML lifecycle:

Data understanding
Cleaning & preprocessing
Feature engineering
Regression
Classification
Clustering
Model evaluation
Production-ready file export

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support