Flight Price Prediction
By: Deepam Purkayastha
Tags: Flight Tickets, Data Science, Machine Learning, Predictive Analytics

Build predictive models to automate the process of predicting the price of the flight ticket for the desired airline.

Table of Contents:

1. Problem Definition.

2. Data Analysis.

3. EDA Concluding Remark.

4. Pre-Processing Pipeline.

5. Building Machine Learning Models.

6. Concluding Remarks.

Problem Definition

Flight ticket prices can be something hard to guess, today we might see a price, check out the price of the same flight tomorrow, it will be a different story. We might have often heard travelers saying that flight ticket prices are so unpredictable.

Objective:

Here, with the help of the prices of flight tickets for various airlines between the months of March and June of 2019 and between various cities, we would be able to predict flight prices based on given parameters.

Type of problem:

The above problem statement clearly explains that the target variable is continuous & it’s a regression problem as we need to predict the price of the flight tickets. So this can be solved by any of the below Regression Machine learning algorithms:

1. Linear Regression.

2. Decision Tree Regressor.

3. Random Forest Regressor.

I have mentioned only a few. We would be performing each of the

techniques later in this blog.

Importing libraries:

Data Analysis

Data Set Description:

There are 2 data sets that are given. One is training data and one is testing data.

1) Train file will be used for training the model, i.e. the model will learn from this file. It contains all the independent variables and the target variable. Size of training set: 10683 records.

2) Test file contains all the independent variables, but not the target variable. We will apply the model to predict the target variable for the test data. Size of test set: 2671 records.

Read the excel file and convert it into a data frame:

DATA SET

Airline: Name of the airline used for traveling

Date_of_Journey: Date at which a person traveled

Source: Starting location of flight

Destination: Ending location of flight

Route: This contains information on starting and ending location of the journey in the standard format used by airlines.

Dep_Time: Departure time of flight from starting location

Arrival_Time: Arrival time of flight at destination

Duration: Duration of flight in hours/minutes

Total_Stops: Number of total stops flight took before landing at the destination.

Additional_Info: Shown any additional information about a flight

Price: Price of the flight.

We have 10683 rows & 11 columns in the dataset.

We have maximum columns as categorical, which is an independent feature.

Handling Missing values:

Out of all these columns, we can see 1 missing value for Route & Total Stops column. As we have a huge dataset & out of which only 1 value is missing, rather than filling it, dropping would be the better option.

After removing the null values, we have 10682 rows & 11 columns.

Let’s check for each independent feature & convert the data type into the required Data type:

Training data set is having all independent features as categorical and the only target variable is numerical, also we can see some special character also being used because of that we need to perform data transformation on it before building the model.

Date_of_Journey: it is given as day-month-year, so we need to use DateTime to convert it & separate it with day & month.

DateTime Code

2) Similarly, we can separate the hour & mins for Dep_Time & Arrival Time.

After dropping unwanted columns as it is of no use now: Arrival Time, Dep_Time & Date_of_Journey, we have:

3) For the duration column, if we see above in the dataset, it is basically the difference between arrival & departure time & it is given as string data type. If we could split then it will be again in string format. So we need to split it & add 0 hours & mins.

EDA Concluding Remark

Handling categorical columns:

So, we have Airline, Source, Destination, Route, Total Stops & Additional Info as categorical columns. Let’s check the relation for each feature with Price.

Airline Vs Price

Jet Airways Business is selling tickets with maximum fare followed by jet airways & multiple careers & least is giving Trujet.

2) Source vs Price

The maximum fare of the flights is departing from Bangalore followed by Delhi.

3) Destination Vs Price

New Delhi is showing the highest average price in terms of destination.

Once we combine New Delhi & Delhi as both are in the same destination city, we got cochin as the highest average price.

4) Total Stops(Airlines) Vs Price

Jet Airways Business is showing the highest price having a single stop followed by 2 stops.

Action Taken based on observation:

We have seen how categorical data is distributed & correlated with the target variable. Also, I have mentioned the findings below each plot.

Dropped columns Route & Additional info as the column ‘Route’ is related to total stops & column ‘additional info’ is showing no information for a maximum number of rows.

Performed One Hot Encoding for Nominal categorical columns: Airline, Source, Destination & performed label encoding for an ordinal categorical column: Total Stops.

No need to check for outliers/skewness as all the independent features are categorical.

Visualizing the correlation in a heat map to check if there is any coefficient of multicollinearity

1)Total stops are more correlated with the target variable.

2)Multicollinearity: total stops & destination hours.

Pre-Processing Pipeline

Divide the data set into test and train

Now that we have converted all the categorical columns into numerical using the encoding technique. The next step is to split the data into test and train and drop the price column from the data set as we need to predict the price.

2) Finding the best Random_ state: We have used Random Forest Regressor to obtain the best random state.

3) Final dimension of data: 10682 rows & 29 columns.

4) Created train test split: We have split the train & test data in 0.2 test size with the best random state.

Building Machine Learning Models

Model Creation:

Now, after performing the train test split, we have x_train, x_test, y_train & y_test, which are required to build Machine learning models. We would build multiple regressor models to get the best R2(coefficient of determination) regression score function out of all the models.

Linear Regression: It is a linear model, e.g. a model that assumes a linear relationship between the input variables (x) and the single output variable (y). More specifically, that y can be calculated from a linear combination of the input variables (x).

2) Decision Tree Regression: Decision Regression trees are needed when the response variable is numeric or continuous. It breaks down a dataset into smaller and smaller subsets while at the same time an associated decision tree is incrementally developed. The final result is a tree with decision nodes and leaf nodes.

3) Random Forest Regressor: It uses the ensemble learning method for regression. The ensemble learning method is a technique that combines predictions from multiple machine learning algorithms to make a more accurate prediction based on majority voting/averaging.

4) AdaBoost Regressor: It is used to boost the performance of any machine learning algorithm. It helps you combine multiple “weak classifiers” into a single “strong classifier”.

5) Bagging Regressor: It is an ensemble meta-estimator that fits base regressors each on random subsets of the original dataset and then aggregates their individual predictions (either by voting or by averaging) to form a final prediction.

6) Gradient Boosting Regressor: It produces a prediction model in the form of an ensemble of weak prediction models, typically decision trees.

1) Observation:

For every model, we have train the data (x_train, y_train) & predict with the help of x_test. Now, with the help of the y_test & prediction test we got the R2 score. So, based on every model we have the best R2 score for Random Forest Regressor with an 84% score, however, this could be because of overfitting. Hence we need to perform cross-validation to check if the model is overfitted or not.

Cross-Validation:

We would perform cross-validation for every model & compare it with the R2 score, whichever model gives less difference between the Cross-validation score & R2 score is the best fit model.

As we have seen, the Ada boost regressor is giving 0 difference but it is having a very low R2 Score. Random Forest Regressor is giving best R2 score with less difference of R2 Score & CV Score, however, let's proceed with model evaluation to get the MSE(mean squared error), RMSE(root mean squared error), & MAE( mean absolute error). These evaluation metrics will help to decide the best model.

Model Evaluation:

Random Forest Regressor is giving the best result as compare to other models. Let's check its test & train score.

Train Score:

Test Score:

As we have seen, we have less difference of train & test score, and the predicted value & test value is normally distributed, Also, in the scatter plot the test value & the prediction value is linearly distributed. The test & prediction values are almost close to each other.

Since Random Forest Regressor is the best model in terms of model score, cross-validation difference, test & train r2 score difference, also as per the evaluation metrics, we choose Random Forest Regressor to be the final model. Let's see if we can increase the score by using hyper-parameter tuning.

Hyper Parameter Tuning:

We would try to increase the model by giving the best parameters (params as seen below) of Random Forest Regressor & again train to get an increased R2 score.

Prediction:

As we can see, Score has increased to 87%.

After performing Hyper Parameter tuning, we have increased the Score, evaluation metrics are showing better results, also test & prediction value is normally distributed. The Test & Prediction value is also linearly related to each other.

Let’s predict & compare the results:

Concluding Remarks

1) Saving the model: The model is ready & we have saved the model in ‘pkl’ format by using “joblib”.

2) Test data: It is having 2672 rows & 10 columns.

3) We need to perform all the same steps as we performed for the training dataset, which includes Data analysis, EDA, & Pre-processing. The test dataset is having all the same columns except the target variable.

4) We don’t need to build any model using the test dataset. Hence no need to perform a train test split, only cleaning the data is required.

5) Once the test dataset is ready, we can load the train data set & predict the price.

6) Comparing the actual target value from the training dataset & predicted test value.

Final conclusion:

As we have seen, the prediction is showing a similar relationship with the actual price from the train data set, which means the model predicted correctly & this could help airlines by predicting what prices they can maintain. It could also help customers to predict future flight prices and plan the journey accordingly because it is difficult for airlines to maintain prices since it changes dynamically due to different conditions. Hence by using Machine learning techniques we can solve this problem.