Predicting Car Price: EDA, Regression, Hypothesis Testing
By: Shubham Pundir
Tags: Machine Learning, Data Science, Towards Data Science, Data Visualization, Business Intelligence

Predicting Description:

I am predicting the selling price of the car based on various features of the cars, including the present price of the cars. I will be using Multiple Linear Regression for building The model.

Data Description:

The dataset is from Cardekho.com, this dataset contains information about cars listed on the website cardekho.com

Source:

Kaggle Data Science Community

Attribute Information:

The datasets consist of several independent variables include:

Car_Name : This column represents the name of the car. Year : This column represents the year in which the car was bought. Selling_Price : This column represents the price the owner wants to sell the car at. Present_Price : This is the present price of the car. Kms_Driven : This is the distance completed by the car in km. Fuel_Type : Fuel type of car i.e Diesel,Petrol,CNG Seller_Type : Defines whether the seller is a dealer or an individual. Transmission : Defines whether the car is manual or automatic. Owner : Defines the number of owners the car has previously had

Exploratory Data Analysis:

Here we can see that:

Most of the cars owned by the people are of Petrol Type.

People are buying more petrol Cars then Diesel and CNG.

Most of the seller type is Dealers only.

But we can see that Individual sellers are coming up with the help of the online marketplace.

Most of the cars are owned by the first owner.

Second-hand cars are less in number.

In most of the year, the Present price of the Car with the 1st owner is high.

When the car is sold at a price the high selling price is with the first car owners rather than the second and third.

The first owners have driven fewer km on average.

It’s Better to buy a car from the first owner.

The dealers are having/selling the cars which are on average driven 40 Thousand km driven which is a lot.

whereas the Individuals who are having/selling the cars which are driven 30 thousand km driven just 10000 less the dealers.

It’s better to buy a car from an Individual if you want less km driven cars.

Automatic cars are most costly than Manual cars.

The dealers are selling the cars at a higher price, as above seen they have the cars which have higher km driven in that case price should be less.

Individuals are selling their cars at a lower price despite being less km driven than seen above.

U nderstanding The Variables:

Let’s dive under to understand the variables and use the correlation matrix to make the process easy.

Correlation Matrix:

The selling price and Present price have a correlation of 0.88

The Selling Price is highly correlated with the Present Price, so we can use any one of the Variables for our analysis but not both.

Outliers:

Now let’s check if we have Outliers in our data.

The Kms_Driven shows that many of the data is Acting like an outlier.

But with the Help of domain Knowledge, we can say that the Cars can we any Kms Driven as it is based on the Owner and the Selling price of that cars Adjust accordingly.

So Rather then removing the outliers values we would like to take log of them.

We have successfully aligned the data and now it’s not acting like outliers.

We have saved valuable information of km Driven.

Hypothesis Testing:

Before Running into the Modeling Part we would do some Hypothesis Testing.

Research Question:

Throughout all the past year Cardekho.com is in the market the average price of the selling cars remained 5.5 lakhs.Cardekho.com claims that the price has not been changed and its the same average selling price.

Population: All the data Cardekho.com has until the present year.

Now we will frame the null and alternate hypothesis as;

Null Hypothesis: The average selling price of a car is 5.5 lakhs.

H0 : myu(u) = 5.5

Alternate Hypothesis: The average selling price of the cars has dropped below and is 4.6 now.

H1: myu(u) != 5.5. # != denotes not equal.

Data: we have the sample data from the Cardekho.com over the various years and will be doing hypothesis testing on that.

so we will perform the t-test(two tail) as we have the sample standard deviation, and will take the confidence level of 95% which gives the alpa=0.05.

P-value (0.05)>(0.0449) so we can conclude that we have sufficient evidence to reject the null hypothesis(H0), and accept the alternate hypothesis(H1).

It means that the average selling price of the car is not 5.5.

Here the type 1 error is 0.05 we kept is as low as possible because type one error is critical for us in the hypothesis.

Research question:

Cardekho.com Being in the business of selling cars says that most of the cars which we buy get sold for just some price high so the average price overall of present price and selling price comes the same at last.

Population: All the data Cardekho.com has till present.

Now we will frame the Null and Alternate Hypothesis.

Null Hypothesis: The average selling and present price is the same.

H0 : myu(u1) = myu(u2)

Alternate Hypothesis: The average selling and present price is not the same.

H1 : myu(u1) != myu(u2) # != means not equal.

Data: we have the sample data of the selling and present price over various years of cardekho.com.

so here we will perform the Annova: One factor as here we need to see the difference between the two means of different features.

Hands down to excel for one click, in this case, so we will do this analysis there and will understand the output for our analysis.

The above, we can see that we have the:

F(26.35) > F crit(3.85)

We have sufficient evidence to reject the Null Hypothesis(H0) and accept the Alternate Hypothesis(H1).

So, the conclusion is the average selling price and average present price of the cars are not the same they are different.

Regression:

Regression analysis consists of a set of machine learning methods that allow us to predict a continuous outcome variable (y) based on the value of one or multiple predictor variables (x). Briefly, the goal of the regression model is to build a mathematical equation that defines y as a function of the x variables.

Feature Scaling:

We will apply StandardScaler as StandardScaler assumes your data is normally distributed within each feature and will scale them so that the distribution is now centered around 0, with a standard deviation of 1.

Now we will do the regression with the Help of Ols from the stats model library:

Here we can see that:

We are achieving the R_squared of 0.918 which is very good.

We need to have a look at Adjusted R_sqr and P-value as it will help us in tunning the model further.

Now we will further tune the model:

Let us remove the Owner variable as it is more than our p-Value i.e.(0.05) acting like insignificant. We also checked our Adj R_sqr should not drop.

Result:

Finally, we have the best fit model with all significant Independent values and a good score of Adj R2 91.5 %.

Conclusion:

We have used regression analysis and have predicted the selling price of the car based on various features of the cars, including the present price of the cars.