Mercedes Benz Greener Manufacturing
Tags: Kaggle, Mercedes Benz

Contents:

Business Problem Problem Statement Source of data About the dataset Machine learning problem formulation Existing solutions First Cut Approach Understanding the dataset Perform Label Encoding of categorical features Select 5 best features using Recursive Feature Elimination Univariate Analysis of all features Bivariate analysis of categorical features and Important features Featurization of the dataset Selecting Important features from the dataset Training a baseline model Training complex models Stacking Ensemble Model Model Comparisons Kaggle Scores Future Works References

1. Business Problem:

Since the first automobile, the Benz Patent Motor Car in 1886, Mercedes-Benz has stood for important automotive innovations. These include, for example, the passenger safety cell with crumple zone, the airbag, and intelligent assistance systems. Mercedes-Benz applies for nearly 2000 patents per year, making the brand the European leader among premium carmakers. Daimler’s Mercedes-Benz cars are leaders in the premium car industry. With a huge selection of features and options, customers can choose the customized Mercedes-Benz of their dreams.

To ensure the safety and reliability of each and every unique car configuration before they hit the road, Daimler’s engineers have developed a robust testing system. But, optimizing the speed of their testing system for so many possible feature combinations is complex and time-consuming without a powerful algorithmic approach. As one of the world’s biggest manufacturers of premium cars, safety and efficiency are paramount on Daimler’s production lines.

In this competition, Daimler is challenging Kagglers to tackle the curse of dimensionality and reduce the time that cars spend on the test bench. Competitors will work with a dataset representing different permutations of Mercedes-Benz car features to predict the time it takes to pass testing. Winning algorithms will contribute to speedier testing, resulting in lower carbon dioxide emissions without reducing Daimler’s standards.

2. Problem Statement:

In this problem, we have to predict the time spent by each Mercedes-Benz cars on the test bench. This will help Mercedes-Benz improve the time taken for testing and therefore reducing the carbon dioxide emissions without reducing their standards.

3. Source of Data:

The dataset is provided by Daimler for the Kaggle competition, and this can be downloaded from here,

4. About the Dataset:

This dataset contains an anonymized set of variables, each representing a custom feature in a Mercedes car. For example, a variable could be 4WD, added air suspension, or a head-up display.

The ground truth is labeled ‘y’ and represents the time (in seconds) that the car took to pass testing for each variable.

There are 378 features in the training dataset (including the y variable) and 4209 rows and the test dataset contains 4209 rows and 377 columns.

5. Machine Learning Problem Formulation:

In this problem we have to predict the time taken in seconds by each car to pass testing and testing is based on 377 different features and the predicted variable y is the time taken in seconds which is continuous so this problem can be considered as a regression problem.

Performance Metric:

The performance metric used for this problem is R2 score (Coefficient of Determination). It is the proportion of the variance in the dependent variable that is predictable from the independent variable.

6. Existing Solutions:

This problem is a Kaggle problem hence there are multiple different solutions in the notebooks section of this problem in Kaggle.

7. First Cut Approach:

Understanding the dataset. Perform Label Encoding of categorical features. Select 5 best features using Recursive Feature Elimination. Univariate Analysis of all features. Bivariate analysis of categorical features and Important features. Featurization of the dataset. Selecting Important features from the dataset. Training a baseline model. Training complex models. Stacking Ensemble Model.

8. Understanding the dataset:

Loading the CSV file using pandas and storing the dataset as a pandas dataframe and printing the head of the dataset to visualize the data.

Checking for missing values in the dataset we find that there are no missing data points in our dataset. Then we find the counts of the datatypes of the features present in the dataset.

From the above snippet of code, we can see that there are 369 features which are of int64 data type and by further processing, we find that all of these columns are binary features, there are 8 features with object datatype which means that there are 8 categorical features and 1 feature which is float64 data type, which is the target variable feature.

After this printing, the count and the feature names of the binary features based on the values they contain.

From the above result, we can see that there are 356 features which contain both 0’s and 1’s as their values and 12 features which contain only 0’s as their value, these features will not add anything to our model so we will remove these features in the further steps.

9. Performing Label Encoding of categorical features:

Label encoding is the process of converting the categorical features into integer values so that they can be easily interpreted by our models.

We are using scikit-learn LabelEncoder to encode our categorical features and storing these features into pickle files for future use.

10. Selecting the Top 5 features using Recursive Feature Elimination:

Recursive feature elimination is the process of recursively selecting smaller and smaller sets of features. First, the estimator is trained on a set of features and the top features from training that estimator using coef_ or feature_importances_ attribute and the least important features are pruned from the set of features. This process is repeated until only the desired amount of features are present in the feature set. Here XGBRegressor is used as the estimator.

As we can see from the results the top 5 features selected by Recursive feature elimination are all binary features.

11.1 Univariate Analysis of Categorical features:

Plotting a count plot of all the categorical features to understand the distribution of all the categories in the features.

Count plot of X0

This plot represents the count of each of the categories in the categorical variable X0. From the plot we can see that all the categories are not equally distributed in X0 variable, categories z and al occur the maximum number of times around 350 times when compared to other categories and categories like g, ab, ac and ao occur the least number of times less than 10 number of times in the entire train data

Count plot of X1

This plot represents the count of each of the categories in the categorical variable X1. As we can see from the plot the categories are not equally distributed over the train data. We can see that category aa occurs the most number of times more than 800 times in the train dataset and categories like g, q, ab, and d occur the least number of times.

Count plot of X2

This plot represents the count of each of the categories in the categorical variable X2. As we can see from the plot the categories are not equally distributed in the train dataset. We can see that category of ‘as’ occurs the most number of times almost 1600 times throughout the dataset and more than 60% of the other categories occur very negligible amount of times as the bar are very small we can assume that these variables occur less than 5 times in the entire dataset.

Count plot of X3

This plot represents the count of each of the categories in the categorical variable X3. As we can see from the plot the categories are not equally distributed throughout the dataset. The category ‘c’ occurs almost 2000 times in the entire dataset and categories like ‘b’ occur around 70 times and category ‘e’ occur around 180 times in the dataset.

Count plot of X4

This plot represents the count of each of the categories in the categorical variable X4. As we can see from the plot one category ‘d’ occurs more than 4000 times in the dataset and other categories ‘b’,’c’ and ‘a’ occur only very very few times.

Count plot of X5

This plot represents the count of each of the categories in the categorical variable X5. As we can see from the plot most of the categories have almost equal counts in the dataset, around 200 times in the dataset. There are certain categories like ‘u’, ‘y’,’x’, ‘h’, ‘g’ which occur very few times around 5 times in the entire dataset.

Count plot of X6

This plot represents the count of each of the categories in the categorical variable X6. From the plot we can see that categories like ‘g’ and ‘j’ occur around 1000 times in the dataset and categories like ‘l’, ‘d’ and ‘i’ occur around 500–600 times in the dataset and categories like ‘c’, ‘k’, ‘e’, ‘f’ and ‘b’ occur very few times in the dataset around 10–50 times.

Count plot of X8

This plot represents the count of each of the categories in the categorical variable X8. From the plot, we can see that each of the variables is almost equally distributed and none of the categories occur very few times in the dataset. We can see that category ‘j’ occurs the most number of times more than 250 times and whereas other variables are also present in and the least count of each category is 100.

11.2 Univariate Analysis of Binary features:

Plotting the count of binary features to understand the number of 0’s and 1’s in the binary features in the dataset.

From the plot, there are certain variables that have an equal distribution of 0’s and 1’s. There are certain variables that contain only 0’s value which is represented by the entire bar in ‘red’ color. There are certain categories which have only very few 0 or 1 value which is presented by either ‘red’ or ‘blue’ color bar.

11.3 Univariate Analysis of target variable:

Plotting a distribution plot of the target variable and from the plot, we can see that there are very few values that are above 150 and most of the values lie in a range below 120. The y variable is very skewed we will overcome this problem in the feature engineering section.

12.1 Bivariate Analysis of Categorical Features:

Plotting a boxplot of all the X0 categorical feature with target variable we can see how each category is distributed with y values.

The categorical value ‘az’ is having the least values for y around 75 with a certain exception represented by the dots. The categorical variable ‘bc’ also has y value around 75 but the spread is much less than ‘az’. The categorical values ‘ac’, ‘g’ and ‘ab’ have a constant value for y around 85, 110, and 115, ‘aa’ categorical variable is having the highest range around 140 to 160.

Similarly, we can plot for all the categorical features and understand the relationship between categories and the target variable.

12.2 Bivariate Analysis of Important Features:

Plotting a violin plot comparing 2 important features together at a time and plotting the target variable as bins so that with the color of the violin plots we can find the range of target variable y.