Choosing the right Encoding method-Label vs OneHot Encoder | by Rahil Shaikh | Towards Data Science

In ML models we are often required to convert the categorical i.e text features to its numeric representation. The two most common ways to do this is to use Label Encoder or OneHot Encoder. However, most of the ML newbies are not familiar with the impact of the choice of encoding has on their model, the accuracy of the model may shift by large numbers by using the right encoding at the right scenario.
Understanding Label and OneHot Encoding.Practical Example with Python on the impact of using Label and One hot encoding on prediction accuracy.
Understanding Label and OneHot Encoding.
Practical Example with Python on the impact of using Label and One hot encoding on prediction accuracy.
Let us understand the working of Label and One hot encoder and further, we will see how to use these encoders in python and see their impact on predictions
Label Encoding in Python can be achieved using Sklearn Library. Sklearn provides a very efficient tool for encoding the levels of categorical features into numeric values. LabelEncoder encode labels with a value between 0 and n_classes-1 where n is the number of distinct labels. If a label repeats it assigns the same value to as assigned earlier.
Consider below example:
If we have to pass this data to the model we need to encode the Country column to its numeric representation by using Label Encoder. After applying Label Encoder we will get a result as seen below
That’s all label encoding is about. But depending on the data, label encoding introduces a new problem. For example, we have encoded a set of country names into numerical data. This is actually categorical data and there is no relation, of any kind, between the rows.
The problem here is since there are different numbers in the same column, the model will misunderstand the data to be in some kind of order, 0 < 1 <2.
The model may derive a correlation like as the country number increases the population increases but this clearly may not be the scenario in some other data or the prediction set. To overcome this problem, we use One Hot Encoder.
Now, as we already discussed, depending on the data we have, we might run into situations where, after label encoding, we might confuse our model into thinking that a column has data with some kind of order or hierarchy when we clearly don’t have it. To avoid this, we ‘OneHotEncode’ that column.
What one hot encoding does is, it takes a column which has categorical data, which has been label encoded and then splits the column into multiple columns. The numbers are replaced by 1s and 0s, depending on which column has what value. In our example, we’ll get four new columns, one for each country — Japan, U.S, India, and China.
For rows which have the first column value as Japan, the ‘Japan’ column will have a ‘1’ and the other three columns will have ‘0’s. Similarly, for rows which have the first column value as the U.S, the ‘U.s’ column will have a ‘1’ and the other three columns will have ‘0’s and so on.
We will be using Medical Cost Personal Datasets to predict the automobile medical charges based on various features you can download the dataset from here. The data used for this blog is just a sample data, but the difference in the prediction results is clearly seen by evaluating the prediction results by the root mean square error, closer the rmse to 0 better the model predictions are.
We will run the xgboost regression algorithm model (you can use any regression algorithm of your choice) and predict the price using Label encoder and then by using One Hot encoder and compare the results.
I have taken out last 37 records from the insurance.csv file and made a new file insuranceTest.csv to predict the charge on unseen data.
Code Snippet:
import pandas as pdimport numpy as npimport xgboostfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import LabelEncoderfrom math import sqrtfrom sklearn.metrics import mean_squared_errordata = pd.read_csv('D://Blogs//insurance.csv')testdata = pd.read_csv('D://Blogs//insuranceTest.csv')mergedata = data.append(testdata)testcount = len(testdata)count = len(mergedata)-testcountX_cat = mergedata.copy()X_cat = mergedata.select_dtypes(include=['object'])X_enc = X_cat.copy()#ONEHOT ENCODING BLOCK#X_enc = pd.get_dummies(X_enc, columns=['sex','region','smoker'])#mergedata = mergedata.drop(['sex','region','smoker'],axis=1)#END ENCODING BLOCK# =============================================================================# #LABEL ENCODING BLOCK# X_enc = X_enc.apply(LabelEncoder().fit_transform) #mergedata = mergedata.drop(X_cat.columns, axis=1)# #END LABEL ENCODING BLOCK# # =============================================================================FinalData = pd.concat([mergedata,X_enc], axis=1)train = FinalData[:count]test = FinalData[count:]trainy = train['charges'].astype('int')trainx = train.drop(['charges'], axis=1)test = test.drop(['charges'], axis=1)X_train,X_test, y_train,y_test = train_test_split(trainx, trainy, test_size=0.3)clf = xgboost.XGBRegressor()clf.fit(X_train,y_train)y_testpred= clf.predict(X_test)y_pred = clf.predict(test)dftestpred = pd.DataFrame(y_testpred)dfpred = pd.DataFrame(y_pred)rms = sqrt(mean_squared_error(y_test, y_testpred))print("RMSE:", rms)
RMSE output: RMSE: 4931.632574106283
Now let's uncomment the One Hot encoder block from our snippet and comment Label Encoder block see the RMSE
import pandas as pdimport numpy as npimport xgboostfrom sklearn.model_selection import train_test_splitfrom sklearn.metrics import accuracy_score, confusion_matrixfrom sklearn.preprocessing import LabelEncoderfrom math import sqrtfrom sklearn.metrics import mean_squared_errordata = pd.read_csv('D://Blogs//insurance.csv')testdata = pd.read_csv('D://Blogs//insuranceTest.csv')mergedata = data.append(testdata)testcount = len(testdata)count = len(mergedata)-testcountX_cat = mergedata.copy()X_cat = mergedata.select_dtypes(include=['object'])X_enc = X_cat.copy()#ONEHOT ENCODING BLOCKX_enc = pd.get_dummies(X_enc, columns=['sex','region','smoker'])mergedata = mergedata.drop(['sex','region','smoker'],axis=1)#END ENCODING BLOCK# =============================================================================# #LABEL ENCODING BLOCK# #X_enc = X_enc.apply(LabelEncoder().fit_transform) ##mergedata = mergedata.drop(X_cat.columns, axis=1)# #END LABEL ENCODING BLOCK# # =============================================================================FinalData = pd.concat([mergedata,X_enc], axis=1)train = FinalData[:count]test = FinalData[count:]trainy = train['charges'].astype('int')trainx = train.drop(['charges'], axis=1)test = test.drop(['charges'], axis=1)X_train,X_test, y_train,y_test = train_test_split(trainx, trainy, test_size=0.3)clf = xgboost.XGBRegressor()clf.fit(X_train,y_train)y_testpred= clf.predict(X_test)y_pred = clf.predict(test)dftestpred = pd.DataFrame(y_testpred)dfpred = pd.DataFrame(y_pred)rms = sqrt(mean_squared_error(y_test, y_testpred))print("RMSE:", rms)
RMSE output: RMSE: 4793.667005276121
RMSE of One Hot Encoder is less than Label Encoder which means using One Hot encoder has given better accuracy as we know closer the RMSE to 0 better the accuracy, again don't be worried for such a large RMSE as I said this is just a sample data which has helped us to understand the impact of Label and OneHot encoder on our model.
If you found this article useful give it a clap and share it with your friends.