Categorical encoding using Label-Encoding and One-Hot-Encoder
By: Dinesh Yadav
Tags: Data Science, Scikit Learn, Label Encoding, Machine Learning Ai

In many Machine-learning or Data Science activities, the data set might contain text or categorical values (basically non-numerical values). For example, color feature having values like red, orange, blue, white etc. Meal plan having values like breakfast, lunch, snacks, dinner, tea etc. Few algorithms such as CATBOAST, decision-trees can handle categorical values very well but most of the algorithms expect numerical values to achieve state-of-the-art results.

Over your learning curve in AI and Machine Learning, one thing you would notice that most of the algorithms work better with numerical inputs. Therefore, the main challenge faced by an analyst is to convert text/categorical data into numerical data and still make an algorithm/model to make sense out of it. Neural networks, which is a base of deep-learning, expects input values to be numerical.

There are many ways to convert categorical values into numerical values. Each approach has its own trade-offs and impact on the feature set. Hereby, I would focus on 2 main methods: One-Hot-Encoding and Label-Encoder. Both of these encoders are part of SciKit-learn library (one of the most widely used Python library) and are used to convert text or categorical data into numerical data which the model expects and perform better with.

Code snippets in this article would be of Python since I am more comfortable with Python. If you need for R (another widely used Machine-Learning language) then say so in comments.

Label Encoding

This approach is very simple and it involves converting each value in a column to a number. Consider a dataset of bridges having a column names bridge-types having below values. Though there will be many more columns in the dataset, to understand label-encoding, we will focus on one categorical column only.

BRIDGE-TYPE

Arch

Beam

Truss

Cantilever

Tied Arch

Suspension

Cable

We choose to encode the text values by putting a running sequence for each text values like below:

With this, we completed the label-encoding of variable bridge-type. That’s all label encoding is about. But depending upon the data values and type of data, label encoding induces a new problem since it uses number sequencing. The problem using the number is that they introduce relation/comparison between them. Apparently, there is no relation between various bridge type, but when looking at the number, one might think that ‘Cable’ bridge type has higher precedence over ‘Arch’ bridge type. The algorithm might misunderstand that data has some kind of hierarchy/order 0 < 1 < 2 … < 6 and might give 6X more weight to ‘Cable’ in calculation then than ‘Arch’ bridge type.

Let’s consider another column named ‘Safety Level’. Performing label encoding of this column also induces order/precedence in number, but in the right way. Here the numerical order does not look out-of-box and it makes sense if the algorithm interprets safety order 0 < 1 < 2 < 3 < 4 i.e. none < low < medium < high < very high.

Label Encoding in Python

Using category codes approach:

This approach requires the category column to be of ‘category’ datatype. By default, a non-numerical column is of ‘object’ type. So you might have to change type to ‘category’ before using this approach.

# import required libraries

import pandas as pd

import numpy as np # creating initial dataframe

bridge_types = ('Arch','Beam','Truss','Cantilever','Tied Arch','Suspension','Cable')

bridge_df = pd.DataFrame(bridge_types, columns=['Bridge_Types']) # converting type of columns to 'category'

bridge_df['Bridge_Types'] = bridge_df['Bridge_Types'].astype('category') # Assigning numerical values and storing in another column

bridge_df['Bridge_Types_Cat'] = bridge_df['Bridge_Types'].cat.codes

bridge_df

Using sci-kit learn library approach:

Another common approach which many data analyst perform label-encoding is by using SciKit learn library.

import pandas as pd

import numpy as np

from sklearn.preprocessing import LabelEncoder # creating initial dataframe

bridge_types = ('Arch','Beam','Truss','Cantilever','Tied Arch','Suspension','Cable')

bridge_df = pd.DataFrame(bridge_types, columns=['Bridge_Types']) # creating instance of labelencoder

labelencoder = LabelEncoder() # Assigning numerical values and storing in another column

bridge_df['Bridge_Types_Cat'] = labelencoder.fit_transform(bridge_df['Bridge_Types'])

bridge_df