Ordinal Encoding Tips | Towards Data Science

I was working with a team relatively new to business/data analytics when the group discussions on feature engineering for machine learning came up. Understandably, it may be confusing and intimidating for people new to machine learning. Working in Jupyter Notebooks and Python, we naturally would refer to 1) the built-in documentation within packages such as Pandas and Sklearn, or 2) online, in-depth technical documentation for these packages. Interestingly, the documentation does not detail the method to resolve the specific issue that the team faced with the dataset — feature engineering of multiple ordinal variables, each with unique ordering sequences.
🔆 Code implementation is arranged after this section.
❓ What are Categorical variables?
📔 Categorical variables are arranged by groups with corresponding value ranges (i.e. group labels). Variables without inherent hierarchy are Nominal variables; group labels are arbitrary. Variables with an ordered sequence are Ordinal variables; group labels are in ascending/ descending order.
❓ How is this related to feature engineering?
📔 For many algorithms (machine learning models), the input must be numerical. Hence, categorical variables need to be converted into numerical values. The code implementation below illustrates some feature engineering techniques and steps to avoid potential pitfalls.
# Begin by importing the librariesimport pandas as pdimport numpy as npfrom sklearn.preprocessing import OrdinalEncoderfrom sklearn.preprocessing import LabelEncoder# Create a dataframe with artifical values# Salary Grade, G to L, G ranked highest# Sector Ranking, SR1 to SR9, SR1 ranked highest# retentionRisks, Low, Med, HighGrades = ['K', 'I', 'G', 'L', 'H', 'G', 'H', 'I', 'K'] # mixed to see effectSRs = ['SR1', 'SR9', 'SR2', 'SR8', 'SR3', 'SR7', 'SR4', 'SR6', 'SR5'] # mixed to see effectretentionRisks = ['Low', 'High', 'Low', 'High', 'Low', 'Med', 'Low', 'High', 'Med']Ex = pd.DataFrame({'grades':Grades,                     'ranks':SRs,                     'retnRisks':retentionRisks                    })Ex
The example dataset comprises three columns (grades, ranks and, retention risk — abbreviated as ‘retnRisks’). All are deliberately categorical for illustrative purposes. Let’s assume the retention risks as the target variable (variable for machine learning classification output).
# Split the datasetX_Ex = Ex.loc[:,:'ranks']y_Ex = Ex['retnRisks']
📓 Some might wonder about the upper and lower case differences here. Generally, it is a way to note the differences between the predictor variables (X_Ex) and target variable (y_Ex). The target variable is a Pandas series. Let’s start with the target variable.
# Let's look at the target variable first# Instantiate the label encoderlabel_encoder = LabelEncoder()# Assign the encoded values to y_ExTy_ExT = label_encoder.fit_transform(y_Ex)# Code block to review encodingy_Exunique = np.unique(y_Ex)y_ExTunique = np.unique(y_ExT)y_encode = dict(zip(y_Exunique, y_ExTunique))print(y_encode)
The encoding is not in the order desired, regardless of ascending or descending order. For the above, the ordering sequence is alphabetical. Ideally, we would want something per High(2) > Med(1) > Low(0). We can do this via Pandas.
# Define a dictionary for encoding target variableenc_dict = {'Low':0,            'Med':1,            'High':2}# Create the mapped values in a new columnEx['target_ordinal'] = Ex['retnRisks'].map(enc_dict)# Review datasetEx
Next, the predictor variables.
# Instantiate ordinal encoderordinal_encoder = OrdinalEncoder()# Assign the encoded values to X_ExTX_ExT = ordinal_encoder.fit_transform(X_Ex)# define function to return encoded values for categorical variable valuesdef enc(array, frame):    for idx in range(array.shape[1]):        X_Exunique =sorted(frame.loc[:,frame.columns[idx]].unique())        X_ExTunique = np.unique(array[:,idx])        encode = dict(zip(X_Exunique, X_ExTunique))        print(encode)# Check encoding using the funcenc(X_ExT, X_Ex)
The encoded sequence is fine except that in the context of the dataset, ‘G’ is ranked higher than ‘L’ for grades. Similar applies for ranks. We need to reverse the order of sequence. We can do that as follows:
# Create lists to hold the order sequence needed# for gradesg = sorted(Ex['grades'].unique(),reverse=True)# for ranksr = sorted(Ex['ranks'].unique(),reverse=True)
Sklearn’s Ordinal encoder takes in a parameter, categories.
‘auto’ — implies the alphabetical ordering
‘list’ — refers to the two lists with our desired sequence. The order in which they are passing into the encoder have to correspond with the order of the variables in the dataset.
# Pass in the correctly-ordered sequence into Ordinal Encoderordinal_encoder = OrdinalEncoder(categories=[g,r])X_ExT2 = ordinal_encoder.fit_transform(X_ex)# gradesX_ExT2_grades_unique = np.unique(X_ExT2[:,0])grades_encode = dict(zip(g, X_ExT2_grades_unique))grades_encode
# rankingX_ExT2_rank_unique = np.unique(X_ExT2[:,1])rank_encode = dict(zip(r, X_ExT2_rank_unique))print(rank_encode)
Data input. Another alternative for data input can be getting the values, then processing them as appropriate.
# Alternative data load approachEx_values = Ex.values# Predictor variablesX_ = Ex_values[:, :-1].astype(str)# target variablesy_ = Ex_values[:,-1].astype(str)
Range of values. For the encoders, the current default is to raise an error if there are new values where the encoder has not seen in the fitting process. This can be configured as per documentation. The other approach is to list out the full range of expected values to be encoded, and pass them into the encoder.
To recap,
regardless of target or independent variables (i.e. predictor variables), we need to understand the correct order sequence, then apply the encoding accordingly. It’s always a good idea to check the encoded values after encoding each columns.
for the target variable, we can opt to use Label Encoder or Pandas.
for the independent variables we can use Ordinal Encoder over Pandas for more efficient process.
where the ordering sequence is unique, these can be defined and passed into the encoders.