Labeling Data with Pandas. Introduction to Data Labeling with... | by Sadrach Pierre, Ph.D. | Towards Data Science

Data labeling is the process of assigning informative tags to subsets of data. There are many examples of labeled data sets. Data containing x-ray images of cancerous and healthy lungs along with their respective tags is an example of labeled data. Another example is consumer credit data that specifies whether or not a consumer has defaulted on a loan. Upon obtaining a labeled data set, machine learning models can be trained on the labeled data and used to predict on new unlabeled examples. Having good labeled data is essential to building high performance machine learning models. In this post, we will discuss the process of generating meaningful labels using the python Pandas library.
Let’s get started!
We will be considering the task of labeling numerical data. For our purposes we will be working with the Red Wine Quality Dataset which can be found here.
To start, let’s read the data into a Pandas data frame:
import pandas as pddf_wine = pd.read_csv("winequality-red.csv")
Next, let’s read the first five rows of data using the ‘.head()’ method.
print(df_wine.head())
The data corresponds to wines with quality scores between 0 and 10. Let’s look at the values in the data:
print("Quality values: ", set(df_wine['quality']))
Suppose instead of classifying wine quality we were interested in another classification problem. Let’s consider the problem of classifying whether or not a wine is above 10% alcohol. Let’s take a look at the minimum and maximum values for alcohol %:
print("Max Alcohol %: ", df_wine['alcohol'].max())print("Min Alcohol %: ", df_wine['alcohol'].min())
let’s also plot the distribution:
import matplotlib.pyplot as pltplt.title("Distribution in Alcohol %")df_wine['alcohol'].hist()
We will label the data with a ‘1’ if the alcohol % is above or equal to 10% and ‘0’ otherwise:
import numpy as npdf_wine['alcohol_class'] = np.where(df_wine['alcohol']>=10.0, '1', '0')
We can now visualize the distribution in binary labels:
from collections import Counterplt.title("Distribution in Alcohol Class Labels")plt.bar(dict(Counter(df_wine['alcohol_class'])).keys(), dict(Counter(df_wine['alcohol_class'])).values())
The data is appropriately labeled for training a binary classification model. Now let’s consider problems beyond binary classification. If we take a look at the minimum and maximum values of the ‘fixed acidity’ we see that the range of values is wider than those for alcohol %:
print("Max fixed acidity %: ", df_wine['fixed acidity'].max())print("Min fixed acidity %: ", df_wine['fixed acidity'].min())
We can use the Pandas ‘.loc[]’ method to assign ternary labels to the data, which would segment the data into three groups. The label of ‘0’ will be assigned to values (4–7), ‘1’ will be assigned to values (7–9), and ‘2’ will be assigned to values (9–16):
df_wine.loc[(df_wine['fixed acidity']>4.0) & (df_wine['fixed acidity']<=7.0), 'acidity_class'] = 0df_wine.loc[(df_wine['fixed acidity']>7.0) & (df_wine['fixed acidity']<=9.0), 'acidity_class'] = 1df_wine.loc[(df_wine['fixed acidity']>9.0) & (df_wine['fixed acidity']<=16.0), 'acidity_class'] = 2
We can now visualize the distribution in ternary labels:
plt.title("Distribution in Alcohol Class Labels")plt.bar(dict(Counter(df_wine['acidity_class'])).keys(), dict(Counter(df_wine['acidity_class'])).values())
The data is now appropriately labeled for training a ternary classification model. This can easily be extended to quaternary, quinary, etc. I’ll stop here but I encourage you to play around with the data and code yourself.
To summarize, in this post we discussed how to use Pandas for labeling data. First, we considered the task of assigning binary labels to wine data that indicates whether a wine is above 10% alcohol by volume. We then took a look at assigning ternary labels that indicate the level of fixed acidity in the wines. I hope you found this post useful/interesting. The code from this post is available on GitHub. Thank you for reading!