Feature Engineering Examples: Binning Numerical Features
By: Max Steele, They Them
Tags: Data Science, Machine Learning, Feature Engineering, Python

Feature Engineering Examples: Binning Numerical Features

Feature engineering focuses on using the variables already present in your dataset to create additional features that are (hopefully) better at representing the underlying structure of your data.

For example, your model performance may benefit from binning numerical features. This essentially means dividing continuous or other numerical features into distinct groups. By applying domain knowledge, you may be able to engineer categories and features that better emphasize important trends in your data.

Photo by Paper Beard on Unsplash

In this post, we’ll walk through three different methods for binning numerical features with specific examples using NumPy and Pandas. We’ll engineer features from a dataset with information about voter demographics and participation. I’ve selected 2 numerical variables to work with:

age : a registered voter’s age at the end of the election year birth_year : the year a registered voter was born

If you want to start applying these methods to your own projects, you’ll just need to make sure you have both NumPy and Pandas installed, then import both.

Using np.where() to Indicate Thresholds

It may be odd to think about, but indicating whether a certain threshold is met by each instance (in this case by each registered voter) is a type of binning.

For example, imagine we’re trying to predict whether each registered voter turned out to vote in the election. Maybe we suspect that younger voters will be more likely to turn out if this is the first time they were eligible to vote in a presidential election. Since the legal voting age is 18, anyone less than 22 years of age during the current presidential election would not have been able to vote in the previous presidential election.

We can create an indicator variable for this threshold using np.where() which takes 3 arguments:

a condition what to return if the condition is met what to return if the condition is not met

The following code creates a new feature, first_pres_elec , based on an individual’s age:

df['first_pres_elec'] = np.where(df['age']<22, 1, 0)

The condition we’re checking is whether or not the individual is less than 22 years of age. If they are below that threshold, np.where() returns a 1 because this was the first presidential election in which they were eligible to vote. If not, then 0 is returned. From our continuous variable age , we have created a new binary categorical variable.

Maybe we also have reason to suspect that senior citizens were more or less likely to turnout to vote. If so, we might want to draw our model’s attention to this threshold by creating another threshold indicator:

df['senior'] = np.where(df['age']>=65, 1, 0)

Now we’ve created two threshold indicators that split the distribution of voter age as shown below. Younger individual’s who are newly eligible to vote in a presidential election are highlighted in red and seniors are highlighted in yellow.

Registered voter age distribution with younger and older thresholds highlighted — Image by author

Applying a Custom Function with apply()

It might make sense to divide our registered voters up into generations based on their year of birth since that often seems so wrapped up in a person’s politics. One way to do this is to write our own custom function delineating the cutoffs for each generation.

Below is one way we could write such a custom function:

And then use Pandas` apply() to create a new feature based on the original birth_year variable:

Now our registered voters are broken up into 5 discrete and meaningful categories. I decided to combine the 2 oldest generations (Greatest and Silent generations) so as not to create 2 rare categories that each make up only a very small portion of the population.

Seaborn countplot showing distribution of voters by generation — Image by author

Defining Bins with pd.cut()

We can also create the same generation bins using pd.cut() instead of writing our own function and applying it. We’ll still need to define the appropriate labels for each group, as well as the bin edges (cut off birth years).

In the last line, we create our new feature by providing pd.cut() with the column we want binned into categories, the bins we want, and how to label each binned category.

Rather than grouping by generation, we could quickly create a range and supply those as our bin edges. For example, if we thought it would be meaningful to group age by decade, we could accomplish that with the following:

The first line defines a range that starts at 10 and continues up to, but not including 110, increasing by 10 at each step. The second line uses that range as bin edges to discretize registered voters by age into the following groups:

Raw count and percentage of registered voters binned based on age in decades — Image by author

The first row shows that 33,349 or 19.84% of our voters are in their 40’s. The parenthesis indicates that the 40 is inclusive, whereas the square bracket indicates the 50 is excluded from the bin. To more easily keep track of what each bin means, we could feed in the following labels to pd.cut() :