5 Basic Fundamental of Feature Engineering for Any Business
By: Pathairush Seeda
Tags: Data Preprocessing, Feature Engineering, Model, Data Science, Machine Learning

FEATURE ENGINEERING

5 Basic Fundamental of Feature Engineering for Any Business

Photo by Malcolm Lightbody on Unsplash

Feature engineering is a fundamental stack of building the data science model. We can use the features both in data analysis and machine learning models. The informative feature guides you to the incredible underlying insight.

You can make an impact decision that you have never imagined before.

We usually start the analysis with the raw data set. The data stores in the transaction level that each row specifies what item and how much quantity has sold. Feature engineering is to

Aggregate the data from 1 level to another (transaction to customer level) Derive the new feature in the same unit of analysis by combining it with the other feature.

The possibility of your feature engineering outcome depends on your data dimension. If you have a high dimensional feature with various kinds of information, you can make many features.

They are two ways to create new features.

Your expertise. Try every possibility (this is what auto-ml does)

The benefit of the first way is that you can add your perspective when deriving the feature.

The resulted feature can be explainable. You won’t make the feature that you can’t intuitively understand.

The explainable feature would be useful for the data analysis task to tell the story to other parties. If you can explain how the feature is come from, the more comfortable you will convince the other to believe it makes sense.

However, for the modeling task, the first approach seems to limit the capability of machine learning.

The useful feature for may not be intuitive. And it’s out of your ideas.

Trying all of the possibilities might be the answer to it. That’s what the auto-ml does nowadays. It skips the intuitive part and takes into account every combined option. That’s why the auto-ml uses so many resources when developing the model.

The good side for the auto-ml is that you may get a new insight that you have never thought about it before.

Problem

it’s all about the money

Photo by Blogging Guide on Unsplash

For the company started to invest in data science capability,

the auto ml seems to be an expensive move for the management team.

They need successful use cases before spending a lot of money on the data infrastructure. Will it worth the money spending? That’s the most critical question to be answered.

That’s why they usually start with the small in-house data analytics team. It can contain a few people. And those data analysts/scientists need to bring the company's value first, and then more investment will come later.

With this situation, we now know why feature engineering is importance because they can generate the insight from both analysis or modeling to help management team monitor, decide, or improve their current business without much cost.

Enough for the introduction

the following will be the basic fundamental of feature engineering I had tried when I worked as a data scientist in the banking industry.

I suffer about the feature engineering’s ideas when I started my career as a data scientist because my lack of business knowledge.

Thus, I hope this will help those who have just started your journey as a data scientist and still need to develop the features that will benefit from this article. I will show you five feature engineering types to create and use with any data set.

Let’s start!

Statistical

Photo by Kelly Sikkema on Unsplash

We learn so many things from the statistics. There is a lot of elements that are useful in feature engineering. You can start with the basic properties such as

Mean, Median, Mode, Standard deviation, Max, Min, Percentile, Rank, Number of unique, Sum, Count, etc.

You can take a look at more options with Pandas here in the Computations / descriptive stats section. I usually try what makes sense in my data set.

For example, If I have a payment amount for each customer, I can aggregate the data over the lifestyle segmentation over a specific period (3, 6, 12 months).

These features are much intuitive.

It shows the behavior over the time or segment that the customer is living in. With more time, we can get a more reliable feature to measure their spending behavior.

Somebody can have high spending for one month, but they can spend it every month for the rich one. This behavior will reflect in the long term period aggregated feature.

The more advanced technique here is that you can fit the regression model to get the slope and intercept as a feature.

You can use the X as the number of the sequential month, let’s say [1,2,3,4,5,6] for six months period, and Y as a based feature value like a spending amount.

The slope will measure the spending trend, whether it’s upwards or downwards, and the intercept can show you a base spending at the starting time.

One thing to concern is for some value. If you aggregate without looking at its underlying meaning, it will lead to the wrong interpretation.

Such as the banking account’s balance. You can sum it over the segmentation, but you cannot add it over time because it will double-count the value of balance, and it doesn’t make any sense in the business.

Time

Photo by Nastya Dulhiier on Unsplash

I partially touch it in the previous section when you aggregate the feature over time. The example shows the aggregation with the basic statistics or using the basic regression model to get the slope or intercept. These can count as a time-domain feature.

In addition to statistical aggregation, the other features are that recency and Month on Book.

Recency is how long since the last time that the customer uses our product/service.

Month on Book is how long since the customer subscribe to our product/service for the first time.

From my experience, the customer with a high value of MOB (Month on Book) and a low value of recency have high loyalty to our brand and likely to take our products if we offer them.

Not every case but almost all of the time, I will use these features with the model, and it mostly shows up as an essential feature.

Other than the product/service calculation, I usually use it with the customer's demographics. For instance, I can derive how long the customer works in a salary-man, self-employ, or freelance position.

This feature shows that the behavior of customers can adapt to how they work in daily life. The mindset of each type of position is so different. We can use it for modeling or describing the persona of our customers.

Another aspect of the time feature is the calendar feature. You can engineer the feature based on the Day of Week, Day of Month, Month of Year, etc. These features can carry on the data’s seasonality to the model. It can use to summarize the information monthly.

Ratio

Image by Peter Kertesz from Pixabay

Something is comparable only when it’s on the same scale.

That’s why we have a ratio here. For instance,

we have two customers who have about 50K $ and 100K $ a month, respectively. If the first customer spends 25K $ and the second customer spends 50K $. There is a large gap in absolute value between them. But it’s the same 50 % of their monthly income.

The interesting point is that we can group them based on the spending/income ratio. This feature gives us another customer’s perspective even if there are ten times difference in the spending amount.

The behavior of spending is the same. The customers have paid half of their income. We can extend these ideas to other domains.

For example

The utilization of their credit card limit.

The spending compared to the median of the segment.

The number of clicks on ads compared to the people in the same geo-location.

The portion of an investment asset (cash, bond, stocks, etc.)

After I have worked for an extended period, I saw that the ratio is the most popular number that will show in any dashboard. It reflects the health of the overall business within one single value. Many financial indicators happen in terms of ratio, such as debt to equity ratio, price to earnings ratio.

Importantly, the management team usually see and discuss to each other about these metrics. Hence, It would be best if you are familiar with.

Crossing

Photo by Jack Patrick on Unsplash

Feature crossing is the process of multiplying two features together. It can use with a numeric or categorical feature.

As an illustration, we have an age feature binning every 10 values [21–30,31–40,41–50] and wealth segment [poor, average, rich]. We can multiply these two features to get [21–30 & poor, 31–40 & poor, …, 41–50 & rich] groups of customers. It will be a new feature making by the feature crossing process.

I think this technique is quite beneficial for the machine learning model. It helps the model understand the underlying pattern easier by explicitly creating it. But if the information has too many distinct values. It could lead the model to be overfitting. You have to be careful when using this kind of feature.

Another example is that you can multiply the binning of latitude and longitude to create the geo-location grid feature explicitly. The model can learn this pattern and give you useful information such as there is the grid that if you plan to establish any building, the price will be up by 10 %

The concept seems like the previous topics, which is the ratio. But you also can have an addition or subtraction between features as well.

To show some examples here, My colleague used to derive a feature called “travel essential” by summing the 1/0 flag from several dimensions such as.

Is the customer booking the hotel recently?

Is the customer booking the flight recently?

Is the customer browsing the travel website?

Is the customer buying travel insurance?

Is the customer going to the embassy?

Then they sum those 1/0 questions to be a new feature called travel essential. As a matter of fact, It plays a vital role in the travel insurance propensity model.

Geo-location

Photo by henry perks on Unsplash

Lastly, we have come so far throughout this article. This one is the latest feature domain that I have discovered. I had a chance to do the health insurance propensity model. And my job is to squeeze every last drop of performance out of the model. That’s when I met the geo-location feature.

The concept is simple. It would be best if you had the latitude and longitude for the point of interest. Then you measure the distance from those points to the customer’s device. If you are a company that already has mobile apps, this is not hard. You can retrieve the information when the customer logs in to your application.

For other ideas, you can also measure each customer's traveling range by using the geo-location data and infer whether the customer often goes abroad. We can mix up the feature with other sections that I mentioned earlier in this article. You can introduce the statical and time-domain feature to make a heuristic view of the customer moving over a certain period.

From my experience, these types of features are usually essential and have a predictive power compared to other domains.

I still don’t get why it is important for some tasks, but you should try if you have a chance—highly recommended!

Final thought

Photo by Jason Briscoe on Unsplash

A lot is covering here. Those five sections are the feature domain that I had generated when I worked in Banking. And I have so much pleasure when I saw others reuse my feature with their model, and It went well. Sometimes, I learned new things based on other’s feature engineering. The ideas about it are unlimited.

I think this is one of the most exciting parts of data science.

Please feel free to share your resource if you have a related article about this kind of stuff.

It took me a lot of time to find an item like this when I started my data science journey. I had to find it from the Kaggle kernel, and It usually not related to my business use cases. I hope this guideline will help you accelerate the part in developing the model and analysis.