Customer Segmentation: Arvato Bertelsmann Project
By: Éverton Bin
Tags: K Means, Arvato, Clustering, Customer Segmentation, Data Science

Customer Segmentation: Arvato Bertelsmann Project

A simplified technical report explaining my solution to building client profiles, targetting customers through demographics data, and communicating findings Éverton Bin Mar 18·15 min read

Photo by Sharon McCutcheon

How about getting to know exactly who your company’s customers are? Where do they live? What do they need? How do they think?

Getting to know more about the customers doesn’t just help to create more personalized products and strategies, but also allows the company to save money and effort when directing their actions to the right targets: the ones that are most likely to convert into becoming customers.

In this project, Bertelsmann Arvato Analytics provided the demographic data related both to the general population and the customers of a mail-order sales company in Germany. The goal was to perform customer segmentation, identifying the parts of the population that best describe the core customer base of the company, and also the proper targets of a marketing campaign, optimizing the customer acquisition for the company.

The project was structured following this track:

Data Overview and Data Cleaning;

Exploratory Data Analysis;

Unsupervised Machine Learning Task: Cluster Analysis;

Customer Segmentation Report;

Supervised Machine Learning Task: Targeting Customers for a Marketing Campaign.

1. Data Overview

This first step consisted of understanding the data structure both for the general population and the customers' datasets. The largest dataset containing the general population demographics data was composed of 891.221 observations (people) and 336 columns (features).

Figure 1. First Rows in the General Population Dataframe (Image by Author)

These features related to different aspects of the population in five main information levels:

person : containing information related to gender, financial and health typologies, social status, mind affinity, and so on;

: containing information related to gender, financial and health typologies, social status, mind affinity, and so on; household : presented number of people in the household, estimated household income, transaction activities, and many more household aspects;

: presented number of people in the household, estimated household income, transaction activities, and many more household aspects; microcell : different CAMEO consumer classifications, most common age of car owners, most common car manufacturer, most common car age, etc.;

: different CAMEO consumer classifications, most common age of car owners, most common car manufacturer, most common car age, etc.; macrocell : density of inhabitants, distance from city center, number of family houses, most common building type, etc.;

: density of inhabitants, distance from city center, number of family houses, most common building type, etc.; community: inhabitants, the share of unemployed people, and so on.

To deal with the amount of different features, this was the adopted strategy:

data cleaning , joining together missing and unknown values;

, joining together missing and unknown values; deleting columns that overpass the defined threshold rate for missing values (considering missing and unknown values together);

deleting a few columns that didn’t relate to the analysis;

treating columns’ types according to their representations;

feature engineer part 1: expanding multiple information contained in the CAMEO_INTL_2015, PRAEGENDE_JUGENDJAHRE, and ALTER_HH features into more representative and informative features;

part 1: expanding multiple information contained in the CAMEO_INTL_2015, PRAEGENDE_JUGENDJAHRE, and ALTER_HH features into more representative and informative features; correlation analysis, deleting columns with similar information and maintaining the ones with a lower rate of missing values.

After these first filters and transformations, the number of features was reduced to 218.

Figure 2. First Rows in the General Population Dataframe after Filters (Image by Author)

After that, the same process was applied to the customers' data.

2. Exploratory Data Analysis

That is when the fun part begins. In this section, the goal was to perform a previous analysis to indicate possible directions and especially extract some insights on customers’ characteristics in comparison to the general population.

A few questions guided this exploratory data analysis task:

How old are the customers?

Where do they live?

How are they classified while consumers?

What about their incomes?

What are their consumption habits, lifestyle, and family composition?

The main findings of the EDA that helped to build an overview of the customers’ main profile are presented below:

2.1 Youth Decade

Figure 3. Bar Plot Comparing Youth Decade for Customers and the General Population (Image by Author)

Youth Decade was one of the features engineered in the first part of the project. It indicates in which decade the person lived his/her youth period, from 4 relating to the 40'ies to 9 indicating the 90'ies.

While younger people are proportionally more frequent in the general population, the opposite happens among customers, indicating that most of the mail-order company’s customers tend to be represented by elder people instead.

2.2 Youth Movement

Figure 4. Bar Plot Comparing Avant-Garde Movement Occurrences for Customers and the General Population (Image by Author)

Another engineered feature, Avant-Garde indicates in which movement people were associated during their youth: avant-garde or mainstream. 1 indicates an association with the avant-garde movement and 0 with the mainstream movement.

Although among customers the share of avant-gardes and mainstreamers are equivalent, there is a clear overrepresentation of avant-gardes in the customers’ group when comparing to the general population.

2.3 Family Status

Figure 5. Bar Plot Comparing Family Status for Customers and the General Population (Image by Author)

The Family Status feature talks about 5 different classes:

1 : wealthy;

: wealthy; 2 : prosperous;

: prosperous; 3 : comfortable;

: comfortable; 4 : less affluent;

: less affluent; 5 : poorer.

While in the general population poorer and less affluent families are prevalent, among clients the most frequent families are wealthy and prosperous, indicating that, although the company reaches out to clients in different segments of the population, status related to richer families are the most common.

2.4 Family Income

Figure 6. Bar Plot Comparing Family Income for Customers and the General Population (Image by Author)

Family Income represents:

1 : highest income;

: highest income; 2 : very high income;

: very high income; 3 : high income;

: high income; 4 : average income;

: average income; 5 : lower-income;

: lower-income; 6 : very low income.

Once again, it’s possible to see a clear overrepresentation of families associated with higher incomes, reinforcing the idea of the previous feature analysis. The underrepresentation happens exactly in the classes related to lower and very low incomes.

3. Unsupervised ML Task: Cluster Analysis

3.1 Data Transformation

To group the population in different clusters, more transformations had to be applied, following this process:

one more feature engineering process was developed, this time on the CAMEO_DEU_2015 feature, replacing its 44 classes for values indicating how overrepresented they were among customers in a scale from 0 to 3;

process was developed, this time on the CAMEO_DEU_2015 feature, replacing its 44 classes for values indicating how overrepresented they were among customers in a scale from 0 to 3; numerical variables had their missing values replaced by the median, preventing outliers’ influence;

categorical features were transformed using the pandas’ get_dummies function; with that, missing values were not imputed, and were indirectly considered in the analysis;

function; with that, missing values were not imputed, and were indirectly considered in the analysis; after treating missing values, the data was standardized , using the scikit-learn StandardScaler ;

, using the scikit-learn ; once all the data was on the same scale, a dimensionality reduction technique was applied (PCA) to reduce the number of features and capture the most important aspects in the data.

In the last step (dimensionality reduction), the level of information discussed earlier was taken into consideration. The assumption was that lower levels of information contained more specific information that could better explain the differences among the groups.

With that in mind, for each level of information, a specific PCA was applied, resulting in pure components. In other words, each level of information was treated separately, and each component could only represent one single level of information.

To decide the number of components for each one of the information levels, the explained variance was studied, prioritizing low levels:

Figure 7. Scree Plot Person Level (Image by Author)

At the person level, 60% of the explained variance was maintained, resulting in 30 components.

Figure 8. Scree Plot Household Level (Image by Author)

At the household level, 20 components would explain about 50% of the data variance.

Figure 9. Scree Plot Microcell Level (Image by Author)

At the microcell level, the chosen number of components was 10, explaining over 40% of the variance.

Figure 10. Scree Plot Macrocell Level (Image by Author)

Finally, at the macrocell level, 10 components explain about 40% of the variance.

The community level was composed of only three features, and because of that, there was no need for dimensionality reduction.

3.2 KMeans Clustering

Once the data was ready to be processed by the KMeans algorithm, the Elbow Method was used to decide on the proper number of clusters.

Figure 11. Elbow Method Plot (Image by Author)

Since there wasn’t a clear elbow indicating the best number of clusters, the line inclination was observed. The most expressive change in the inclination happens before the number of 20 clusters. With that, it was decided to go forward with 18 clusters.

With the general population classified into 18 different groups, the algorithm predicted to which cluster each customer's observation belonged. After that, it was possible to compare the distribution of the clusters both in the general and customer population.

Figure 12. Bar Plot Comparing Cluster Occurrences for Customers and the General Population (Image by Author)

These are the highlighted clusters, indicating overrepresented and underrepresented groups among customers:

Figure 13. Clusters Over and Underrepresentation (Image by Author)

4. Customer Segmentation Report

At this stage, these clusters need to be explained so we can understand which profiles are more likely to become customers for the company. To explain what each cluster represents, the cluster centers indicated the most important components, and these components were linked to the most important features, allowing cluster description.

4.1 Overrepresented Clusters

4.1.1 Green Dreamers (Cluster #2)

Photo by Agnieszka Boeske

their vacation habits can be classified as nature fans, already emphasizing the green tendency as an important aspect present in people represented by this cluster;

the green aspect is reinforced with the avant-garde aspect, relating this cluster to people with the avant-garde mindset, which also relates them to the green avant-garde movements;

they don’t seem to be familiar minded;

this cluster is representing mostly men that are not single, not critical-minded but are dreamers and social-minded.

4.1.2 Traditional High-Society Elders (Cluster #9)

Photo by Ravi Patel

they are in advanced life stages, being represented as families or multiperson households, and also related to higher incomes;

their social status can be classified as independents, house owners, and top-earners;

they are not dreamers, nor eventful oriented, but they can be represented as being critical and traditional-minded, as well as religious;

the cluster describes elder people.

4.1.3 Empty Nest, Full Wallet (Cluster #2)

Photo by Toa Heftiba

it represents mostly average to high-income couples;

it could also represent house owner couples and top earner-couples of higher age;

the third component brings aspects seen before, as being traditional-minded, and religious, but not eventful oriented;

it’s possible to say that some aspects seen before are now applied mostly to couples with high-incomes from different ages, and also mature and top earner-couples.

4.2 Underrepresented Clusters

4.2.1 Less Affortunate Beginners (Cluster #0)

Photo by Zou Meng

it represents people that could be classified as less affluent or poorer, mostly indicating pre-family couples and singles or young couples with children;

their household incomes can be considered as lower or very low incomes, and their insurance typology is mostly classified as social safety-driven;

in terms of health and shopping typologies, they could be described as sanitary affine and demanding shoppers;

the microcell component indicates areas with a high share of cares built before 1994 that had two or more pre-owners, creating the picture of people with lower incomes that live in poorer areas.

4.2.2 Multi-Generation Money-Savers (Cluster #16)

Photo by Andre Taissin

this cluster describes people with low-income and average earners of higher age from multiperson households, and families classified as two-generation household;

there is also a strong aspect of singles with low-incomes or average-earners families, and also average earners of younger age from multiperson households. This profile tends to be related to people living in the Westside;

their financial typology to be classified as low financial interest or money-savers.

4.2.3 Second-Hand Car Cells (Cluster #14)

Photo by Nikolai Justesen

it represents people that could be classified as less affluent or poorer, mostly indicating pre-family couples and singles or young couples with children;

the microcell component indicates areas with a high share of cares built before 1994 that had two or more pre-owners, creating the picture of people with lower incomes that live in poorer areas;

it also describes areas with a high share of family houses, with a very low share of cars per household, and also close to city centers;

most of these aspects were already highlighted in other clusters, but in this case, it seems to emphasize more the micro and macrocells aspects than the ones related to the person.

4.3 Call to Action

Although there is a clear profile indicating specific segments of the population that are more likely to become customers of the company, it is important to highlight the fact that when it comes to the income, age, or social classes analysis, the base core customer represents the smaller portion of the population.

It could also be seen as a great opportunity for the company, once there could be launched different and simpler versions of the current products that could be, for example, purchased by people with lower incomes.

At the same time, it’s possible to reach out to younger people that are especially concerned about environmentalism issues, investing in directed marketing campaigns that speak with these new generations. They could potentially become customers for a lifetime.

5. Supervised ML Task: Targeting Customers for Marketing Campaign

In this last part of the project, different approaches were used to find the best model for predicting how likely it is for a person to be responsive when exposed to a marketing campaign.

The main sequence consisted of:

developing data transformation and machine learning pipelines to avoid data leakage;

and to avoid data leakage; testing different approaches during data transformation: PCA, TruncatedSVD, SMOTE, and so on;

parameter tuning with GridSearchCV and BayesSearchCV ;

and ; scoring predictions on the Kaggle platform to check how well the model performs on unknown data.

5.1 Evaluation Metric

Different metrics could be used to evaluate the model performance. In this specific case, the training data is a typical case presenting class imbalance. It means that most of the observations correspond to people that didn’t respond to the marketing campaign.

In this situation, it possible that the algorithm learns a lot about one of the classes, but isn’t capable of ‘understanding’ the other class. Since we care equally for both classes, the roc_auc score shows up as a proper metric to be used since it represents a tradeoff between the true positive rate and the false positive rate.

5.2 Learning Curves

The learning curve was evaluated for a few algorithms to make one first choice. In this analysis, the Gradient Boosting Classifier was the one with the best performance.

Figure 14. Learning Curves Gradient Boosting Classifier (Image by Author)

The learning curves presented convergence and there was no sign of overfitting. Because of that, the first attempt was based on this machine learning algorithm.

After the basic transformations, the data pipeline consisted of:

selecting numerical features and imputing missing values with the median , and after that, standardizing to data to a common scale;

and imputing missing values with the , and after that, to data to a common scale; missing values in categorical features were imputed with the most common value (which could be interpreted as the mode), and then the features were one-hot encoded ;

were imputed with the (which could be interpreted as the mode), and then the features were ; binary features with missing values followed the same procedure as the categorical features, and binary features with no missing values didn’t suffer any further transformation.

On the other hand, the machine learning pipeline consisted of:

preprocessing the data on the test set following the pattern presented above;

the data on the test set following the pattern presented above; training the algorithm testing different combinations of parameters through GridSearchCV ;

the algorithm testing different combinations of parameters through ; predicting on the test set for different cross-validation folds;

on the test set for different cross-validation folds; evaluating the model through the roc_auc score, given the high class unbalance shown in the data;

the model through the score, given the high class unbalance shown in the data; returning the best model.

5.3 First Approach: Gradient Boosting Classifier + GridSearchCV

In this first attempt, two model’s parameters were fixed:

learning_rate : set to 0.1;

: set to 0.1; n_estimators : set to 150.

GridSeachCV searched for the best combination of three parameters: min_samples_split , max_depth , and max_features .

Figure 15. Parameter Tuning for Gradient Boosting Classifier (Image by Author)

min_samples_split refers to the minimum number of samples required to split an internal node, and two values were tested here (2 and 4).

max_depth indicates the maximum depth of the individual regression estimators, limiting the number of nodes in the tree. 3 and 5 were the values tested here.

max_features specifies the number of features to consider when looking for the best split. If None, all the features are candidates to be chosen, if auto, the model restricts the number of features that can be selected, in this case, the squared root of the total number of features.

Considering these parameters, and the roc_auc score, that was the best combination:

Figure 16. Parameter Tuning Result for Gradient Boosting Classifier (Image by Author)

Finally, this model was used to make predictions on new unseen data. After submitting these predictions on the Kaggle platform, the actual model score could be obtained.

The first model achieved a reasonable roc_auc score of 0.79488. Considering the Kaggle rank, this first approach was positioned among the top 150 models.

After that, different approaches were tested in order to improve this first model. Basically, different algorithms were tested, including Ada Boost Classifier and LightGBM Classifier, and also a few variations on the data preprocessing pipeline.

Among these attempts, one single model overcame the first one. The same data transformation and machine learning pipelines were applied. The Gradient Boosting Classifier was replaced by the XGBoost Classifier, while the GridSearchCV gave space to the BayesSearchCV.

5.4 Best Approach: XGBoost Classifier + BayesSearchCV

While the GridSearchCV combines the different parameters’ values to find the combination that optimizes the result, the BayesSeachCV receives a range of values (Integer, Real or Categorical) related to each one of the parameters.

Once one improvement is found, it explores that region more deeply to achieve higher improvements.

Figure 17. Parameter Tuning for XGBoost Classifier (Image by Author)

In this attempt, no parameter was previously fixed. Gbtree and dart built-in boosters were tested, and also two different tree methods: auto and hist.

The minimum loss reduction required to make a further leaf node partition ( gamma ) was set in the range from 0 to 100, while the learning_rate was limited in the range of 0.01 to 0.3. Different regularization terms were also tested ( lambda and alpha ).