Take Banking Personalization to the Next Level with 3 effective ML practices
By: Srdjan Jovanovic
Tags: Personalization, Data Science, Banking, Machine Learning, Recommendation System

Take Banking Personalization to the Next Level with 3 effective ML practices Srdjan Jovanovic Sep 8·21 min read

Source: pinterest.com

You can hear the word “personalization” at every turn nowadays. Everyone has their reasoning about the definition of this term (fun fact). In the original sense, this term signifies tailoring the service or product to accommodate it to the specific individuals or groups of similar individuals. An informal definition would be trying to match the user (individual) with the item (product or service) that fits their needs as much as possible. So, whether it’s about salesmen in the leather goods store who offers belts to male visitors and purses to women, telecommunication companies offering tariff packages for different target populations based on age, or suggesting adequate music on Spotify to each person based on its preferences, we are speaking about different levels of personalization, i.e. more or less personalized depending on the type of business they are used for.

Source: tealium.com/blog/customer-centricity

One common thing for all of them is that personalization quality could be measured by tracking customer satisfaction (structured or unstructured feedback) via conversion rates, the number of visits, given ratings, or even through the sentiment of their comments to the suggested content. Thanks to this we can say whether personalization was justified or not. It is clear that depending on the quality of the recommended content, the metrics would be better and therefore business would get better engagement (trust) from its customers.

A bank is an excellent example of a business where proper interaction with customers is of high importance. If done properly, it is possible to form long-lasting relationships with them and establish strong loyalty. This will certainly ensure customers take more products or services from the bank and contribute to the business growth in general. So, the bank should recognize their customers’ needs in order to suggest to them the right actions from which they will benefit.

Traditional approaches such as bulk campaigning for RFM (recency-frequency-monetary based) segments offering Cash Credit once in 6 months, targeting 20% of highest-paid customers (select top * from … order by) with premium products followed by generic message or just showing spending distribution across categories (TRAVEL, VEHICLES, GROCERIES, …) without any relevant content and call to action are not going to encompass customer needs at that particular moment and persuade them to make a decision.

People need more than that! They want to get noticed by a specific product that is relevant to their current life situation, plans, or wishes. Therefore, a sophisticated and empathetic approach is a must.

Some trends that are present now, influence how the access towards the customers should be developed.

Customers are becoming more and more digitally aware (could be contacted through digital channels)

(could be contacted through digital channels) Hunger for timely, precise, and appropriate content arises as never before (at the right moment specific for each person, otherwise it is called spam)

(at the right moment specific for each person, otherwise it is called spam) Patience for waiting in a row converges to zero (response time as an ultimate request).

Source: behance.net

These factors are the main causes that transform the traditional ways of building relationships with the customer. In this article, however, I will focus on the second of them (“timely, precise, and appropriate content”), or personalization, in one word!

This is the moment where banks ask for Machine Learning to appropriately respond to these challenges!

Luckily, besides gigantic reservoirs of money, banks’ greatest asset is data. All the executed transactions, taken products, occurred activities, call center calls made, mbank loggings, website visits, demographic info, and many others are being recorded and stored in the databases. This represents a valuable resource for getting improved insights on customer behavior.

In this article, I will try to analyze how personalization is (or could be) powered by machine learning in the banking industry using 3 different approaches: segmentation, prediction, recommendation.

But first of all, let’s dive into data…

Data

The bank’s data is quite broad and various but these types of data are the most probable you would find there:

financial — contains all the information of the financial state of the client including monthly payments, income/outcome, debts, salary, account balances, internal classifications etc.

— contains all the information of the financial state of the client including monthly payments, income/outcome, debts, salary, account balances, internal classifications etc. product — includes information of all the products that customer has (loans, overdrafts, services, savings accounts, mortgages, etc.)

— includes information of all the products that customer has (loans, overdrafts, services, savings accounts, mortgages, etc.) demographic — stores information on personal information regarding the age, employment, living conditions, education, residence location etc.

— stores information on personal information regarding the age, employment, living conditions, education, residence location etc. digital channels — exposes information on social interaction between the client and the internal or external services related to the bank (ebank/mbank application, social network profile of the bank, website, chat bot conversations, call center requests)

— exposes information on social interaction between the client and the internal or external services related to the bank (ebank/mbank application, social network profile of the bank, website, chat bot conversations, call center requests) transactional — keeps information on all the transactions that the client made including purchases from merchants, money transfers, cash deposits, fees, etc.

Now the main question: how ML attacks these problems?

Segmentation

Source: nateqnews.com

The idea is to cluster customers based on the chosen features. Depending on the business problem that is being solved and the type of products the bank has, different target groups (clusters) should be detected, such as young people at the beginning of their career, large families with kids, people that enjoy traveling, fitness and new trends, etc. In this sense, appropriate features that describe these types of people are created and clustering is done. Target groups are then detected based on the most dominant features in the cluster and after analyzing them manually (using the domain knowledge) labels are assigned to each of them.

Distribution of features for 3 different clusters

In this example, the three distributions may seem similar and even between some of them the difference is only in one feature, the meaning is different for each of them. The first cluster has high spending and we may assume that it is about some wealthy family with a lot of members. The second cluster is quite similar to the first one, except that salary and spending power are much smaller, while spending is mainly focused on basic necessary things. The last one represents the younger population that is employed, just bought an apartment, and prefers enjoying life (expensive traveling, etc.).

Note that the label may consist of more “small” labels (properties), for example: instead of the label name average family with an attitude toward traveling and house decoration, this could be presented with three properties #average_family #traveling #house_decoration. Following this, there may exist clusters with a similar one or more properties and the described type of labels allows to target clients based on some trait or interest without the need to list all clusters manually, but only by choosing the property (increased flexibility and granularity). In order to properly cluster clients based on the desired output, it is of huge importance to choose adequate features.

For example, if the bank wants to offer products related to traveling, house reparation, fitness services, or technology, the main features could be extracted from transactional data based on the customer spending toward different merchants. If the bank decides to target customers with products related to their current life situation and monetary power (high spenders, average loyalists, career starters, schooling family, digital boomers) then features could be found in financial and demographic data primarily.

Features that are extracted could be:

numerical : average monthly income (outcome), the average amount of money spent on different categories of spending, number of transactions per month, annuity amount, number of loan installments, age, location (longitude/latitude), number of monthly website visits/clicks per item, etc., or

: average monthly income (outcome), the average amount of money spent on different categories of spending, number of transactions per month, annuity amount, number of loan installments, age, location (longitude/latitude), number of monthly website visits/clicks per item, etc., or categorical: internal classification of the client, types of opened products/services, employment status, living conditions, education level, chatbot intents.

In case of numerical features that are changing over time (such as income amount, spending on some specific merchant, etc.), these are created almost always by applying some aggregate function to the value for some period of time (average monthly income, total spending to specific merchant in the last 3 months). On the other hand, categorical features from some variables are created by applying an aggregate function to all possible values of that variable (pivoting), which may result in a few to dozens of new features.

After the features are prepared, the next step is to perform clustering.

Many clustering algorithms such as KMeans, DBSCAN, Gaussian Mixture Models, Agglomerative Hierarchical Clustering, and many others could be used. I wouldn’t get much into this because there are many tutorials and articles where each of them is thoroughly explained and implemented. Here, I will focus more on how KMeans clustering could be exploited to give the best possible results.

It is obvious that KMeans applied to the pure input won’t lead towards satisfying results because we may have a large number of features (especially in categorical variables) and also not all features are of the same importance.

A solution to this case would be to perform some kind of dimensionality reduction (we don’t want to drop off features manually, but rather let the model do it) which will transform the input to a smaller dimension while keeping all the information (or most of it). There are many techniques to do this: PCA, matrix factorization (SVD, NMF, etc.), Linear Discriminant Analysis (LDA), autoencoders. Again, there are many blogs on how each of these is used so I won’t get much into details to explain it, but rather just to expose the main difference between them. All of them try to compress input dimensions to a smaller number of dimensions so that the original input could be reconstructed from the compressed data. While PCA, Matrix factorization algorithms and LDA are completely linear (because it is matrix multiplication in the basic terms), autoencoders are fully connected neural networks with non-linear activation functions between layers (matrix multiplication plus non-linearities). They can catch some relations between features that are non-trivial or easily observable, which wouldn’t be the case with PCA or SVD for example. This is quite preferable considering clients’ unpredictable behavior most of the time. For example, clients’ spending on insurance and paying the rate to the travel agency has a completely understandable intention (traveling :)) and relation between spending on insurance and travel agency would be detected by linear algorithms most probably, while client spending too much on fitness and paying huge bills is not so clear so this relation may be only recognized by autoencoder. This is the reason why autoencoders are well suited for this kind of problem (dimensionality reduction).

Here are a few more words on its architecture. Autoencoders are special types of neural networks that try to compress (unlabelled) data to its latent-space representation and to reconstruct the output from this representation. In this way, an autoencoder tries to learn the distribution in the input and to store it in a lower number of dimensions. The first part is called encoder and it is a simple (deep) feed-forward neural network resulting in embedding (or code), while the second part is a decoder (mirrored encoder) which tries to reconstruct some defined output (could be original input, input without noise, etc.) only using the knowledge stored in the embedding. Depending on the defined loss function, this architecture ends up learning the embedding while minimizing loss. Since FF neural networks contain non-linearities between layers, the encoder can learn complex relations between input features (which is the reason why they perform the best in dimensionality reduction problems) and therefore, find the best low-dimensional representation that describes input data.

Autoencoder architecture consisting of Encoder and Decoder

After embedding from each input row (user) is extracted, KMeans clustering is applied as presented in the image below.

Here is a list of blogs on this approach applied to different kinds of segmentation:

TSNE

Speaking about non-linearities in compressing the information, it is worth mentioning one more method that is doing the same thing. T-distributed Stochastic Neighbor Embedding or TSNE is a method that converts the similarities between data points to joint probabilities and tries to minimize Kullback-Leibner distance between joint distributions of low-dimensional embedding and the high-dimensional data. It is therefore capable of compressing the input to low dimensional embedding, as previously mentioned techniques. Since TSNE has a cost function that is not convex, every time it is initialized the method will provide different results (probabilistic) and that’s why it is not advised to use this for dimensionality reduction. However, this is a perfect tool to get a visual sense of the data representations if data is compressed to 2 dimensions. Usually, a large number of features are combined (using FeatureTools package, for example) resulting in a few hundred new features. This kind of processed data is then fed into TSNE, compressed to 2 dimensions, and represented in an image. We can understand this as if two points are closer in 2D representation, they are much more similar.

Also, if the bank is interested to discover only a particular group of users with clearly distinctive traits (such as people that are high spenders and follow new trends, or exclusive clients that spend a lot and have a tendency toward traveling), it is possible by inspecting certain groups on 2D representation and their most dominant traits. If this group is unambiguously noticeable on 2D TSNE representation, all the clients that belong to this group are easily labeled. This is very important because the bank can easily label its customers and use them to make a classification model (instead of clustering). Again, TSNE is a quite powerful technique for visual representations of the input but since it is a probabilistic model it is not recommended to be used in getting the embedding from data!

Here are some of the articles on this topic:

Pros and cons of segmentation approach

Besides the manual work that needs to be done (analyzing and assigning labels) and the lack of metrics during the development phase that could tell how good the model is (this could be checked only after the model is deployed in production since the model is unsupervised), this approach is also quite inert. Since the features used in clustering are often aggregated (average, count, sum) over the last few months, this means that the segmentation will only change if some “huge” event happens in the customer behavior or if “smaller” unusual events happen frequently over a shorter period of time. In other words, as long as the user stays in its comfort zone, there is no migration between the clusters, and the bank is unable to address the customer in a timely manner for some specific life event. A potential fix to this would be decreasing time intervals (aggregate features over smaller time periods) or choosing the features that are related to the time components (e.g. recency).

Even though its real accuracy could be checked only after the model is deployed to production, segmentation based on clustering is still great to start with and the bank that decides to introduce personalization in its business should definitely try with this one first. It is quite easy to implement, very intuitive and understandable, adaptable and adjustable to different requirements for segments and the results would most probably satisfy initial expectations :)

Prediction

The idea is to make a prediction model which will tell whether the customer will take a product or service in the next period. These types of models try to attack the problem of timely, precise, and appropriate content. In basic terms, this is a classification model that classifies customer behavior for some period to positive (product taken after that period) or negative class (not taken). Depending on the number of products (or services) the bank wants to recommend, the model will have an output of that size with a binary classifier for each class at the end (ex. sigmoid). It is advised to have a binary classifier at the end (instead of softmax for example) since distribution across product counts is not uniform and we don’t want to bias less frequent products. In this way, each product is considered independently. Labels for this approach are extracted from the historical data such that considering a particular time point in history, customer behavior before that moment (during the period of an arbitrary number of months) is used to build features, while a few months after that as prediction period. If the user took the product during the prediction period, the label for that class is positive, otherwise negative.

If one of the bank’s requirements in these cases is explainability, then some of the models that should be considered are: multiclass logistic regression, SVM, boosting algorithms (AdaBoost, XGB, Gradient Boosting, ..), etc. These models can represent feature importance and therefore explain how much each feature contributes to the final score. This is also something that should be started with since they are quite simple yet effective models that should give some sense of metrics and serve as a baseline.

On the other hand, if this is not the requirement, then feed-forward and recurrent neural networks (RNNs) could be used. These networks can learn non-linear relations between features in the input which can result in higher accuracies. Moreover, RNNs are capable of learning sequential information and model customer behavior much better than one-time aggregated representation. The challenge here is, however, to appropriately represent each timestamp of customer behavior (topic for itself)!

As we want our model to be significantly sensitive to the dynamics of the users’ behavior, so that the offer is sent as soon as possible, special attention should be committed to features creation. Unlike the segmentation where we wanted to have a “smoothed” overall picture of the client for some longer period, here we want to give an advantage to the latest events, shorten time periods for aggregation function and focus more on each individual client’s action (rather than their average). Here are some examples: maximal withdrawal of money from foreign currency in the last month, latest spend on travel agencies/car dealers/house material, increase in salary compared to previous month, number of website visits in the last 24h, etc.

Note that here highly imbalanced classes are expected since some products are naturally less frequent than others and the action of taking the product is much rarer than not taking. This module is strongly dependent on the data quality and there must be enough examples per each class for successful training. One of the trivial solutions to this would be resampling (up/down sampling). However, this is not the only option.

Predicting whether the customer will take the product could be seen from a data and time perspective. The explained approach in extracting the labels and features (data) at a certain point in time could be multiplied to many time points. If we slide the point through time in regular time intervals (e.g. one month) we may get similar features (probably not identical) for the same client and the same product but from a different time view. In this way, we can augment our dataset and increase the number of examples for positive rare classes. Also, at each time interval, we get an enormous number of negative examples, so it is advised to use negative sampling to reduce their number.

Shifting time window and multiplicating data

Considering class imbalance, metrics that are observed for this task are Precision, Recall, F1 and ROC AUC curve. More on these could be found here.

Prediction models try to answer when and what should be offered to the customer. Some cons of this model are:

Not detailed enough for each product

Content is dependant on the already existing product (could not be applied to some new product without history)

Imbalanced classes (some products are quite rare so there is no enough positive samples)

Pros:

Focuses on time component, so the engagement with the customer is made at the right moment

Clear and precise metrics (F1, AUC, accuracy, …)

In case when the bank doesn’t have clear granularity in product types (for example, under Loans are assumed all the loans without split regarding its purpose to the car loan, house reparation loan, travel loan, etc. ) the segmentation could be combined so that particular content of interest is generated for each client (or groups of clients). For example, if a person is a student starting undergraduate studies, then the content of the Loan may be related to that special life event, or if a person is just starting a career, then the content of the Loan may be related to house/car purchase, etc. This is the combination of ML-based models with manually defined rules.

The next level of personalization would be to suggest products (such as Loan, Overdraft, Mortgage, or Insurance) with concrete amounts of money, the number of installments, purpose (Car purchase, summer vacation planning, house decoration, travel insurance, etc.) generated by machine learning models considering client financial state and interests. Instead of predicting whether the person will take a Loan or not, the model could also predict these additional product properties combined with information on taking the product. This could be implemented as a combination of two or more loss functions that should be optimized (loss for binary classifier, regression loss on continual values such as amount, number of installments, and cross-entropy loss for multiclass classifier (purpose)). Also, some extra input features that could be added here may be the results from segmentation (mentioned previously). Moreover, the bank could focus on detecting important life events such as Child Birth, Starting studies, Buying a house, Car purchases, Starting a new job, Salary Increase by simply combining prediction and segmentation results.

Recommendation

In cases where a bank has an enormous number of products (or items of any kind, in general) and there is potential to target its customers often (mistakes are not penalized hardly by customer satisfaction) use of a recommendation system is a natural choice. Items are preferred to be of the same type, repeatable, and easily accessible for users. The question that one may ask here is: Why would the bank have such a gigantic number of products or items? This depends on the bank’s business strategy and the product portfolio they want to offer. I will give a few examples:

Products such as Credit Cards could be intended for particular spending purposes which will allow the users to easily pay for it. Car, appliances, tech equipment, house, apartment, insurance, traveling, etc., are just some of them. A combination of the type of a credit card and any of the spending categories is a separate product. The number of these could be quite large and suggesting the customer the right one becomes a recommendation system problem.

As part of the business strategy, the bank may recommend to the client products or stores from its corporate clients. In this way, the bank may directly influence the business of companies while at the same time helps its customers make better purchases (through vouchers, discounts, promo codes for shopping in those companies). In this case, an item is a concrete store or branch with a location (unique name and address combination). For example, Health Food Store 1st Ave New York, Nike Store 12 Rue Linois Paris, Artour Tourist Agency Via del Colosseo 27 Rome, etc. Since the bank already has information on which customer interacted with which item, it is suitable for a recommendation system where on the one side there are companies containing different stores while on the other there are clients interested in some of the stores. Furthermore, if the bank has a client’s location, it could recommend the closest items in real-time through digital channels.

Bank, Clients and Companies creating an ecosystem

Deep recommendation systems

This is quite an extensive field as there are many kinds of these systems. Some of the most successful are deep recommendation systems, mainly run nowadays by companies such as Netflix, Spotify, Facebook, etc reporting outstanding results. Traditionally speaking, there are content-based and collaborative-based recommendation systems. The first one is matching the user with items based on the features from both of them, while the second one is matching the user with items based on the interactions of similar users to similar items. There are also hybrid approaches content-collaborative where both of these are taken into consideration.

Collaborative filtering

In this article, I will focus on one of the basic yet powerful architectures for collaborative filtering introduced in this paper.

Source: arxiv.org

The idea is to represent user and item inputs as sparse one-hot encodings of dimensions M (number of users) and N (number of items), respectively. After the encoding, there goes the embeddings layer (dimension K) which tries to learn a user/item latent representation. These two representations are then merged (concatenation or scalar dot-product) and fed to a feed-forward neural network (Neural CF Layers). The output is a sigmoid function resulting in the probability that processed user and item interacted actually. If there is an interaction then the target is a positive class 1, otherwise negative 0. Looking at all possible combinations of user-item pairs, it is obvious that just a few of them are positive while most are negative. Because of this disbalance, during training, not all negative pairs are used, but rather those chosen by the “negative sampling” approach (for each positive sample randomly choose n negatives, ration 1:n). Train and validation datasets are formed from the user perspective. For each user, all interactions with items except the last one are used as positive samples in the train set, while the last one is used for validation. Negative samples are added to both datasets using the above-mentioned approach, while for validation there is an extra condition that negative samples are not matching those included in the train set. The collaborative filtering approach assumes that all users are present during the training procedure so that each user is able to form an embedding. This might be a problem for new users since “cold start” would occur until users interact with some of the items. Moreover, the model needs to be trained all over again since new users don’t have learned embeddings. This is where content-based recommenders come in rescue :)

Content-based recommender

While collaborative filtering represents the user or item as one-hot encoding and based on the interactions learns its representations, content-based systems tend to represent it using features that describe it better. Depending on the nature of the features, there might be multiple embeddings, each of them learning representation for the particular type of features. These are all merged in some way (cosine similarity, scalar dot-product, concatenation) resulting in one embedding for a user and one for an item. Features that represent user and item don’t have to be of the same type, in general. The rest of the architecture is more or less the same as for collaborative filtering.

Embedding is learned from user/item features

This recommender is particularly popular since it doesn’t require interactions between user and item in order to make a recommendation. As a user is represented with features that are independent of items or other users, the system is able to extract user embedding and calculate a probability of interacting with all existing items. The “cold start” problem is therefore solved!

The important thing here is that special attention should be committed to features creation since the model performance is dependant on this to great extent. Besides internal information that the bank has for each company/item (and therefore its stores/branches/products), additional information could be obtained from external sources such as Google Places, OpenCage Geocoder, MapQuest, etc. There is information such as the type of the store, rating, pricing level, activity level, open hours, customer reviews, etc. This type of data that specifies users’ preferences combined with their geo-location could enable meaningful recommendations in real-time, too.

Pros:

The model is able to generate recommendations for new users, as well as to be retrained only for new users (not the whole set)

It can capture specific “fine” user interests and suggest corresponding “rare” items, which very few other users are interested in

Cons:

The performance of the model mostly depends on the proper feature selection and demands strong domain knowledge

Recommendations are usually related to the user interests (not those that might be interesting for the user based on similar users)

In cases where both approaches are combined and embeddings are formed from features, as well as from interactions, this is called hybrid systems.

Metrics used for this type of problem are usually Hit Ratio and NDCG (Normalized Discounted Cumulative Gain), but more on this in the article.

Here are some blogs on deep recommendation systems types:

Conclusion

The goal of this article was to analyze current approaches on how personalization is done in the banking industry and to try to list some parts of ML which could be used in it. In practice, there is no general solution to these types of problems, but a lot of experimentation and data wrangling is needed in order to get to the winning formula. Moreover, the results depend a lot on the data quality and sometimes even the best state-of-the-art models are not able to give satisfying results. These are all factors that should be considered when starting a personalization system in the banking industry.

On the other hand, understanding customer needs and approaching each in a unique way offering services and products that are convenient, relevant, and valuable at a certain moment is a MUST. Even though it requires extra effort to be done, it is worth committing to it in the long run. Banks that are focusing on their customers have a SUCCESSFUL strategy and Machine Learning is the best tool to help them accomplish this.

Hope you enjoyed reading this article! I would be grateful for any kind of feedback that could help me in my further writing, as well as to hear your thoughts on this topic :) Here is my mail srdjanjovanovic1408@gmail.com and Linkedin profile, so feel free to contact me anytime.