Using Machine Learning To Detect Fraud | by Kurtis Pykes | Towards Data Science

Introduction
The Fraud detection problem
The Data
Building the Framework
Machine Learning is a subset of Artificial Intelligence (AI) that provides systems with the ability to automatically learn and improve from experience without being explicitly programmed. In saying that, it has been made possible for us (humans) to supply a large set of data to a computer for the computer to learn patterns so that it may learn how to make a decision when faced with a new instance or instances — When I discovered this insight I instantly knew the world is about to change.
It reveals that fraud is costing the global economy £3.89 trillion, with losses rising by 56% in the past decade. — Crowe UK
Having been a victim of fraud, coming up with ideas to prevent it from ever happening to me (and anyone else for that matter) again essentially contributed to me wondering into a totally different field from what I was accustomed to.
In Machine Learning terminology, problems such as the Fraud Detection problem may be framed as a classification problem, of which the goal is to predict the discrete label 0 or 1 where 0 generally suggest that a transaction is non-fraudulent and 1 suggest that the transaction seems to be fraudulent.
Hence, this problem requires practitioners to build models that are intelligent enough to be capable of accurately detecting fraudulent and non-fraudulent transactions given various users transaction data — which is often anonymous to protect users privacy.
Since depending exclusively on rule-based systems is not the most effective strategy, Machine learning has been the approach many financial institutes are taking to combat the problem.
What makes this problem (fraud detection) so challenging is that when we model it in the real world, the majority of transactions that occur are genuine transactions and only a very small portion account for fraudulent behavior. This means we deal with the problem of imbalanced data — My post on Oversampling and Undersampling is one approach to dealing with this problem. However, for this post our main focus will be on beginning our machine learning framework to detect fraud — If you are unfamiliar with building your own framework you may want to read through Structuring Machine Learning Projects before completing this post.
towardsdatascience.com
towardsdatascience.com
The data was put together by researchers from IEEE Computational Intelligence Society (IEEE-CIS) for the task of predicting the probability that an online transaction is fraudulent, as denoted by the binary target isFraud.
Note: The data section is copied from Kaggle Data section for this competition.
The data is broken into two files identity and transaction, which are joined by TransactionID. Not all transactions have corresponding identity information.
Categorical Features — Transaction
ProductCD
card1 - card6
addr1, addr2
P_emaildomain
R_emaildomain
M1 - M9
Categorical Features — Identity
DeviceType
DeviceInfo
id_12 - id_38
The TransactionDT feature is a timedelta from a given reference datetime (not an actual timestamp).
You can read more about the data from this post by the competition host.
Files
train_{transaction, identity}.csv — the training set
test_{transaction, identity}.csv — the test set (you must predict the isFraud value for these observations)
sample_submission.csv — a sample submission file in the correct format
The first step of when working on any machine learning task is to build a reliable Cross Validation strategy.
towardsdatascience.com
Note: The general idea behind the framework was taken from no other than Abhishek Thakur — Github
A general approach to take when facing a imbalanced data problem is to use StratifiedKFold which splits the data randomly in such a way that we maintain the same class distribution.
I implemented out create folds as part of preprocessing.py.
This code merges the identity and transaction data from the train and test set then renames the columns names in the merged_test data since the id columns use “-” instead of “_” — which will cause problems later when we check to ensure we have the exact same column names in our test. Next, we add a column name called kfold to our training data and set the index in accordance with the fold it is in then save to a csv file.
You may of noticed that we import config and use it as a path to various things. All config is, is variables in another script so that we don’t have to repeatedly call those variables in different scripts.
It is very important to build your pipeline fast and in a manner that allows for fast iterations when working on a machine learning problem, so the next script we will build is our model_dispatcher.py of which we call our classifiers and train.py where we train our model.
Let’s start with model_dispatcher.py
Here, we have simply imported a Logistic Regression and Random Forest that and created a dictionary so we can call the algorithms into our train script by doing models["logistic_regression"] which will run the Logistic Regression model.
The train script is as follows...
Hopefully you can read code but if you can not, to sum up what is going on, we set training data as the values in the column kfold that and the values that are equal to the fold we passed is the test set. Then we do Label encoding to the categorical variables, and fill all missing values with 0 before training our data on a logistic regression model.
We get predictions on the current fold and print the ROC_AUC.
towardsdatascience.com
Note: As things stand, the code is does not run itself, so we’d have to pass the values for fold and model when we run each fold.
Let’s see the outputs of our Logistic Regression Model.
### Logistic Regression # Fold 0 ROC_AUC_SCORE: 0.7446056326560758# Fold 1ROC_AUC_SCORE: 0.7476247589462117# Fold 2ROC_AUC_SCORE: 0.7395710927094167# Fold 3ROC_AUC_SCORE: 0.7365641912867861# Fold 4ROC_AUC_SCORE: 0.7115696956435416
Those are pretty decent results to start with, but let’s use the more powerful random forest model and see if we can improve.
### Random Forest# Fold 0 ROC_AUC_SCORE: 0.9280242455299264# Fold 1ROC_AUC_SCORE: 0.9281600723876517# Fold 2ROC_AUC_SCORE: 0.9265254015330469# Fold 3ROC_AUC_SCORE: 0.9224746067992484# Fold 4ROC_AUC_SCORE: 0.9196977372298685
It’s pretty clear the random forest model is yielding much better results. Let’s do a late submission on to Kaggle and see where we rank on the leaderboards. This is of the most important parts — To do this we must run inference.py.
Note: The process of submitting to Kaggle is beyond the scope of this discussion so I will the score model gets and how that does on the Leaderboard.
Given this score translated to Kaggle’s private leaderboard (since it is the score on the public leaderboard), we would of ranked 3875/6351 (Top 61%) on Kaggle’s private leaderboard. Although, It doesn’t look so good from a Kaggle perspective, in a real world scenario we’d probably settle with this score depending on the task.
However, the goal of this project is not to come up with the best model, it is to create our own API which we will come back to in later post.
For building a quick pipeline for fast iteration, the code we have is okay but if we want to deploy this model we’d have to do plenty of clean up work so that we follow the Software Engineering Best Practices.
towardsdatascience.com
Detecting Fraud is a very common and challenging problem in the real world and increasing the accuracy is very important to prevent the embarrassment of customers cards being declined when making a genuine transaction at a store. We’ve have built a very simple approach using label encoding for categorical variables, filling all missing values with 0, and a random forest without any tuning or methods to handle the imbalance in data but our model still scores pretty highly. To improve the model, we may want to first look at the important features from the Random Forest model and drop less important features, or we can use other more powerful models like Light Gradient Boosting Machine and Neural Networks
Note: At the time of writing this script, the modules aren’t the best but it is in a format that would allow me to quickly iterate. In future work, I plan on deploying this model as an API on a cloud server.
Let’s continue the conversation on LinkedIn...