Credit Card Fraud Detection. Fraud detection using the multivariate... | by Christos Mousmoulas | Towards Data Science

Nowadays most of the transactions take place online, meaning that credit cards and other online payment systems are involved. This method is convenient both for the company and for the consumer. Consumers save time because they don’t have to go to the store to make their purchases and companies save money by not owing physical stores and avoiding expensive rental payments. It seems that the digital age brought some highly useful features which changed the way that both companies and consumers interact with each other but with one cost... Companies need to hire skilled software engineers and penetration testers to make sure that all the transactions are legal and non-fraudulent. Those people are designing the company’s servers in a way that the client has no control over critical transaction parts such as payment amount. With careful designing most (if not all) of the problems can be eliminated but even the framework which was used to create the server is not perfect. For example, if you follow the django framework release notes, you will see that there are many bug fixes among versions, thus a company should not rely only on its engineering skills.
In this article, I will present a way to detect if someone bypasses the security walls and makes an illegal transaction. You can find the code and the dataset in my github repo but I strongly encourage you to build it along with the instructions of this article.
The dataset consists of credit card transactions whose features are the product of PCA analysis and thus we don’t know what they represent except from the ‘Amount’, ‘Time’ and ‘Class’ ones. ‘Amount’ is the price of each transaction, ‘Time’ is the <<seconds elapsed between each transaction and the first transaction>> and ‘Class’ represents a fraud transaction when its value equals 1 and a valid transaction when its value equals 0. Initially, we will install the required packages so grub the ‘requirements.txt’ file from my GitHub repo, create a virtual environment install them:
pip install -r requirements.txt
Initially create a ‘main.py’ file and import the following:
import numpy as npimport pandas as pdimport sklearnfrom scipy.stats import normfrom scipy.stats import multivariate_normalfrom sklearn.preprocessing import MinMaxScalerimport matplotlib.pyplot as pltimport seaborn as sns
Now we will read the dataset and check if there is any missing value:
df = pd.read_csv('creditcardfraud/creditcard.csv')# missing valuesprint("missing values:", df.isnull().values.any())
There are no missing values so we can proceed with the visualizations to better understand our data. Let’s plot how balanced our dataset is:
# plot normal and fraudcount_classes = pd.value_counts(df['Class'], sort=True)count_classes.plot(kind='bar', rot=0)plt.title("Distributed Transactions")plt.xticks(range(2), ['Normal', 'Fraud'])plt.xlabel("Class")plt.ylabel("Frequency")plt.show()
We see that our data is highly imbalanced. Therefore, we cannot use any supervised learning algorithm directly because it will overfit based on the ‘Normal’ examples. At this point we are still in the data investigation-analysis stage so let’s plot the heatmap:
# heatmapsns.heatmap(df.corr(), vmin=-1)plt.show()
It seems that we don’t have highly correlated features although there is a slightly negative correlation between ‘V2’ and ‘Amount’ (label not shown in the above heatmap but it is near the ‘Class’ feature) features. The ‘Amount’ feature is slightly correlated with the other features as well, meaning that it can partially be calculated by them so we can try dropping it (from my tests, this gave a nice amount of improvement to the final score). There is also a little correlation between ‘Time’ and other features but we will see a more important reason to drop it in the aftermath of this article.
Until now, we did some nice data analysis but we didn’t do the most important part which is, learning “where our data came from”. Thus the next graph will be the data distribution one:
fig, axs = plt.subplots(6, 5, squeeze=False)for i, ax in enumerate(axs.flatten()):    ax.set_facecolor('xkcd:charcoal')    ax.set_title(df.columns[i])    sns.distplot(df.iloc[:, i], ax=ax, fit=norm,                 color="#DC143C", fit_kws={"color": "#4e8ef5"})    ax.set_xlabel('')fig.tight_layout(h_pad=-1.5, w_pad=-1.5)plt.show()
I encourage you to execute the above code snippet in your local machine to see the results better. The blue line indicates the actual Gaussian distribution, while the red one our data’s probability density function. We see that almost every feature comes from the normal (or gaussian) distribution except from the ‘Time’ one. Therefore, this will be our motivation to do fraud detection with the multivariate Gaussian distribution. This method works only for features from the gaussian distribution, thus if your data is not gaussian-like you can transform it to be with a method that I describe in this article. For example, you can try this technique on features ‘V26’, ‘V4’, ‘V1’ and see if you get improvements in our final score. The ‘Time’ feature comes from the bimodal distribution which can’t be transformed to gaussian-like, thus we will discard it. Another reason to discard ‘Time’ feature is that it doesn’t seem to contain extreme values like the features shown in the other graphs.
Moreover, the algorithm which we will be using is sensitive to distance metrics so if we scale the features to a fixed range, we will obtain much better results. Add the following code snippet which will drop the features mentioned above and scale the others so that they take values between 0 and 1:
classes = df['Class']df.drop(['Time', 'Class', 'Amount'], axis=1, inplace=True)cols = df.columns.difference(['Class'])MMscaller = MinMaxScaler()df = MMscaller.fit_transform(df)df = pd.DataFrame(data=df, columns=cols)df = pd.concat([df, classes], axis=1)
Note that MinMaxScaler won’t change the shape of the distribution so the outliers will still be on the right place.
At this point, we finished with our data analysis and preprocessing and we are ready to dive into the algorithm’s implementation details. The steps of the algorithm are the following:
Find features that may contain anomalous examples (done)Calculate the mean of each feature in the training set which contains usually the 60% of the normal transactionsCalculate the covariance matrix on the training setCalculate the multivariate normal pdf (probability density function) on the training set (given below)Calculate the multivariate normal pdf on the validation set (contains 50% of the fraudulent transactions and usually 20% of the normal ones)Calculate the multivariate normal pdf on the test set (contains 50% of the fraudulent transactions and usually 20% of the normal ones)Find a threshold based on the pdf from validation set which indicates that the pdf values (of any set) which are smaller than the threshold are outliersCalculate the outliers on the test set which are the sum of the pdf values that are smaller than the previous threshold
Find features that may contain anomalous examples (done)
Calculate the mean of each feature in the training set which contains usually the 60% of the normal transactions
Calculate the covariance matrix on the training set
Calculate the multivariate normal pdf (probability density function) on the training set (given below)
Calculate the multivariate normal pdf on the validation set (contains 50% of the fraudulent transactions and usually 20% of the normal ones)
Calculate the multivariate normal pdf on the test set (contains 50% of the fraudulent transactions and usually 20% of the normal ones)
Find a threshold based on the pdf from validation set which indicates that the pdf values (of any set) which are smaller than the threshold are outliers
Calculate the outliers on the test set which are the sum of the pdf values that are smaller than the previous threshold
where μ is the mean, det is the determinant, Σ is the covariance matrix, and κ is the dimension of the space where x takes values.
In general, this pdf returns the <<confidence>> that a transaction is normal. If that number is lower than the threshold then the transaction is an outlier.
And yes, I know that the above pdf hurts your eyes (just like mine) but it is already implemented in the scipy package...
Continuing with our code, create a file called ‘functions.py’ where we will add useful functions to implement certain stages of the algorithm and place the following function which will split our dataset into train, validation and test sets:
import pandas as pdimport numpy as npdef train_validation_splits(df):    # Fraud Transactions    fraud = df[df['Class'] == 1]    # Normal Transactions    normal = df[df['Class'] == 0]    print('normal:', normal.shape[0])    print('fraud:', fraud.shape[0])    normal_test_start = int(normal.shape[0] * .2)    fraud_test_start = int(fraud.shape[0] * .5)    normal_train_start = normal_test_start * 2    val_normal = normal[:normal_test_start]    val_fraud = fraud[:fraud_test_start]    validation_set = pd.concat([val_normal, val_fraud], axis=0)    test_normal = normal[normal_test_start:normal_train_start]    test_fraud = fraud[fraud_test_start:fraud.shape[0]]    test_set = pd.concat([test_normal, test_fraud], axis=0)    Xval = validation_set.iloc[:, :-1]    Yval = validation_set.iloc[:, -1]    Xtest = test_set.iloc[:, :-1]    Ytest = test_set.iloc[:, -1]    train_set = normal[normal_train_start:normal.shape[0]]    Xtrain = train_set.iloc[:, :-1]    return Xtrain.to_numpy(), Xtest.to_numpy(), Xval.to_numpy(), Ytest.to_numpy(), Yval.to_numpy()
Now add the function below which will calculate the mean and the covariance matrix:
def estimate_gaussian_params(X):    """    Calculates the mean and the covariance for each feature.    Arguments:    X: dataset    """    mu = np.mean(X, axis=0)    sigma = np.cov(X.T)    return mu, sigma
Back to our “main.py”, import and call the above functions along with the multivariate normal pdf for each of our sets:
(Xtrain, Xtest, Xval, Ytest, Yval) = train_validation_splits(df)(mu, sigma) = estimate_gaussian_params(Xtrain)# calculate gaussian pdfp = multivariate_normal.pdf(Xtrain, mu, sigma)pval = multivariate_normal.pdf(Xval, mu, sigma)ptest = multivariate_normal.pdf(Xtest, mu, sigma)
Now it’s time to refer to the threshold (or ‘epsilon’). In general, a good practice is to initialize the threshold with the minimum value of the pdf and increment it with a small step until you reach the maximum pdf while saving each threshold value in a vector. For our problem, I found that the values from the pdf can effectively be used to create the threshold vector. After creating the vector, we create a ‘for’ loop and iterate over it. In every iteration, we compare the current threshold with the values of the pdf which yield our predictions. We then calculate the F1 score based on our predictions and the ground truth values and if the F1 score that we found is greater than the previous one, we override a “best threshold” variable. At the end of the ‘for’ loop, we have the epsilon value that generated the best F1 score.
Note that we can’t use accuracy as our metric! If we identify all the transactions as ‘normal’ we could have 99% accuracy and a useless algorithm.
To implement the above, add the following functions to the “functions.py”:
def metrics(y, predictions):    fp = np.sum(np.all([predictions == 1, y == 0], axis=0))    tp = np.sum(np.all([predictions == 1, y == 1], axis=0))    fn = np.sum(np.all([predictions == 0, y == 1], axis=0))    precision = (tp / (tp + fp)) if (tp + fp) > 0 else 0    recall = (tp / (tp + fn)) if (tp + fn) > 0 else 0    F1 = (2 * precision * recall) / (precision +                                     recall) if (precision + recall) > 0 else 0    return precision, recall, F1def selectThreshold(yval, pval):    e_values = pval    bestF1 = 0    bestEpsilon = 0    for epsilon in e_values:        predictions = pval < epsilon        (precision, recall, F1) = metrics(yval, predictions)        if F1 > bestF1:            bestF1 = F1            bestEpsilon = epsilon    return bestEpsilon, bestF1
Finally, import the functions in our “main.py” file and call them to return our threshold and the F1 score on the validation set, and to evaluate our model on our test set:
(epsilon, F1) = selectThreshold(Yval, pval)print("Best epsilon found:", epsilon)print("Best F1 on cross validation set:", F1)(test_precision, test_recall, test_F1) = metrics(Ytest, ptest < epsilon)print("Outliers found:", np.sum(ptest < epsilon))print("Test set Precision:", test_precision)print("Test set Recall:", test_recall)print("Test set F1 score:", test_F1)
The results are:
Best epsilon found: 5e-324 Best F1 on cross validation set: 0.7852998065764023 Outliers found: 210 Test set Precision: 0.9095238095238095 Test set Recall: 0.7764227642276422 Test set F1 score: 0.837719298245614
which are pretty good!
You reached the end of this tutorial, I hope that you found this information useful. Please feel free to comment, and ask about things you didn’t understand and I will try to reply as soon as possible!
Originally published at Fraud detection in Python (devnal.com).