Credit Card Fraud Detection using Autoencoders in H2O | by Maneesha Rajaratne | Towards Data Science

Frauds in the finance field are very rare to be identified. Because of that, it can do a severe damage to the financial field. It is estimated that fraud costs at least $80 billion a year across all lines of insurance. If there is a small possibility of detecting fraudulent activities, that can do a major impact on annual losses. That is why financial companies invest in machine learning as a preemptive approach to tackling fraud.
The benefits of using a machine learning approach are that,
It helps to find hidden and implicit correlations in data.
Faster data processing and less manual work
Automatic detection of possible fraud scenarios.
The best way to detect frauds is anomaly detection.
Anomaly detection is a technique to identify unusual patterns that do not conform to the expected behaviors, called outliers. It has many applications in business from fraud detection in credit card transactions to fault detection in operating environments. Machine learning approaches for Anomaly detection;
K-Nearest Neighbor
Autoencoders — Deep neural network
K-means
Support Vector Machine
Naive Bayes
Today we will be using Autoencoders to train the model.
Most of us are not familiar with this model. Autoencoders is an unsupervised Neural Network. It is a data compression algorithm which takes the input and going through a compressed representation and gives the reconstructed output.
As for the dataset we will be using Credit Card Transaction dataset provided by Kaggle: https://www.kaggle.com/mlg-ulb/creditcardfraud
The dataset includes 284,807 transactions. among them, 492 transactions are labeled as frauds. Because of this, the dataset is highly imbalanced. It contains only numerical variables. Feature ‘Time’ contains the seconds elapsed between each transaction and the first transaction in the dataset. The feature ‘Amount’ is the transaction Amount, this feature can be used for example-dependent cost-sensitive learning. Feature ‘Class’ is the response variable and it takes value 1 in case of fraud and 0 otherwise.
You can find my Kaggle Kernel here: https://www.kaggle.com/maneesha96/credit-card-fraud-detection-using-autoencoders
Full code: https://github.com/Mash96/Credit-Card-Fraud-Detection
Then Let's get started!!!
We will be using H2O as the ML platform today. You can find more info here: https://www.h2o.ai
import h2oimport matplotlib.pyplot as pltfrom pylab import rcParamsimport numpy as np # linear algebraimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)import osfrom h2o.estimators.deeplearning import H2OAutoEncoderEstimator, H2ODeepLearningEstimator
Initialize H2O server
h2o.init(max_mem_size = 2) # initializing h2o serverh2o.remove_all()
Loading dataset using pandas data frame
creditData = pd.read_csv(r"File_Path\creditcard.csv") creditData.describe()# H2O method# creditData_df = h2o.import_file(r"File_Path\creditcard.csv") 
creditData.shape> (284807, 31)
Checking for null values in the dataset
creditData.isnull().values.any() # pandas method# creditData_h2o.na_omit() # h2o method# creditData_h2o.nacnt() # no missing values found> False
In order to proceed we need to convert the pandas data frame to H2O data frame
# Turns python pandas frame into an H2OFramecreditData_h2o  = h2o.H2OFrame(creditData)# Let’s plot the Transaction class against the Frequencylabels = [‘normal’,’fraud’]classes = pd.value_counts(creditData[‘Class’], sort = True)classes.plot(kind = ‘bar’, rot=0)plt.title(“Transaction class distribution”)plt.xticks(range(2), labels)plt.xlabel(“Class”)plt.ylabel(“Frequency”)
fraud = creditData[creditData.Class == 1]normal = creditData[creditData.Class == 0]# Amount vs Classf, (ax1, ax2) = plt.subplots(2,1,sharex=True)f.suptitle('Amount per transaction by class')ax1.hist(fraud.Amount, bins = 50)ax1.set_title('Fraud List')ax2.hist(normal.Amount, bins = 50)ax2.set_title('Normal')plt.xlabel('Amount')plt.ylabel('Number of Transactions')plt.xlim((0, 10000))plt.yscale('log')plt.show()
# time vs Amountf, (ax1, ax2) = plt.subplots(2, 1, sharex=True)f.suptitle('Time of transaction vs Amount by class')ax1.scatter(fraud.Time, fraud.Amount)ax1.set_title('Fraud List')ax2.scatter(normal.Time, normal.Amount)ax2.set_title('Normal')plt.xlabel('Time (in seconds)')plt.ylabel('Amount')plt.show()
#plotting the dataset considering the classcolor = {1:'red', 0:'yellow'}fraudlist = creditData[creditData.Class == 1]normal = creditData[creditData.Class == 0]fig,axes = plt.subplots(1,2)axes[0].scatter(list(range(1,fraudlist.shape[0] + 1)), fraudlist.Amount,color='red')axes[1].scatter(list(range(1, normal.shape[0] + 1)), normal.Amount,color='yellow')plt.show()
The Time variable is not giving an impact on the model prediction. This can figure out from data visualization. Before moving on to the training part, we need to figure out which variables are important and which are not. So we can drop the unwanted variables.
features= creditData_h2o.drop(['Time'], axis=1)
Split the data frame as training set and testing set keeping 80% for the training set and rest to the testing set.
train, test = features.split_frame([0.8])print(train.shape)print(test.shape)> (227722, 30)> (57085, 30)
Our dataset has a lot of non-fraud transactions. Because of this for the model training, we only send non-fraud transactions. So that the model will learn the pattern of normal transactions.
# converting to pandas dataframetrain_df = train.as_data_frame()test_df = test.as_data_frame()train_df = train_df[train_df['Class'] == 0]# drop the Class variabletrain_df = train_df.drop(['Class'], axis=1)Y_test_df = test_df['Class'] # true labels of the testing settest_df = test_df.drop(['Class'], axis=1)train_df.shape> (227335, 29)
train_h2o = h2o.H2OFrame(train_df) # converting to h2o frametest_h2o = h2o.H2OFrame(test_df)x = train_h2o.columns
When building the model, 4 fully connected hidden layers were chosen with, [14,7,7,14] number of nodes for each layer. First two for the encoder and last two for the decoder.
anomaly_model = H2ODeepLearningEstimator(activation = "Tanh",                               hidden = [14,7,7,14],                               epochs = 100,                               standardize = True,                                stopping_metric = 'MSE',                                 loss = 'automatic',                                train_samples_per_iteration = 32,                                shuffle_training_data = True,                                    autoencoder = True,                               l1 = 10e-5)anomaly_model.train(x=x, training_frame = train_h2o)
Variable Importance : In H2O there is a special way of analyzing which variables are giving higher impact on the model.
anomaly_model._model_json['output']['variable_importances'].as_data_frame()
Visualization
# plotting the variable importancercParams['figure.figsize'] = 14, 8#plt.rcdefaults()fig, ax = plt.subplots()variables = anomaly_model._model_json['output']['variable_importances']['variable']var = variables[0:15]y_pos = np.arange(len(var))scaled_importance = anomaly_model._model_json['output']['variable_importances']['scaled_importance']sc = scaled_importance[0:15]ax.barh(y_pos, sc, align='center', color='green', ecolor='black')ax.set_yticks(y_pos)ax.set_yticklabels(variables)ax.invert_yaxis()ax.set_xlabel('Scaled Importance')ax.set_title('Variable Importance')plt.show()
# plotting the lossscoring_history = anomaly_model.score_history()%matplotlib inlinercParams['figure.figsize'] = 14, 8plt.plot(scoring_history['training_mse'])plt.title('model loss')plt.ylabel('loss')plt.xlabel('epoch')
The testing set has both normal and fraud transactions in it. The Autoencoder will learn to identify the pattern of the input data. If an anomalous test point does not match the learned pattern, the autoencoder will likely have a high error rate in reconstructing this data, indicating anomalous data. So that we can identify the anomalies of the data. To calculate the error, it uses Mean Squared Error(MSE)
test_rec_error = anomaly_model.anomaly(test_h2o) # anomaly is a H2O function which calculates the error for the dataset# converting to pandas dataframetest_rec_error_df = test_rec_error.as_data_frame()# plotting the testing dataset against the errortest_rec_error_df['id']=test_rec_error_df.indexrcParams['figure.figsize'] = 14, 8test_rec_error_df.plot(kind="scatter", x='id', y="Reconstruction.MSE")plt.show()
# predicting the class for the testing datasetpredictions = anomaly_model.predict(test_h2o)error_df = pd.DataFrame({'reconstruction_error': test_rec_error_df['Reconstruction.MSE'],                        'true_class': Y_test_df})error_df.describe()
# reconstruction error for the normal transactions in the testing datasetfig = plt.figure()ax = fig.add_subplot(111)rcParams['figure.figsize'] = 14, 8normal_error_df = error_df[(error_df['true_class']== 0) & (error_df['reconstruction_error'] < 10)]_ = ax.hist(normal_error_df.reconstruction_error.values, bins=10)
# reconstruction error for the fraud transactions in the testing datasetfig = plt.figure()ax = fig.add_subplot(111)rcParams['figure.figsize'] = 14, 8fraud_error_df = error_df[error_df['true_class'] == 1]_ = ax.hist(fraud_error_df.reconstruction_error.values, bins=10)
from sklearn.metrics import (confusion_matrix, precision_recall_curve, auc,                             roc_curve, recall_score, classification_report, f1_score,                             precision_recall_fscore_support)fpr, tpr, thresholds = roc_curve(error_df.true_class, error_df.reconstruction_error)roc_auc = auc(fpr, tpr)plt.title('Receiver Operating Characteristic')plt.plot(fpr, tpr, label='AUC = %0.4f'% roc_auc)plt.legend(loc='lower right')plt.plot([0,1],[0,1],'r--')plt.xlim([-0.001, 1])plt.ylim([0, 1.001])plt.ylabel('True Positive Rate')plt.xlabel('False Positive Rate')plt.show();
The accuracy is 0.9718
Since the data is highly imbalanced, it cannot be measured only by using accuracy. Precision vs Recall was chosen as the matrix for the classification task.
Precision: Measuring the relevancy of obtained results.
[ True positives / (True positives + False positives)]
Recall: Measuring how many relevant results are returned.
[ True positives / (True positives + False negatives)]
True Positives — Number of actual frauds predicted as frauds
False Positives — Number of non-frauds predicted as frauds
False Negatives — Number of frauds predicted as non-frauds.
precision, recall, th = precision_recall_curve(error_df.true_class, error_df.reconstruction_error)plt.plot(recall, precision, 'b', label='Precision-Recall curve')plt.title('Recall vs Precision')plt.xlabel('Recall')plt.ylabel('Precision')plt.show()
We need to find a better threshold that can separate the anomalies from normal. This can be done by calculating the intersection of the Precision/Recall vs Threshold graph.
plt.plot(th, precision[1:], label="Precision",linewidth=5)plt.plot(th, recall[1:], label="Recall",linewidth=5)plt.title('Precision and recall for different threshold values')plt.xlabel('Threshold')plt.ylabel('Precision/Recall')plt.legend()plt.show()
# plot the testing set with the thresholdthreshold = 0.01groups = error_df.groupby('true_class')fig, ax = plt.subplots()for name, group in groups:    ax.plot(group.index, group.reconstruction_error, marker='o', ms=3.5, linestyle='',            label= "Fraud" if name == 1 else "Normal")ax.hlines(threshold, ax.get_xlim()[0], ax.get_xlim()[1], colors="r", zorder=100, label='Threshold')ax.legend()plt.title("Reconstruction error for different classes")plt.ylabel("Reconstruction error")plt.xlabel("Data point index")plt.show();
import seaborn as snsLABELS = ['Normal', 'Fraud']y_pred = [1 if e > threshold else 0 for e in error_df.reconstruction_error.values]conf_matrix = confusion_matrix(error_df.true_class, y_pred)plt.figure(figsize=(12, 12))sns.heatmap(conf_matrix, xticklabels=LABELS, yticklabels=LABELS, annot=True, fmt="d");plt.title("Confusion matrix")plt.ylabel('True class')plt.xlabel('Predicted class')plt.show()
csr = classification_report(error_df.true_class, y_pred)print(csr)
Our model is catching most of the fraudulent data. In Autoencoders, it gives a good accuracy. But if we look into Precision and Recall of the dataset, it is not performing enough. As I mentioned earlier, there are other anomaly detection methods that perform well in highly imbalanced datasets.
I have tried more methods on this dataset. So I will see you soon with those. :)