Supervised Machine Learning Technique for Anomaly Detection: Logistic Regression
By: Mahbubul Alam
Tags: Supervised Learning, Machine Learning, Logistic Regression, Anomaly Detection, Classification

Typically in previous articles, I create a small synthetic dataset on the fly and implement the algorithms with bare minimum codes to give an intuition on how they work. It’s going to be different today since it is a supervised classification problem and I have to follow all the essential steps. As you will see, data preparation alone is going to take quite a bit of space.

Step 1: Import libraries

We need quite a few libraries for this exercise for data wrangling, preparation of model inputs, model building and validation — all libraries coming from three big packages: pandas , nunpy and sklearn .

# data wrangling

import pandas as pd

import numpy as np # inputs data preparation

from sklearn.preprocessing import RobustScaler

from sklearn.model_selection import train_test_split # modeling

from sklearn.linear_model import LogisticRegression # model validation

from sklearn.metrics import classification_report

from sklearn.metrics import confusion_matrix

Step 2: data wrangling

I am using a popular dataset from Kaggle on credit card fraud detection. The dataset was released in the public domain by European cardholders after removing any user identifier. Let’s import and take a look at the datasets and see the features.

# import data

df = pd.read_csv("..\creditcard.csv") # view the column names

df.columns

# view the column names

df.columns >> Index(['Time', 'V1', 'V2', 'V3', 'V4', 'V5', 'V6', 'V7', 'V8', 'V9', 'V10', 'V11', 'V12', 'V13', 'V14', 'V15', 'V16', 'V17', 'V18', 'V19', 'V20', 'V21', 'V22', 'V23', 'V24', 'V25', 'V26', 'V27', 'V28', 'Amount', 'Class']

The dataset has 31 columns. The first column “Time” is the transaction timestamp, the second last column “Amount” is the transaction amount and the last column “Class” designates whether the transaction is fraudulent or not (fraud = 1, non-fraud = 0). The rest of the columns, “V1” to “V28” are unknown features and the values were scaled.

Now check out the number of fraud and no-fraud cases.

# number of fraud and non-fraud observations

frauds = len(df[df.Class == 1])

nonfrauds = len(df[df.Class == 0]) print("Frauds", frauds); print("Non-frauds", nonfrauds) >> Frauds 492

>> Non-frauds 284315

As I said, that’s quite an unbalanced dataset, only 492 fraud cases out of a quarter of a million observations. This represents only a 0.17% fraud cases in all of the transactions.

Step 3: Inputs data preparation

The dataset is pretty clean, so in the next step, we are just going to scale the “Amount” and “Time” columns to make sure they are of a similar format to the other columns.

## scaling the "Amount" and "Time" columns

rob_scaler = RobustScaler() df['scaled_amount'] = rob_scaler.fit_transform(df['Amount'].values.reshape(-1,1))

df['scaled_time'] = rob_scaler.fit_transform(df['Time'].values.reshape(-1,1)) # now drop the original columns

df.drop(['Time','Amount'], axis=1, inplace=True)

Now let’s define the X and y input variables.

# define X and y variables

X = df.loc[:, df.columns != 'Class']

y = df.loc[:, df.columns == 'Class']

Since this is an unbalanced dataset, we need to take an extra step to take a subsample — known as undersampling — so the machine doesn’t have to find a needle in the haystack to learn from examples.

# number of fraud cases

frauds = len(df[df.Class == 1]) # selecting the indices of the non-fraud classes

fraud_indices = df[df.Class == 1].index

nonfraud_indices = df[df.Class == 0].index # from all non-fraud observations, randomly select observations equal to number of fraud observations

random_nonfraud_indices = np.random.choice(nonfraud_indices, frauds, replace = False)

random_nonfraud_indices = np.array(random_nonfraud_indices) # appending the 2 indices

under_sample_indices = np.concatenate([fraud_indices,random_nonfraud_indices]) # undersample dataset

under_sample_data = df.iloc[under_sample_indices,:] # now split X, y variables from the under sample data

X_undersample = under_sample_data.loc[:, under_sample_data.columns != 'Class']

y_undersample = under_sample_data.loc[:, under_sample_data.columns == 'Class']

Splitting data into training and testing sets before feeding into the model.

# split dataset

X_train_undersample, X_test_undersample, y_train_undersample, y_test_undersample = train_test_split(X_undersample, y_undersample, test_size = 0.3, random_state = 0)

Step 4: Model building

As usual, the actual model building takes only 3 lines of code, to instantiate, fit and predict on the given dataset. Since this is for demonstration purposes only, we are going to use default parameters without tuning anything.

# instantiate model

model = LogisticRegression() # fit

model.fit(X_train_undersample, y_train_undersample) # predict

y_pred = model.predict(X_test_undersample)

Step 5: Model evaluation

As a standard model evaluation metric, we are producing a classification report and confusion metrix.