Combining Actual Data with Simulated Data in Machine Learning | by Benjamin Obi Tayo Ph.D. | Towards Data Science

With machine learning, you need a lot of data, so sometimes it makes sense to combine actual data with simulated data to find the right patterns in the data or for model building.
Central limit says that the larger the size of the dataset the better, as population mean approximates sample mean when sample size becomes really large. By generating replicated copies of the sample dataset, we can estimate the sample mean across different populations to ensure that the mean value is stable. We can also estimate the variance (uncertainty) in the prediction of the mean value.
In order to simulate data from a given sample data, we need to be able to identify patterns in the sample data and also have some knowledge about the features and how they are distributed. For example, if I have a small sample of male heights, and I know that heights follow a normal distribution, I could generate a dataset of male heights using the following:
mu = mean(male_heights)sd = sd(male_heights)N = 1000simulated_data = rnorm(N, mu,sd)
This article will discuss how Monte-Carlo (MC) simulation can be used for generating approximate replicated copies of the sample dataset. Each of these blocks of datasets is then trained. Then the overall performance of the model is then calculated by performing an ensemble average over the data blocks including actual and simulated data.
Introduction: Predicting the status of a loan is an important problem in risk assessment. A bank or financial organization has to be able to estimate the risk involved before granting a loan to a customer. Data Science and predictive analytics play an important role in building models that can be used for predicting the probability of loan default. In this project, we are provided with the loan_timing.csv dataset containing 50000 data points. Each data point represents a loan, and two features are provided as follows:
The column with header “days since origination” indicates the number of days that elapsed between origination and the date when the data was collected.For loans that charged off before the data was collected, the column with header “days from origination to charge-off” indicates the number of days that elapsed between origination and charge-off. For all other loans, this column is blank.
The column with header “days since origination” indicates the number of days that elapsed between origination and the date when the data was collected.
For loans that charged off before the data was collected, the column with header “days from origination to charge-off” indicates the number of days that elapsed between origination and charge-off. For all other loans, this column is blank.
Definition of Technical Terms
Origination: This refers to the date when a borrower receives a loan from a lender.Charge-off (loan default) Status: The borrower then makes regular repayments, until the borrower stops making payments, typically due to financial hardship, before the end of the loan term. This event is called charge-off, and the loan is then said to have charged off or in default state.Current or Active Status: The borrower continues making repayments over the entire loan term. At this point, the debt has been fully repaid.Loan Term: Period over which a loan agreement is in force, and before or at the end of which the loan should either be repaid or renegotiated for another term. In this example, we consider a loan with a term or duration of 3 years.
Origination: This refers to the date when a borrower receives a loan from a lender.
Charge-off (loan default) Status: The borrower then makes regular repayments, until the borrower stops making payments, typically due to financial hardship, before the end of the loan term. This event is called charge-off, and the loan is then said to have charged off or in default state.
Current or Active Status: The borrower continues making repayments over the entire loan term. At this point, the debt has been fully repaid.
Loan Term: Period over which a loan agreement is in force, and before or at the end of which the loan should either be repaid or renegotiated for another term. In this example, we consider a loan with a term or duration of 3 years.
Project Objective: The goal of this project is to use techniques of data science to estimate what fraction of these loans (50,000 customer records in the loan_timing.csv dataset) will have charged off during the 3-year loan term.
The dataset and R code for this article can be downloaded from this repository: https://github.com/bot13956/Monte_Carlo_Simulation_Loan_Status.
library(readr)library(tidyverse)library(broom)library(caret)
df<-read_csv("loan_timing.csv",na="NA")names(df)=c("origination","chargeoff")# partition data set into two: default (charged off ) and currentindex<-which(!(df$chargeoff=="NA"))default<-df%>%slice(index)current<-df%>%slice(-index)
A) Actual Data
# Plot of days to charge-off vs. days since origination for defaulted loans using actual datadefault%>%ggplot(aes(origination,chargeoff))+geom_point()+xlab('days since origination')+ ylab('days to charge-off')+ggtitle("days to charge-off vs. days since origination")+theme(plot.title = element_text(color="black", size=12, hjust=0.5, face="bold"),axis.title.x = element_text(color="black", size=12, face="bold"),axis.title.y = element_text(color="black", size=12, face="bold"),legend.title = element_blank())
B) Simulated Data
# Monte Carlo Simulation of Defaulted Loansset.seed(2)N <- 3*365 # loan duration in daysdf_MC<-data.frame(u=round(runif(15500,0,N)),v=round(runif(15500,0,N)))df_MC<-df_MC%>%filter(v<=u)df_MC<-df_MC%>%filter(u<=730 & v<=730) #select loans within first 2 yearsdf_MC[1:nrow(default),]%>%ggplot(aes(u,v))+geom_point()+xlab('days since origination')+ylab('days to charge-off')+ggtitle("MC simulation of days to charge-off vs. days since origination")+theme(plot.title = element_text(color="black", size=12, hjust=0.5, face="bold"),axis.title.x = element_text(color="black", size=12, face="bold"),axis.title.y = element_text(color="black", size=12, face="bold"),legend.title = element_blank())
Because there is randomness associated with the charge-off of a loan, we see that MC simulation provides a good approximation for the distribution of defaulted loans.
Predictions: Since we have demonstrated that the relationship between days to charge-off and days since origination in the first 2 years (i.e. 0 to 730 days) can be approximated using an MC simulation, we can predict the fraction of loans that will be charged off by the time all of their 3-year terms are finished using MC simulation.
The total number of charged-off loans in our dataset is 3,305. This means that there are 46,695 loans that are currently active. Of these active loans, a certain proportion will default over the 3-year period. To estimate the total fraction of defaulted loans, we simulated defaulted loans with charge-off and days since origination covering the entire duration of the loan (i.e. 0 to 1095 days), then by appropriate scaling, we computed the fraction of loans that will have charged off after the 3-year term i.e., 1095 days.
# Predicting fraction of these loans will have charged off by the time all of their 3-year term is finished.set.seed(2)B<-1000fraction<-replicate(B, {df2<-data.frame(u=round(runif(50000,0,N)),v=round(runif(50000,0,N)))df2<-df2%>%filter(v<=u)b2<-(df2%>%filter(u<=730 & v<=730))total<-(nrow(df2)/nrow(b2))*nrow(default)100.0*(total/50000.0)})mean(fraction)# Histogram of total fraction of charged off loansfdf<-data.frame(fraction=fraction)fdf%>%ggplot(aes(fraction))+geom_histogram(color="white",fill="skyblue")+xlab('fraction of charged off loans after 3-year term')+ylab('count')+ggtitle("Histogram of total fraction of charged off loans")+theme(plot.title = element_text(color="black", size=12, hjust=0.5, face="bold"),axis.title.x = element_text(color="black", size=12, face="bold"),axis.title.y = element_text(color="black", size=12, face="bold"),legend.title = element_blank())# Calculate Confidence Interval of Percentage of Defaulted Loans after 3-year termmean<-mean(fraction)sd<-sd(fraction)confidence_interval<-c(mean-2*sd, mean+2*sd)confidence_interval
By creating N = 1000 random trials, we obtained the following distribution for the fraction of defaulted loans 3-year term:
Figure 6: Histogram for fraction of charged-off loans after 3-year term using N = 1000 samples.
Based on our calculations, the 95% confidence interval for the fraction of loans that will have charged off after the 3-year loan duration is accordingly 14.8% +/- 0.2%. So if 50,000 loans were given out with a loan term of 3 years, approximately 15% of these loans will default.
Conclusions: We have presented a simple model based on the MC simulation for predicting the fraction of loans that will default at the end of the 3-year loan duration period. Monte Carlo simulation is an important method that can be used in prescriptive analytics for the prescribing course of action to be taken in cases where the dataset is very stochastic in nature.
The dataset and R code for this article can be downloaded from this repository: https://github.com/bot13956/Monte_Carlo_Simulation_Loan_Status.