Home Credit Loan Default Risk?
By: Winston Fernandes
Tags: Kaggle, Artificial Intelligence, Deployment, Data Science, Machine Learning

5.1 Target Count

The target variable is 0 if the client will repay the loan and 1 if the client will default. It is clear looking at the graph that the data set is imbalanced.

Some features that were explored …

Amount credit (Credit amount of the loan): Amount Credit is the amount requested by the applicant.

5.2 Amount Credit

Analysis

The density is high for the amount less than 10⁶ for both types of applicants who can pay and have difficulty in paying loans.

The graph looks right-skewed.

Conclusion

It is clear that the data is highly overlapping between the people who are able to pay loan on time and the people who had difficulty in paying loan back.

So it is not of much use.

Amount Income Total (Income of the client):

5.3 Amount Income

Analysis:

AMT_INCOME_TOTAL is the income of the client.

The income of both types of clients is less than 20000000.0 except for one client who has an income of 117000000.0 and still is not able to pay the loan.

If we remove it and plot a graph of AMT_INCOME_TOTAL <(0.2X1e8) we removed the one outlier but we can’t clearly make out whether the majority of both classes are overlapping or not.

After plotting AMT_INCOME_TOTAL<(1X1e6) it is clear that majority values are highly overlapping for both classes

Conclusion:

Since the majority values of these features in both classes are overlapping this feature is not of much use.

Days employed (in years)

5.4 Days employed (in years)

Analysis:

After plotting the number of years employed, there are some clients in both targets who are working for 1000 years which are an outlier.

After removing the value 365243 in the number of days employed now the maximum number of years a person has worked in 50 years.

It is visible that the density of clients who have less than 10 years of experience had difficulties in repaying loans.

Conclusion:

There is slightly less overlapping in the maximum density of two target values.

So this feature will be useful.

Days_birth (in years)

5.5 Days birth(in years)

Analysis:

There is a majority of young age group between (20–40) are having difficulty in repaying the loan.

As the age increases, we see the age group (50–70) have the majority of repaying the loan back.

Conclusion:

There is slightly less overlapping in the maximum density of two target values. Younger clients are more likely to default

So this feature will be useful.

Feature Engineering is an important skill in such problems but creating features that actually affect the target variable is more important. My main task was finding the features in every table that is affecting the target variable.

Let's see the number of the document submitted by a person who was able to pay the loan and who wasn’t

5.6 Target 0

5.7 Target 1

Conclusion: Both actually submit the same proportion of the number of documents during the application. There is one client submitting 4 documents in the target 1 section which is different but not much helpful.

5.8 flag document 3

Analysis

Both types of clients mostly submit only 1 document which is document 3.

DOCUMENT_3 is the most common document submitted by the applicants.

FLAG_DOCUMENT_3==0 means not submitted document 3

FLAG_DOCUMENT_3==1 means submitted document 3

FLAG_DOCUMENT_3 is mostly submitted by the clients who can repay the loan and not submitted by the clients who cannot repay the loan but this could be also possible because the data is imbalanced.

Conclusion

So FLAG_DOCUMENT_3 could be a useful feature.

EXT_SOURCE_1, EXT_SOURCE_2, EXT_SOURCE_3 (Normalized score from external data source)

5.9 External Source

Analysis

External source 1 < 0.4 then the client will default and External source 1 > 0.4 client will repay, there is a visible separation between two classes.

External source 2 < 0.5 then the client will default and External source 2> 0.5 client will repay, there is a visible separation between two classes.

External source 3< 0.4 then the client will default and External source 3 > 0.4 client will repay, there is a visible separation between two classes.

Conclusion

EXT_SOURCE_1, EXT_SOURCE_2, EXT_SOURCE_3 these features are very useful

Encoding of Categorical values :

There a mainly two ways of encoding Label Encoder and One Hot Encoding. The label encoder gives an ordinal value to the variable e.g. 1,2,3 etc which is not valid in categorical columns of this data so I used One hot encoding.

One hot encoding can be done using pandas.get_dummies..

How to find the relation of features with the target variable??

1st Approach: Pearson correlation of coefficient and find the top 20 features negatively or positively correlated with the target.

5.10 Pearson’s Correlation of application train data with target variable

2nd Approach: Simply train a model on the data without any hyperparameter tuning and get the feature importance. I knew two of the library doing this one is Random Forest and the other Light Gradient Boosting Mechanism. I selected Light Gradient Boosting because it gave me a high ROC-AUC score on train data.

Light GBM is a fast, distributed, high-performance gradient boosting framework based on decision tree algorithm, used for ranking, classification and many other machine learning tasks.

Top 30 features after training LGBM on application train data

5.11 Lgbm top 30 feature on application train data

I considered the 2nd Approach as a priority because correlation is found out between two continuous data variable but here my target variable is discrete i.e {0,1} so the results are not completely trustworthy and when I created some new features using the top features from the 2nd Approach we can see the ROC-AUC score improving, which didn’t happen when used the 1st approach.

when only trained on the application train data with LGBM model gave the ROC-AUC Score: 0.8024315069244328

Let us look at the feature of the highest importance by lgbm in fig 5.11 which is Payment rate Inverse

Payment rate Inverse:

It is the ratio of Amount Credit (amount given by home credit)and Amount Annuity(Amount to pay back every month).

5.12 Payment Rate Inverse

Analysis:

Clients having a value of less than 12 will repay the loan.

Clients having a value greater than 12 and less than 18 will be a defaulter.

Also having a payment range between 22 and 35 there is a high probability that the client will repay the loan.

Conclusion:

Payment Rate Inverse feature is very useful.

b. bureau.csv & bureau_balance.csv

bureau_balance.csv:

Monthly balances of previous credits in Credit Bureau. This table has one row for each month of history of every previous credit reported to the Credit Bureau.

5.13 bureau_balance head

The two features in this table are :-

MONTHS_BALANCE:- Month of balance relative to the application date (-1 means the freshest balance date)

STATUS:- Status of Credit Bureau loan during the month (active, closed, DPD0–30,… [C means closed, X means status unknown, 0 means no DPD, 1 means maximal did during the month between 1–30, 2 means DPD 31–60,… 5 means DPD 120+ or sold or written off ] )

I have applied some aggregations on the bureau_balance.csv like min, max, sum.

Month wise aggregation of STATUS==C which means a number of credit account closed within a particular number of months.

bureau.csv:

All client’s previous credits provided by other financial institutions that were reported to Credit Bureau (for clients who have a loan in our sample). For every loan in our sample, there are as many rows as a number of credits the client had in the Credit Bureau before the application date.