Logistic Regression to Predict Extra Marital Affair
By: Akhil Anand
Tags: Machie Learning, Predictions, Data Science, Logistic Regression

Overview :

The dataset I chose is the affairs dataset that comes with Stats models. It was derived from a survey of women in 1974 by Red book magazine, in which married women were asked about their participation in extramarital affairs.I decided to treat this as a classification problem by creating a new binary variable affair (did the woman have at least one affair?) and trying to predict the classification for each woman.Variables that is present in the dataset for prediction are :-rate_marriage(women’s rating for her marriage) ,age(women’s age),yrs_married(number of years married), children(no. of children she has),religious(on the scale of 1 -5 whether she believe in religion or not),education(level of education), occupation ,occupation of husband, affairs

Step 1 : Importing all the essential libraries and acquiring data set

import numpy as np

import pandas as pd

import statsmodels.api as sm

import matplotlib.pyplot as plt

import seaborn as sns

from patsy import dmatrices

from sklearn.linear_model import LogisticRegression

from sklearn.model_selection import train_test_split

from sklearn import metrics

Affair dataset is present inside the stats models ,so we will be going to extract that library.

df=sm.datasets.fair.load_pandas().data

df.head()

output of first five rows

Step 2 : Data Preprocessing

We want to make it classification problem for that we need to convert target variable (affairs) into binary class where if the value will be 0 then women had no any affair and 1 means women had been involved in several extra marital affairs.

(df.affairs>0).astype(int)#converting into binary features

df.affairs.unique() [out]>> array([1, 0], dtype=int64)

Here we will use pasty.dmatrices, it’s main task is to analyze the independent variable and dependent variable and add some data if it is needed based on target variable.

y,X=dmatrices(affairs~rate_marriage + age + yrs_married + children + religious + educ + C(occupation)+C(occupation_husb)',df

return_type="dataframe")

x.head()

Output of first five rows

Renaming the features present in X so that it can be understandable by all.

X = X.rename(columns ={'C(occupation)[1.0]':'occ_1','C(occupation)[2.0]':'occ_2','C(occupation)[3.0]' : 'occ_3' ,'C(occupation[4.0]' : 'occ_4','C(occupation[5.0]':'occ_5','C(occupation)[6.0]':'occ_6',

'C(occupation_husb)[T.2.0]':'occ_husb_2', 'C(occupation_husb[T.3.0]' :'occ_husb_3','C(occupation_husb)[T.4.0]':'occ_husb_4' , 'C(occupation_husb)[T.5.0]':'occ_husb_5', 'C(occupation_husb)[T.6.0]':'occ_husb_6'})

X.head()

output of renamed X

Checking is there any null value present in dataset or not .

df=pd.concat([X,y],axis=1)

df.isnull().values.any() [out]>> False #no null value present in dataset

Now next step would be to check whether the dataset is balanced or not.

print(df[df["affairs"]==0].shape)

print(df[df["affairs"]==1].shape) [out]>> (4313, 18)

(2053, 18)

--------------------------------------------------------------------

plt.figure(figsize=(8,6))

sns.set_style("darkgrid")

sns.countplot(np.ravel(y))

plt.yticks(range(0,5000,250))

plt.show()

Figure 1

As we see the visual the dataset is imbalanced because class 0 is almost double of class 1 . we will do up sampling to balance the dataset.

from imblearn.random_sampling import RandomOverSampler

rdm=RandomOverSampler()

X,y=rdm.fit_sample(X,y)

df=pd.concat([X,y],axis=1) #updating the balanced form of dataset

print(df[df["affairs"]==0].shape)

print(df[df["affairs"]==1].shape) [out]>> (4313, 18)

(4313, 18) --------------------------------------------------------------------

plt.figure(figsize=(8,6))

sns.set_style("darkgrid")

sns.countplot(np.ravel(y))# converting y into array form to plot

plt.yticks(range(0,5000,250))

plt.show()

Figure 2

Statistical Description of data

df.describe()

Statistical description

Step 3 : Data Visualization

How children affects the husband-wife relationship.

plt.figure(figsize=(8,6))

sns.set_style("darkgrid")

sns.countplot("children",data=df,hue="affairs")

plt.title("How children affects marriage")

plt.show()

Figure 3

How years of marriage affects extra married life.

Figure 5

As the married couple spend more and more years together there would be great chance that his wife would be in extra marital relation.Best result of trust and negligible extra marital affair seen in newly married couple.

How women rate their married life on a scale of 1 to 5.

plt.figure(figsize=(8,6))

plt.hist(df["rate_marriage"])

plt.xlabel("Marriage rating",fontsize=20)

plt.ylabel("count",fontsize=20)

plt.show()

Figure 6

From above figure we can say large bunch of women is satisfied with their married life irrespective of their affairs.

Let’s see how education make an impact on married life

plt.figure(figsize=(8,6))

sns.set_style("darkgrid")

sns.countplot("educ",hue="affairs",data=df,order=df["educ"].value_counts().index)

plt.xlabel("education",fontsize=20)

plt.ylabel("count",fontsize=20)

plt.show()

Figure 7

Women who is doing white collar jobs having less chance of affair and women with graduation and higher educational background also has less chance of affair but for those women who is in college and have lower literacy has shown more interest in extra marital affairs.

How the marriage of women affect by religiousness

plt.figure(figsize=(8,6))

sns.set_style("darkgrid")

sns.countplot("religious",hue="affairs",data=df)

plt.xlabel("religious",fontsize=20)

plt.ylabel("count",fontsize=20)

plt.show()

Figure 7

More the women will be religious lesser will be the chance that women will have extra marital affair.

Step 4 : Modeling

splitting the data into train and test set

x_train,x_test,y_train,y_test=train_test_split(X,y,test_size=0.2,random_state=1)

print(x_train.shape)

print(y_train.shape)

print(x_test.shape)

print(y_test.shape) [out]>> (6900, 17)

(6900, 1)

(1726, 17)

(1726, 1)

Making model and doing prediction

clf=LogisticRegression()

clf.fit(x_train,y_train)

y_predicted=clf.predict(x_test)

y_train_predicted=clf.predict(x_train)

Step 5 : Evaluation

from sklearn.metrics import accuracy_score, confusion_matrix,roc_curve,classification_report

print("train set accuracy score: ",accuracy_score(y_train,y_train_predicted))

print("test set accuracy score :", accuracy_score(y_test,y_predicted)) [out] >> train set accuracy score: 0.6918840579710145

test set accuracy score : 0.6853997682502897

As we see there is not large deviation between dataset hence model is in normal state.

Accuracy matrix to check TP ,TN ,FP ,FN values.

ax=splt.metrics.plot_confusion_matrix(y_test,y_predicted,text_fontsize="large",figsize=(8,6))

plt.xlabel("Predicted label",fontsize=20)

plt.ylabel("True label",fontsize=20)

bottom, top = ax.get_ylim()

ax.set_ylim(bottom + 0.5, top - 0.5)

plt.show()

Figure 8

Here we should be careful about FP because our model will predict the women having affair but in actual she won’t have any affair which may leads to divorce for the mistake she has not done so we should show some carefulness about FP value when we are dealing with such sensitive kind of prediction.

Finding Precision,Recall F1- Score

from sklearn.metrics import classification_report

print(classification_report(y_test,y_predicted))

Figure 9

ROC CURVE

pred_probab=clf.predict_proba(x_test)

splt.metrics.plot_roc_curve(y_test,pred_probab,text_fontsize="large",figsize=(8,6))

Figure 10

Final Testing of Model :

print("Prediction chance of your model :")

print(clf.predict_proba(np.array([[0,0,0,0,0,1,1,0,0,0,0,4,32,5,2,3,17]]))*100)

Prediction chance for the upper given test set is 27.7 percent and the predicted result would be;

print("your prediction is :",clf.predict(np.array([[0,0,0,0,0,1,1,0,0,0,0,4,32,5,2,3,17]]))) [out] >> your prediction is : [0.]

Prediction result for given data is that the women doesn’t have affair.

Conclusion :

Hope this blog gave you better idea about how logistic regression works in classification problem . Keep Learning Keep Exploring ………