Happiness and Life Satisfaction
By: Xuankhanh Nguyen
Tags: Machine Learning, Python, Happiness, Data Virtualization, Data Science

3. How Happiness Score is distributed

As we can see below, the Happiness Score has values above 2.85 and below 7.76. So there is no single country which has a Happiness Score above 8.

4. The relationship between different features with Happiness Score

We want to predict Happiness Score, so our dependent variable here is Score other features such as GPD Support Health , etc., are our independent variables.

GDP per capita

We first use scatter plots to observe relationships between variables.

Gif by Author

'''Happiness score vs gdp per capital'''

px.scatter(finaldf, x="GDP", y="Score", animation_frame="Year",

animation_group="Country",

size="Rank", color="Country", hover_name="Country",

trendline= "ols") train_data, test_data = train_test_split(finaldf, train_size = 0.8, random_state = 3)

lr = LinearRegression()

X_train = np.array(train_data['GDP'],

dtype = pd.Series).reshape(-1,1)

y_train = np.array(train_data['Score'], dtype = pd.Series)

lr.fit(X_train, y_train) X_test = np.array(test_data['GDP'],

dtype = pd.Series).reshape(-1,1)

y_test = np.array(test_data['Score'], dtype = pd.Series) pred = lr.predict(X_test) #ROOT MEAN SQUARED ERROR

rmsesm = float(format(np.sqrt(metrics.mean_squared_error(y_test,pred)),'.3f')) #R-SQUARED (TRAINING)

rtrsm = float(format(lr.score(X_train, y_train),'.3f')) #R-SQUARED (TEST)

rtesm = float(format(lr.score(X_test, y_test),'.3f')) cv = float(format(cross_val_score(lr,finaldf[['GDP']],finaldf['Score'],cv=5).mean(),'.3f')) print ("Average Score for Test Data: {:.3f}".format(y_test.mean()))

print('Intercept: {}'.format(lr.intercept_))

print('Coefficient: {}'.format(lr.coef_)) r = evaluation.shape[0]

evaluation.loc[r] = ['Simple Linear Regression','-',rmsesm,rtrsm,'-',rtesm,'-',cv]

evaluation

By using these values and the below definition, we can estimate the Happiness Score manually. The equation we use for our estimations is called hypothesis function and defined as

We also printed the intercept and coefficient for the simple linear regression.

Let’s show the result, shall we?

Since we have just two dimensions at the simple regression, it is easy to draw it. The below chart determines the result of the simple regression. It does not look like a perfect fit, but when we work with real-world datasets, having an ideal fit is not easy.

seabornInstance.set_style(style='whitegrid') plt.figure(figsize=(12,6))

plt.scatter(X_test,y_test,color='blue',label="Data", s = 12)

plt.plot(X_test,lr.predict(X_test),color="red",label="Predicted Regression Line")

plt.xlabel("GDP per Captita", fontsize=15)

plt.ylabel("Happiness Score", fontsize=15)

plt.xticks(fontsize=13)

plt.yticks(fontsize=13)

plt.legend() plt.gca().spines['right'].set_visible(False)

plt.gca().spines['top'].set_visible(False)

The relationship between GDP per capita(Economy of the country) has a strong positive correlation with Happiness Score, that is, if the GDP per capita of a country is higher than the Happiness Score of that country, it is also more likely to be high.

Support

To keep the article short, I won’t include the code in this part. The code is similar to the GDP feature above. I recommend you try to implement yourself. I will include the link at the end of this article for reference.

Social Support of countries also has a strong and positive relationship with Happiness Score. So, it makes sense that we need social support to be happy. People are also wired for emotions, and we experience those emotions within a social context.

Healthy life expectancy

A healthy life expectancy has a strong and positive relationship with the Happiness Score, that is, if the country has a High Life Expectancy, it can also have a high Happiness Score. Being happy doesn’t just improve the quality of a person’s life. It may increase the quantity of our life as well. I will also be happy if I get a long healthy life. You?

Freedom to make life choices

Freedom to make life choices has a positive relationship with Happiness Score. Choice and autonomy are more directly related to happiness than having lots of money. It gives us options to pursue meaning in our life, finding activities that stimulate and excite us. This is an essential aspect of feeling happy.

Generosity

Generosity has a fragile linear relationship with the Happiness Score. Why the charity has no direct relationship with happiness score? Generosity scores are calculated based on the countries which give the most to nonprofits around the world. Countries that are not generous that does not mean they are not happy.

Perceptions of corruption

Distribution of Perceptions of corruption rightly skewed that means very less number of the country has high perceptions of corruption. That means most of the country has corruption problems.

How corruption feature impact on the Happiness Score?

Perceptions of corruption data have highly skewed no wonder why the data has a weak linear relationship. Still, as we can see in the scatter plot, most of the data points are on the left side, and most of the countries with low perceptions of corruption have a Happiness Score between 4 to 6. Countries with high perception scores have a high Happiness Score above 7.

5. Visualize and Examine Data

We do not have big data with too many features. Thus, we have a chance to plot most of them and reach some useful analytical results. Drawing charts and examining the data before applying a model is a good practice because we may detect some possible outliers or decide to do normalization. This step is not a must but gets to know the data is always useful. We start with the histograms of dataframe .

# DISTRIBUTION OF ALL NUMERIC DATA

plt.rcParams['figure.figsize'] = (15, 15)

df1 = finaldf[['GDP', 'Health', 'Freedom',

'Generosity','Corruption']] h = df1.hist(bins = 25, figsize = (16,16),

xlabelsize = '10', ylabelsize = '10')

seabornInstance.despine(left = True, bottom = True)

[x.title.set_size(12) for x in h.ravel()];

[x.yaxis.tick_left() for x in h.ravel()]

Next, to give us a more appealing view of where each country is placed in the World ranking report, we use darker blue for countries that have the highest rating on the report (i.e., are the “happiest”), while the lighter blue represents countries with a lower ranking. We can see that countries in the European and Americas regions have a reasonably high ranking than ones in the Asian and African areas.

'''World Map

Happiness Rank Accross the World''' happiness_rank = dict(type = 'choropleth',

locations = finaldf['Country'],

locationmode = 'country names',

z = finaldf['Rank'],

text = finaldf['Country'],

colorscale = 'Blues_',

autocolorscale=False,

reversescale=True,

marker_line_color='darkgray',

marker_line_width=0.5)

layout = dict(title = 'Happiness Rank Across the World',

geo = dict(showframe = False,

projection = {'type': 'equirectangular'}))

world_map_1 = go.Figure(data = [happiness_rank], layout=layout)

iplot(world_map_1)

Let’s check which countries are better positioned in each of the aspects being analyzed.

fig, axes = plt.subplots(nrows=3, ncols=2,constrained_layout=True,figsize=(10,10)) seabornInstance.barplot(x='GDP',y='Country',

data=finaldf.nlargest(10,'GDP'),

ax=axes[0,0],palette="Blues_r")

seabornInstance.barplot(x='Health' ,y='Country',

data=finaldf.nlargest(10,'Health'),

ax=axes[0,1],palette='Blues_r')

seabornInstance.barplot(x='Score' ,y='Country',

data=finaldf.nlargest(10,'Score'),

ax=axes[1,0],palette='Blues_r')

seabornInstance.barplot(x='Generosity' ,y='Country',

data=finaldf.nlargest(10,'Generosity'),

ax=axes[1,1],palette='Blues_r')

seabornInstance.barplot(x='Freedom' ,y='Country',

data=finaldf.nlargest(10,'Freedom'),

ax=axes[2,0],palette='Blues_r')

seabornInstance.barplot(x='Corruption' ,y='Country',

data=finaldf.nlargest(10,'Corruption'),

ax=axes[2,1],palette='Blues_r')

Checking Out the Correlation Among Explanatory Variables

mask = np.zeros_like(finaldf[usecols].corr(), dtype=np.bool)

mask[np.triu_indices_from(mask)] = True f, ax = plt.subplots(figsize=(16, 12))

plt.title('Pearson Correlation Matrix',fontsize=25) seabornInstance.heatmap(finaldf[usecols].corr(),

linewidths=0.25,vmax=0.7,square=True,cmap="Blues",

linecolor='w',annot=True,annot_kws={"size":8},mask=mask,cbar_kws={"shrink": .9});

It looks like GDP , Health , and Support are strongly correlated with the Happiness score. Freedom correlates quite well with the Happiness score; however, Freedom connects quite well with all data. Corruption still has a mediocre correlation with the Happiness score.

Beyond Simple Correlation

In the scatterplots, we see that GDP , Health , and Support are quite linearly correlated with some noise. We find the auto-correlation of Corruption fascinating here, where everything is terrible, but if the corruption is high, the distribution is all over the place. It seems to be just a negative indicator of a threshold.

I found an exciting package by Ian Ozsvald that uses. It trains random forests to predict features from each other, going a bit beyond simple correlation.

# visualize hidden relationships in data

classifier_overrides = set()

df_results = discover.discover(finaldf.drop(['target', 'target_n'],axis=1).sample(frac=1),

classifier_overrides)

We use heat maps here to visualize how our features are clustered or vary over space.

fig, ax = plt.subplots(ncols=2,figsize=(24, 8))

seabornInstance.heatmap(df_results.pivot(index = 'target',

columns = 'feature',

values = 'score').fillna(1).loc[finaldf.drop(

['target', 'target_n'],axis = 1).columns,finaldf.drop(

['target', 'target_n'],axis = 1).columns],

annot=True, center = 0, ax = ax[0], vmin = -1, vmax = 1, cmap = "Blues")

seabornInstance.heatmap(df_results.pivot(index = 'target',

columns = 'feature',

values = 'score').fillna(1).loc[finaldf.drop(

['target', 'target_n'],axis=1).columns,finaldf.drop(

['target', 'target_n'],axis=1).columns],

annot=True, center=0, ax=ax[1], vmin=-0.25, vmax=1, cmap="Blues_r")

plt.plot()

This gets more interesting. Corruption is a better predictor of the Happiness Score than Support. Possibly because of the ‘threshold’ we previously discovered?

Moreover, although Social Support correlated quite well, it does not have substantial predictive value. I guess this is because all the distributions of the quartiles are quite close in the scatterplot.

6. Multiple Linear Regression

In the thirst section of this article, we used a simple linear regression to examine the relationships between the Happiness Score and other features. We found a poor fit. To improve this model, we want to add more features. Now, it is time to create some complex models.

We determined features at first sight by looking at the previous sections and used them in our first multiple linear regression. As in the simple regression, we printed the coefficients which the model uses for the predictions. However, this time we must use the below definition for our predictions if we want to make calculations manually.

We create a model with all features.

# MULTIPLE LINEAR REGRESSION 1

train_data_dm,test_data_dm = train_test_split(finaldf,train_size = 0.8,random_state=3) independent_var = ['GDP','Health','Freedom','Support','Generosity','Corruption']

complex_model_1 = LinearRegression()

complex_model_1.fit(train_data_dm[independent_var],train_data_dm['Score']) print('Intercept: {}'.format(complex_model_1.intercept_))

print('Coefficients: {}'.format(complex_model_1.coef_))

print('Happiness score = ',np.round(complex_model_1.intercept_,4),

'+',np.round(complex_model_1.coef_[0],4),'∗ Support',

'+',np.round(complex_model_1.coef_[1],4),'* GDP',

'+',np.round(complex_model_1.coef_[2],4),'* Health',

'+',np.round(complex_model_1.coef_[3],4),'* Freedom',

'+',np.round(complex_model_1.coef_[4],4),'* Generosity',

'+',np.round(complex_model_1.coef_[5],4),'* Corrption') pred = complex_model_1.predict(test_data_dm[independent_var])

rmsecm = float(format(np.sqrt(metrics.mean_squared_error(

test_data_dm['Score'],pred)),'.3f'))

rtrcm = float(format(complex_model_1.score(

train_data_dm[independent_var],

train_data_dm['Score']),'.3f'))

artrcm = float(format(adjustedR2(complex_model_1.score(

train_data_dm[independent_var],

train_data_dm['Score']),

train_data_dm.shape[0],

len(independent_var)),'.3f'))

rtecm = float(format(complex_model_1.score(

test_data_dm[independent_var],

test_data_dm['Score']),'.3f'))

artecm = float(format(adjustedR2(complex_model_1.score(

test_data_dm[independent_var],test_data['Score']),

test_data_dm.shape[0],

len(independent_var)),'.3f'))

cv = float(format(cross_val_score(complex_model_1,

finaldf[independent_var],

finaldf['Score'],cv=5).mean(),'.3f')) r = evaluation.shape[0]

evaluation.loc[r] = ['Multiple Linear Regression-1','selected features',rmsecm,rtrcm,artrcm,rtecm,artecm,cv]

evaluation.sort_values(by = '5-Fold Cross Validation', ascending=False)

We knew that GDP , Support , and Health are quite linearly correlated. This time, we create a model with these three features.

# MULTIPLE LINEAR REGRESSION 2

train_data_dm,test_data_dm = train_test_split(finaldf,train_size = 0.8,random_state=3) independent_var = ['GDP','Health','Support']

complex_model_2 = LinearRegression()

complex_model_2.fit(train_data_dm[independent_var],train_data_dm['Score']) print('Intercept: {}'.format(complex_model_2.intercept_))

print('Coefficients: {}'.format(complex_model_2.coef_))

print('Happiness score = ',np.round(complex_model_2.intercept_,4),

'+',np.round(complex_model_2.coef_[0],4),'∗ Support',

'+',np.round(complex_model_2.coef_[1],4),'* GDP',

'+',np.round(complex_model_2.coef_[2],4),'* Health') pred = complex_model_2.predict(test_data_dm[independent_var])

rmsecm = float(format(np.sqrt(metrics.mean_squared_error(

test_data_dm['Score'],pred)),'.3f'))

rtrcm = float(format(complex_model_2.score(

train_data_dm[independent_var],

train_data_dm['Score']),'.3f'))

artrcm = float(format(adjustedR2(complex_model_2.score(

train_data_dm[independent_var],

train_data_dm['Score']),

train_data_dm.shape[0],

len(independent_var)),'.3f'))

rtecm = float(format(complex_model_2.score(

test_data_dm[independent_var],

test_data_dm['Score']),'.3f'))

artecm = float(format(adjustedR2(complex_model_2.score(

test_data_dm[independent_var],test_data['Score']),

test_data_dm.shape[0],

len(independent_var)),'.3f'))

cv = float(format(cross_val_score(complex_model_2,

finaldf[independent_var],

finaldf['Score'],cv=5).mean(),'.3f')) r = evaluation.shape[0]

evaluation.loc[r] = ['Multiple Linear Regression-2','selected features',rmsecm,rtrcm,artrcm,rtecm,artecm,cv]

evaluation.sort_values(by = '5-Fold Cross Validation', ascending=False)

When we look at the evaluation table, multiple linear regression -2 (selected features) is the best. However, I have doubts about its reliability, and I would prefer the multiple linear regression with all elements.