Finding Donors for Charity using machine learning(supervised learning)
By: Mr. Buck Man
Tags: Supervised Learning, Machine Learning, Data Science

Using the one-hot encoding

This is one of the interesting parts in solving the problem. Let me start by answering the question why do we use one-hot encoding? the answer is

Typically, learning algorithms expect input to be numeric, Generally learning algorithms expect the input variables to be in numeric form. So we need to convert the non numeric features.Widely applied way to make this conversion by using the one-hot encoding method.

It creates dummy variables for each possible category of each non numeric features. let me throw an example to make this clear let’s take a sample feature spl_feature which has two possible entries in it X,Y . after encoding this becomes spl_feature_X, spl_feature_Y

Additionally, as with the non-numeric features, we need to convert the non-numeric target label, 'income' to numerical values for the learning algorithm to work properly. Since there are only two categories for this label (“<=50K” and “>50K”), we can skip the One-Hot encoding step and simply encode these two categories as 0 and 1(zero and one).

In order to do so we are using the pandas.get__dummies(), Convert the target label 'income_raw' to numerical entries and applying “<=50K” to 0 and records with ">50K" to 1

# applying pd.get_dummies() on the 'features_log_minmax_transform' data

features_final =pd.get_dummies(features_log_minmax_transform) # Changing the 'income_raw' data to numerical values

income = income_raw.replace(["<=50K",">50K"],[0,1]) # Print the number of features after transforming

encoded = list(features_final.columns)

print("{} total features after one-hot encoding.".format(len(encoded))) #checking the data

print (encoded)

The output is shown below

103 total features after one-hot encoding.

['age', 'education-num', 'capital-gain', 'capital-loss', 'hours-per-week', 'workclass_ Federal-gov', 'workclass_ Local-gov', 'workclass_ Private', 'workclass_ Self-emp-inc', 'workclass_ Self-emp-not-inc', 'workclass_ State-gov', 'workclass_ Without-pay', 'education_level_ 10th', 'education_level_ 11th', 'education_level_ 12th', 'education_level_ 1st-4th', 'education_level_ 5th-6th', 'education_level_ 7th-8th', 'education_level_ 9th', 'education_level_ Assoc-acdm', 'education_level_ Assoc-voc', 'education_level_ Bachelors', 'education_level_ Doctorate', 'education_level_ HS-grad', 'education_level_ Masters', 'education_level_ Preschool', 'education_level_ Prof-school', 'education_level_ Some-college', 'marital-status_ Divorced', 'marital-status_ Married-AF-spouse', 'marital-status_ Married-civ-spouse', 'marital-status_ Married-spouse-absent', 'marital-status_ Never-married', 'marital-status_ Separated', 'marital-status_ Widowed', 'occupation_ Adm-clerical', 'occupation_ Armed-Forces', 'occupation_ Craft-repair', 'occupation_ Exec-managerial', 'occupation_ Farming-fishing', 'occupation_ Handlers-cleaners', 'occupation_ Machine-op-inspct', 'occupation_ Other-service', 'occupation_ Priv-house-serv', 'occupation_ Prof-specialty', 'occupation_ Protective-serv', 'occupation_ Sales', 'occupation_ Tech-support', 'occupation_ Transport-moving', 'relationship_ Husband', 'relationship_ Not-in-family', 'relationship_ Other-relative', 'relationship_ Own-child', 'relationship_ Unmarried', 'relationship_ Wife', 'race_ Amer-Indian-Eskimo', 'race_ Asian-Pac-Islander', 'race_ Black', 'race_ Other', 'race_ White', 'sex_ Female', 'sex_ Male', 'native-country_ Cambodia', 'native-country_ Canada', 'native-country_ China', 'native-country_ Columbia', 'native-country_ Cuba', 'native-country_ Dominican-Republic', 'native-country_ Ecuador', 'native-country_ El-Salvador', 'native-country_ England', 'native-country_ France', 'native-country_ Germany', 'native-country_ Greece', 'native-country_ Guatemala', 'native-country_ Haiti', 'native-country_ Holand-Netherlands', 'native-country_ Honduras', 'native-country_ Hong', 'native-country_ Hungary', 'native-country_ India', 'native-country_ Iran', 'native-country_ Ireland', 'native-country_ Italy', 'native-country_ Jamaica', 'native-country_ Japan', 'native-country_ Laos', 'native-country_ Mexico', 'native-country_ Nicaragua', 'native-country_ Outlying-US(Guam-USVI-etc)', 'native-country_ Peru', 'native-country_ Philippines', 'native-country_ Poland', 'native-country_ Portugal', 'native-country_ Puerto-Rico', 'native-country_ Scotland', 'native-country_ South', 'native-country_ Taiwan', 'native-country_ Thailand', 'native-country_ Trinadad&Tobago', 'native-country_ United-States', 'native-country_ Vietnam', 'native-country_ Yugoslavia']

Splitting the training and testing data

Now in this step we will split the data into Training and Testing data. In this case we are using 80% of the data into training and other 20% into testing the data.

# Import train_test_split

from sklearn.cross_validation import train_test_split # Split the 'features' and 'income' data into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(features_final,

income,

test_size = 0.2,

random_state = 0) # Show the results of the split

print("Training set has {} samples.".format(X_train.shape[0]))

print("Testing set has {} samples.".format(X_test.shape[0])) '''Ignore deprecation and future warnings.'''

def suppress_warnings():

import warnings

warnings.filterwarnings('ignore', category = DeprecationWarning)

warnings.filterwarnings('ignore', category = FutureWarning)

#This will be the output for when i run the above code Training set has 36177 samples.

Testing set has 9045 samples.

Evaluating Model Performance

Here we are using the F-beta score as a metric that considers both precision and recall.

Accuracy : it can be defined as the measure of how often the model makes the correct prediction or simply put as the ratio of the number of correct predictions to the total number of predictions.

Precision: simply put precision as the ratio of true positives to all positives.

It is a ratio of true positives(words classified as spam, and which are actually spam) to all positives.

Below is the code used to calculate the accuracy , precision and recall

# Calculating the accuracy, precision and recall

TP = np.sum(income)

FP = income.count() - TP

TN = 0

FN = 0

accuracy = float(TP)/(TP+FP)

recall = float(TP)/(TP+FN)

precision = accuracy # Calculating the F-score using the formula above for beta = 0.5 and correct values for precision and recall.

beta=0.5

fscore = (1 + beta ** 2)*(precision * recall)/(beta ** 2 *precision + recall) # Print the results

print("Naive Predictor: [Accuracy score: {:.4f}, F-score: {:.4f}]".format(accuracy, fscore))

we get the output with accuracy score of 0.247 , F.score of 0.2917

Implementation — Creating a Training and Predicting Pipeline

To evaluate the model which we build its important to create a training and predicting pipeline which allows us to train the model with different sizes of training data and apply those predictions on test data.

# TODO: Import two metrics from sklearn - fbeta_score and accuracy_score

from sklearn.metrics import fbeta_score def train_predict(learner, sample_size, X_train, y_train, X_test, y_test):

'''

inputs:

- learner: the learning algorithm to be trained and predicted on

- sample_size: the size of samples (number) to be drawn from training set

- X_train: features training set

- y_train: income training set

- X_test: features testing set

- y_test: income testing set

'''



results = {}



# TODO: Fit the learner to the training data using slicing with 'sample_size'

start = time() # Get start time

learner.fit(X_train[:sample_size],y_train[:sample_size])

end = time() # Get end time



# TODO: Calculate the training time

results['train_time'] = end - start



# TODO: Get the predictions on the test set,

# then get predictions on the first 300 training samples

start = time() # Get start time

predictions_test = learner.predict(X_test)

predictions_train = learner.predict(X_train[:300])

end = time() # Get end time



# TODO: Calculate the total prediction time

results['pred_time'] = end-start



# TODO: Compute accuracy on the first 300 training samples

from sklearn.metrics import accuracy_score

results['acc_train'] = accuracy_score(y_train[:300],predictions_train)



# TODO: Compute accuracy on test set

results['acc_test'] = accuracy_score(y_test,predictions_test)



# TODO: Compute F-score on the the first 300 training samples

results['f_train'] = fbeta_score(y_train[:300],predictions_train,beta=0.5)



# TODO: Compute F-score on the test set

results['f_test'] = fbeta_score(y_test,predictions_test,beta=0.5)

In order to improve the results we can apply different techniques if necessary and see which one works best.

After optimising the model we get the below scores

The optimized model’s accuracy and F-score are respectively 0.8676 and 0.7448. The unoptimized model’s accuracy and F-score are respectively 0.8576 and 0.7246.

Describing the Model in Layman’s Terms

Explaining week learning

A weak learner is any machine learning algorithm that gives better accuracy than simply guessing. For instance, if you are trying to classify animals at a zoo, you might have an algorithm that can correctly identify zebras most of the time, but it simply guesses for any other animal. That algorithm would be a weak learner because it is better than guessing.

If you had an algorithm that identified every animal as a zebra, then that probably is not better than guessing and so it would not be a weak learner.

For boosting problems, the best kinds of weak learners are ones that are very accurate, even if it is only over a limited scope of the problem. For instance the algorithm that correctly identifies zebras would be good. It allows you to confidently identify as least most of the zebras, allowing other weak learners to focus on the remaining animals.

explaining how Are Weak Learners Combined?

If there are any errors caused by the first algorithm, we give higher attention to observations having prediction error.Later we apply the next algorithm.

We keep repeating the above mentioned step until a higher accuracy is achieved.

Finally it combines all the week iterations to form one strong algorithm, which increases the ability to predict the power of the model

For the above problem, using the rule created for the individuals who have income more than 50k or less than 50k the ada boost algorithm can predict the individuals who make more than 50k income.

source https://prateekvjoshi.com/2014/05/05/what-is-adaboost/