Classification using the Tree-based method in R
By: Mario Camacho
Tags: Classification, Facebook, R, Marketing, Data Science

One of the biggest problems in different industries is the classification of customers to create more segmented marketing campaigns.

The importance of a segmented marketing campaign is the ability to get a better conversions rate, which can become a real challenge.

Following the previous idea, through this article, we are going to make an introduction of how to use the decision tree method for the classification of customers, users, and publications in R.

First of all, it is important to understand the concept and purpose of using a “Decision Tree Method”. The tree-based method segments an observation in a number of regions, to make this prediction it uses the mean or the mode of the training observations in the respective region. For example, a tree-based method could help us to segment a customer by their qualitative variables, like income, age, gender, preferences, helps us see different relations that we can’t see by observing the data.

For example, with a Facebook dataset in which there are 500 publications on Facebook of a cosmetics company with the following data of likes, shares, comments and the most important, a variable name “category”, which we don’t know what it’s meaning.

figure 1 Facebook dataset in R

First of all, we need to understand how to identify the variables that determine each category.

As a first step, we Install and use our libraries to work with

#facebook Data ---- install.packages("digest")

install.packages("parsnip")

install.packages("rsample")

install.packages("yardstick")

install.packages("rpart")

install.packages("xgboost")

install.packages("rpart.plot")

install.packages("tidyverse")

install.packages("Ckmeans.1d.dp")

# LIBRARIES ---- library(parsnip) # machine learning

library(rsample) # testing/training splits

library(recipes) # preprocessing

library(yardstick) # ROC/AUC

library(xgboost) # xgboost

library(Ckmeans.1d.dp) # feature importance plot

library(rpart) # decision tree

library(rpart.plot) # decision tree plot

library(ggplot2) # Graphic

library(tidyverse)

Next, clean the data by deleting de NA fields using the following code.

fb_data <- na.omit(fb_data)

It’s important to preprocess data where we will have our training and test data to see how it goes our model.

set.seed(1)

split_rsample <- fb_data %>%

rsample::initial_split(prop = 0.80) train_tbl <- rsample::training(split_rsample)

test_tbl <- rsample::testing(split_rsample) preprocessing_pipeline <- recipe(Category ~ ., data = train_tbl)

#to implement the decision Tree #design a matrix with the preprocess Data

train_processed_tbl <- bake(preprocessing_pipeline, train_tbl)

test_processed_tbl <- bake(preprocessing_pipeline, test_tbl)

Once we prepare data, we define our machine learning model, that in this case is boosting our classification model.

set.seed(123)

model_xgb <- boost_tree(

mode = "classification",

mtry = 80,

trees = 1000,

min_n = 3,

tree_depth = 8,

learn_rate = 0.01,

loss_reduction = 0.01) %>%

# runs xgboost under the hood

set_engine("xgboost") %>%

# model_spec is an object. fit similar to bake, in the sense

# That it is applying the fit to the training data

fit.model_spec(as.factor(Category) ~ ., data = train_processed_tbl)

After having our model created, we create our prediction variable to see if our model is good for classified our facebook posts.

predictions_test_tbl <- model_xgb %>%

predict.model_fit(new_data = test_processed_tbl, type = "prob") %>%

bind_cols(test_tbl) %>%

mutate(Category = as.factor(Category))

Then, we create our decision tree with our model and plot it

fit <- rpart(as.factor(Category) ~ ., method = "class", data = train_processed_tbl) ### prune tree

pfit<-prune(fit, cp=0.015) rpart.plot(pfit, extra=104,

main="Category Tree - Facebook")

We got this result:

Decision Tree

Finally we test our model with ROC,

predictions_test_tbl %>%

roc_auc(Category, .pred_1, .pred_2, .pred_3) predictions_test_tbl %>%

roc_curve(Category, .pred_1, .pred_2, .pred_3) %>%

ggplot(aes(x = 1-specificity, y = sensitivity)) +

geom_path() +

geom_point() +

labs(title = "ROC Curve")

And we got the next result:

ROC Curve

We got a 0.828 which gives a really good result to predict our results, reducing false positives and false negatives.

In conclusion, this is one method to make the classification of users, customers, publications, it’s really helpful to segment our marketing campaign, like in our example, we can classify our publications and see which ones are getting more engagement with the Facebook user.

Later on, I will be writing about other classification methods, let me know if you like to know about a specific method and any other conclusion you could get in the comment section.