Tree-Based Algorithms Approach on Predicting Customer Satisfaction | by Zikry Adjie Nugraha | Towards Data Science

Customer satisfaction is one of the methods that businesses can use to effectively manage and monitor their operations. Every product can survive in the marketplace for a long time as long as the customer satisfaction level remains high. Good quality of the product is not the only factor that can make a product can have a high level of customer satisfaction. Many factors, such as delivery times, additional payment, product price, and etc. can have a significant impact on customer satisfaction. Customer satisfaction is critical for customer retention. A satisfied customer is more likely to consider purchasing the same product and using the same service again, resulting in a large profit for the company. Meanwhile, a dissatisfied customer is likely to avoid purchasing the same product in the future due to a previous negative experience with the company’s customer service[1].
Customer satisfaction evaluation is required to assist businesses in determining the next marketing strategy that will bring a profit to the company. Predicting customer satisfaction can be difficult due to how unpredictable each customer’s behavior can be. However, in order to overcome this, there are numerous statistical methods that will undoubtedly assist businesses in analyzing and forecasting customer satisfaction based on specific factors[2]. In this article, we will primarily focus on predicting customer satisfaction from e-commerce datasets using tree-based machine learning algorithms.
The dataset for this modeling prediction is Brazilian E-Commerce Public Dataset by olist from Kaggle.
The author has granted permission for this dataset to be used, shared, and modified publicly. The license can be found here.
The goal of this hands-on project is to predict customer satisfaction from an e-commerce dataset. The tree-based machine learning modeling techniques that we will employ are decision tree, random forest, and gradient boosting. Each of the models will be evaluated, and we will compare them to determine which of the three is the best model for predicting customer satisfaction.
Decision trees are one of the supervised machine learning methods for classification and regression. In this hands-on project, we will concentrate on the decision tree classifier and its various types, which will provide us with a readable classification model. The tree node on the decision tree classifier represents an attribute test. The branches emerge from that node, with each branch representing one of the previous attribute’s possible values. The leaf grows from those branches, with each leaf corresponding to the class labels associated with the instance. The instances in the training set are directed from the decision tree’s root to the leaf [3].
The model is easy to understand and interpret.The model requires minimal data preparation and does not support missing values.Statistical tests can be used to validate the model’s reliability.This model can use both numerical and categorical data, but it does not support categorical data in scikit-learn.This model can be used to solve multi-output problems.
The model is easy to understand and interpret.
The model requires minimal data preparation and does not support missing values.
Statistical tests can be used to validate the model’s reliability.
This model can use both numerical and categorical data, but it does not support categorical data in scikit-learn.
This model can be used to solve multi-output problems.
Overfitting occurs frequently when the model contains overly complex trees that do not generalize the data well.Small variations in the data can cause the decision tree to become unstable, resulting in the different generated trees than the previous one.If some classes dominate, the model may produce biased trees.
Overfitting occurs frequently when the model contains overly complex trees that do not generalize the data well.
Small variations in the data can cause the decision tree to become unstable, resulting in the different generated trees than the previous one.
If some classes dominate, the model may produce biased trees.
Decision tree classifier uses two arrays of array X and array Y. Array X holds the training sample with the shape of (n_samples, n_features). Array Y holds the class labels for the training sample with the shape of (n_samples).
Decision tree variants of random forest and gradient boosting are types of modeling that employ the use of multiple trees to achieve a better result than a single tree. Both models can also be used to prevent overfitting.
Random forest constructs each tree independently and then combines the prediction results of the average value of each tree.Gradient boosting constructs one tree at a time, combining prediction results along the way.
Random forest constructs each tree independently and then combines the prediction results of the average value of each tree.
Gradient boosting constructs one tree at a time, combining prediction results along the way.
Import LibrariesData UnderstandingData cleaning and preprocessingEDA (Exploratory Data Analysis)Feature EngineeringMachine learning modelingConclusion
Import Libraries
Data Understanding
Data cleaning and preprocessing
EDA (Exploratory Data Analysis)
Feature Engineering
Machine learning modeling
Conclusion
It is important to import the library that we will be using during this modeling process. We will primarily employ various types of libraries in our data analysis and modeling processes. We will use pandas and NumPy as data processing and transformation tools. To get a better visualization of our data, we will use seaborn and matplotlib as data visualization tools. We will primarily use the scikit-learn library, which is divided into various types of modules, for the modeling process tools of our prediction model.
#Import libraries for data processing & transformation
import pandas as pd
import numpy as np
#Import libraries for data visualization process
import seaborn as sns
import matplotlib.pyplot as plt
#Import libraries for modelling process
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay, classification_report, accuracy_score
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
Before proceeding with the process, the raw dataset must be combined. We will only include the dataset with order-related columns, product-related columns, payment-related columns, and, specifically, review score columns. Each dataset’s combination process is depicted below.
# load the customer datasets
customer = pd.read_csv('dataset/olist_customers_dataset.csv')
customer.head()
# load the order items datasets
order_items = pd.read_csv('dataset/olist_order_items_dataset.csv')
order_items.head()
# load the order payments datasets
order_payments = pd.read_csv('dataset/olist_order_payments_dataset.csv')
order_payments.head()
# load the order review datasets
order_reviews = pd.read_csv('dataset/olist_order_reviews_dataset.csv')
order_reviews.head()
# load the orders datasets
orders = pd.read_csv('dataset/olist_orders_dataset.csv')
orders.head()
# load the products datasets
products = pd.read_csv('dataset/olist_products_dataset.csv')
products.head()
# load the sellers datasets
sellers = pd.read_csv('dataset/olist_sellers_dataset.csv')
sellers.head()
# load the product translation datasets
product_translation = pd.read_csv('dataset/product_category_name_translation.csv')
product_translation
#Merging all customer related datasets
A = pd.merge(orders,order_reviews,on='order_id')
A = pd.merge(A,order_payments,on='order_id')
A = pd.merge(A,customer,on='customer_id')
A.head()
#Merging all seller related datasets
B = pd.merge(order_items,products,on='product_id')
B = pd.merge(B,sellers,on='seller_id')
B = pd.merge(B,product_translation,on='product_category_name')
B.head()
#Merging customer and seller datasets
df_ecommerce = pd.merge(A,B, on = 'order_id')
df_ecommerce.head()
#Choosing only the important column
df_ecommerce = df_ecommerce[['order_status','order_purchase_timestamp','order_delivered_customer_date','order_estimated_delivery_date',
                              'shipping_limit_date','payment_sequential','payment_type','payment_installments','payment_value',
                              'price','freight_value','product_category_name_english','product_name_lenght','product_description_lenght',
                              'product_photos_qty','review_score']]

#Fixing typos and column names
df_ecommerce = df_ecommerce.rename(columns={'product_name_lenght': 'product_name_length', 'product_description_lenght': 'product_description_length',
                                            'product_category_name_english': 'product_category'})

df_ecommerce.head()
df_ecommerce.shape
(115609, 16)
There are 115,689 rows and 16 columns in total. Each column is labeled with its own description. The unused column from the original column has been removed, and the 16 columns are the only columns that will be used for our analysis and modeling.
order_status: This is a reference to the order status (delivered or canceled).order_purchase_timestamp: Displays the timestamp of each item’s purchase.order_delivered_customer_date: Displays the customer’s actual order delivery date.order_estimated_delievy_date: Displays the estimated delivery date that was provided to the customer at the time of purchase.shipping_limit_date: Displays the seller’s shipping limit date for transferring the order to the logistic partner.payment_sequential: A customer may pay for an order using multiple payment methods.payment_type: The customer’s preferred method of payment.payment_installments: The customer’s preferred number of payment installments.payment_value: The transaction’s value.price: The cost of each item.freight_value: The cost of transportation for each item (if an order has more than one item the freight value is split between items).product_category: Each item’s category.product_name_length: The number of characters extracted from the product name.product_description_length: The number of characters extracted from the product description.product_photos_qty: The number of product photos that have been published.review_score: A rating given by a customer on a satisfaction survey ranging from 1 to 5.
order_status: This is a reference to the order status (delivered or canceled).
order_purchase_timestamp: Displays the timestamp of each item’s purchase.
order_delivered_customer_date: Displays the customer’s actual order delivery date.
order_estimated_delievy_date: Displays the estimated delivery date that was provided to the customer at the time of purchase.
shipping_limit_date: Displays the seller’s shipping limit date for transferring the order to the logistic partner.
payment_sequential: A customer may pay for an order using multiple payment methods.
payment_type: The customer’s preferred method of payment.
payment_installments: The customer’s preferred number of payment installments.
payment_value: The transaction’s value.
price: The cost of each item.
freight_value: The cost of transportation for each item (if an order has more than one item the freight value is split between items).
product_category: Each item’s category.
product_name_length: The number of characters extracted from the product name.
product_description_length: The number of characters extracted from the product description.
product_photos_qty: The number of product photos that have been published.
review_score: A rating given by a customer on a satisfaction survey ranging from 1 to 5.
As previously stated, decision tree modeling does not support missing values. Therefore, data preprocessing will include a data cleaning process to detect any missing values in each column.
#Detecting every NaN value on every columns
df_ecommerce.isnull().sum()
order_status                        0
order_purchase_timestamp            0
order_delivered_customer_date    2400
order_estimated_delivery_date       0
shipping_limit_date                 0
payment_sequential                  0
payment_type                        0
payment_installments                0
payment_value                       0
price                               0
freight_value                       0
product_category                    0
product_name_length                 0
product_description_length          0
product_photos_qty                  0
review_score                        0
dtype: int64
#Removing data with NaN value
prev_size = df_ecommerce.shape[0]
df_ecommerce.dropna(how='any',inplace=True)
current_size = df_ecommerce.shape[0]

df_ecommerce.isnull().values.any()
print(df_ecommerce.shape)
print('From the data cleaning, we remove {}% of NaN value data'.format(round(((prev_size - current_size)/prev_size)*100,2)))
On the “order delivered customer date” column alone, there are 2,400 NaN values. The data cleaning process reduces 115,609 rows to 113,209 rows by removing 2.08 percent of the NaN value data.
The timestamp data from the date value column has been converted to the new column with integral data for further analysis on the machine learning model.
Exploratory data analysis is used to gain a business understanding of each correlated column that influences the customer review. Here are some business questions regarding the various factors that will influence the customer review:
What are the top ten most popular products among customers?When it comes to paying the payment value, which payment method would the majority of customers prefer?How much will the payment value factor influence the customer review?How much will the freight value factor influence the customer review?How much influence will the price factor have on the customer review?How much will the relationship between payment value and price factor influence the customer review?How much influence will the correlation between estimated and delivery days have on the customer review?
What are the top ten most popular products among customers?
When it comes to paying the payment value, which payment method would the majority of customers prefer?
How much will the payment value factor influence the customer review?
How much will the freight value factor influence the customer review?
How much influence will the price factor have on the customer review?
How much will the relationship between payment value and price factor influence the customer review?
How much influence will the correlation between estimated and delivery days have on the customer review?
The top ten most purchased products are from the product categories of bed bath table, health beauty, sport leisure, furniture decoration, computer accessories, housewares, watches gifts, telephony, garden tools, and auto.Customers’ most popular product is the bed bath table, which has received over 10,000 orders.More than 8,000 orders have been placed in the categories of health and beauty, sport and leisure, furniture and decoration, and computer accessories.More than 6,000 orders were placed for housewares and watches gifts.More than 3,000 orders have been placed in the categories of telecommunications, garden tools, and auto.These top ten products played an important role in determining customer satisfaction, and they will be used as the parameter to do the one-hot encoding process later on during the feature engineering process.
The top ten most purchased products are from the product categories of bed bath table, health beauty, sport leisure, furniture decoration, computer accessories, housewares, watches gifts, telephony, garden tools, and auto.
Customers’ most popular product is the bed bath table, which has received over 10,000 orders.
More than 8,000 orders have been placed in the categories of health and beauty, sport and leisure, furniture and decoration, and computer accessories.
More than 6,000 orders were placed for housewares and watches gifts.
More than 3,000 orders have been placed in the categories of telecommunications, garden tools, and auto.
These top ten products played an important role in determining customer satisfaction, and they will be used as the parameter to do the one-hot encoding process later on during the feature engineering process.
Credit cards have the highest payment value, followed by boleto, debit cards, and vouchers.Both credit card and boleto payments have a payment value of more than 175.The payment value using debit card is lower than both credit card and boleto as it has the value of 150.Voucher has the lowest payment value among the others because half of the actual price of the product can be paid by customers using redeemed voucher.
Credit cards have the highest payment value, followed by boleto, debit cards, and vouchers.
Both credit card and boleto payments have a payment value of more than 175.
The payment value using debit card is lower than both credit card and boleto as it has the value of 150.
Voucher has the lowest payment value among the others because half of the actual price of the product can be paid by customers using redeemed voucher.
Review score is increase slightly as the payment value is decrease.Review score with value of 1 occurs when the payment value is more than 200.Review score with value of 2 occurs as the payment value decrease into around 175.Review score with value of 3, 4, and 5 occurs when the payment value is around 160 which makes the payment value of 160 will be the best option for making customer make high review score.
Review score is increase slightly as the payment value is decrease.
Review score with value of 1 occurs when the payment value is more than 200.
Review score with value of 2 occurs as the payment value decrease into around 175.
Review score with value of 3, 4, and 5 occurs when the payment value is around 160 which makes the payment value of 160 will be the best option for making customer make high review score.
Customer will likely to give better review score as the freight value decrease.Freight value of more than 20 gives the variety of customer review from 1 to 4.Customer will give the best review score of 5 when the freight value is less than 20.The lowest freight value makes the best review score because customers will be happy when they only have to pay a little for the additional payment of freight value.
Customer will likely to give better review score as the freight value decrease.
Freight value of more than 20 gives the variety of customer review from 1 to 4.
Customer will give the best review score of 5 when the freight value is less than 20.
The lowest freight value makes the best review score because customers will be happy when they only have to pay a little for the additional payment of freight value.
The lower price value has no discernible effect on the review score.A product with a price tag of more than 120 can elicit both positive and negative feedback from customers.The review score of 1 can occur when a product has a high price but the quality of the product does not meet the customer’s expectations (The high price can also mean high quality product).When a product has a high price and the quality of the product meets the customer’s expectations, a review score of 5 is possible.The review score of 3 has the lowest price among the others, with a value of around 110.
The lower price value has no discernible effect on the review score.
A product with a price tag of more than 120 can elicit both positive and negative feedback from customers.
The review score of 1 can occur when a product has a high price but the quality of the product does not meet the customer’s expectations (The high price can also mean high quality product).
When a product has a high price and the quality of the product meets the customer’s expectations, a review score of 5 is possible.
The review score of 3 has the lowest price among the others, with a value of around 110.
Most of the good review score of 4 and 5 occurs when the price and payment value has the same value.The correlated review score of the same value between price and payment value occurs when the customer only has to pay the price of a product without any additional value that will increase the payment value.Most of the review score of 5 also happened when the payment value is lower than the actual price of the products.Most of the review score of 1 happened when the payment value is higher than the actual price of the products.
Most of the good review score of 4 and 5 occurs when the price and payment value has the same value.
The correlated review score of the same value between price and payment value occurs when the customer only has to pay the price of a product without any additional value that will increase the payment value.
Most of the review score of 5 also happened when the payment value is lower than the actual price of the products.
Most of the review score of 1 happened when the payment value is higher than the actual price of the products.
The majority of the good review scores of 4 and 5 have long estimated days, but the actual delivery days of the product arrive early or on time, making the product arrive at the customer’s hand faster than the expected time.The majority of the negative review scores of 1 and 2 have short estimated delivery days, but the actual delivery days of the product are longer than the expected time.
The majority of the good review scores of 4 and 5 have long estimated days, but the actual delivery days of the product arrive early or on time, making the product arrive at the customer’s hand faster than the expected time.
The majority of the negative review scores of 1 and 2 have short estimated delivery days, but the actual delivery days of the product are longer than the expected time.
To make the data more precise, feature engineering is used to create new features such as new columns and dummy variables. The explanation for this project’s feature engineering can be found further down below.
We create three new columns from the feature engineering and delete the old ones because the new columns will contribute more to the model prediction.
arrival_time: Specify how many days the product should take to reach the customer based on estimated and delivery days.delivery_arrival: Based on the arrival time, determine whether the delivery is on time or late (if the negative value on arrival time means late and the positive value on arrival time means on time).Score: Determine whether a customer provides a positive or negative review (0 for review score = 1–2, 1 for review score = 4–5, and neutral value of review score 3 is removed).
arrival_time: Specify how many days the product should take to reach the customer based on estimated and delivery days.
delivery_arrival: Based on the arrival time, determine whether the delivery is on time or late (if the negative value on arrival time means late and the positive value on arrival time means on time).
Score: Determine whether a customer provides a positive or negative review (0 for review score = 1–2, 1 for review score = 4–5, and neutral value of review score 3 is removed).
There are some columns that must be handled, such as those with categorical values. As a result, label encoding and one-hot encoding must be used to transform those values into int values for machine learning modeling.
Label encoding on the distinct value of 2: Convert the negative value to 0 and the positive value to 1.One-hot encoding for distinct values greater than 2: One-hot encoding process on a specific column using the pd.get dummies function to get the dummy variable.One-hot encoding on a distinct value greater than 10: Define the top ten most frequently appearing values and then apply one-hot encoding to those specific values.
Label encoding on the distinct value of 2: Convert the negative value to 0 and the positive value to 1.
One-hot encoding for distinct values greater than 2: One-hot encoding process on a specific column using the pd.get dummies function to get the dummy variable.
One-hot encoding on a distinct value greater than 10: Define the top ten most frequently appearing values and then apply one-hot encoding to those specific values.
Before we begin the modeling process, we must divide the data into X and Y arrays, and then divide it again into train and test data. The X array contains a features column of order-related, payment-related, and product-related information that we previously processed. The y array only contains the “Score” feature engineered column. A more detailed explanation of the code is provided below.
To find the optimal parameter for each modeling phase, hyperparameter tuning and cross-validation techniques are performed. Furthermore, to gain more information on the model evaluation, we will utilize the confusion matrix and evaluate the model correctness to determine whether or not our data is overfitting. We will also compare actual train data and predicted train data to gain further evidence of how our statistical machine learning method can accurately predict customer satisfaction.
The decision tree model’s Hyperparameter tuning process of cross-validation takes only about 16 seconds.The best parameter for the decision tree uses max_depth value of 4, the min_samples_split value of 100, and mean test score of 0.871051.The accuracy of the train data is 0.8714%, while the accuracy of the test data is 0.8673%.When comparing actual train data to predicted train data, the decision tree model predicts 88 out of 100 correct predictions.The decision tree model has an F1-score of 0.93%.
The decision tree model’s Hyperparameter tuning process of cross-validation takes only about 16 seconds.
The best parameter for the decision tree uses max_depth value of 4, the min_samples_split value of 100, and mean test score of 0.871051.
The accuracy of the train data is 0.8714%, while the accuracy of the test data is 0.8673%.
When comparing actual train data to predicted train data, the decision tree model predicts 88 out of 100 correct predictions.
The decision tree model has an F1-score of 0.93%.
The Hyperparameter tuning process of cross-validation for the decision tree model only takes around 2 minutes, takes longer than the decision tree model.The best parameter for the random forest model uses max_depth value of 5, n_estimator value of 50, and mean test score of 0.869954.The train data accuracy is 0.8701%, while the test data accuracy is 0.8664%.The random forest model predicts 88 out of 100 correct predictions when comparing actual train data with predicted train data.The F1-score of this modeling is 0.93%, which is the same as the decision tree model.
The Hyperparameter tuning process of cross-validation for the decision tree model only takes around 2 minutes, takes longer than the decision tree model.
The best parameter for the random forest model uses max_depth value of 5, n_estimator value of 50, and mean test score of 0.869954.
The train data accuracy is 0.8701%, while the test data accuracy is 0.8664%.
The random forest model predicts 88 out of 100 correct predictions when comparing actual train data with predicted train data.
The F1-score of this modeling is 0.93%, which is the same as the decision tree model.
The Hyperparameter tuning process of cross-validation for the gradient boosting model takes more than 11 minutes, The longest than the other models.The best parameter for the random forest model uses max_depth value of 5, n_estimator value of 50, and mean test score of 0.874234.Train data accuracy is 0.8775% while the test data accuracy is 0.8709%, making this model the most fitted than other models.The random forest model predicts 88 out of 100 correct predictions when comparing actual train data with predicted train data.This model’s F1-score is 0.93%, which is the same as the other models.
The Hyperparameter tuning process of cross-validation for the gradient boosting model takes more than 11 minutes, The longest than the other models.
The best parameter for the random forest model uses max_depth value of 5, n_estimator value of 50, and mean test score of 0.874234.
Train data accuracy is 0.8775% while the test data accuracy is 0.8709%, making this model the most fitted than other models.
The random forest model predicts 88 out of 100 correct predictions when comparing actual train data with predicted train data.
This model’s F1-score is 0.93%, which is the same as the other models.
After we have evaluated all of the modeling methods, we will compare the effectiveness of each model. Each model has advantages and disadvantages; we will decide which of these models is the best to use in predicting customer satisfaction.
The F1-score of all models has a score of 0.93%.Based on the comparison of actual train and predicted train data, we can conclude that all of our models correctly predict 88 out of 100 times.The gradient boosting model has the most fitted data results compared to the other models, but it has the longest cross-validation process for hyperparameter tuning, lasting more than 11 minutes.The decision tree model has the quickest hyperparameter tuning cross-validation process, taking only 16 seconds.The decision tree model is recommended for predicting customer satisfaction because it has the quickest modeling process with the same accuracy and high F-1 score as the other models.
The F1-score of all models has a score of 0.93%.
Based on the comparison of actual train and predicted train data, we can conclude that all of our models correctly predict 88 out of 100 times.
The gradient boosting model has the most fitted data results compared to the other models, but it has the longest cross-validation process for hyperparameter tuning, lasting more than 11 minutes.
The decision tree model has the quickest hyperparameter tuning cross-validation process, taking only 16 seconds.
The decision tree model is recommended for predicting customer satisfaction because it has the quickest modeling process with the same accuracy and high F-1 score as the other models.
[1] D. R. Allen and T. R. Rao, Analysis of customer satisfaction data: A comprehensive guide to multivariate statistical analysis in customer satisfaction, loyalty, and service quality research (2000), ASQ Quality Press.
[2] P. K. Kapur, A. Anand and N. Sachdeva, Profit estimation for a product under warranty: an evaluation based on customer’s expectation and satisfaction (2014), International Journal of Reliability, Quality and Safety Engineering.
[3] C. Dobre and F. Xhafa, Pervasive computing next-generation platforms for intelligent data collection (2016), Academic Press.
[4] https://scikit-learn.org/stable/modules/tree.html
[5] https://www.datasciencecentral.com/profiles/blogs/decision-tree-vs-random-forest-vs-boosted-trees-explained
If you want to get more details on the code used in this modeling method, you can check on my Github repositories here.
To simplify the explanation of this modeling, I have also created the PowerPoint so that you can get a clear explanation. You can check the PowerPoint here.