4 Easy Steps for Implementing CatBoost | Towards Data Science

IntroductionInstallation and ImportsDefine DatasetApply ModelPredictSummaryReferences
Introduction
Installation and Imports
Define Dataset
Apply Model
Predict
Summary
References
CatBoost [2] has beaten many other popular machine learning algorithms on benchmark datasets where logloss was the error metric. It beat mainly LightGBM and XGBoost, which have recently been the standard before in not only data science competitions, but also in professional settings as well. Now is the time to learn this powerful library, and below is how you can implement it in four easy steps.
This tutorial will be using popular data science tools like Python and Jupyter Notebook. First, we will start off with the three simple installation commands, then move on to all the necessary imports that you would need for your first, basic CatBoost regression model — which, as you will see, might be your first and last, because that is how impressively great CatBoost is without much tuning or additional code.
Here are the main installation commands:
!pip install catboost!pip install ipywidgets!jupyter nbextension enable  — py widgetsnbextension
Here are the main import commands:
from catboost import CatBoostRegressorfrom sklearn.model_selection import train_test_splitimport numpy as npimport pandas as pd
As you can see, there are only just a few lines of code that you need for installing and importing. We will import other common useful libraries like numpy, pandas, and sklearn as well. Next, we will discuss the dataset process.
There are several ways to import a dataset, but the most common one that I do or see is importing a CSV file with pandas. You will establish your x attributes and your y variable, the variable you are trying to predict. You can isolate your dataset in the common way of a 75:25, for your train and test split. The random state is so that you can reproduce the same split in the future, or when collaborating with others.
Here are the main dataset defining commands:
dataframe = pd.read_csv('file_path_to_your_dataset.csv')X = dataframe[['X_feature_1', 'X_feature_2', etc.,]]y = dataframe['target_variable']X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.75, random_state=42)
When applying the CatBoost model, it works similarly to other sklearn approaches. However, the most important part is to designate your categorical variables, so that you can get the most out of your CatBoost model. Also, CatBoost has some awesome plots that visualize the error metric improvements over iterations, so if you want to visually see how the train and test set compare over time, setting the plot to True is a great approach. I have also included a common loss function and evaluation metric of RMSE for a regression model. The default n_estimators is 100, which is a great place to start, if you see that the model is still improving on the last round, then you can up the number of iterations.
Here are the main model commands:
CAT_FEATURES = [] #list of your categorical features# set up the modelcatboost_model = CatBoostRegressor(n_estimators=100,                                   loss_function = 'RMSE',                                   eval_metric = 'RMSE',                                   cat_features = CAT_FEATURES)# fit modelcatboost_model.fit(X_train, y_trian,                    eval_set = (X_test, y_test),                   use_best_model = True,                   plot = True)
Perhaps the easiest and most rewarding step is to see your predictions. Some other useful ways that you can use the predict method is to merge your original dataframe with your predictions with an inner join with both indexes set to true, so that you can see the features that led to those predictions. This command also follows the typical sklearn approach that most data scientists are used to.
Here is the main predict command:
# get your predictionspreds = catboost_model.predict(X_test)
I hope you enjoyed this short tutorial on CatBoost. It really is a powerful machine learning algorithm and library that anyone can implement. It is designed so that parameter tuning is usually not necessary, and the default parameters are the best. Additionally, there are other key visualizations that you can play around with, which are made from the popular SHAP library. The purpose of this article was to introduce you to CatBoost, and all you need moving forward is this code and your dataset to start making your first model predictions.
Here are all of the steps once more, summarized:
* Installation and Imports* Define Dataset* Apply Model* Predict
I hope you found my article both interesting and useful! Please feel free to comment down below if you have used CatBoost before. Has it helped you in your Data Science career now? Do you agree or disagree, and why?
Please feel free to check out my profile and other articles, as well as reach out to me on LinkedIn. I have no affiliations with these mentioned companies.
Thank you for reading!
[1] Photo by Manja Vitolic on Unsplash, (2018)
[2] Yandex, CatBoost, (2021)
[3] Photo by Christopher Gower on Unsplash, (2017)
[4] Photo by krakenimages on Unsplash, (2020)