Package: chefboost
ChefBoostChefBoostis a lightweight decision tree framework for Pythonwith categorical feature support. It covers regular decision tree algorithms:ID3,C4.5,CART,CHAIDandregression tree; also some advanved techniques:gradient boosting,random forestandadaboost. You just need to writea few lines of codeto build decision trees with Chefboost.Installation-DemoThe easiest way to install ChefBoost framework is to download it fromfrom PyPI. It's going to install the library itself and its prerequisites as well.pip install chefboostThen, you will be able to import the library and use its functionalitiesfromchefboostimportChefboostaschefUsage-DemoBasically, you just need to pass the dataset as pandas data frame and the optional tree configurations as illustrated below.importpandasaspddf=pd.read_csv("dataset/golf.txt")config={'algorithm':'C4.5'}model=chef.fit(df,config=config,target_label='Decision')Pre-processingChefboost handles the both numeric and nominal features and target values in contrast to its alternatives. So, you don't have to apply any pre-processing to build trees.OutcomesBuilt decision trees are stored as python if statements in thetests/outputs/rulesdirectory. A sample of decision rules is demonstrated below.deffindDecision(Outlook,Temperature,Humidity,Wind):ifOutlook=='Rain':ifWind=='Weak':return'Yes'elifWind=='Strong':return'No'else:return'No'elifOutlook=='Sunny':ifHumidity=='High':return'No'elifHumidity=='Normal':return'Yes'else:return'Yes'elifOutlook=='Overcast':return'Yes'else:return'Yes'Testing for custom instancesDecision rules will be stored inoutputs/rules/folder when you build decision trees. You can run the built decision tree for new instances as illustrated below.prediction=chef.predict(model,param=['Sunny','Hot','High','Weak'])You can consume built decision trees directly as well. In this way, you can restore already built decision trees and skip learning steps, or applytransfer learning. Loaded trees offer you findDecision method to test for new instances.moduleName="outputs/rules/rules"#this will load outputs/rules/rules.pytree=chef.restoreTree(moduleName)prediction=tree.findDecision(['Sunny','Hot','High','Weak'])tests/global-unit-test.py will guide you how to build a different decision trees and make predictions.Model save and restorationYou can save your trained models. This makes your model ready for transfer learning.chef.save_model(model,"model.pkl")In this way, you can use the same model later to just make predictions. This skips the training steps. Restoration requires to store .py and .pkl files underoutputs/rules.model=chef.load_model("model.pkl")prediction=chef.predict(model,['Sunny',85,85,'Weak'])Sample configurationsChefBoost supports several decision tree, bagging and boosting algorithms. You just need to pass the configuration to use different algorithms.Regular Decision TreesRegular decision tree algorithms find the best feature and the best split point maximizing the information gain. It builds decision trees recursively in child nodes.config={'algorithm':'C4.5'}#Set algorithm to ID3, C4.5, CART, CHAID or Regressionmodel=chef.fit(df,config)The following regular decision tree algorithms are wrapped in the library.AlgorithmMetricTutorialDemoID3Entropy, Information GainTutorialDemoC4.5Entropy, Gain RatioTutorialDemoCARTGINITutorialDemoCHAIDChi SquareTutorialDemoRegressionStandard DeviationTutorialDemoGradient BoostingTutorial,DemoGradient boosting is basically based on building a tree, and then building another based on the previous one's error. In this way, it boosts results. Predictions will be the sum of each tree'e prediction result.config={'enableGBM':True,'epochs':7,'learning_rate':1,'max_depth':5}Random ForestTutorial,DemoRandom forest basically splits the data set into several sub data sets and builds different data set for those sub data sets. Predictions will be the average of each tree's prediction result.config={'enableRandomForest':True,'num_of_trees':5}AdaboostTutorial,DemoAdaboost applies a decision stump instead of a decision tree. This is a weak classifier and aims to get min 50% score. It then increases the unclassified ones and decreases the classified ones. In this way, it aims to have a high score with weak classifiers.config={'enableAdaboost':True,'num_of_weak_classifier':4}Feature Importance-DemoDecision trees are naturally interpretable and explainable algorithms. A decision is clear made by a single tree. Still we need some extra layers to understand the built models. Besides, random forest and GBM are hard to explain. Herein,feature importanceis one of the most common way to see the big picture and understand built models.df=chef.feature_importance("outputs/rules/rules.py")featurefinal_importanceHumidity0.3688Wind0.3688Outlook0.2624Temperature0.0000ParalellismChefBoost offers parallelism to speed model building up. Branches of a decision tree will be created in parallel in this way. You should set enableParallelism argument to True in the configuration. Its default value is False. It allocates half of the total number of cores in your environment if parallelism is enabled.if__name__=='__main__':config={'algorithm':'C4.5','enableParallelism':True,'num_cores':2}model=chef.fit(df,config)Notice that you have to locate training step in an if block and it should check you are in main.ContributingPull requests are welcome. You should run the unit tests locally by runningtest/global-unit-test.py. Please share the unit test result logs in the PR.SupportThere are many ways to support a project - starring⭐️ the GitHub repos is just one 🙏You can also support this work onPatreonCitationPlease citeChefBoostin your publications if it helps your research. Here is an example BibTeX entry:@misc{serengil2021chefboost,author={Serengil, Sefik Ilkin},title={ChefBoost: A Lightweight Boosted Decision Tree Framework},month=oct,year=2021,publisher={Zenodo},doi={10.5281/zenodo.5576203},howpublished={https://doi.org/10.5281/zenodo.5576203}}Also, if you use chefboost in your GitHub projects, please add chefboost in the requirements.txt.LicenceChefBoost is licensed under the MIT License - seeLICENSEfor more details.