title: ÐÐ±ÑÐ¸Ðµ ÑÐ»ÑÑÑÐµÐ½Ð¸Ñ Ð² Ð³ÑÐ°Ð´Ð¸ÐµÐ½ÑÐ½Ð¾Ð¼ Ð±ÑÑÑÐ¸Ð½Ð³Ðµ
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ°
titleType: Subtitle
title: General gradient boosting improvements
Creators: name: Ð¡ÑÐ°ÑÑÐ»Ð¸Ð²ÑÐµÐ², ÐÐ¸ÐºÐ¸ÑÐ°
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2021
Subjects: Ð³ÑÐ°Ð´Ð¸ÐµÐ½ÑÐ½ÑÐ¹ Ð±ÑÑÑÐ¸Ð½Ð³
Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ðµ Ð¾Ð±ÑÑÐµÐ½Ð¸Ðµ
Ð´ÐµÑÐµÐ²ÑÑ ÑÐµÑÐµÐ½Ð¸Ð¹
Ð·Ð°Ð´Ð°ÑÐ° ÑÐµÐ³ÑÐµÑÑÐ¸Ð¸
Ð³Ð¸ÑÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ñ Ð¿ÑÐ¸Ð·Ð½Ð°ÐºÐ¾Ð²
ÑÐ°ÑÑÐ¸ÑÐ½Ð¾-ÑÐ»ÑÑÐ°Ð¹Ð½ÑÐµ Ð´ÐµÑÐµÐ²ÑÑ ÑÐµÑÐµÐ½Ð¸Ð¹
gradient boosting
machine learning
decision trees
regression problem
features histograms
partially randomized decision trees

description: ÐÐ°Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð¿Ð¾ÑÐ²ÑÑÐµÐ½Ð° Ð¿Ð¾Ð²ÑÑÐµÐ½Ð¸Ñ ÐºÐ°ÑÐµÑÑÐ²Ð° Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð° Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ, Ð¸Ð·Ð²ÐµÑÑÐ½Ð¾Ð³Ð¾ ÐºÐ°Ðº Ð³ÑÐ°Ð´Ð¸ÐµÐ½ÑÐ½ÑÐ¹ Ð±ÑÑÑÐ¸Ð½Ð³. ÐÐ°Ð´Ð°ÑÐ¸, ÐºÐ¾ÑÐ¾ÑÑÐµ ÑÐµÑÐ°Ð»Ð¸ÑÑ Ð² ÑÐ¾Ð´Ðµ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ: 1. Ð Ð°Ð·Ð±Ð¾Ñ ÑÐ°Ð±Ð¾ÑÑ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð° Ð³ÑÐ°Ð´Ð¸ÐµÐ½ÑÐ½Ð¾Ð³Ð¾ Ð±ÑÑÑÐ¸Ð½Ð³Ð° Ð½Ð° Ð´ÐµÑÐµÐ²ÑÑÑ ÑÐµÑÐµÐ½Ð¸Ð¹. 2. Ð ÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ñ Ð³ÑÐ°Ð´Ð¸ÐµÐ½ÑÐ½Ð¾Ð³Ð¾ Ð±ÑÑÑÐ¸Ð½Ð³Ð° Ð½Ð° Ð´ÐµÑÐµÐ²ÑÑÑ ÑÐµÑÐµÐ½Ð¸Ð¹ Ñ ÑÐ»ÑÑÑÐµÐ½Ð¸ÑÐ¼Ð¸ Ð´Ð»Ñ ÑÐµÑÐµÐ½Ð¸Ñ Ð·Ð°Ð´Ð°ÑÐ¸ ÑÐµÐ³ÑÐµÑÑÐ¸Ð¸. 3. Ð¡ÑÐ°Ð²Ð½ÐµÐ½Ð¸Ðµ ÐºÐ°ÑÐµÑÑÐ²Ð° Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹ Ð³ÑÐ°Ð´Ð¸ÐµÐ½ÑÐ½Ð¾Ð³Ð¾ Ð±ÑÑÑÐ¸Ð½Ð³Ð° ÑÐ²Ð¾ÐµÐ¹ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ð¸Ð¸ Ñ Ð¸Ð·Ð²ÐµÑÑÐ½ÑÐ¼Ð¸ Ð´Ð¾ÑÑÑÐ¿Ð½ÑÐ¼Ð¸ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸ÑÐ¼Ð¸. 4. ÐÐ·ÑÑÐµÐ½Ð¸Ðµ Ð²Ð»Ð¸ÑÐ½Ð¸Ñ ÑÐµÐ°Ð»Ð¸Ð·Ð¾Ð²Ð°Ð½Ð½ÑÑ Ð¸Ð´ÐµÐ¹ Ð¿Ð¾ ÑÐ»ÑÑÑÐµÐ½Ð¸Ñ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð° Ð½Ð° ÐµÐ³Ð¾ ÐºÐ°ÑÐµÑÑÐ²Ð¾. Ð¡Ð¿Ð¸ÑÐ¾Ðº Ð¾ÑÐ½Ð¾Ð²Ð½ÑÑ Ð¸ÑÑÐ»ÐµÐ´ÑÐµÐ¼ÑÑ ÑÐ»ÑÑÑÐµÐ½Ð¸Ð¹ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð°: â¢ Ð§Ð°ÑÑÐ¸ÑÐ½Ð¾-ÑÐ»ÑÑÐ°Ð¹Ð½ÑÐµ Ð¿Ð¾ÑÐ¾Ð³Ð¸ Ð¿ÑÐ¸Ð·Ð½Ð°ÐºÐ¾Ð². â¢ ÐÐ¸ÑÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ñ Ð¿ÑÐ¸Ð·Ð½Ð°ÐºÐ¾Ð² Ñ Ð¿ÐµÑÐµÐ¼ÐµÐ½Ð½Ð¾Ð¹ ÑÐµÑÐºÐ¾Ð¹. â¢ Ð¡Ð»ÑÑÐ°Ð¹Ð½Ð°Ñ Ð´Ð¾Ð±Ð°Ð²ÐºÐ° Ðº ÑÐµÐ½Ðµ ÑÐ°Ð·Ð±Ð¸ÐµÐ½Ð¸Ñ Ð¿ÑÐ¸ Ð¿Ð¾ÑÑÑÐ¾ÐµÐ½Ð¸Ð¸ Ð´ÐµÑÐµÐ²ÑÐµÐ² ÑÐµÑÐµÐ½Ð¸Ð¹. Ð ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐµ Ð±ÑÐ» Ð½Ð°Ð¿Ð¸ÑÐ°Ð½ Ð¿ÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ð½ÑÐ¹ Ð¼Ð¾Ð´ÑÐ»Ñ Ð½Ð° ÑÐ·ÑÐºÐµ C++ Ð´Ð»Ñ ÑÐ·ÑÐºÐ° Python 3, ÐºÐ¾ÑÐ¾ÑÑÐ¹ ÑÐ¾Ð´ÐµÑÐ¶Ð¸Ñ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ñ Ð³ÑÐ°Ð´Ð¸ÐµÐ½ÑÐ½Ð¾Ð³Ð¾ Ð±ÑÑÑÐ¸Ð½Ð³Ð° Ñ ÑÐ»ÑÑÑÐµÐ½Ð¸ÑÐ¼Ð¸. ÐÑÐ»Ð¾ Ð¿ÑÐ¾Ð²ÐµÐ´ÐµÐ½Ð¾ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ðµ ÐºÐ°ÑÐµÑÑÐ²Ð° Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹ Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ, Ð¿Ð¾Ð»ÑÑÐµÐ½Ð½ÑÑ Ð² ÑÐ¾Ð´Ðµ ÑÐ²Ð¾ÐµÐ¹ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ð¸, Ñ Ð¸Ð·Ð²ÐµÑÑÐ½ÑÐ¼Ð¸ Ð´Ð¾ÑÑÑÐ¿Ð½ÑÐ¼Ð¸ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸ÑÐ¼ Ð½Ð° Ð¿ÑÐµÐ´Ð¼ÐµÑ ÐºÐ°ÑÐµÑÑÐ²Ð° Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹. ÐÑÐ»Ð¾ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¾ Ð²Ð»Ð¸ÑÐ½Ð¸Ðµ ÑÐ»ÑÑÑÐµÐ½Ð¸Ð¹, Ð¿ÑÐµÐ´ÑÑÐ°Ð²Ð»ÐµÐ½Ð½ÑÑ Ð² ÑÐ°Ð±Ð¾ÑÐµ, Ð½Ð° ÐºÐ°ÑÐµÑÑÐ²Ð¾ Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹.
description: The given work is devoted to the quality improvement of the machine learning algorithm known as gradient boosting. The research set the following goals: 1. Understanding of the work of algorithm of gradient boosting based on regression trees in application to the solution of the regression problem. 2. Gradient boosting based on regression trees algorithm implementation. 3. Comparison of quality of gradient boosting models of the proposed implementation with known available implementations. 4. Research of impact of the implemented ideas of algorithm improvements on its quality. The list of main researched algorithm improvements: â¢ Partially randomized feature thresholds. â¢ Feature histograms with variable grid. â¢ Random additive to the score of splits during decision tree fit. As the result, the program module with gradient boosting algorithm with im-provements implementation has been written in C++ language to use in Python 3 programming language. Comparison between the quality of machine learning models got with the proposed implementation and known available implementations has been done. The impact of the proposed improvements on the quality of the models has been studied.