# Interactive 3D Vase Design Based on Gradient Boosting Decision Trees
## 2 Algorithm and Principle
### CatBoost Algorithm  
The CatBoost [30] algorithm was developed by the Yandex team and optimized specifically for datasets containing classification features. The CatBoost algorithm is competitive in performance and accuracy and can handle large-scale datasets. The chief characteristic of the CatBoost algorithm is that it specifically processes classification features without the need for tedious preprocessing. It adopts the idea of gradient boosting but has been optimized in implementation to handle high-dimensional and large-scale datasets.  
The simplest way to handle the classification features in a GBDT is to replace them with the average value of the labels corresponding to the classification features. In the decision tree, the average value of labels will serve as the criterion for node splitting. This method is called Greedy Target-based Statistics, as shown in Equation (6):  
\[x_{i,k}=\frac{\sum_{j=1}^{n}\left[x_{j,k}=x_{i,k}\right]\cdot Y_{i}}{\sum_{j=1}^ {n}\left[x_{j,k}=x_{i,k}\right]}. \tag{6}\]  
This method has an obvious drawback, which is that the features usually contain more information than the labels. If the average value of the labels is forcibly used to represent the features, there will be a condition shift problem when the training dataset and test dataset have different data structures and distributions. CatBoost uses Ordered TargetStatistics (OTS) to reduce the impact of noise and low-frequency categorical data on data distribution. OST is shown in Equation (7):  
\[x_{i,k}=\frac{\sum_{j=1}^{p-1}\Big{[}x_{\sigma j,k}=x_{\sigma p,k}\Big{]}\cdot Y _{j}+a\cdot p}{\sum_{j=1}^{p-1}\Big{[}x_{\sigma j,k}=x_{\sigma p,k}\Big{]}+a}, \tag{7}\]  
where \(p\) is the added prior term, and \(a\) is usually a weight coefficient greater than 0. Adding prior terms is a common practice, which can reduce noisy data for features with fewer categories.  
The specific steps of feature processing are as follows: Firstly, it randomly sorts the dataset to reduce the model's dependence on specific features and increase its generalization ability. Secondly, for each category of features, CatBoost performs OTS calculations to convert the statistical information of target variables for each category into numerical features. This transformation method is often based on the average value of the target variable in that category, and the features are mapped to numerical values so that the model can directly use these transformed features for training. In addition, CatBoost also introduces the concept of priority, which calculates weight coefficients based on statistical information of features and is used to weight different categories to improve the stability and generalization ability of the model. These feature-processing strategies enable CatBoost to process classification features more effectively, improving the performance and generalization ability of the model. The pseudocode for CatBoost is shown in Algorithm 2.  
```
1:Input: train_data, test_data, hyperparameters
2:Output: predictions
3: catboost_model = CatBoostRegressor(learning_rate = hyperparameters['learning_rate'], max_depth = hyperparameters['max_depth'], subsample = hyperparameters['subsample'], colsample_bylevel = hyperparameters['colsample_bylevel'], n_estimators = hyperparameters['n_estimators'], random_seed = 42)
4: catboost_model.fit(train_data['X'], train_data['Y'], eval_set = (test_data['X'], test_data['Y']))
5: predictions = catboost_model.predict(test_data['X'])
6:return predictions
```  
**Algorithm 2**CatBoost Algorithm