## 4 Machine Learning Techniques
In our study, we use all of the three most common types of neural networks: convolutional (CNNs) that find patterns in the images (we have not trained them to recognize specific body parts, they only keep track on the surface heat distribution. ), recurrent (RNNs) that analyze the text in our datasets (as a substitute for encoding, e.g., in the feature importance estimates) and the usual artificial neural networks (ANNs), which give predictions in our stack ensemble based on tabular values. However, those algorithms have become an integral part of our everyday life with social media, chat-bots, and face recognition apps, so they probably do not need much explanation. Ensemble learning techniques, on the other hand, seem to be popular mostly within the data science community as they tend to solve more specific tasks; thus, we shall comment on them. Ensemble learning promotes the concept of collective decisions, showing in practice that a large number of rough and biased models (weak learners) often beat the accuracies of highly sophisticated algorithms. The most commonly used example, Random Forest (RF), has been around for almost three decades. It uses an ensemble of decision trees trained in parallel with biased data--each tree, with its own sample and subset of features via bootstrapping (sampling with replacement). Then, simple majority vote (aggregation) takes place for the output, although statistical weights can also be introduced in some modifications. Combining these two stages, we end up with what in ML slang is called a bagging model. It is powerful and unpretentious at the same time, good for parallel processing and resistant to overfitting, but it might need lots of resources, especially when it comes to hyperparameter tuning and optimization. Boosting algorithms provide significant improvement in that respect, relying on the iterative corrections of statistical weights assigned to each data instance in the samples. The ones that lead to wrong predictions are promoted by increasing the corresponding weight, so that they  
Figure 5: Merging and stacking of multiple models in AutoML. are more likely to appear in the next iteration, giving the model a chance to learn and to minimize the error, much like how musicians spend more time on the difficult parts of the pieces. That is the basis of adaptive boosting (AdaBoost), but there are certainly other variations to this idea. For example, gradient boosting algorithms, such as XGBoost, use gradient descent to optimize the weights at each iteration, just like as with neural networks do. This makes it quite fast and successful in balancing the so-called 'bias-variance trade-off'. Just like bagging algorithms, it does not require data normalization and works well even with missing values, but it seems to be more sensitive to outliers, and so one needs to pay attention. Nevertheless, it is one of the best known solutions to both the classification and regression machine learning problems. CatBoost is another gradient boosting algorithm that is specifically designed for handling categorical data in a more efficient manner. It works well even with smaller samples, it does not require much tuning, and it takes special measures to prevent overfitting. Another difference is the decision tree topology:'stumps' for AdaBoost, symmetric trees for CatBoost, and leaf-wise growing ones for LightGBM.