# Machine Learning Based Surrogate Models for the Thermal Behavior of Multi-Plate Clutches
## 2 Methodology
### Model Development  
Five different machine learning methods are investigated to construct the surrogate model. These are explained in greater depth below. Unless otherwise noted, more information on each algorithm can be found in the work by [PERSON] [21]. 1. Polynomial Regression (PR): Polynomial regression is a subclass of linear regression, in which a basis function expansion is performed with polynomial functions [8; 21]. The use of higher order polynomial functions enables non-linear relationships to be modeled. The PR model is given by: \[f(x)=w^{T}\phi(x)\] (1) with \[\phi(x)=\left[1,\ x,\ x^{2},\ldots,\ x^{d}\right]\] (2) where \(w\) is the vector with the weight factors and \(d\) is the selected polynomial order. Although PR models are very popular due to their simple calculation and the option of deriving conclusions about the influences of the individual input parameters [8], these models suffer from the disadvantage of being prone to overfitting for high polynomial degrees [21]. 2. Decision Tree (DT): Decision trees are methods that divide the input space into several areas. These subdivisions take place along the individual axes of the input space by means of the CART algorithm and can be represented by means of a tree. The mean value \(w_{i}\) is calculated for each of the regions \(R_{i}\) created after the subdivision. The output value of the model is given by the following expression: \[f(x)=\sum\limits_{i=1}^{n}w_{i}\mathbb{I}(x\in R_{i})\] (3)
3. Support Vector Regression (SVR): Support vector regression is a parametric model that uses kernels and considers only a portion of the training dataset to generate predictions. SVR models for a predefined kernel \(\kappa\) can generically be defined by the following equation: \[f(x)=w_{0}+\sum\limits_{i=1}^{n}w_{i}\kappa(x_{i},x)\] (4) where \(w\) are the parameters to be calculated. A combination of \(\ell_{2}\) regularization and epsilon insensitive loss function is used as the cost function. With the use of the epsilon insensitive loss function, all data points located within a band of width epsilon are not penalized. As a result, the model can be obtained by solving the constrained optimization problem given by Equation (5) [21]:  
\[ \begin{split}\min\frac{1}{2}|w|^{2}+C\sum\limits_{i=1}^{n}\left( \xi_{i}^{+}-\xi_{i}^{-}\right)\\ s.t.\ \left\{\begin{matrix}y_{i}-f(x_{i})\leq\epsilon+\xi_{i}^{+}\\ f(x_{i})-y_{i}\leq\epsilon+\xi_{i}^{-}\\ \xi_{i}^{+}\cdot\xi_{i}^{-}\geq 0\end{matrix}\right.\end{split}  \tag{5}\]  
where \(\xi_{i}^{+}\) and \(\xi_{i}^{-}\) are introduced as slack variables and indicate the extent to which a data point lies outside the \(\epsilon\)-band. The hyperparameter \(C\) regulates the tradeoff between the flatness of the model and the degree to which deviations larger than \(\epsilon\) are tolerated. One of the drawbacks of these methods is the high computational cost of constructing the models. Interested readers can refer to [PERSON] and [PERSON] [22] for more information on SVR. Gaussian Process (GP): Gaussian processes are non-parametric methods having the inference of distributions over functions as a basic principle [21]. If the data is noise-free, then GPs have the capability to interpolate the data points exactly. This is advantageous when creating surrogate models of deterministic FE models [23]. For the measured values \(f\) at the sample points \(X\) and the values \(f_{*}\) being predicted at the points \(X_{*}\), the joint probability distribution is given by:  
\[ \begin{pmatrix}f\\ f_{*}\end{pmatrix} \sim\mathcal{N}\bigg{(} \begin{pmatrix}\mu\\ \mu_{*}\end{pmatrix} , \begin{pmatrix}K&K_{*}\\ K_{*}^{T}&K_{**}\end{pmatrix} \bigg{)} \tag{6}\]  
with  
\[K=\kappa(X,X),\ K_{*}=\kappa(X,X_{*}),\ K_{**}=\kappa(X_{*},X_{*}) \tag{7}\]  
where \(\kappa\) is the selected kernel or covariance function.