# Prediction of Flotation Deinking Performance: A Comparative Analysis of Machine Learning Techniques
## 2 Materials and Methods  
### Machine Learning Techniques for Prediction of Flotation Deinking Performance  
#### 2.1.1 Support Vector Regression  
Support Vector Machines (SVMs) are a supervised machine learning method originally developed for solving binary classification problems. While the output variable is discrete in classification, it is continuous in regression (real number). Therefore, it is not possible to give an exact prediction as in classification, and an error (deviation) \(\varepsilon\) is introduced [18], SVM for regression problems is usually denoted as Support Vector Regression (SVR).  
Suppose a training dataset \(D=\left\{(x_{i},y_{i}),i=1,\ldots,N\right\}\) which consists of \(N\) training pairs, where \(x_{i}\in R^{n}\) is the \(n\)-dimensional vector denoting the model's inputs and \(y_{i}\in R\) are the observed responses to these inputs (model's outputs). The goal of SVR is to determine a function \(f(x)\) that deviates from \(y_{i}\) by a value not greater than \(\varepsilon\) for each training data point \(x_{i}\).  
\[f(x)=\left\langle\omega,x\right\rangle+b;\omega\in R^{n},b\in R \tag{1}\]  
where \(\omega\) is the weight, b is the bias and \(\left\langle\omega,x\right\rangle\) denotes the dot product. Values \(\omega\) and \(b\) are determined from the training data by maximizing the margin \(2/\left\|\omega\right\|\), or equivalently by minimizing \(\frac{1}{2}\left\|\omega\right\|^{2}\), where the factor \(\frac{1}{2}\) is used for mathematical convenience only [19]:  
\[argmin\frac{1}{2}\left\|\omega\right\|^{2}+C\sum_{i=1}^{N}(\xi_{i}+\xi_{i}^{*} );\text{ subjectto}\left\{\begin{array}{l}y_{i}-f(x_{i})\leq\varepsilon+\xi_{i} \\ f(x_{i})-y_{i}\leq\varepsilon+\xi_{i}^{*}\\ \xi_{i},\xi_{i}^{*}\geq 0\end{array}\right. \tag{2}\]  
where constant \(C>0\) defines the amount of error larger than \(\varepsilon\) that is tolerated and \(\xi_{i},\ \xi_{i}^{*}\) are the error tolerances. The solution of (2) is found using Lagrange multipliers with the dual set of variables. To obtain the dual formula, a Lagrange function is constructed from the primal function by introducing non-negative Lagrange multipliers \(\alpha_{i}\), \(\alpha_{i}^{*}\), \(\eta_{i}\), \(\eta_{i}^{*}\) for each training data point \(x_{i}\).  
\[\omega=\sum_{i=1}^{N}(\alpha_{i}-\alpha_{i}^{*})\cdot x_{i}\]  
\[b=y_{i}-\left\langle\omega,x_{i}\right\rangle-\varepsilon,0<\alpha_{i}<C \tag{3}\]  
\[b=y_{i}-\left\langle\omega,x_{i}\right\rangle+\varepsilon,0<\alpha_{i}^{*}<C\]  
The function used to predict new values then becomes:  
\[f(x)=\sum_{i=1}^{N}(\alpha_{i}-\alpha_{i}^{*})\cdot\left\langle x_{i},x\right\rangle+b \tag{4}\]  
where the bias b is defined in (5).  
When the linear model is not adequate, the Lagrange dual formulation can be extended to nonlinear functions by replacing the dot product \(\left\langle x_{i},x\right\rangle\) with a nonlinear kernel function \(\left\langle x_{i},x\right\rangle=\varphi(x_{i})\cdot\varphi(x)\), where each data point \(x_{i}\) is mapped to a higher-dimensional space using the transform \(\Phi:x_{i}\rightarrow\varphi(x_{i})\). The solution to the optimization problem for the nonlinear case becomes:  
\[\omega=\sum_{i=1}^{N}(\alpha_{i}-\alpha_{i}^{*})\cdot\varphi(x_{i})\]  
\[f(x)=\sum_{i=1}^{N}(\alpha_{i}-\alpha_{i}^{*})\cdot k(x_{i},x)+b \tag{5}\]  
Common kernel functions are the linear, polynomial, sigmoid and radial basis function (RBF). The RBF kernel is used in this paper [20]:  
\[k\left\langle x_{i},x\right\rangle=exp\left(-\gamma\left\|x_{i}-x\right\|^{2}\right) \tag{6}\]  
where \(\gamma>0\) is the regularization parameter which determines the trade-off between the fitting error minimization and the smoothness of the estimated function.