# Copper Price Prediction Using Support Vector Regression Technique
## 3 Support Vector Regression Model  
Given a data set of \(N\) elements \(\{(X_{i},y_{i})\}_{i=1}^{N}\), where \(X_{i}\) is the \(i\)-th element in a space of \(n\) dimensions, \(X_{i}=[x_{1,i},\ldots,x_{n,i}]\in\mathcal{R}^{n}\) and \(y_{i}\) (\(y_{i}\in\mathcal{R}\)) is the actual value for \(X_{i}\), a nonlinear function is defined as \(\phi:\mathcal{R}^{n}\rightarrow\mathcal{R}^{n_{h}}\). To map the entry data, \(X_{i}\) is an \(\mathcal{R}^{n_{h}}\) space of high dimension called space of features that determines the nonlinear transformation \(\phi\). So, in a high-dimensional space, there exists a linear function \(f\) that makes it possible to relate the entry data \(X_{i}\) and output \(y_{i}\). That linear function, the SVR function, is presented in Equation (1),  
\[f(X)=W^{T}\cdot\phi(X)+b \tag{1}\]  
where \(f(X)\) represents the foretold values; \(W\in\mathcal{R}^{n}\) and \(b\in\mathcal{R}\). The SVR minimizes the empiric risk, shown in Equation (2)  
\[R_{reg}(f)=C\sum_{i=1}^{N}\Theta_{\epsilon}(y_{i}-f(X_{i}))+\frac{1}{2}\left\| W^{T}\right\| \tag{2}\]  
where \(\Theta_{\epsilon}(y_{i}-f(X_{i}))\) is a cost function. In the case of the \(\epsilon\)-SVR, a loss function \(\epsilon\)-insensitive is used [10; 24], defined in Equation (3),  
\[\Theta_{\epsilon}(y-f(X))=\begin{cases}|y-f(X)|-\epsilon&\quad\text{If}\ \ |y-f(X)|\geqslant \epsilon\\ 0&\quad\text{In another case}\end{cases} \tag{3}\]  
\(\Theta_{\epsilon}\) is used to determine the nonlinear function \(\phi\) in the \(\mathcal{R}^{n_{h}}\) space to find a function that can fit current training data with a deviation less than or equal to \(\epsilon\) (see Figure 0(a)). This function minimizes the training error between the data training, and the function \(\epsilon\)-insensitive is provided by Equation (4) [11; 25].  
\[\min_{W,b,\xi^{*},\xi}R_{reg}(W,\xi^{*},\xi)=\frac{1}{2}W^{T}W+C\sum_{i=1}^{N} (\xi_{i}^{*}+\xi_{i}) \tag{4}\]  
subject to restrictions (for all, \(i=1,\ldots,N\)):  
\[Y_{i}-W^{T}\phi(X_{i})-b \leqslant\epsilon+\xi_{i}^{*}\] \[-Y_{i}-W^{T}\phi(X_{i})+b \leqslant\epsilon+\xi_{i}\] \[\xi_{i}^{*} \geqslant 0 \tag{5}\] \[\xi_{i} \geqslant 0\]  
Equation (4) punishes the training errors of \(f(X)\) and \(Y\) through the function \(\epsilon\)-insensitive (Figure 0(b)). The parameter \(C\) determines the compromise between the complexity of the model, expressed by the vector \(W\) and the points that fulfill the condition \(|f(X)-y|\geqslant\epsilon\) in Equation (3). If \(C\rightarrow\infty\), the model has a small margin and is adjusted to the data. If \(C\to 0\), the model has a big margin, which is why it is softened. Finally, \(\xi_{i}^{*}\) represents the training errors greater than \(\epsilon\) and \(\xi_{i}\) the errors less than \(-\epsilon\) (see Figure 0(a)).  
To solve this regression problem, we can replace the internal product of Equation (1) by functions of kernel \(K()\). This makes it possible to perform such an operation in a superior dimension, using low-dimensional space data input without knowing the transformation \(\phi\)[26], as it is shown in Equation (6). This is called the kernel trick.  
\[f(X)=\sum_{i=1}^{N}(\beta^{*}-\beta)\cdot K(X_{i},X)+b \tag{6}\]  
The parameters \(\beta^{*}\) and \(\beta\) are Lagrange multipliers associated with the problem of quadratic optimization. Several types of functions can be used as kernel [27], but in this work, we will be using the Gaussian function of a radial base (RBF) [28]:  
\[K(X_{i},X)=exp(-\gamma||X_{i}-X||^{2}) \tag{7}\]  
The parameters \(\gamma\) of the kernel function, the regularization constant \(C\) and \(\varepsilon\) of the loss function are considered the parameters of design for the SVR to use. Furthermore, they are obtained from a data set that is different from the training data.