## 2 \(\varepsilon\)-In SVR and Its Dual Problem
### The Nonsmooth Version of \(\varepsilon\)-In SVR  
The primal optimization problem of \(\varepsilon\)-In SVR in (1) can be equivalently formulated with equality constraints as follows. \[\min_{\mathbf{w}\in R^{n},b\in R^{1}} \frac{1}{2}\|\mathbf{w}\|^{2}+C\sum_{s=1}^{L}l_{\varepsilon}\big{(} \xi_{s}\big{)}\] (8) subject to \[\zeta_{s}=y_{s}-\mathbf{w}^{T}\varphi(\mathbf{x}_{s})-b,\forall s\in\{1, \dots,L\}\]  
The continuously differentiable \(\varepsilon\)-insensitive loss function is denoted by the following:  
\[l_{\varepsilon}(x;\eta_{1},\eta_{2})= \begin{cases}0,&\text{if }|x|<\varepsilon\\ \frac{1}{\eta_{1}}\ln\big{(}\cosh(\eta_{2}(|x|-\varepsilon))\big{)},&\text{ otherwise}\end{cases}  \tag{9}\]  
where the penalty parameter is represented by \(C\) and the insensitiveness region is determined by \(\varepsilon\). The Lagrangian of this problem (8) is then obtained as follows:  
\[\mathcal{L}(\mathbf{w},b,\mathbf{\xi},\mathbf{\alpha})=\frac{1}{2}\|\mathbf{w}\|^{2}+C\sum_{s=1 }^{L}l_{\varepsilon}\big{(}\xi_{s}\big{)}-\sum_{s=1}^{L}\alpha_{s}\big{(}\xi_{ s}-y_{s}+\mathbf{w}^{T}\varphi(\mathbf{x}_{s})+b\big{)} \tag{10}\]  
and the optimality conditions become as follows:  
\[\partial_{b}\mathcal{L} =\sum_{s=1}^{L}\alpha_{s}=0 \tag{11}\] \[\partial_{\mathbf{w}}\mathcal{L} =\mathbf{w}-\sum_{s=1}^{L}\alpha_{s}\varphi(\mathbf{x}_{s})=0\] (12) \[\partial_{\xi_{s}}\mathcal{L} =\alpha_{s}-\frac{C\eta_{2}}{\eta_{1}}\tanh_{\varepsilon}(\xi_{s}; \eta_{2})=0 \tag{13}\]  
where \(\tanh_{\varepsilon}(x;\eta_{2})\)\(\stackrel{{\text{def}}}{{=}}\)\( \begin{cases}0,&\text{if }|x|<\varepsilon\\ \tanh\big{(}\eta_{2}(x-\varepsilon)\big{)},&\text{if }x\geq\varepsilon\\ \tanh\big{(}\eta_{2}(x+\varepsilon)\big{)},&\text{otherwise}\end{cases} \). Equation (13) implies \(\xi_{s}=\frac{1}{\eta_{2}}\tanh^{-1}(\frac{\eta_{1}\alpha_{s}}{\eta_{2}C})+ \varepsilon sign^{*}(\alpha_{s})\) where \(sign^{*}(\alpha_{s})\) is defined as \( \begin{cases}1,&\text{if }\alpha_{s}>0\\ -1,&\text{if }\alpha_{s}=0\end{cases} \). Substituting it with (11) and (12) into (10), the following dual nonsmooth optimization problem is obtained as follows:  
\[ \begin{array}{ll}\min_{\mathbf{\alpha}\in\mathbb{R}^{L}}J(\mathbf{\alpha})&= \frac{1}{2}\sum_{s=1}^{L}\sum_{r=1}^{L}\alpha_{s}K(\mathbf{x}_{s},\mathbf{x}_{r})\alpha _{r}-\sum_{s=1}^{L}y_{s}\alpha_{s}-\sum_{s=1}^{L}\left(\frac{C}{\eta_{1}}\ln \Big{[}\cosh\big{(}\tanh^{-1}(\frac{\eta_{1}\alpha_{s}}{\eta_{2}C})\big{)} \Big{]}\right.\\ &\left.-\frac{\alpha_{s}}{\eta_{2}}\tanh^{-1}(\frac{\eta_{1}\alpha_{s}}{\eta_ {2}C}\Big{)}\right)+\varepsilon\sum_{s=1}^{L}|\alpha_{s}|\\ \text{subject to}&\sum_{s=1}^{L}\alpha_{s}=0,\\ &-\frac{\eta_{2}}{\eta_{1}}C\leq\alpha_{s}\leq\frac{\eta_{2}}{\eta_{1}}C, \forall s\in\{1,\dots,L\}\end{array}  \tag{14}\]  
where \(K(\mathbf{x}_{s},\mathbf{x}_{r})=\varphi(\mathbf{x}_{s})^{T}\varphi(\mathbf{x}_{r})\) is the kernel function and \(\mathbf{\alpha}=[\alpha_{1}\cdots\alpha_{L}]\in R^{L}\) are the Lagrange multipliers. It is worth mentioning that the loss function in (9) becomes optimal in the maximum likelihood sense to a family of PHS distributions as described in [49]. These family of distributions have the following probability density function:  
\[p(x;\eta_{1},\eta_{2},\varepsilon)=\frac{\eta_{2}}{B(\frac{1}{2\eta_{1}}, \frac{1}{2})+2\varepsilon\eta_{2}}[sech_{\varepsilon}(x;\eta_{2})]^{\frac{1} {\eta_{1}}} \tag{15}\]  
where \(B(k_{1},k_{2})=\int_{0}^{1}k^{k_{1}-1}(1-t)^{k_{2}-1}dt\) is the Beta function, \(sech_{\varepsilon}(x;\eta_{2})\stackrel{{\text{def}}}{{=}} \begin{cases} 1,&\text{if }|x|<\varepsilon\\ sech(\eta_{2}(|x|-\varepsilon)),&\text{otherwise}\end{cases} \) and \(\eta_{1}>0\). The study in [49] leads to the following proposition, which highlights the impact of the parameters \(\eta_{1}\) and \(\eta_{2}\) such that the probability density function (15) becomes equivalent to Laplacian, Gaussian and hyperbolic secant distributions by adjusting these parameters. **Proposition 1**.