## 3 The SMO-like Algorithm for the Nonsmooth Dual Problem of \(\varepsilon\)-In SVR
### Working Set Selection
Then, by the definition of strong convexity [51], it can be written that  
\[\hat{f}(d_{j}^{*})-\hat{f}(0)\leq z_{d_{j}^{*}}d_{j}^{*}-\frac{1}{\eta_{2}\hat{ C}}(d_{j}^{*})^{2}=-\frac{1}{\eta_{2}\hat{C}}(d_{j}^{*})^{2} \tag{43}\]  
where \(z_{d_{j}^{*}}\in\partial\hat{f}(d_{j}^{*})\) is the subgradient of \(\hat{f}\) at \(d_{j}^{*}\) and \(z_{d_{j}^{*}}=0\) since \(d_{j}^{*}\) is the optimal point. The proof concludes with \(J\Big{(}\mathbf{\alpha}^{k+1}\Big{)}-J\Big{(}\mathbf{\alpha}^{k}\Big{)}=\hat{f}(d_{j}^ {*})-\hat{f}(0)\leq-\frac{1}{\eta_{2}\hat{C}}(d_{j}^{*})^{2}=-\frac{\left\| \mathbf{\alpha}^{k+1}-\mathbf{\alpha}^{k}\right\|^{2}}{2\eta_{2}\hat{C}}\). ```
input :Training data \(\left\{\ (x_{1},y_{1})\ \ \cdots\ \ (x_{L},y_{L})\ \right\}\) output :\(\mathbf{\alpha},b\) Initialize by setting : \(\mathbf{\alpha}=0\), \(\
abla f_{q}(\mathbf{\alpha})=\mathbf{p}\), \(T(\mathbf{\alpha}_{s})=0\), \(g_{s}(\mathbf{\alpha}):=-\
abla f(\mathbf{\alpha})_{s}-esign^{+}(\mathbf{\alpha}_{s})\) and \(G_{s}(\mathbf{\alpha}):=-\
abla f(\mathbf{\alpha})_{s}-esign^{-}(\mathbf{\alpha}_{s})\ \forall s\in\left\{1,\cdots,L\right\}\) repeat /* Select the working set \(\left\{i,j\right\}\) as Equations (36) and (37): */ \(g_{max}=-INF_{i}\_candidate=-1\); for\(t=1\) to \(L\)do if\(\alpha_{t}<\hat{C}-\delta\)then \(g_{t}(\mathbf{\alpha})>g_{max}\)then \(g_{max}=g_{t}(\mathbf{\alpha})\); \(i\_candidate=t\);  end if  end for \(i=i\_candidate\); \(upper\_bound\_min=INF\); \(j\_candidate=-1\); for\(t=1\) to \(L\)do if\(\alpha_{t}>-\hat{C}+\delta\)then if\(g_{max}-G_{t}(\mathbf{\alpha})>0\)then \(h_{it}=g_{i}(\mathbf{\alpha})-G_{t}(\mathbf{\alpha})\); \(a_{it}=K_{ii}+K_{jj}-2\mathbf{K}_{jj}\); if\(\frac{-h_{i}^{2}}{a_{it}}<upper\_bound\_min\)then \(upper\_bound\_min=\frac{-h_{i}^{2}}{a_{it}}\); \(j\_candidate=t\);  end if  end for  end for  end for
```  
**Algorithm 1**SMO-like algorithm for the nonsmooth nonlinear dual problem  
```
input :Training data \(\left\{\ (x_{1},y_{1})\ \ \cdots\ \ (x_{L},y_{L})\ \right\}\) output :\(\mathbf{\alpha},b\) Initialize by setting : \(\mathbf{\alpha}=0\), \(\
abla f_{q}(\mathbf{\alpha})=\mathbf{p}\), \(T(\mathbf{\alpha}_{s})=0\), \(g_{s}(\mathbf{\alpha}):=-\
abla f(\mathbf{\alpha



## 4 Experiments

This section considers the dual problem (14) of the proposed \(\varepsilon\)-ln SVR, where a specific relationship holds between parameters \(\eta_{1}\) and \(\eta_{2}\) such as \(\eta_{2}=\sqrt{\frac{1}{2}\psi_{1}(\frac{1}{2\eta_{1}})}\). Tuning \(\eta_{1}\) optimizes the loss function for a family of PHS distributions, as detailed in Section 2.2. To solve this challenging nonsmooth nonlinear dual problem (14) including linear equality and box constraints, a novel SMO-like algorithm with an efficient WSS procedure is introduced. This approach involves minimizing an upper bound on the second-order approximation, derived from the Taylor series expansion between consecutive loss function values. For evaluating the \(\varepsilon\)-ln SVR, comparisons are drawn against the \(\varepsilon\)-\(l_{2}\) SVR and the \(\varepsilon\)-SVR, both respectively solved by SMO algorithms [21; 52]. The proposed WSS procedure with SO-like information, which is one of the parts of the SMO-like algorithm that most affects the convergence time, is compared with its FO counterpart. Additionally, the proposed SMO algorithm for the nonsmooth dual problem is compared with the SMO algorithm for its smooth counterpart (7). All implementations are written in C++. The SMO-like algorithm presented in Algorithm 1 is adapted for use with the well-known LIBSVM library [52] to take advantage of its efficient kernel computation and caching mechanisms. All experiments were carried out on a PC equipped with an Intel Core i5-12450H processor and 16 GB of RAM, operating on a 64-bit Windows 11 system. The RBF kernel, defined as \(K(x_{s},x_{r})=\exp(-\|x_{s}-x_{r}\|^{2}/2\sigma^{2})\), was used. A stopping criterion (35) with \(\tau=10^{-3}\) and a cache size of 100 MB was set across all SMO algorithm implementations.