## 3 The SMO-like Algorithm for the Nonsmooth Dual Problem of \(\varepsilon\)-In SVR
### Working Set Selection
Moreover, as indicated in (20) and (21), the Incosh loss approaches Vapnik's and \(\varepsilon\)-insensitive \(l_{2}\) losses for the limit values of \(\eta_{1}\). For other values of \(\eta_{1}\), it continues to exhibit its inherent properties, including robustness to noise and outliers. So, it effectively addresses outliers induced by Cauchy noise while also handling small noises mostly exhibiting Gaussian distribution. Therefore, the \(\varepsilon\)-ln SVR has better test RMSE compared to \(\varepsilon\)-SVR and \(\varepsilon\)-\(l_{2}\) SVR, as shown in Table 1, thanks to its ability to be optimal for different noise distributions by tuning \(\eta_{1}\). While these results are an extension of the previous work [40], a computationally efficient SMO-like algorithm is provided to solve the nonsmooth nonlinear dual problem of \(\varepsilon\)-ln SVR in order to handle larger datasets. Furthermore, it has been observed that solving such a complex nonsmooth nonlinear optimization problem (14)

\[\min_{d_{j}}\hat{J}_{-}^{\prime}(0)d_{j}+\frac{\hat{J}_{-}^{\prime\prime}(0)}{ 2}d_{j}^{2}<-\frac{h_{ij}^{2}}{a_{ij}} \tag{39}\]0 Comparison of the proposed SMO algorithm for solving smooth (7) vs. nonsmooth (14) problems. Iteration and training time ratios are presented for (**a**) the optimal hyperparameters specified in Table 1 and (**b**) the hyperparameter selection procedure. \[\min_{d_{j}}\hat{J}_{-}^{\prime}(0)d_{j}+\frac{\hat{J}_{-}^{\prime\prime}(0)}{ 2}d_{j}^{2}<-\frac{h_{ij}^{2}}{a_{ij}} \tag{39}\]1 Comparison of the proposed SMO algorithm with the novel WSS procedure, which provides SO-like information, and the SMO with the traditional WSS procedure, which provides FO information. Iteration and training time ratios are presented for (**a**) the optimal hyperparameters specified in Table 1 and (**b**) the hyperparameter selection procedure. requires comparable training times as seen in Table 1 compared to solving the piecewise QP problems of \(\varepsilon\)-\(l_{2}\) SVR [21] and QP problems of \(\varepsilon\)-SVR [52]. \(\varepsilon\)-SVR and \(\varepsilon\)-\(l_{2}\) SVR have a well-defined analytical solution at each iteration for solving the subproblem consisting of two optimization parameters. Despite the lack of a direct analytical solution, the proposed SMO algorithm uses [PERSON]'s method at each iteration to address this complex problem and provides acceptable training times through its easily computable WSS procedure. Due to its fast nature and single execution per iteration as demonstrated in Algorithm 1, [PERSON]'s method contributes minimally to the overall running time of the algorithm. The SMO algorithm solves the optimization problem iteratively by updating only two Lagrangian multipliers at each step. The choice of the right pair, known as the working set selection procedure, is crucial as it significantly affects the convergence speed. Therefore, the proposed WSS procedure with SO-like information, which is designed to minimize an upper bound on the second-order Taylor polynomial approximation of consecutive loss function values, is compared with the traditional WSS procedure with FO information. It is observed that the proposed SMO with easily computable WSS procedure significantly improves both the number of iterations and the training times compared to its FO counterpart, as shown in \[\min_{d_{j}}\hat{J}_{-}^{\prime}(0)d_{j}+\frac{\hat{J}_{-}^{\prime\prime}(0)}{ 2}d_{j}^{2}<-\frac{h_{ij}^{2}}{a_{ij}} \tag{39}\]2 This is evident in all datasets, where both time and iteration ratios are consistently less than 1. The strength of the WSS procedure lies in its efficient retrieval of SO-like information. This allows it to prioritize pairs that maximize the decrease in consecutive loss function values. It achieves this by estimating an upper bound on the Taylor polynomial approximation, unlike the FO counterpart which simply uses gradient information. This often results in many more iterations for the FO counterpart because it lacks insight into the potential reduction in consecutive losses.