## 3 Sparse Logistic Regression (SPLR)
### Optimization of Algorithm Subsection  
The objective Function (5) is a convex function. Solving the coefficient \(w\) is a regularized convex optimization problem. The illustration for \(\ell 1\)-norm regularization is shown in Figure 3. Here, the elliptical contour of this method is shown through full curves and at the center ordinary logistic regression \(w\) estimates. However, the constraints area is the rotated square. The \(\ell 1\)-norm regularization  
Figure 2: Sparse logistic regression (SPLR) (Lasso) feature selection. The \(X_{i}\) are the feature sets. The \(w\) is sparse coefficient vector and white element in \(w\) the stand for zero elements (sparse data) and rest of all are selected feature.  
is the first place that the contour touches the square. In the last decade, regularized convex optimization has been intensely studied and led to many efficient results [26]. In short, these approaches can seem as a natural extension of the gradient technique. Here an objective function is to be minimized that leads to a non-smooth component. In this paper, the accelerated proximal gradient descent method is used, which reduces computational cost and leads to linear convergence [27]. Suppose a regularized convex optimization problem of (5), with a cost function \(l(w)\) and a regularization function \(\lambda g(w)\), the accelerated proximal gradient method solve this iteratively and every iteration, indexed by \(j+1\), it comprises of two key stages.  
Here, the first stage is a descent step for the function \(l(w)\), in order to accelerate the convergence, we start the first step from search point, and then the adaptive backtracking line search scheme is used to determine the step size. This involves starting with a relatively large estimate of step size with respect to the search direction and repeatedly shrinkage in the step size (backtracking) until reduce of object function is achieved. Which is an affine combination of \(w^{(j+1)}\) and \(w^{(j)}\).  
\[s^{j}=w^{j}+\alpha^{j}(w^{j}-w^{(j+1)}) \tag{6}\]  
\[u^{(j+1)}=s^{(j)}+t^{(j)}\
abla f(s^{(j)}) \tag{7}\]  
Here, \(\alpha^{j}\) is a tuning parameter. The approximate solution \(w^{(j+1)}\) will be measured as a gradient step. Now, adaptive backtracking line search [28] is used to select a proper step size \(t^{(j)}\).  
The second stage is to project \(u^{j}\) into regularized space while a proximal operator is applied, the proximal operator is defined as  
\[prox_{(\lambda)}g(u)=\text{argmin}_{(u)}(\frac{1}{2}|u-w|_{2}^{2}+\lambda g(u)) \tag{8}\]  
For \(\ell 1\)-regularization, an analytical solution for every variable \(w\) can be derived as,  
\[\begin{array}{ll}w^{(j+1)}&=prox_{\lambda}(u^{(j+1)})\\ &=\text{sgn}(u^{(j+1)})\text{max}(\left|u^{(j+1)}\right|-\lambda,0)\end{array} \tag{9}\]  
Repeatedly applying the accelerated gradient method and proximal operator the convergence of the algorithm leads to the optimum results. This technique is effective corresponding to the accelerated gradient descent and analytical solution of the proximal operator. A detailed summary of SPLR algorithm shown in Algorithm 1.  
Figure 3: Estimation picture for the \(\ell 1\)-norm regularization (Lasso).  
```
Input: Sparse function \(f(.)\) and sparse regularization function \(g(.)\) with regularization parameter \(\lambda\). Initialize: Step size \(t^{(0)}\) and affine combination parameter \(w^{(0)}\) Output: Optimum Result \(\hat{w}\) \(w\gets 0;j\gets 0\) \(j\gets j+1\);  Compute the search point \(s^{j}\) by Equation (6)  Compute the gradient descent point \(u^{(j+1)}\) by Equation (7) with adaptive step size.  Apply the proximal operator to compute \(w^{(j+1)}\) by Equation (8)  Update \(t^{(j+1)}\) and \(w^{(j+1)}\) for next iteration.  Repeat the above steps till the change between \(w^{(j+1)}\) and \(w^{(j)}\) is smaller than a threshold.  Return \(w^{(*)}=w^{(j+1)}\)
```  
**Algorithm 1** Pseudo code for SPLR (Lasso) Algorithm.