# Data-Driven Identification of Crane Dynamics Using Regularized Genetic Programming
## 2 Materials and Methods
### Regularization  
Once an individual \(j\) is generated by the evolutionary process, the semantics of the model terms are stored in a design matrix \(\phi_{j}\), and sparse regression is applied in order to select the appropriate model terms and obtain their coefficients. In this paper, the \(l_{0}\)-penalized linear least squares problem is used:  
\[L(\theta)=f(\theta)+g(\theta)=\frac{1}{2}\|y-\phi\theta\|_{2}^{2}+\lambda\| \theta\|_{0} \tag{5}\]  
where the vector is \(y\in R^{n}\), and the design matrix is \(\phi\in R^{n\times q}\). This problem has been proven to be NP-hard [39], and approximations to (5) have been based on greedy algorithms or relaxation. One commonly applied method of relaxation is basis pursuit denoising (BPDN), which replaces the \(l_{0}\) penalty with the convex \(l_{1}\) penalty. Other methods of solving (5) include sequential threshold least squares [40], which has been used to find sparse dynamical models from a library of candidate functions, and sparse relaxed regularized regression (SR3) [41]. The mAPG algorithm was developed in [35] to find a critical point of a function \(F(\theta)=f(\theta)+g(\theta)\), where it is a proper function with Lipschitz continuous gradients, \(g(\theta)\) is nonsmooth and can be either convex or nonconvex but is lower semicontinuous, and \(f(\theta)+g(\theta)\) is coercive. These assumptions are satisfied in (5) and, therefore, every accumulation point obtained by Algorithm 1 is a critical point of (5). A proof that the accumulation point of Algorithm 1 is a critical point of \(f(\theta)+g(\theta)\), satisfying the aforementioned assumptions, can be found in [35]. Monotone accelerated proximal gradient descent is an extension of APG developed in [42], which extends [PERSON]'s accelerated gradient descent to the nonsmooth case.  
```
Input:\(\phi,y,\lambda\) Initialize:\(\rho<1,\ \delta,\mathbf{z}_{1}=\theta_{1}=\theta_{0},\ t_{1}=1,\ \ t_{0}=0,\ k=1\) while not converged do \(k\gets k+1\) \(w_{k}=\theta_{k}+\frac{l_{k-1}}{l_{k}}(z_{k}-\theta_{k})+\frac{l_{k-1}-1}{l_ {k}}(\theta_{k}-\theta_{k-1})\)  Initialize step size \(\eta_{w}\) and \(\eta_{\theta}\) using Barzilai-Borwein method while\(F(z_{k+1})\geq F(w_{k})-\delta\|z_{k+1}-w_{k}\|_{2}^{2}\)do \(z_{k+1}=\text{prox}_{\eta_{w}\lambda\|\cdot\|_{0}}(w_{k}-\eta_{w}\
abla f(w_{ k}))\) \(\eta_{w}=\rho\eta_{w}\) endwhile while\(F(v_{k+1})\geq F(\theta_{k})-\delta\|v_{k+1}-\theta_{k}\|_{2}^{2}\)do \(v_{k+1}=\text{prox}_{\eta_{k}\lambda\|\cdot\|_{0}}(\theta_{k}-\eta_{\theta} \
abla f(\theta_{k}))\) \(\eta_{\theta}=\rho\eta_{\theta}\) endwhile \(t_{k+1}=\frac{\sqrt{4l_{k}^{2}+1}+1}{2}\) \(\theta_{k+1}=\begin{cases}z_{k+1}\ \ \text{if}\ F(z_{k+1})\leq F(v_{k+1})\\ v_{k+1}\ \ \text{otherwise}\end{cases}\) endwhile Output:\(\theta\)
```  
**Algorithm 1**mAPGAlgorithm 1 shows the implementation of mAPG. mAPG monitors the extrapolated variable \(w\) and corrects it when it has the possibility to fail while also ensuring convergence to a critical point of (5). In this work, we use a backtracking line search initialized with the Barzilai-Borwein method to find the initial step size \(\eta\). The proximal operator for the \(l_{0}\) pseudo-norm (Algorithm 1) is the hard thresholding operator (6). The flowchart of the proposed G3P-SR algorithm is shown in Figure 4.  
\[\text{prox}_{\gamma\|\cdot\|_{0}}v=\begin{cases}0&|v|\leq\sqrt{2\gamma}\\ v&|v|>\sqrt{2\gamma}\end{cases} \tag{6}\]