# An Efficient Algorithm for Ill-Conditioned Separable Nonlinear Least Squares

Authors: Jiayan Wang; Lanlan Guo; Zongmin Li; Xueqin Wang; Zhengqing Fu
Journal: Journal of Mathematics
 (2021)

---

## Abstract

For separable nonlinear least squares models, a variable projection algorithm based on matrix factorization is studied, and the ill-conditioning of the model parameters is considered in the specific solution process of the model. When the linear parameters are estimated, the Tikhonov regularization method is used to solve the ill-conditioned problems. When the nonlinear parameters are estimated, the QR decomposition, Gram–Schmidt orthogonalization decomposition, and SVD are applied in the Jacobian matrix. These methods are then compared with the method in which the variables are not separated. Numerical experiments are performed using RBF neural network data, and the experimental results are analyzed in terms of both qualitative and quantitative indicators. The results show that the proposed algorithms are effective and robust.


---

## Body

## 1. Introduction

The nonlinear least squares model has been applied to solve several practical problems in many subject areas. The separable nonlinear least squares (SNLLS) problem is a special type of nonlinear least squares problem. The corresponding estimation model can be expressed as a linear combination of nonlinear problems. Based on the special structure of this model, Golub and Pereyra proposed a variable projection (VP) algorithm to solve the model in 1973 [1]. This algorithm can eliminate the linear parameters, transform the problem into an estimation problem with only nonlinear parameters, and reduce the complexity of the model. The VP algorithm can reduce the dimensions of the parameter space and the number of iterations, thus improving the efficiency of finding the global optimal solution to the SNLLS problem.SNLLS models have a wide range of applications in engineering. Researchers have combined the VP algorithm with matrix decomposition or with rank recognition algorithms, which can be used to estimate different system models. Machine learning, neural networks and their many variants, and neural fuzzy systems [2] are all linear combinations of problems, where the problem is transformed into a nonlinear basis function. Prony converted a signal processing problem into a combination of complex exponential functions and used this method to analyze the frequency component of the signal. In time-series analyses, the smooth transition autoregressive model can be used as the separable least squares model. RBF-AR neural network models can be processed with linear combinations of exponential functions [3]. These examples are only part of the problem that can be solved using the SNLLS models.In recent years, many improvements have been made to the VP algorithm. Krogh presented a more efficient VP algorithm. Kaufman introduced a simplified Jacobian matrix calculation method in the VP algorithm, which improved the calculation efficiency [4]. Ruhe and Wedin analyzed the asymptotic convergence of the VP algorithm [5]. Gan proposed a new VP formula that can solve optimization problems in the absence of a derivative [6]. Most forms of the VP method require matrix decomposition. O’Leary and Rust proposed a robust singular value decomposition (SVD) method [7]. Ruano et al. proposed an error function based on QR classification [8]. Gan [9] and Wang [10] proposed a classical and improved Gram–Schmidt orthogonalization (GSO) method.Regarding the SNLLS problem, there are few studies on the potential ill-conditioning of the parameters to be estimated. Regularization is a commonly used method when ill-conditioned problems are solved. The use of regularization can make the regression coefficients have lower variance values, thereby reducing potential ill-conditioned problems [11, 12]. Commonly used regularization methods include the Tikhonov regularization method, the truncated singular value method, the kernel-based regularization method, and the norm-based regularization method [13–15].In this study, the Levenberg–Marquardt (LM) algorithm is used to estimate the nonlinear parameters, and the Jacobian matrix in the algorithm is in the form given by Kaufman. During the iteration process, the SVD, QR decomposition, and GSO decomposition are applied to the matrix to improve the iteration efficiency. When the linear parameters are estimated, the Tikhonov regularization method is used to solve the potential ill-conditioned problems. Combining the advantages of the two methods improves not only the calculation efficiency of the algorithm but also the accuracy of the results. With RBF neural network data, the method is verified through both qualitative and quantitative analyses.


## 2. Variable Projection Model and Its Parameter Estimation Method

### 2.1. Variable Projection Model

The SNLLS model can be expressed as follows:(1)yt=∑j=1mθL,jϕjθN;t,where ϕjθN;tj=1,2,…,m,t=1,2,…,n are nonlinear functions. The linear parameters θL=θL,1,θL,2,…,θL,a and nonlinear parameters θN=θN,1,θN,2,…,θN,b are the parameters that need to be estimated.The parameter estimatorsθ^L and θ^N can be obtained by solving the following formula:(2)θ^L,θ^N=minθ^L,θ^N∑t=1nyt−∑jmθLϕjθN;t2.The estimator can be expressed in a matrix form as(3)θ^L,θ^N=minθ^L,θ^Ny−ΦθNθL2,where ΦθN is composed of a column vector ϕjθN;t. The component of the vector y is yt. ⋅2 is the Euclidean norm of the vector.We first estimate the nonlinear parameters and then estimate the linear parameters using the following nonlinear least squares method:(4)θ^L=ΦθNTΦθN−1ΦθNTy=ΦθN+y.In the above formula,ΦθN+ is the pseudo-inverse of ΦθN.In this paper, the separable nonlinear model can be expressed as follows:(5)y=ΦθNθL=ΦθL,where Φ∈Rm×a, θL∈Ra×1, and y∈Rm×1.The LM iterative algorithm is used to estimate the nonlinear parameters. In the calculation process, the calculation of the Jacobian matrix is important. For a matrixFθNk,θLk=y−ΦθNθL, the Jacobian matrix is its first-derivative matrix.(6)JθNk=∂FθNk,θLk∂θNkT,PΦθNk⊥=I−ΦθNkΦθNk+.Thus, we have(7)FθNk,θLk=I−ΦθNΦθNk+y.In this model, the Jacobian matrix can be calculated using the method proposed by Kaufman [3].(8)JKAU=−PΦ⊥DΦΦ−y,where D is the Fréchet derivative of the map, Φ− is the symmetric pseudo-inverse of Φ that satisfies ΦΦ−Φ=Φ, and ΦΦ−T=ΦΦ−.(9)rθN=y−ΦθNkΦθNk+y2=I−PΦy2=PΦ⊥y2.


### 2.2. Nonlinear Parameter Estimation Method

To improve the calculation efficiency, the matrix is decomposed first, and then iterative calculations are performed with the decomposed matrix. The SVD, QR decomposition, and GSO decomposition are used.


#### 2.2.1. SVD

The specific decomposition form is as follows:(10)Φ=U1U2Σ000V1,V2T,where rankU1=rankV1=rankΦ=r and Σ is a diagonal matrix of order r. Through SVD, we can obtain(11)Φ−=V1,V2Σ−1000U1,U2T.The corresponding residual function and Jacobian matrix can be expressed as follows:(12)rSVDθN=PΦ⊥y2=U1U2000Im−rV1,V2Ty2=U2V2y2JSVD=−U2U2TDΦV1Σ−1U1Ty.The algorithm based on the SVD is denoted byVPSVD.


#### 2.2.2. GSO Decomposition

The specific decomposition form is as follows:(13)Φ=ST11T1200PT=S1,S2T11T1200P1,P2T,where S is an orthogonal matrix, P is a permutation matrix, and T11 is an upper triangular matrix. Here, ranks=m,rankP=n,and rankT11=r. Through GSO decomposition, we can obtain(14)Φ−=P1P2T11−1000S1,S2T.The corresponding residual function and Jacobian matrix can be expressed as follows:(15)rTθNA=PΦ⊥y2=S2S2Ty2,JGSO=−S2S2TDΦP1T11−1S1Ty.The algorithm based on the GSO decomposition is denoted byVPGSO.


#### 2.2.3. QR Decomposition

The specific decomposition form is as follows:(16)Φ=QR=Q1,Q2R10,where Q is an orthogonal matrix and R and R1 are upper triangular matrices. The corresponding residual function and Jacobian matrix can be expressed as follows:(17)rQR=Q2Q2Ty2,JQR=−Q2Q2TDΦR1−1Q1y.The algorithm based on the QR decomposition is denoted byVPQR.The following iterative methods can be used to estimate the nonlinear parameters:(18)θNk+1=θNk+βkdk,where βk is the step size, which can make the objective function y−ΦθNθ˜L2 decrease.dk is the search direction, which can be determined by solving the following equation:(19)JθNkTJθNk+γkIdk=−JθNkTFθNk,θ˜Lk,where γk is the damping factor.In the LM algorithm,βk can be calculated using the formula(20)rθk+ρmkdk≤rθk+ρmkgkTdk.mk in (20) is the nonnegative minimum integer satisfying equation (20). Here, ρ=0.5, gk=Jθkrθk, and βk=ρmk.


### 2.3. Linear Parameter Estimation Method

The SVD of matrixΦ can be expressed as(21)Φ=USVT=∑i=1luiσiviT,where U and V are the unitary matrices composed of column vectors.S=sij∈Ra×b, sij=σii=j0i≠j, σ1≥σ2≥⋯≥σl>0,σl+1=σl+2=⋯=σb=0, and σii=1l is the singular value of Φ. Solving equation (21) using the SVD method yields(22)θ^L=∑i=1luiTyσivi.If the singular value decreases and reaches zero, then (21) may be an ill-conditioned problem. If the singular value cannot be zero, when σ1/σl is very high and the condition number of Φ is also high, then (21) may be an ill-conditioned problem.With the addition of filtering factors, the ill-conditioning is controlled to obtain ideal parameter estimates. After adding the filter factor, the SVD can be expressed as follows:(23)θ^L=∑i=1lfiuiTyσivi,where the filter factor is fi=σi2/σi2+σ12 and θ^L is obtained using the Tikhonov method. Equation (27) can be expressed as follows:(24)θ^L=∑i=1lσi2σi2+σ12uiTyσivi.By combining the linear parameter and nonlinear parameter estimation methods, three algorithms can be obtained. The LM algorithm is used to estimate the nonlinear parameters. During the iterative process, the Jacobian matrix takes three forms—JSVD, JGSO, and JQR—for the SVD, GSO, and QR methods, respectively. The least squares method is used to estimate the linear parameters. During the iterative process, the Tikhonov regularization method is used to control the ill-conditioning. After the combination, the following estimation method can be obtained:(25)θ^L,θ^N=argminθ^Ny−ΦθNθ^L22,θ^L=∑i=1lσi2σi2+σ12uiTyσivi.The specific calculation process of the algorithm is as follows:(1)
Set the initial values of the nonlinear parameterθN0, control error ε, and maximum number of iteration steps kmax(2)
Use equation (21) to calculate the initial value of the linear parameter θL0(3)
Calculate the linear and nonlinear parameters sequentially and iteratively(4)
Repeat the third step until it meets the conditionθNk+1−θNk<ε or reaches the maximum number of iterations kmax


## 2.1. Variable Projection Model

The SNLLS model can be expressed as follows:(1)yt=∑j=1mθL,jϕjθN;t,where ϕjθN;tj=1,2,…,m,t=1,2,…,n are nonlinear functions. The linear parameters θL=θL,1,θL,2,…,θL,a and nonlinear parameters θN=θN,1,θN,2,…,θN,b are the parameters that need to be estimated.The parameter estimatorsθ^L and θ^N can be obtained by solving the following formula:(2)θ^L,θ^N=minθ^L,θ^N∑t=1nyt−∑jmθLϕjθN;t2.The estimator can be expressed in a matrix form as(3)θ^L,θ^N=minθ^L,θ^Ny−ΦθNθL2,where ΦθN is composed of a column vector ϕjθN;t. The component of the vector y is yt. ⋅2 is the Euclidean norm of the vector.We first estimate the nonlinear parameters and then estimate the linear parameters using the following nonlinear least squares method:(4)θ^L=ΦθNTΦθN−1ΦθNTy=ΦθN+y.In the above formula,ΦθN+ is the pseudo-inverse of ΦθN.In this paper, the separable nonlinear model can be expressed as follows:(5)y=ΦθNθL=ΦθL,where Φ∈Rm×a, θL∈Ra×1, and y∈Rm×1.The LM iterative algorithm is used to estimate the nonlinear parameters. In the calculation process, the calculation of the Jacobian matrix is important. For a matrixFθNk,θLk=y−ΦθNθL, the Jacobian matrix is its first-derivative matrix.(6)JθNk=∂FθNk,θLk∂θNkT,PΦθNk⊥=I−ΦθNkΦθNk+.Thus, we have(7)FθNk,θLk=I−ΦθNΦθNk+y.In this model, the Jacobian matrix can be calculated using the method proposed by Kaufman [3].(8)JKAU=−PΦ⊥DΦΦ−y,where D is the Fréchet derivative of the map, Φ− is the symmetric pseudo-inverse of Φ that satisfies ΦΦ−Φ=Φ, and ΦΦ−T=ΦΦ−.(9)rθN=y−ΦθNkΦθNk+y2=I−PΦy2=PΦ⊥y2.


## 2.2. Nonlinear Parameter Estimation Method

To improve the calculation efficiency, the matrix is decomposed first, and then iterative calculations are performed with the decomposed matrix. The SVD, QR decomposition, and GSO decomposition are used.


### 2.2.1. SVD

The specific decomposition form is as follows:(10)Φ=U1U2Σ000V1,V2T,where rankU1=rankV1=rankΦ=r and Σ is a diagonal matrix of order r. Through SVD, we can obtain(11)Φ−=V1,V2Σ−1000U1,U2T.The corresponding residual function and Jacobian matrix can be expressed as follows:(12)rSVDθN=PΦ⊥y2=U1U2000Im−rV1,V2Ty2=U2V2y2JSVD=−U2U2TDΦV1Σ−1U1Ty.The algorithm based on the SVD is denoted byVPSVD.


### 2.2.2. GSO Decomposition

The specific decomposition form is as follows:(13)Φ=ST11T1200PT=S1,S2T11T1200P1,P2T,where S is an orthogonal matrix, P is a permutation matrix, and T11 is an upper triangular matrix. Here, ranks=m,rankP=n,and rankT11=r. Through GSO decomposition, we can obtain(14)Φ−=P1P2T11−1000S1,S2T.The corresponding residual function and Jacobian matrix can be expressed as follows:(15)rTθNA=PΦ⊥y2=S2S2Ty2,JGSO=−S2S2TDΦP1T11−1S1Ty.The algorithm based on the GSO decomposition is denoted byVPGSO.


### 2.2.3. QR Decomposition

The specific decomposition form is as follows:(16)Φ=QR=Q1,Q2R10,where Q is an orthogonal matrix and R and R1 are upper triangular matrices. The corresponding residual function and Jacobian matrix can be expressed as follows:(17)rQR=Q2Q2Ty2,JQR=−Q2Q2TDΦR1−1Q1y.The algorithm based on the QR decomposition is denoted byVPQR.The following iterative methods can be used to estimate the nonlinear parameters:(18)θNk+1=θNk+βkdk,where βk is the step size, which can make the objective function y−ΦθNθ˜L2 decrease.dk is the search direction, which can be determined by solving the following equation:(19)JθNkTJθNk+γkIdk=−JθNkTFθNk,θ˜Lk,where γk is the damping factor.In the LM algorithm,βk can be calculated using the formula(20)rθk+ρmkdk≤rθk+ρmkgkTdk.mk in (20) is the nonnegative minimum integer satisfying equation (20). Here, ρ=0.5, gk=Jθkrθk, and βk=ρmk.


## 2.2.1. SVD

The specific decomposition form is as follows:(10)Φ=U1U2Σ000V1,V2T,where rankU1=rankV1=rankΦ=r and Σ is a diagonal matrix of order r. Through SVD, we can obtain(11)Φ−=V1,V2Σ−1000U1,U2T.The corresponding residual function and Jacobian matrix can be expressed as follows:(12)rSVDθN=PΦ⊥y2=U1U2000Im−rV1,V2Ty2=U2V2y2JSVD=−U2U2TDΦV1Σ−1U1Ty.The algorithm based on the SVD is denoted byVPSVD.


## 2.2.2. GSO Decomposition

The specific decomposition form is as follows:(13)Φ=ST11T1200PT=S1,S2T11T1200P1,P2T,where S is an orthogonal matrix, P is a permutation matrix, and T11 is an upper triangular matrix. Here, ranks=m,rankP=n,and rankT11=r. Through GSO decomposition, we can obtain(14)Φ−=P1P2T11−1000S1,S2T.The corresponding residual function and Jacobian matrix can be expressed as follows:(15)rTθNA=PΦ⊥y2=S2S2Ty2,JGSO=−S2S2TDΦP1T11−1S1Ty.The algorithm based on the GSO decomposition is denoted byVPGSO.


## 2.2.3. QR Decomposition

The specific decomposition form is as follows:(16)Φ=QR=Q1,Q2R10,where Q is an orthogonal matrix and R and R1 are upper triangular matrices. The corresponding residual function and Jacobian matrix can be expressed as follows:(17)rQR=Q2Q2Ty2,JQR=−Q2Q2TDΦR1−1Q1y.The algorithm based on the QR decomposition is denoted byVPQR.The following iterative methods can be used to estimate the nonlinear parameters:(18)θNk+1=θNk+βkdk,where βk is the step size, which can make the objective function y−ΦθNθ˜L2 decrease.dk is the search direction, which can be determined by solving the following equation:(19)JθNkTJθNk+γkIdk=−JθNkTFθNk,θ˜Lk,where γk is the damping factor.In the LM algorithm,βk can be calculated using the formula(20)rθk+ρmkdk≤rθk+ρmkgkTdk.mk in (20) is the nonnegative minimum integer satisfying equation (20). Here, ρ=0.5, gk=Jθkrθk, and βk=ρmk.


## 2.3. Linear Parameter Estimation Method

The SVD of matrixΦ can be expressed as(21)Φ=USVT=∑i=1luiσiviT,where U and V are the unitary matrices composed of column vectors.S=sij∈Ra×b, sij=σii=j0i≠j, σ1≥σ2≥⋯≥σl>0,σl+1=σl+2=⋯=σb=0, and σii=1l is the singular value of Φ. Solving equation (21) using the SVD method yields(22)θ^L=∑i=1luiTyσivi.If the singular value decreases and reaches zero, then (21) may be an ill-conditioned problem. If the singular value cannot be zero, when σ1/σl is very high and the condition number of Φ is also high, then (21) may be an ill-conditioned problem.With the addition of filtering factors, the ill-conditioning is controlled to obtain ideal parameter estimates. After adding the filter factor, the SVD can be expressed as follows:(23)θ^L=∑i=1lfiuiTyσivi,where the filter factor is fi=σi2/σi2+σ12 and θ^L is obtained using the Tikhonov method. Equation (27) can be expressed as follows:(24)θ^L=∑i=1lσi2σi2+σ12uiTyσivi.By combining the linear parameter and nonlinear parameter estimation methods, three algorithms can be obtained. The LM algorithm is used to estimate the nonlinear parameters. During the iterative process, the Jacobian matrix takes three forms—JSVD, JGSO, and JQR—for the SVD, GSO, and QR methods, respectively. The least squares method is used to estimate the linear parameters. During the iterative process, the Tikhonov regularization method is used to control the ill-conditioning. After the combination, the following estimation method can be obtained:(25)θ^L,θ^N=argminθ^Ny−ΦθNθ^L22,θ^L=∑i=1lσi2σi2+σ12uiTyσivi.The specific calculation process of the algorithm is as follows:(1)
Set the initial values of the nonlinear parameterθN0, control error ε, and maximum number of iteration steps kmax(2)
Use equation (21) to calculate the initial value of the linear parameter θL0(3)
Calculate the linear and nonlinear parameters sequentially and iteratively(4)
Repeat the third step until it meets the conditionθNk+1−θNk<ε or reaches the maximum number of iterations kmax


## 3. Numerical Experiments

The experiments were conducted on a PC with a 2.30 GHz processor and 4 GB of memory, running on Windows 10. Software used was MATLAB 2016b. The experiment was conducted to verify the three algorithms proposed in this paper.In this experiment, the Mackey–Glass differential equation was used to generate a sequence of points.(26)dytdt=ayt−τ1+yct−τ−byt,where the values of the parameter are a=0.2,b=0.1,c=10,andτ=10. The fourth-order Runge–Kutta method was used to intercept 500 points to test the SNLLS model. Figure 1 shows the specific point chart. The horizontal and vertical axes are the value points of x and y, respectively.Figure 1
Mackey–Glass time series.First, the first 200 points were used to learn and train the model and estimate the linear and nonlinear parameters. Subsequently, the remaining points were used to verify the identified model.TheRBF−ARm,d model is a linear combination of Gaussian RBFs, expressed as(27)yt=θL,1+∑k=1mθL,k+1exp−λktt−zk2+ξt=∑j=1m+1θL,jϕj−1θN;tt+ξt,where m,d∈Z+ is the model number, λk is the model scaling parameters, zk, k=0,1,…,m is the center of the model, θL represents the linear parameters, and θN represents the nonlinear parameters. The number of linear parameters in the RBF-AR model is often greater than the number of nonlinear parameters. The parameter estimation using the VP theory is suitable. The scaling parameter λk can be calculated using the following formula:(28)λk=−logεmaxtt−zk2,0.0001≤ε≤0.1.When the model is solved, the following four methods are used: parameter nonseparation method, VPQR, VPGSO, and VPSVD. Figure 2 shows the curves of the training and prediction points obtained using different methods. The first half of the curve represents the parameter training curve, containing 200 points. The parameters were estimated through these points, and a neural network model was established. The latter half of the curve represents the prediction curve obtained using the estimated model.Figure 2
Curves of training and prediction points: (a) not separated, (b) QR, (c) GSO, and (d) SVD.
(a)(b)(c)(d)Figure2 shows that the curves obtained using the four methods are consistent with the Mackey–Glass time-series curve. This shows that the four models are accurate and that they can predict the curve well.The residual error and RMSE values of the four methods are low. The calculation formula of RMSE is shown in the following:(29)RMSE=∑i=1nθNo,i−θNe,i2n,where θno,i and θne,i are the observed value and estimated value.Table1 lists the quantitative indicators.Table 1
Quantitative indicators.
Not separatedVPQRVPSVDVPGSOResiduals of training points0.13401.5349 × 10−80.02600.0421RMSE of training points0.41776.1945 × 10−50.08060.1025Residuals of predicted points0.28315.8095 × 10−80.09360.3554RMSE of predicted points0.59491.2051 × 10−40.15300.2981Iteration time (s)17368719542528As listed in Table1, this shows that they yield accurate estimation parameters and prediction curves. The residual and RMSE of the three proposed methods are lower than those of the method without matrix decomposition. The proposed methods have higher accuracy. Therefore, the three methods proposed in this article are compared in the following part.Figure3 shows the RMSE curves of the multistep prediction of the three methods. The horizontal axis is the number of prediction points. The vertical axis is the RMSE value.Figure 3
RMSE values of the multistep prediction of three methods: (a) QR, (b) GSO, and (c) SVD.
(a)(b)(c)As shown, with the increase in the number of prediction points, the three methods have a higher error growth rate in the early stages and a lower error growth rate in the later stages. This shows that the proposed methods can accurately estimate the model parameters. The RMSE is the lowest for VPQR and highest for VPSVD; however, the difference is small, indicating the reliability of the parameters obtained by using the three methods.With a random initial point, the experiment was repeated 30 times, and the box plots of the RMSE were obtained, as shown in Figure4. The vertical axis is the RMSE value. From left to right are the box plots for the VPSVD, VPGSO, and VPQR methods.Figure 4
Box plots of the RMSE for three methods.The residuals of the prediction points of the three methods are relatively stable. From the graph, it is found that the prediction point residuals obtained by using the VPQR method are the most stable.The LM algorithm was used to estimate the nonlinear parameters. Figure5 shows the iterative convergence curves of the nonlinear parameters of the three methods.Figure 5
Iterative convergence curves: (a) VPQR, (b) VPGSO, and (c) VPSVD.
(a)(b)(c)From the iterative convergence of the nonlinear parameters, it is found that, after multiple iterations, the nonlinear parameters yield stable estimates. The three matrix decompositions are all orthogonal decompositions. When solving the model, the three methods require different numbers of iterations. The VPGSO method requires fewer iterations to reach a stable estimate. When using the LM algorithm to estimate the nonlinear parameters, the total residual change after each iteration of the parameter is calculated. The rise or fall of the curve during the iteration process does not mean that the algorithm is unstable. All the three parameter estimation methods stably converge to the optimal estimate.


## 4. Conclusion

This paper reports a variable projection algorithm to solve SNLLS problems. In the solution, the Jacobian matrix was decomposed to improve the iteration efficiency and stability of the algorithm. When solving the linear parameters, the Tikhonov regularization method was used to control the ill-conditioning of the model parameters. The RBF neural network data were used for the experiments, and the algorithm was verified through qualitative and quantitative indicators. The experimental results were compared with those obtained by using the variable projection-based matrix nondecomposition method. The experimental results showed that the proposed methods are more efficient than the matrix nondecomposition method. The three proposed methods have their own advantages. The VPQR algorithm is the best in terms of the accuracy, the VPGSO algorithm has the highest computational efficiency, and the VPSVD algorithm is the best in terms of the prediction point residual stability.



---

*Source: 7625175-2021-11-10.xml*