# Calibration for Parameter Estimation of Signals with Complex Noise via Nonstationarity Measure

Authors: Zhiming Zhou; Zhengyun Zhou; Liang Wu
Journal: Complexity
 (2021)

---

## Abstract

The signals in numerous complex systems of engineering can be regarded as nonlinear parameter trend with noise which is identically distributed random signals or deterministic stationary chaotic signals. The commonly used methods for parameter estimation of nonlinear trend in signals are mainly based on least squares. It can cause inaccurate estimation results when the noise is complex (such as non-Gaussian noise, strong noise, and chaotic noise). This paper proposes a calibration method for this issue in the case of single parameter via nonstationarity measure from the perspective of the stationarity of residual sequence. Some numerical studies are conducted for validation. Results of numerical studies show that the proposed calibration method performs well for various models with different noise strengths and types (including random noise and chaotic noise) and can significantly improve the accuracy of initial estimates obtained by least squares method. This is the first time that the nonstationarity measure is applied to the parameter calibration. All these results will be a guide for future studies of other parameter calibrations.


---

## Body

## 1. Introduction

The signals in numerous complex systems of engineering can be regarded as nonlinear parameter trend with noise such as the multiexponential model for nuclear magnetic resonance (NMR) signal [1], the optical-sensing membership function model [2], the autoregressive exogenous (ARX) model [3], the separable nonlinear function models [4], and the hydrodynamic model for water level and water flow [5]. The noise can be identically distributed random signals or deterministic stationary chaotic signals [6–10]. The parameter estimation (or fitting) of nonlinear parameter trend with noise has been an important topic for signal analysis in complex systems of engineering. There are some classical methods and their improvements for this topic such as Gauss–Newton method and Levenberg–Marquardt method [11–13]. Most of these methods are based on least squares. But if the nonlinear trend has complex noise such as strong noise, non-Gaussian noise (e.g., power-law and Cauchy noise) and chaotic noise, there are no effective methods for its parameter estimation. It is rather difficult for parameter estimation in this case. For non-Gaussian noise, due to the heavy tails of power-law and Cauchy distribution, the variance of noise does not exist. This causes that the sum of squared residuals (the optimization objective of least squares) is always large and not sensitive to the search of parameter. It can also suffer from the same problem when the noise is strong. It is well known that some chaotic dynamical system exhibit stochastic behaviors, which can be regarded as chaotic noise [14]. For chaotic noise, due to deterministic inherent quantitative relations in noise, the dependence in noise is strong, which can pollute the function signal and may produce spurious trend. These problems eventually lead to inaccurate estimation results obtained by least-squares-based methods.The noise considered in this paper is stationary, which means that the joint probability distribution of noise does not change over time. Otherwise, it is nonstationary. For signals with stationary noise, an accurate estimate makes its residual sequence more like noise and be stationary. From this perspective, the stationarity of estimated residual sequence can be used for the calibration of inaccurate estimate in the case of complex non-Gaussian noise. To measure the stationarity of whole sequence, Ding et al. [15–17] proposed the nonstationarity measure (NS), which is constructed based on ergodic theory and Shannon entropy, and is sensitive to nonstationary signals. The values of NS are in 0,1. The smaller the NS value of sequence, the more stationary the sequence. Lan et al. [18] used nonstationarity measure to select irrigation water consumption model. Tan and Ding [19] analysed the lottery data using nonstationarity measure. Wu [17] analysed the Stock Returns using nonstationarity measure. Tan et al. [20] studied the decomposition of noise and trend via nonstationarity measure. Liu et al. [21] obtained an objective estimate of protein dynamic states with single molecule fluorescence data via nonstationarity measure. Zhao [22] conducted a nonstationarity analysis of high frequency water level data based on nonstationarity measure.Due to NS’s sensitivity to nonstationary signals, we propose a calibration method to achieve accurate estimation results via NS from the perspective of the stationarity of residual sequence. This paper considers single parameter in order to introduce our calibration method conveniently at the first time. After obtaining the least square estimate, the proposed calibration method increases or decreases the obtained estimate to decrease NS value of residual sequence and stops when the NS value satisfies a given value condition. Various models with different noise strengths and types are simulated and the simulated data are used to validate the proposed calibration method. Results of validation show that the proposed calibration method performs well for all the models and can significantly improve the accuracy of initial estimates obtained by least squares method. It can not only solve the problem of large sum of squared residuals but also deal with strong dependent deterministic noise, which both cause inaccurate estimation results. The nonstationarity measure is first applied to parameter calibration in this study. All these results will be a guide for future studies of other parameter calibrations.The remaining parts of the paper are organized as follows: the methods are introduced in Section2. The numerical results are reported and discussed in Sections 3 and 4. Our work is concluded in Section 5.


## 2. Methodology

### 2.1. Parameter Estimation for Single Parameter Trend in Signal

Letyt be the observed signal which can be expressed as follows:(1)yt=ft;θ+εt,where ft;θ is a nonlinear function of t depending on parameter θ and denotes the trend in signal; εt is the noise signal. Here, the single parameter θ is considered in order to introduce our calibration method conveniently at the first time.It is important to estimate the parameterθ of signal with single parameter trend. The most classical method for the estimation of θ is Gauss–Newton method. This method linearizes the nonlinear function ft;θ via the first order term of its Taylor series to use linear least squares. The algorithm can be described as follows.LetY=yt1,…,ytnT be the observed signal series. For given an initial value θ0, the ft;θ can be approximately expressed in the following linear form via Taylor expansion in the neighborhood of θ0,(2)ft;θ≈ft;θ0+f′t;θ0Δθ,θ=θ0+Δθ,f′t;θ0=∂ft;θ∂θ|θ=θ0.The approximate linear form is further used to obtain the input of next iteration via linear least squares, that is to replaceft;θ in the sum of squared residuals 1/2∑i=1nfti;θ−yti2, and minimize the following approximate form with respect to Δθ,(3)R=12∑i=1nfti;θ0+f′ti;θ0Δθ−yti2.LetE=yt1−ft1;θ0,…,ytn−ftn;θ0T and X=f′t1;θ0,…,f′tn;θ0T. The minimiser Δθmin=XTX−1XTE. Thus, the input of next iteration θ1=θ0+Δθmin.Repeat the abovementioned steps until convergence. Letθk be the obtained value in the k step. The limit of θk is the estimate of θ. We denote it by θ^.This estimation method is mainly based on the least squares and linear approximation. When the noise signal is large or non-Gaussian (heavy tails), the sum of squared residuals is always large. The estimate obtained by similar least square methods is inaccurate in these cases. Thus, after obtaining the least square estimate, the parameter calibration is considered from another perspective via the stationarity of estimated residual series.


### 2.2. Model Selection Based on Residual Sequence

Model selection is the task of selecting a model from a set of candidate models after giving data. In its basic forms, model selection is one of the fundamental tasks of scientific inquiry. Usually, the data points are collected under random noise, and the model we are interested in is as follows:(4)y=ft+ε,ε∼i.i.d.where f stands for the relationship between T and Y, and ε is the random noise. i.i.d. means independent and identically distributed. The residual sequence for the ith observation in the dataset is(5)ε^i=yi−f^ti,where f^ is the estimate of f. If we rewrite the residual sequence as(6)ε^i=yi−f^ti=yi−fti+fti−f^ti,ε^=ε^i can be seen as an estimation of the i.i.d., random noise ε=εi [23]. If the model fits to the data correctly, the residual sequence would be closed to the random noise. If the residual sequence appears to behave randomly, it suggests that the model fits the data well. On the other hand, if nonrandom structure trend is evident in the residual sequence, it is a clear sign that the model fits the data poorly. In fact, any trend in the residual sequence indicates the deviations between f^t and ft.To check if a fitf^t is good or not, the nonstationary measure (NS) can be computed from the perspective of the stationarity to check if the residual sequence behaves randomly. The nonstationarity measure (NS) is first introduced by Ding et al. [15, 16] and constructed based on ergodic theory and Shannon entropy. It is used to measure the stationarity of sequence and is sensitive to nonstationary signals. The values of NS are in 0,1. The smaller the NS value of sequence, the more stationary the sequence will be.Thus, NS can be regarded as an index to illustrate the deviation of the estimated modelf^t and true model ft. If NS is not small, the deviation between ft and f^t cannot be neglected; the estimation should be improved. In the single parameter estimation model, the model is characterized by the parameter θ; to select good model is equivalent to find the best parameter. The deviation between the estimate θ^ and true parameter θ will be inflected by the NS value of the residual sequence ε^i. If NSε^i is larger than a prescribed threshold c, the θ^ is not closed to θ because such a deviation of θ and θ^ induces an appeared trend ft,θ−ft,θ^ in the noise ε. The estimated parameter θ^ must be calibrated to reduce the effect of the trend ft,θ−ft,θ^ till NSε^i≤c.


### 2.3. Parameter Calibration via Nonstationarity Measure

Based on the perspective of model selection via residual sequence, we introduce a calibration method for the inaccurate least square estimate under complex noise via nonstationarity measure.Due to its sensitivity to nonstationary signals, the NS can be used for the calibration of least squares estimate via measuring the nonstationarity of estimated residual sequence. Based on previous studies [15, 16, 20], the values of 0.05 can be used as the threshold to determine whether there is any trend in the estimated residual sequence. If the NS value of estimated residual sequence is less than 0.05, it indicates there is no trend in the residual sequence and the corresponding estimate of θ is accuracy.Letε^tθ^=yt−ft;θ^ be the estimated residual sequence with estimate θ^. NSθ^ denotes the NS value of ε^tθ^. The calibration procedure is described as follows:(i)
Obtain the estimateθ^ of noise-bearing model via Gauss–Newton method as the initial value for parameter calibration. Redenote it by θ^0.(ii)
Compute the NS valueNSθ^0 (greater than 0.05 in general).(iii)
Increase or decrease the value ofθ^0 by a fixed small size (set 0.01) along the overall direction of decreasing NS value.(iv)
Letθ^k be the obtained value in the k step. Repeat the previous step until the following case happens: NSθ^k≤0.05,NSθ^k+1≤0.05,…,NSθ^k+i−1≤0.05 and NSθ^k+i>0.05.(v)
The obtained corrected estimateθ′=1/iθ^k+θ^k+1+⋯+θ^k+i−1.


## 2.1. Parameter Estimation for Single Parameter Trend in Signal

Letyt be the observed signal which can be expressed as follows:(1)yt=ft;θ+εt,where ft;θ is a nonlinear function of t depending on parameter θ and denotes the trend in signal; εt is the noise signal. Here, the single parameter θ is considered in order to introduce our calibration method conveniently at the first time.It is important to estimate the parameterθ of signal with single parameter trend. The most classical method for the estimation of θ is Gauss–Newton method. This method linearizes the nonlinear function ft;θ via the first order term of its Taylor series to use linear least squares. The algorithm can be described as follows.LetY=yt1,…,ytnT be the observed signal series. For given an initial value θ0, the ft;θ can be approximately expressed in the following linear form via Taylor expansion in the neighborhood of θ0,(2)ft;θ≈ft;θ0+f′t;θ0Δθ,θ=θ0+Δθ,f′t;θ0=∂ft;θ∂θ|θ=θ0.The approximate linear form is further used to obtain the input of next iteration via linear least squares, that is to replaceft;θ in the sum of squared residuals 1/2∑i=1nfti;θ−yti2, and minimize the following approximate form with respect to Δθ,(3)R=12∑i=1nfti;θ0+f′ti;θ0Δθ−yti2.LetE=yt1−ft1;θ0,…,ytn−ftn;θ0T and X=f′t1;θ0,…,f′tn;θ0T. The minimiser Δθmin=XTX−1XTE. Thus, the input of next iteration θ1=θ0+Δθmin.Repeat the abovementioned steps until convergence. Letθk be the obtained value in the k step. The limit of θk is the estimate of θ. We denote it by θ^.This estimation method is mainly based on the least squares and linear approximation. When the noise signal is large or non-Gaussian (heavy tails), the sum of squared residuals is always large. The estimate obtained by similar least square methods is inaccurate in these cases. Thus, after obtaining the least square estimate, the parameter calibration is considered from another perspective via the stationarity of estimated residual series.


## 2.2. Model Selection Based on Residual Sequence

Model selection is the task of selecting a model from a set of candidate models after giving data. In its basic forms, model selection is one of the fundamental tasks of scientific inquiry. Usually, the data points are collected under random noise, and the model we are interested in is as follows:(4)y=ft+ε,ε∼i.i.d.where f stands for the relationship between T and Y, and ε is the random noise. i.i.d. means independent and identically distributed. The residual sequence for the ith observation in the dataset is(5)ε^i=yi−f^ti,where f^ is the estimate of f. If we rewrite the residual sequence as(6)ε^i=yi−f^ti=yi−fti+fti−f^ti,ε^=ε^i can be seen as an estimation of the i.i.d., random noise ε=εi [23]. If the model fits to the data correctly, the residual sequence would be closed to the random noise. If the residual sequence appears to behave randomly, it suggests that the model fits the data well. On the other hand, if nonrandom structure trend is evident in the residual sequence, it is a clear sign that the model fits the data poorly. In fact, any trend in the residual sequence indicates the deviations between f^t and ft.To check if a fitf^t is good or not, the nonstationary measure (NS) can be computed from the perspective of the stationarity to check if the residual sequence behaves randomly. The nonstationarity measure (NS) is first introduced by Ding et al. [15, 16] and constructed based on ergodic theory and Shannon entropy. It is used to measure the stationarity of sequence and is sensitive to nonstationary signals. The values of NS are in 0,1. The smaller the NS value of sequence, the more stationary the sequence will be.Thus, NS can be regarded as an index to illustrate the deviation of the estimated modelf^t and true model ft. If NS is not small, the deviation between ft and f^t cannot be neglected; the estimation should be improved. In the single parameter estimation model, the model is characterized by the parameter θ; to select good model is equivalent to find the best parameter. The deviation between the estimate θ^ and true parameter θ will be inflected by the NS value of the residual sequence ε^i. If NSε^i is larger than a prescribed threshold c, the θ^ is not closed to θ because such a deviation of θ and θ^ induces an appeared trend ft,θ−ft,θ^ in the noise ε. The estimated parameter θ^ must be calibrated to reduce the effect of the trend ft,θ−ft,θ^ till NSε^i≤c.


## 2.3. Parameter Calibration via Nonstationarity Measure

Based on the perspective of model selection via residual sequence, we introduce a calibration method for the inaccurate least square estimate under complex noise via nonstationarity measure.Due to its sensitivity to nonstationary signals, the NS can be used for the calibration of least squares estimate via measuring the nonstationarity of estimated residual sequence. Based on previous studies [15, 16, 20], the values of 0.05 can be used as the threshold to determine whether there is any trend in the estimated residual sequence. If the NS value of estimated residual sequence is less than 0.05, it indicates there is no trend in the residual sequence and the corresponding estimate of θ is accuracy.Letε^tθ^=yt−ft;θ^ be the estimated residual sequence with estimate θ^. NSθ^ denotes the NS value of ε^tθ^. The calibration procedure is described as follows:(i)
Obtain the estimateθ^ of noise-bearing model via Gauss–Newton method as the initial value for parameter calibration. Redenote it by θ^0.(ii)
Compute the NS valueNSθ^0 (greater than 0.05 in general).(iii)
Increase or decrease the value ofθ^0 by a fixed small size (set 0.01) along the overall direction of decreasing NS value.(iv)
Letθ^k be the obtained value in the k step. Repeat the previous step until the following case happens: NSθ^k≤0.05,NSθ^k+1≤0.05,…,NSθ^k+i−1≤0.05 and NSθ^k+i>0.05.(v)
The obtained corrected estimateθ′=1/iθ^k+θ^k+1+⋯+θ^k+i−1.


## 3. Numerical Analysis with Random Noise

This section focuses on the numerical study of the proposed calibration procedure in Section2.3 in the case of random noise. The signals for numerical study are generated according to equation (1). If not specified in this section, the noise added to signals is independent and identically distributed standard Cauchy variables, whose probability density function is(7)px;x0,γ=1πγ1+x−x0/γ2=1πγx−x02+γ2,with γ=1 and x0=0.The signal-to-noise ratio (SNR) is used to measure the signal and noise strength in this paper and is defined as follows:(8)SNR=10lg∑i=1nfti;θ2∑i=1nεti2.The relative error (RE) is defined as follows for measuring the accuracy of estimation and calibration:(9)REθ^=θ−θ^θand REθ′=θ−θ′θ.


### 3.1. Performance of Calibration

The following three models offt;θ are adopted to validate the performance of proposed calibration:(10)Model1:ft;θ=θt3−1+12θsinθsint,Model2:ft;θ=logt−2θ+0.01⋅θt2,Model3:ft;θ=−θ−θ2⋅t+θ22⋅t2−θ3⋅sint.Model 1 is not smooth, it admits a fold point att=1. Model 2 has a log singularity. Model 3 has a complex trend. They can represent typical situations in applications. For Model 1 and Model 2, the signals are generated at t every 0.1 from −3 to 3. For Model 3, the signals are generated at t every 0.1 from −10 to 10. For further analysing the effects of parameter values on calibration, we choose four values of θ in Model 3 for validation. One generated signal is shown in Figure 1. Results of calibration for the three models are shown in Table 1. All the results are obtained via the average of 200 generated signals.Figure 1
Generated signals for different models. The curves denoteft;θ. The circles denote yt. (a) Model 1, θ=2. (b) Model 2, θ = 2. (c) Model 3, θ = 0.2. (d) Model 3, θ = 0.5. (e) Model 3, θ = 0.7. (f) Model 3, θ = 1.
(a)(b)(c)(d)(e)(f)Table 1
Calibration performance for different models.
θSNRθ^REθ^ (%)NSθ^θ′REθ^ (%)Model 12−10.461.575811.440.24831.40.99Model 22−27.62072.990549.5250.41302.126Model 30.2−21.2840.39950.74780.2150.5−21.60.948789.740.91380.500.7−8.7171.6249132.10.960.7018.41392.76591770.987810All the results are obtained via the average of 200 generated signals.From Table1, the value of NS is bigger than 0.05, and the value of θ^ should be adjusted close to the true value. It can also be seen that the proposed calibration via NS improves the accuracy of estimate significantly for all three models in Table 1. For the effects of parameter values, it is found that the initial estimated values deviate from the true parameters with the increase of parameters in Model 3. Figure 2 shows one of the calibration procedures for Model 3. It can be seen in Figure 2 that the estimates of different parameter can be calibrated to the exact value. When the calibration stops, NS<0.05. SNR is negative when the parameter is small, but the parameters can be still calibrated to the exact value via NS.Figure 2
Calibration procedure for Model 3.The abovementioned analysis validates the proposed NS-based calibration method. Next, we fix Model 3 withθ=0.2 for analysing the effects of various factors on the proposed calibration method.


### 3.2. Effects of Noise Strength

In this subsection, the added Cauchy noise is set tox0=0,γ=0.5, x0=0,γ=1, and x0=0,γ=1.5 to obtain different noise strengths. Model 3 with θ=0.2 is chosen for validation. Figure 3 shows one generated signal with each noise strength. Results of calibration are shown in Table 2. All the results are obtained via the average of 200 generated signals. One of the calibration procedures is shown in Figure 4.Figure 3
Generated signals with different noise strengths. The curves denoteft;θ. The circles denote yt. (a) x0 = 0, γ = 0.5. (b) x0 = 0, γ = 1. (c) x0 = 0, γ = 1.5.
(a)(b)(c)Table 2
Calibration performance for different noise strengths.
γSNRθ^REθ^ (%)NSθ^θ′REθ′ (%)0.5−21.1350.332666.30.80910.2151−23.0760.385392.60.73560.2151.5−26.9680.4267113.30.7160.215All the results are obtained via the average of 200 generated signals.Figure 4
Calibration procedure for different noise strengths.From Table2, it can be seen that the smaller the SNR, the larger the RE of θ^ will be. This indicates that the strong noise can reduce the accuracy of estimation. The calibration method performs well for all noise strengths. Figure 4 indicates that the estimates for different noise strengths can be calibrated to the exact value. When the calibration stops, NS<0.05. Figure 4 also shows a fast convergence rate which confirms the successful calibration procedure.


### 3.3. Effects of Noise Type

In this subsection, three types of noise are added to signals for analysing effects of noise type. The selected noises are independent and identically distributed and have uniform, Cauchy and power-law distributions. The Cauchy noise is set tox0=0,γ=0.5. The uniform noise is distributed as U−10,10. The power-law noise is distributed as the following density:(11)px=Cx−2,x≥0.1,where C is a positive constant and valued such that px is a density. Model 3 with θ=0.2 is chosen for validation.Figure5 shows one generated signal with each noise type. Results of calibration are shown in Table 3. All the results are obtained via the average of 200 generated signals. One of the calibration procedures is shown in Figure 6.Figure 5
Generated signals with different noise types. The curves denoteft;θ. The circles denote yt. (a) Uniform noise. (b) Cauchy noise. (c) Power-law noise.
(a)(b)(c)Table 3
Calibration performance for different noise types.
Noise typeSNRθ^REθ^ (%)NSθ^θ′REθ′ (%)Cauchy−15.680.3588790.760.195Uniform−14.680.279039.50.06950.1857.5Power-law−12.66370.263731.850.88270.20All the results are obtained via the average of 200 generated signals.Figure 6
Calibration procedure for different noise types.From Table3, it can be seen that the RE of initial estimate is large in the case of Cauchy noise. Comparing with other noise, it suffers from a more inaccurate estimate via least squares method. After calibration using NS, the estimates for Cauchy and power-law noise are corrected almost equal to the true parameter. The accuracy in the case of uniform noise is also improved by more than 30%. It is shown in Figure 6 that the calibration for power-law noise can distinguish the accuracy of a smaller step. Figure 6 also reveals a consistent conclusion that the NS for uniform noise decreases slowly in calibration procedure.


### 3.4. Effects of Sample Size

We fix the range oft from −10 to 10 and change the sampling interval to obtain different sample sizes. The sample size means the number of points were sampled uniformly in the range of t. Model 3 with θ=0.2 is chosen for validation. One generated signal with each sample size is shown in Figure 7. Results of calibration are shown in Table 4. All the results are obtained via the average of 200 generated signals. One of the calibration procedures is shown in Figure 8.Figure 7
Generated signals with different sample sizes. The curves denoteft;θ. The circles denote yt. (a) n = 20. (b) n = 40. (c) n = 50. (d) n = 100. (e) n = 200. (f) n = 1000.
(a)(b)(c)(d)(e)(f)Table 4
Calibration performance for different sample sizes.
nSNRθ^REθ^ (%)NSθ^θ′REθ′ (%)20−14.270.384792.350.18150.19540−30.060.4021010.45680.2050−28.310.398299.10.47840.1952.5100−14.580.385292.60.61570.1857.5200−23.630.38894.40.760.1952.51000−47.30.4174108.70.88800.20All the results are obtained via the average of 200 generated signals.Figure 8
Calibration procedure for different sample sizes.From Table4, it is found that the more the sample points, the larger the NS values. It indicates that when the sample points are sufficient, the NS is more sensitive to the estimated value. This method can distinguish the estimate from the real parameter and then correct the parameter more accurately. This conclusion is consistent with results of Figure 8, which shows changes in estimate of parameter during calibration.


### 3.5. Effects of Model Complexity

In this subsection, a more complex model extended from Model 3 is considered for effects of model complexity. It is denoted by Model 4 as follows:(12)Model4:ft;θ=−θ−θ2⋅t+θ22⋅t2−θ3⋅sint+sinθ⋅cossint,where θ is set to 0.2. The signals are generated at t every 0.1 from −10 to 10. Different types of noise are used for this analysis. The settings of noise are the same as in Section 3.3. One generated signal for this model is shown in Figure 9. Results of calibration are shown in Table 5. All the results are obtained via the average of 200 generated signals. One of the calibration procedures is shown in Figure 10.Figure 9
Generated signals for complex model. The curves denoteft;θ. The circles denote yt. (a) Uniform noise. (b) Cauchy noise. (c) Power-law noise.
(a)(b)(c)Table 5
Calibration performance for complex model.
Noise typeSNRθ^REθ^ (%)NSθ^θ′REθ′ (%)Cauchy−13.4840.308454.20.83450.20Uniform−15.4550.2720360.06760.195Power-law−15.59350.24431.850.86320.20All the results are obtained via the average of 200 generated signals.Figure 10
Calibration procedure for complex model.From Tables5 and 3, there is no significant difference between the results of Model 3 and Model 4. Comparing with results of Model 3, the accuracy of initial parameter estimation is improved slightly. But the accuracy of calibration is slightly inferior to the simple model. Figure 10 shows changes in estimate of parameter during calibration. It can be seen in Figure 10 that parameters can be calibrated to an accurate value under various noises, which indicates the successful calibration procedures for complex model with multitype noise.


## 3.1. Performance of Calibration

The following three models offt;θ are adopted to validate the performance of proposed calibration:(10)Model1:ft;θ=θt3−1+12θsinθsint,Model2:ft;θ=logt−2θ+0.01⋅θt2,Model3:ft;θ=−θ−θ2⋅t+θ22⋅t2−θ3⋅sint.Model 1 is not smooth, it admits a fold point att=1. Model 2 has a log singularity. Model 3 has a complex trend. They can represent typical situations in applications. For Model 1 and Model 2, the signals are generated at t every 0.1 from −3 to 3. For Model 3, the signals are generated at t every 0.1 from −10 to 10. For further analysing the effects of parameter values on calibration, we choose four values of θ in Model 3 for validation. One generated signal is shown in Figure 1. Results of calibration for the three models are shown in Table 1. All the results are obtained via the average of 200 generated signals.Figure 1
Generated signals for different models. The curves denoteft;θ. The circles denote yt. (a) Model 1, θ=2. (b) Model 2, θ = 2. (c) Model 3, θ = 0.2. (d) Model 3, θ = 0.5. (e) Model 3, θ = 0.7. (f) Model 3, θ = 1.
(a)(b)(c)(d)(e)(f)Table 1
Calibration performance for different models.
θSNRθ^REθ^ (%)NSθ^θ′REθ^ (%)Model 12−10.461.575811.440.24831.40.99Model 22−27.62072.990549.5250.41302.126Model 30.2−21.2840.39950.74780.2150.5−21.60.948789.740.91380.500.7−8.7171.6249132.10.960.7018.41392.76591770.987810All the results are obtained via the average of 200 generated signals.From Table1, the value of NS is bigger than 0.05, and the value of θ^ should be adjusted close to the true value. It can also be seen that the proposed calibration via NS improves the accuracy of estimate significantly for all three models in Table 1. For the effects of parameter values, it is found that the initial estimated values deviate from the true parameters with the increase of parameters in Model 3. Figure 2 shows one of the calibration procedures for Model 3. It can be seen in Figure 2 that the estimates of different parameter can be calibrated to the exact value. When the calibration stops, NS<0.05. SNR is negative when the parameter is small, but the parameters can be still calibrated to the exact value via NS.Figure 2
Calibration procedure for Model 3.The abovementioned analysis validates the proposed NS-based calibration method. Next, we fix Model 3 withθ=0.2 for analysing the effects of various factors on the proposed calibration method.


## 3.2. Effects of Noise Strength

In this subsection, the added Cauchy noise is set tox0=0,γ=0.5, x0=0,γ=1, and x0=0,γ=1.5 to obtain different noise strengths. Model 3 with θ=0.2 is chosen for validation. Figure 3 shows one generated signal with each noise strength. Results of calibration are shown in Table 2. All the results are obtained via the average of 200 generated signals. One of the calibration procedures is shown in Figure 4.Figure 3
Generated signals with different noise strengths. The curves denoteft;θ. The circles denote yt. (a) x0 = 0, γ = 0.5. (b) x0 = 0, γ = 1. (c) x0 = 0, γ = 1.5.
(a)(b)(c)Table 2
Calibration performance for different noise strengths.
γSNRθ^REθ^ (%)NSθ^θ′REθ′ (%)0.5−21.1350.332666.30.80910.2151−23.0760.385392.60.73560.2151.5−26.9680.4267113.30.7160.215All the results are obtained via the average of 200 generated signals.Figure 4
Calibration procedure for different noise strengths.From Table2, it can be seen that the smaller the SNR, the larger the RE of θ^ will be. This indicates that the strong noise can reduce the accuracy of estimation. The calibration method performs well for all noise strengths. Figure 4 indicates that the estimates for different noise strengths can be calibrated to the exact value. When the calibration stops, NS<0.05. Figure 4 also shows a fast convergence rate which confirms the successful calibration procedure.


## 3.3. Effects of Noise Type

In this subsection, three types of noise are added to signals for analysing effects of noise type. The selected noises are independent and identically distributed and have uniform, Cauchy and power-law distributions. The Cauchy noise is set tox0=0,γ=0.5. The uniform noise is distributed as U−10,10. The power-law noise is distributed as the following density:(11)px=Cx−2,x≥0.1,where C is a positive constant and valued such that px is a density. Model 3 with θ=0.2 is chosen for validation.Figure5 shows one generated signal with each noise type. Results of calibration are shown in Table 3. All the results are obtained via the average of 200 generated signals. One of the calibration procedures is shown in Figure 6.Figure 5
Generated signals with different noise types. The curves denoteft;θ. The circles denote yt. (a) Uniform noise. (b) Cauchy noise. (c) Power-law noise.
(a)(b)(c)Table 3
Calibration performance for different noise types.
Noise typeSNRθ^REθ^ (%)NSθ^θ′REθ′ (%)Cauchy−15.680.3588790.760.195Uniform−14.680.279039.50.06950.1857.5Power-law−12.66370.263731.850.88270.20All the results are obtained via the average of 200 generated signals.Figure 6
Calibration procedure for different noise types.From Table3, it can be seen that the RE of initial estimate is large in the case of Cauchy noise. Comparing with other noise, it suffers from a more inaccurate estimate via least squares method. After calibration using NS, the estimates for Cauchy and power-law noise are corrected almost equal to the true parameter. The accuracy in the case of uniform noise is also improved by more than 30%. It is shown in Figure 6 that the calibration for power-law noise can distinguish the accuracy of a smaller step. Figure 6 also reveals a consistent conclusion that the NS for uniform noise decreases slowly in calibration procedure.


## 3.4. Effects of Sample Size

We fix the range oft from −10 to 10 and change the sampling interval to obtain different sample sizes. The sample size means the number of points were sampled uniformly in the range of t. Model 3 with θ=0.2 is chosen for validation. One generated signal with each sample size is shown in Figure 7. Results of calibration are shown in Table 4. All the results are obtained via the average of 200 generated signals. One of the calibration procedures is shown in Figure 8.Figure 7
Generated signals with different sample sizes. The curves denoteft;θ. The circles denote yt. (a) n = 20. (b) n = 40. (c) n = 50. (d) n = 100. (e) n = 200. (f) n = 1000.
(a)(b)(c)(d)(e)(f)Table 4
Calibration performance for different sample sizes.
nSNRθ^REθ^ (%)NSθ^θ′REθ′ (%)20−14.270.384792.350.18150.19540−30.060.4021010.45680.2050−28.310.398299.10.47840.1952.5100−14.580.385292.60.61570.1857.5200−23.630.38894.40.760.1952.51000−47.30.4174108.70.88800.20All the results are obtained via the average of 200 generated signals.Figure 8
Calibration procedure for different sample sizes.From Table4, it is found that the more the sample points, the larger the NS values. It indicates that when the sample points are sufficient, the NS is more sensitive to the estimated value. This method can distinguish the estimate from the real parameter and then correct the parameter more accurately. This conclusion is consistent with results of Figure 8, which shows changes in estimate of parameter during calibration.


## 3.5. Effects of Model Complexity

In this subsection, a more complex model extended from Model 3 is considered for effects of model complexity. It is denoted by Model 4 as follows:(12)Model4:ft;θ=−θ−θ2⋅t+θ22⋅t2−θ3⋅sint+sinθ⋅cossint,where θ is set to 0.2. The signals are generated at t every 0.1 from −10 to 10. Different types of noise are used for this analysis. The settings of noise are the same as in Section 3.3. One generated signal for this model is shown in Figure 9. Results of calibration are shown in Table 5. All the results are obtained via the average of 200 generated signals. One of the calibration procedures is shown in Figure 10.Figure 9
Generated signals for complex model. The curves denoteft;θ. The circles denote yt. (a) Uniform noise. (b) Cauchy noise. (c) Power-law noise.
(a)(b)(c)Table 5
Calibration performance for complex model.
Noise typeSNRθ^REθ^ (%)NSθ^θ′REθ′ (%)Cauchy−13.4840.308454.20.83450.20Uniform−15.4550.2720360.06760.195Power-law−15.59350.24431.850.86320.20All the results are obtained via the average of 200 generated signals.Figure 10
Calibration procedure for complex model.From Tables5 and 3, there is no significant difference between the results of Model 3 and Model 4. Comparing with results of Model 3, the accuracy of initial parameter estimation is improved slightly. But the accuracy of calibration is slightly inferior to the simple model. Figure 10 shows changes in estimate of parameter during calibration. It can be seen in Figure 10 that parameters can be calibrated to an accurate value under various noises, which indicates the successful calibration procedures for complex model with multitype noise.


## 4. Numerical Analysis with Chaotic Noise

First, we introduce the chaotic noise. LetS:0,1⟶0,1 be a transformation. The iterations of S are a discrete dynamical system. After giving an initial value x0∈0,1, put(13)xi+1=Sxi,i=0,1,2,…,n.The obtainedxii=0n is an orbit of S with initial value x0 and length n+1. Suppose S is chaotic and admits an absolutely continuous invariant probability measure (distribution) PS whose density function is hx, i.e.,(14)PSx1,x2=∫x1x2hxdx,0≤x1<x2≤1.The distributionPS is stationary because it is invariant. According to the classical Birkhoff individual ergodic theorem, for almost all initial value x0, the histogram of vxii=0n will approach to hx as n⟶∞. So, almost all long orbits of S exhibit stochastic behavior. Typically, xii=0n looks like a sample of the distribution PS; it can be regarded as a chaotic noise.It is known that the chaotic noise has deterministic inherent quantitative relations which result in strong dependence in noise and can disturb the function signal. Thus, parameter calibration via NS under chaotic noise is challenging and must be able to deal with such strong dependence in noise. In this section, we focus on the numerical analysis of the proposed NS-based calibration method in the case of chaotic noise.Model 2 with independent standard Gaussian noise is chosen for this numerical study. Three values of parameterθ are selected. The signals are generated at t every 0.01 from 2 to 8. Two different chaotic noises are further added to this setting for numerical analysis with chaotic noise. The analysed model is expressed as follows:(15)yt=logt−2θ+0.01⋅θt2+εt,where εt=εg,t+εc,t denotes the noise signal, εg,t denotes the independent standard Gaussian noise, and εc,t denotes the added chaotic noise which is generated respectively via the following two kinds of iteration. The initial input of iteration is randomly selected from the uniform distribution U0,1.(16)Chaotic noise1:xi+1=4xi1−xi,xi∈0,1,Chaotic noise2:xi+1=3xi+1−32,xi∈0,12,3xi−12,xi∈12,1.The transformation of the first dynamical system isSx=4x1−x,x∈0,1. In 1947, Ulam and von Neumann [24] showed that S admits an absolutely continuous invariant probability measure whose density is(17)hx=1πx1−x.So,S:0,1⟶0,1 is chaotic discrete dynamical system and its typical orbit exhibits stochastic behavior. The transformation of the second dynamical system is(18)Tx=3x+1−32,x∈0,12,3x−12,x∈12,1.It is a piecewise linear Lorenz map with constant slope3 [25]. T admits an absolutely continuous invariant probability measure PT. Furthermore, the mean value of PT is 1/2 because of the equality Tx+T1−x=1. The mean value of the first invariant probability measure PS is also 1/2 because of the equality Sx=S1−x. In both cases, let εc,ti=xi−1/2; εc,ti is zero mean chaotic noise for almost all initial values x0.Due to disturbance from strong dependence of chaotic noise, the estimator via least-squares-based method in this case is rather inaccurate and very far from the truth parameter. In order to focus on the validation of proposed calibration method and speed up calibration, the initial value is directly given near the truthθ for calibration instead of least-squares-based method. Results of analysis and calibration are shown in Figure 11 and Table 6. All the results are obtained via the average of 200 generated signals.Figure 11
Change of NS with given parameterθ for chaotic noise. The truth values of θ are 2, 2.71, and 3.14 from left to right. All the results are obtained via the average of 200 generated signals. (a) Choatic noise 1. (b) Choatic noise 2.
(a)(b)Table 6
Calibration performance with given initial value for chaotic noise.
θSNRθ˜θ′RE (θ′) (%)Chaotic noise 126.39111.82.00500.25Chaotic noise 12.714.83432.52.70670.12Chaotic noise 13.145.249433.13600.13Chaotic noise 226.83841.81.99500.25Chaotic noise 22.715.31182.52.72000.37Chaotic noise 23.145.597533.14500.16θ˜ denotes the initial value near the truth θ which is given directly. All the results are obtained via the average of 200 generated signals.Figure11 shows the change of NS with given parameter near the truth parameter θ. It can be seen that although the NS for θ=2.71θ changes slightly at locations that are larger than the truth, all the minimums of NS values occur at location that is very close to the true θ. This validates the availability of our proposed calibration method under chaotic noise. Table 6 shows the results of calibration with the given initial value. It can be seen that the calibration makes the given initial value close to the truth θ and improves the accuracy of given initial value, which indicates that the proposed calibration method is also effective under chaotic noise as the case of random noise.


## 5. Conclusions

It is important to estimate the parameter of single-parameter nonlinear trend in signals. The commonly used methods for this topic are mainly based on least squares. It can cause inaccurate estimation results when suffering from large noise and complex non-Gaussian noise. Thus, we proposed a calibration method for this issue via nonstationarity measure (NS) from the perspective of the stationarity of residual sequence. Some numerical studies were conducted for validation. Results of numerical studies showed that the proposed calibration method performed well for various models with different noise strengths and types could significantly improve the accuracy of initial estimates obtained by least squares method. It can not only solve the problem of large sum of squared residuals but also deal with strong dependent deterministic noise, which cause inaccurate estimation results. For some detailed conclusions, the performance under uniform noise is worse than those under Cauchy and power-law noise. When the sample points are sufficient, the proposed calibration method is sensitive to the estimated value and becomes more accurate. The strength of model complexity can improve the accuracy of calibration slightly. The proposed calibration method is also effective under chaotic noise as the case of random noise despite the rather inaccuracy of least-squares-based method. The nonstationarity measure is first applied to parameter calibration in this study. All these results will be a guide for future studies of other parameter calibrations such as the bias correction of Hurst parameter estimations [26–28].



---

*Source: 8840757-2021-01-29.xml*