# A note on fixed- and discrete-time estimation via the DREM method

Marina Korotina<fnref n="1" />,<fnref n="2" />, Stanislav Aranovskiy<fnref n="1" />, Rosane Ushirobira<fnref n="3" />, Denis Efimov<fnref n="3" />, and Jian Wang<fnref n="4" />

**Abstract**—A simple fixed-time converging estimation algorithm is presented for a linear regression using the dynamic regressor extension and mixing (DREM) method within a discrete-time setting, with a persistently excited regressor and bounded measurement noises. The solution is based on Kreisselmeier's filters, and it is computationally simpler than the existing analogs.

## I. INTRODUCTION

In this note, we consider the parameter estimation problem for a discrete-time linear regression equation (LRE)

$$ \bar{y}_k = \bar{\phi}_k^\top \bar{\theta} + \bar{v}_k, \quad k \in \mathbb{N}, \quad (1) $$

where $\bar{y}_k, \bar{v}_k \in \mathbb{R}$ are the output and the bounded measurement noise, $\bar{\phi}_k, \bar{\theta} = (\theta_1 \dots \theta_n)^\top \in \mathbb{R}^n$ are the bounded regressor and the unknown parameter vectors. The goal is to estimate $\bar{\theta}$ given the online measurements $\bar{y}_k$ and $\bar{\phi}_k$.

The parameter estimation problem for dynamic or static systems is an essential issue in many scientific disciplines, and many well-matured approaches to solving this problem are available [1], [2]. Despite various batch methods based on post-processing of collected measurements, two popular iterative parameter estimation methods are the recursive least-squares and gradient-descent estimators [3]. The fundamental result in parameter estimation claims that if the regressor $\bar{\phi}$ is persistently excited (PE), that is uniformly excited in time (see Section III for a formal definition of this property), then the recursive least-squares and gradient-descent iterative estimators converge exponentially. They are also input-to-state stable (ISS) and provide filtering of the noise $\bar{v}_k$.

Numerous recent studies address a possible relaxation of the PE requirement in both continuous- and discrete-time settings. These results include the composite [4] and concurrent [5] learning, directional forgetting [6], fixed-time estimation [7], and the recent regularization-based results [8], [9]. However, it is widely understood that if the regressor $\bar{\phi}$ is not uniformly excited in time but only on a particular interval (interval excitation), then the additive noise cannot be efficiently filtered, and the PE-relaxation methods typically consider the noise-free scenario, $\bar{v} \equiv 0$. In this note, due to the presence of noise $\bar{v}_k$ in (1), we are focused on the uniform-in-time excitation, i.e., when the regressor $\bar{\phi}$ is PE.

Standard iterative estimators also have some known shortfalls, such as the difficulty of accelerating the convergence process (this property is mainly predefined by the level of excitation in the regressor) and the non-monotonicity of the

convergence [10]–[12]. The latter means that despite a decreasing norm of the parameter vector estimation error, the respective discrepancies for each parameter may demonstrate a complex oscillatory behavior before settling down in a vicinity of the true value, which may postpone a real-time utilization of the obtained estimates. That is why a solution to these drawbacks recently proposed by the dynamical regressor extension and mixing (DREM) method [13] has quickly gained popularity [14]–[17].

The DREM procedure consists of two steps: extension and mixing. In the first step, the model (1) is extended to get a new linear regression with a square $n \times n$ regressor matrix. Then, at the mixing step, the single extended LRE for a vector of unknown parameters is transformed into a set of $n$ scalar LREs for each element of the vector $\theta$ independently. Then an iterative estimator, e.g., the gradient-descent one, is applied to each of the new scalar LREs separately, yielding the element-wise improved transients with simple and transparent estimator tuning rules.

In this note, we argue that using iterative estimators in the DREM procedure is more a custom than a need. We show that a proper choice of an LRE extension at the first step of the DREM procedure ensures that the novel regressor is strictly positive, allowing for pointwise (algebraic) estimation. A pointwise estimator can be of limited practical interest due to its noise sensitivity, even if it provides the fixed-time (independent in initial conditions) convergence. To this end, we study the worst-case bound on the parameter estimation error and propose tuning guidelines minimizing this bound.

To summarize, the contribution of this paper is as follows:

* we show that a particular choice of filters for an LRE extension ensures strict positiveness of the novel regressor;

* we analyze noise propagation for the pointwise (algebraic) estimator and derive the worst-case bound on estimation error.

Finally, we address the argument that standard iterative estimators provide a good trade-off between the transient time and the steady-state noise attenuation (filtering). We show that we obtain faster transients with a comparable steady-state performance by putting the pointwise estimator in a serial connection with a (non-)linear filter. Together, these contributions support our claim that, under the PE condition, the DREM procedure can function as a fixed-time estimator without requiring subsequent gradient-type algorithms.

The rest of the paper is organized as follows. In Section II we briefly describe the DREM procedure. In Section III, we discuss how the initial excitation can be preserved by the DREM procedure. Section IV presents this note's main result, and Section V contains simulations illustrating our results. Finally, Section VI contains some concluding remarks.

<footnote>This work was supported by the China 111 Project D17019.</footnote>

<footnote n="1">¹CentraleSupélec–IETR, 35576 Cesson-Sévigné, France.</footnote>

<footnote n="2">²ITMO University, 197101 Saint Petersburg, Russia.</footnote>

<footnote n="3">³Inria, Univ. Lille, CNRS, UMR 9189 - CRISTAL, F-59000 Lille, France.</footnote>

<footnote n="4">⁴Hangzhou Dianzi University, Hangzhou, China. Corresponding author, wangjian119@hdu.edu.cn</footnote>
## Notation

* The sets of nonnegative real and integer numbers are denoted by $\mathbb{R}_+$ and $\mathbb{N}$, respectively. Also, $\mathbb{N}^* := \mathbb{N} \setminus \{0\}$. The set of real $n \times m$-matrices is denoted by $\mathbb{R}^{n \times m}$. The $n$-identity matrix is denoted by $I_n$.

* For a vector $x \in \mathbb{R}^n$, $\|x\|$ denotes its Euclidean norm.

* The rounding function to the greatest integer smaller than $s \in \mathbb{R}$ is denoted by $\lfloor s \rfloor = \text{floor}(s)$.

## II. DYNAMICAL REGRESSOR EXTENSION AND MIXING

The original DREM procedure in [13] starts with a dynamics extension step. It consists in finding a (dynamic) transformation that applies to the original LRE (1) and leads to a new extended regression

$$Y_k = \Phi_k \bar{\theta} + \bar{V}_k, \quad \bar{V}_k = V_k + \epsilon_k, \quad k \in \mathbb{N}, \quad (2)$$

where $Y_k \in \mathbb{R}^n$ and $\Phi_k \in \mathbb{R}^{n \times n}$ are measured signals generated by the extension of the dynamics, $V_k \in \mathbb{R}^n$ results from the noise $\bar{v}_k$ propagation, and $\epsilon_k \in \mathbb{R}^n$ is an exponentially decaying term arising due to the initialization of the dynamics extension. Such an extension can be performed, for instance, via delay operators or linear filters, whose auxiliary role is also to filter the noise, see [13]. A particular choice of the dynamics extension used in this paper is presented in Section III.

For the extended regression (2), we define

$$\phi_k = \det(\Phi_k), \quad y_k = \operatorname{adj}(\Phi_k) Y_k, \quad v_k = \operatorname{adj}(\Phi_k) \bar{V}_k \quad (3)$$

where $\operatorname{adj}(M)$ denotes the adjugate matrix of a matrix $M$. This yields the element-wise scalar linear regression:

$$y_{k,i} = \phi_k \theta_i + v_{k,i}, \quad k \in \mathbb{N}, \quad i \in \{1, \dots, n\}, \quad (4)$$

where $y_{k,i} \in \mathbb{R}$ and $\phi_k \in \mathbb{R}$ are known signals, $\theta_i \in \mathbb{R}$ is the unknown constant parameter to be estimated, and $v_{k,i} \in \mathbb{R}$ is an unknown bounded measurement distortion. Note that the estimation of each $\theta_i$ is now explicitly independent of other components of $\bar{\theta}$.

Let $\tilde{\theta}_k \in \mathbb{R}^n$ denote an estimate of $\bar{\theta}$ at time $k \in \mathbb{N}$. Define $e_k := y_k - \phi_k \tilde{\theta}_k = \phi_k \bar{\theta}_k + v_k$ as the measured regression error, where $\tilde{\theta}_k := \bar{\theta} - \tilde{\theta}_k$ is the parameter estimation error. It remains to design an algorithm to compute $\tilde{\theta}_k$, which is a simpler problem than (1), since in (4) the estimation of each $\tilde{\theta}_i$ can be treated separately. Moreover, in scalar linear regression, the convergence of $\tilde{\theta}_k$ is element-wise monotone and can be accelerated by tuning the estimation procedures [13].

One of the main issues of the DREM methodology is the excitation of the common regressor $\phi_k$, and its relation with the excitation of $\bar{\phi}_k$ in the original problem statement, which is obviously predefined by the choice of the dynamics extension generating $\Phi_k$. Let us present a solution to this problem.

## III. KREISSELMEIER'S REGRESSOR EXTENSION

Let us show how the DREM procedure can keep the excitation level of the original regression for (4). To this end, let us characterize the admissible excitation levels [10]:

**Definition 1 (PE).** Let $\ell \in \mathbb{N}^*$ and $\mu > 0$. A bounded signal $\bar{\phi}: \mathbb{N} \to \mathbb{R}^n$ is called $(\ell, \mu)$-persistently excited (PE) if

$$\sum_{i=0}^{\ell-1} \bar{\phi}_{k+i} \bar{\phi}_{k+i}^{\top} \geq \mu I_n, \quad \forall k \in \mathbb{N}.$$

It is said to be intervally excited (IE) if the above inequality is satisfied for $k = 0$ only.

It is worth noting that in the noise-free case, interval excitation can be related with the conventional identifiability/observability condition for $\bar{\theta}$ in (1).

To keep the excitation, we proceed as follows. Instead of applying a dynamic extension directly to (1), consider the auxiliary regression problem

$$\bar{\phi}_k \bar{y}_k = \bar{\phi}_k \bar{\phi}_k^{\top} \bar{\theta} + \bar{\phi}_k \bar{v}_k$$

and apply an auxiliary filter $H(z) = \frac{1-\lambda}{z^{1-\lambda}}$, where $\lambda \in (0, 1)$. Then for

$$Y_k = H(z) \bar{\phi}_k \bar{y}_k, \quad \Phi_k = H(z) \bar{\phi}_k \bar{\phi}_k^{\top}, \quad V_k = H(z) \bar{\phi}_k \bar{v}_k,$$

we obtain the extended LRE (2). The state-space implementation of this extension is known as Kreisselmeier's dynamic regressor extension [18] and is given by

$$\Phi_{k+1} = \lambda \Phi_k + (1 - \lambda) \bar{\phi}_k \bar{\phi}_k^{\top}, \quad (5)$$

$$Y_{k+1} = \lambda Y_k + (1 - \lambda) \bar{\phi}_k \bar{y}_k,$$

where $Y_0 \in \mathbb{R}^n$ and $\Phi_0 \in \mathbb{R}^{n \times n}$, $\Phi_0 = \Phi_0^\top \geq 0$. After the mixing step, it yields the decoupled linear regression equation (4). Specifically, the choice $\Phi_0 = 0$ and $Y_0 = 0$ guarantees that $\epsilon_k = 0$ in (2) for all $k \in \mathbb{N}$.

Kreisselmeier's extension was first used within the DREM context in [19] under the name of memory regressor extension. It is known that (under a proper choice of $\Phi_0$), the matrix $\Phi$ is semi-positive definite, and $\Phi$ is positive definite if $\bar{\phi}$ is PE, see [20]. The following Lemma further extends this result, providing a lower bound on the excitation characteristics of the signal $\phi$ defined in (3).

**Lemma 1.** Consider a $(\ell, \mu)$-PE signal $\bar{\phi}$ and the Kreisselmeier's dynamic extension (5). Then the signal $\phi$ defined in (3) is $(\ell, \alpha)$-PE, with

$$\alpha = (\mu(1 - \lambda)\lambda^{\ell-1})^n. \quad (6)$$

Moreover,

$$\phi_k \geq \alpha, \quad \forall k \geq \ell; \quad \liminf_{k \to \infty} \phi_k \geq \alpha \left( \frac{1}{1 - \lambda^\ell} \right)^n.$$

*Proof:* The solution of (5) is

$$\Phi_k = \lambda^k \Phi_0 + (1 - \lambda) \sum_{i=0}^{k-1} \psi(k, i), \quad (7)$$

with $\psi(k, i) = \lambda^{k-1-i} \bar{\phi}_i \bar{\phi}_i^{\top}$. Set $q = \lfloor \frac{k}{\ell} \rfloor$ and $v = k - q\ell$, so $k = q\ell + v$. Then for all $k \geq \ell$

$$\sum_{i=0}^{k-1} \psi(k, i) = \sum_{j=1}^{q} \left( \sum_{i=k-j\ell}^{k-1-(j-1)\ell} \psi(k, i) \right) + \sum_{i=0}^{v-1} \psi(k, i).$$
Since $\psi(k, i) \ge 0$ for all $k, i \in \mathbb{N}$ and $\Phi_0 \ge 0$, it follows from (7) that for all $k \ge \ell$

$$ \Phi_k \ge (1 - \lambda) \sum_{j=1}^{q} \left( \sum_{i=k-j\ell}^{k-1-(j-1)\ell} \psi(k, i) \right). $$

Recalling that $\lambda < 1$, for all $k \ge \ell$ and $j \in \{1, \dots, q\}$, it holds

$$ \sum_{i=k-j\ell}^{k-1-(j-1)\ell} \psi(k, i) \ge \lambda^{j\ell-1} \sum_{i=k-j\ell}^{k-1-(j-1)\ell} \bar{\phi}_i \bar{\phi}_i^\top \\ = \lambda^{j\ell-1} \sum_{i=0}^{\ell-1} \bar{\phi}_{k-j\ell+i} \bar{\phi}_{k-j\ell+i}^\top, $$

and using that $\bar{\phi}$ is $(\ell, \mu)$-PE, we obtain

$$ \sum_{i=k-j\ell}^{k-1-(j-1)\ell} \psi(k, i) \ge \lambda^{j\ell-1} \mu I_n. $$

Thus

$$ \Phi_k \ge \mu(1 - \lambda) \sum_{j=1}^{q} \lambda^{j\ell-1} I_n = \mu(1 - \lambda) \lambda^{\ell-1} \frac{1 - \lambda^{q\ell}}{1 - \lambda^{\ell}} I_n. $$

Then for $\phi_k = \det(\Phi_k)$, we obtain for all $k \ge \ell$ (and since $q \ge 1$)

$$ \phi_k \ge (\mu(1 - \lambda)\lambda^{\ell-1})^n = \alpha. $$

Moreover, $\lambda^{q\ell} \to 0$ as $k \to \infty$, and consequently

$$ \liminf_{k \to \infty} \phi_k \ge \left( \mu (1 - \lambda) \frac{\lambda^{\ell-1}}{1 - \lambda^\ell} \right)^n. $$

In the continuous-time setting, a similar result on the choice of filters is given in [21]. Of course, another filter selection is possible, providing the excitation properties of (1) in (4).

**Remark 1.** From Lemma 1, it follows that if $\bar{\phi}$ in (1) is PE, then $\phi$ is strictly positive for all $k \ge \ell$. Moreover, if $\bar{\phi}$ in (1) is IE, then $\phi$ resulting from (3), (5) cannot be identically zero. In both cases, some $k \in \mathbb{N}$ exists when $\phi_k \ne 0$. In the following, we consider only such steps and assume that $\phi_k \ne 0$. Otherwise, the linear regression (4) reads $y_k = v_k$, and there is no reason to update the estimate for such samples, i.e., $\hat{\theta}_k = \hat{\theta}_{k-1}$ for $\phi_k = 0$ (recall that the signal $\phi$ is known).

## IV. NOISE PROPAGATION

In the noise-free case, after applying the DREM method and obtaining the decoupled linear regression (4), the estimation problem has a trivial pointwise solution

$$ \hat{\theta}_k = \frac{y_k}{\phi_k}, \qquad (8) $$

which can be unconditionally applied as soon as $\phi_k \ne 0$ for some $k \in \mathbb{N}$. Such a condition is always verified with an IE-signal $\phi$.

In the presence of noise, the noise propagation can be studied using the established properties of Kreisselmeier's filters (5). Lemma 1 helps to prove the following simple result constituting the main outcome of this note:

**Theorem 1.** Assume that the regressor signal $\bar{\phi}$ in (1) is $(\ell, \mu)$-PE and Kreisselmeier's dynamic extension (5) is applied to yield (2). Then the estimate $\hat{\theta}_k$ can be computed by (3), (8), for all $k \ge \ell$, and

$$ \sup_{k \ge \ell} \|\tilde{\theta}_k\| \le \frac{\sup_{k \ge \ell} \|v_k\|}{\alpha}, \\ \limsup_{k \to +\infty} \|\tilde{\theta}_k\| \le (1 - \lambda^\ell)^n \frac{\limsup_{k \to +\infty} \|v_k\|}{\alpha}. $$

*Proof:* According to Lemma 1, for all $k \ge \ell$, $\phi_k \ge \alpha = (\mu(1 - \lambda)\lambda^{\ell-1})^n$, hence the division in (8) is well defined. By definition,

$$ \tilde{\theta}_k = \bar{\theta} - \frac{y_k}{\phi_k} = \frac{v_k}{\phi_k} $$

giving the required estimates by the properties of $\phi_k$. ■

Thus, in the noise-free case, the DREM-based algorithm (5), (3), (8) guarantees a fixed-time convergence of the estimate $\hat{\theta}_k$ to the true value $\bar{\theta}$, while in the presence of noises, the estimation error will be settled at a vicinity of the origin, whose size is proportional to the noise $v$ amplitude. Note that the convergence (settling) time $\ell \ge n$ is independent of initial conditions in both cases.

**Remark 2.** According to the estimates given in Theorem 1, the choice $\lambda^* = \frac{\ell-1}{\ell}$ minimizes the parameter estimation error gain $\alpha^{-1}$ with respect to the noise, i.e., it maximizes $\alpha$ given in (6). Augmenting the value of $\lambda$ can improve the asymptotic precision of the proposed algorithm, i.e., for $k \to +\infty$.

Several final comments are in order:

1) Roughly speaking, the DREM method provides a direct measurement of the unknown parameter vector $\bar{\theta}$ for $k \ge \ell$:

$$ \mathcal{Y}_k = \bar{\theta} + \mathcal{V}_k, \quad \text{with} \quad \mathcal{Y}_k = \phi_k^{-1} y_k, \quad \mathcal{V}_k = \phi_k^{-1} v_k. $$

Note also that (3), (5), (8) implicitly realize exponentially-weighted least-squares estimation with a forgetting factor $\lambda$.

2) Any additional noise filtering can be applied to $\mathcal{Y}_k$ attenuating the influence of $\mathcal{V}_k$. In the existing literature, gradient algorithms are frequently used for (4) [7], [16], [22], [23], which serve merely for implicit noise-canceling, being probably not the best choice for such a purpose. Indeed, the conventional selection is the least-squares algorithm:

$$ \hat{\theta}_k = \hat{\theta}_{k-1} + \gamma_k \phi_k (y_k - \phi_k \hat{\theta}_{k-1}), \quad \gamma_k \in (0, 2\phi_k^{-2}). $$

For $\gamma_k = \phi_k^{-2}$, it is reduced to (8), while for $\gamma_k = \varrho\phi_k^{-2}$ with $\varrho \in (0, 1)$, to a low-pass filtering of (8). In general, many more options are available, as the moving-average filter or nonlinear designs that cannot be directly applied to the linear regression (4), e.g., the median filter. See Section V for an illustrative comparison of selected methods.

3) Note that Kreisselmeier's dynamic extension (5) filters the noise $\bar{v}_k$ already, and other preliminary noise filters can be applied to get (1).

The DREM method was proposed to process regressor matrices that can be singular (or close to it): multiplying by the adjugate matrix is a viable operation, and the zero
(or arbitrarily small) determinant is addressed by a (gradient) estimator. If the original regressor is persistently excited, handling it with DREM after Kreisselmeier's extension, we know that the matrix is not singular after the first samples. Then, applying DREM and dividing by the determinant is the same as multiplying by the inverse. We choose an augmented computational complexity of the DREM framework due to all its advantages, e.g., the possibility of estimating a single element of the vector of unknown parameters with a controlled performance. Another message of this note is that the noise attenuation of the DREM method can be further improved by complementing it with a suitable filtering solution.

V. SIMULATIONS

A. General description

We consider a standard example where we estimate param-
eters of a sinusoidal signal:

$$
\bar{y}_k = A_0 + A \sin(\omega k + \psi) + \bar{v}_k, \quad k \in \mathbb{N}, \quad (9)
$$

which can be written as (1) with

$$
\begin{align*}
\bar{\phi}_k &= (1 \quad \sin(\omega k) \quad \cos(\omega k))^{\top}, \\
\bar{\theta} &= (A_0 \quad A \cos(\psi) \quad A \sin(\psi))^{\top},
\end{align*}
$$

where $\omega = \frac{2\pi}{T}$, $T \in \mathbb{N}$, $T \ge 3$, is the known period, and $A_0 \in \mathbb{R}$, $A \in \mathbb{R}_+$, $\psi \in [0, 2\pi)$ are unknown constant parameters. For the conciseness of simulations, we focus below on estimating the parameter $\theta_1 = A_0$ only.

The DREM procedure (5) with zero initial conditions yields
the element-wise scalar linear regression (4), where we are
interested in the equation corresponding to i = 1 only, namely

$$
y_{k,1} = \phi_k \theta_1 + v_{k,1}. \tag{10}
$$

Note that the DREM procedure allows for the single parameter
estimation; otherwise, the whole vector $\bar{\theta}$ has to be estimated.

It is straightforward to show that the regressor $\bar{\phi}$ is $(\ell, \mu)$-PE with $\ell = T$ and $\mu = \frac{T}{2}$, and Lemma 1 applies. The signal $\phi$ is thus also PE and it is strictly separated from zero for $k \ge T$. Then the direct estimation (8) is applied. To summarize, the DREM procedure, taking the measurements $\bar{\phi}_k$ and $\bar{y}_k$ as inputs and producing the direct estimate of the first element of the vector $\theta$ as an output, is given by

$$
\begin{align*}
\Phi_{k+1} &= \lambda\Phi_k + (1-\lambda)\bar{\phi}_k \bar{\phi}_k^\top, \quad \Phi_0 = 0, \\
Y_{k+1} &= \lambda Y_k + (1-\lambda)\bar{\phi}_k \bar{y}_k, \quad Y_0 = 0, \\
\phi_k &= \det(\Phi_k), \quad y_k = \operatorname{adj}(\Phi_k) Y_k, \\
\hat{\theta}_{k,1}^{\text{dir}} &= \frac{y_{k,1}}{\max\{\phi_k, \varepsilon_\phi\}}, \tag{11}
\end{align*}
$$

where $\varepsilon_\phi > 0$ is a small constant introduced to ensure the
feasibility of implementing the direct estimation for the initial
transients of (5).

In the sequel, we consider various filtering techniques
applied to the direct estimate (11) and compare them with esti-
mators used in the element-wise linear regression model (10).
The results illustrate that fixed-time direct estimation followed
by filtering performs better than estimators with asymptotic
convergence.

B. Used approaches

By abuse of notation, we use $y_k$, $\theta$, and $\hat{\theta}_k$ to denote $y_{k,1}$,
$\theta_1$, and $\hat{\theta}_{k,1}$, resp., since we are interested only in $\hat{\theta}_1$.

For the element-wise linear regression (10), we consider the
following two estimators:

• the gradient estimator given by

$$
\hat{\theta}_{k}^{\text{gr}} = \hat{\theta}_{k-1}^{\text{gr}} + \frac{\gamma_0}{1 + \gamma_0 \phi_k^2} (y_k - \phi_k \hat{\theta}_{k-1}^{\text{gr}}), \quad (12)
$$

where $\gamma_0 > 0$ is the tuning coefficient;

• the least-squares estimator with forgetting given by

$$
p_k = \frac{p_{k-1}}{\lambda_{ls} + \phi_k^2 p_{k-1}}, \quad \hat{\theta}_k^{\text{ls}} = \hat{\theta}_{k-1}^{\text{ls}} + p_k \phi_k (y_k - \phi_k \hat{\theta}_{k-1}^{\text{ls}}), \quad (13)
$$

with $p_0 > 0$ and $\lambda_{ls} \in (0, 1)$ is the forgetting factor.

For the direct estimate, we consider the following filtering
methods applied to $\hat{\theta}_k^{\text{dir}}$ given in (11):

• the low-pass filtering given by

$$
\hat{\theta}_{k}^{\text{low}} = \varrho \hat{\theta}_{k-1}^{\text{low}} + (1 - \varrho) \hat{\theta}_{k}^{\text{dir}}, \quad (14)
$$

where $\varrho \in (0, 1)$ is the tuning coefficient;

• the moving-average filtering given by

$$
\hat{\theta}_k^{\text{ma}} = \frac{1}{N} \sum_{i=0}^{N-1} \hat{\theta}_{k-i}^{\text{dir}}, \quad (15)
$$

where $N = \min\{k, N_{\text{ma}}\}$, and $N_{\text{ma}}$ is the length of the
averaging window;

• the median filter given by

$$
\hat{\theta}_k^{\text{med}} = \text{median}\left\{\hat{\theta}_{k-N+1}^{\text{dir}}, \dots, \hat{\theta}_k^{\text{dir}}\right\}, \quad (16)
$$

where $N = \min\{k, N_{\text{med}}\}$, and $N_{\text{med}}$ is the number of
samples used in the median computation,

• the Kalman filter given by

$$
\begin{align}
\hat{\theta}_k^{\text{kal}} &= \hat{\theta}_{k-1}^{\text{kal}} + \frac{P_{k-1} + Q}{P_{k-1} + Q + R_k} \left( \frac{y_k}{\phi_k} - \hat{\theta}_{k-1}^{\text{kal}} \right), \nonumber \\
P_k &= \frac{R_k}{P_{k-1} + Q + R_k} (P_{k-1} + Q), \tag{17} \\
R_k &= \frac{R_0}{\max\{\phi_k^2, \varepsilon_\phi^2\}}, \nonumber
\end{align}
$$

where $Q \ge 0$ and $R_0 > 0$ are the process and observation
noise covariances. When $Q \ne 0$, the case of time-varying
parameters can be treated.

C. Simulation results

Consider the signal (9), where $A_0 = -3$, $A = 5$, $\psi = \frac{\pi}{3}$,
and $T = 117$. The measurement noise $\bar{v}$ is chosen as a uniform
random variable, $\bar{v} \sim U(-1, 1)$. Following Remark 2, the
value $\lambda$ in (5) is chosen as $\lambda = \frac{T-1}{T}$ and the initial conditions
in (5) are zero. Then the lower bound $\alpha$ defined in Lemma 1
is $\alpha = 0.0063$ and the asymptotic lower bound of $\phi$ is 0.025.

For the direct estimation, the value $\varepsilon_\phi$ in (11) is chosen
as $\varepsilon_\phi = 10^{-8}$. The initial values in the estimators (12), (13)
and filters (14), (17) are zero; the initial values of $p_0$ in (13)
and $P_0$ in (17) equal to one. The parameters of the estimators
<figure><img src="image_1.png" /><figcaption>Fig. 1. Estimation error transients in the noise-free case: the direct estimation $\hat{\theta}^{\text{dir}}$ (11), the gradient estimator $\hat{\theta}^{\text{gr}}$ (12), the least-squares estimator $\hat{\theta}^{\text{ls}}$ (13), the low-pass filtering $\hat{\theta}^{\text{low}}$ (14), the moving-average filtering $\hat{\theta}^{\text{ma}}$ (15), the median filtering $\hat{\theta}^{\text{med}}$ (16), the Kalman filter $\hat{\theta}^{\text{kal}}$ (17). The dashed line corresponds to 5% of the initial error value; the estimators are tuned to have approximately equal transient time. For the median filter, the number of samples is the same as for the moving average.</figcaption></figure>

<table><caption>TABLE I
MSE AND MAE OF THE SELECTED METHODS.</caption><thead><tr><th></th><th>θ̂<sup>gr</sup></th><th>θ̂<sup>ls</sup></th><th>θ̂<sup>low</sup></th><th>θ̂<sup>ma</sup></th><th>θ̂<sup>med</sup></th><th>θ̂<sup>kal</sup></th></tr></thead><tbody><tr><td>MSE · 10<sup>3</sup></td><td>1.4</td><td>1.3</td><td>1.2</td><td>1.2</td><td>1.2</td><td>1.1</td></tr><tr><td>MAE · 10<sup>2</sup></td><td>3.0</td><td>3.0</td><td>2.7</td><td>2.7</td><td>2.7</td><td>2.6</td></tr></tbody></table>

(12), (13) and filters (14), (15), (17) are chosen to provide equal transient time in the ideal noise-free scenario, where the transient time is measured as the step number when the estimation errors decays to 5% of the initial value. The values are $\gamma_0 = 10^3$, $\lambda_{ls} = e^{-3/8}$, $\rho = e^{-3/3}$, $N_{ma} = 76$, and $Q = 2.5 \cdot 10^{-6}$, $R_0 = 1.75 \cdot 10^{-4}$. For the median filter (16), the number of used samples is equal to the length of the averaging window in (15), $N_{med} = N_{ma}$.

Fig. 1 depicts the estimation error transients of the considered methods in the ideal noise-free scenario. Denote the step number when the value $\phi_k$ overcomes the threshold $\varepsilon_\phi$ as $T_{\text{fix}}$, where $T_{\text{fix}} = 14$ in the considered example. The direct estimate (11) converges in the fixed time $T_{\text{fix}}$. The median filter also has the fixed-time convergence property with the convergence time $\min\{2T_{\text{fix}}, T_{\text{fix}} + [\frac{N_{\text{med}}}{2}]\}$. The moving-average filter converges in the fixed time $T_{\text{fix}} + N_{ma} - 1$, and other estimates converge asymptotically.

Fig. 2 depicts the estimation error transients of the considered methods for noisy measurements. The direct estimate exhibits sensitivity to the noise in the initial transient when the value of $\phi$ is small; this sensitivity can be reduced by increasing $\varepsilon_\phi$. Nevertheless, all considered methods perform well in alleviating the noise, where filtering of the direct estimate converges to a vicinity of the true value faster than the estimators applied to the linear regression model.

The steady-state performance of the considered methods is summarized in Table I, where the mean squared error (MSE) and the mean absolute error (MAE) are given; these values are computed over $10^6$ samples after the transients.

To summarize, under equivalent tuning, the direct estimate (8) provides fixed-time convergence, and followed by a filter

<figure><img src="image_2.png" /><figcaption>Fig. 2. Estimation error transients for noised measurements: the direct estimation $\hat{\theta}^{\text{dir}}$ (11), the gradient estimator $\hat{\theta}^{\text{gr}}$ (12), the least-squares estimator $\hat{\theta}^{\text{ls}}$ (13), the low-pass filtering $\hat{\theta}^{\text{low}}$ (14), the moving-average filtering $\hat{\theta}^{\text{ma}}$ (15), the median filtering $\hat{\theta}^{\text{med}}$ (16), the Kalman filter $\hat{\theta}^{\text{kal}}$ (17).</figcaption></figure>

provides parameter estimation with an improvement in the noise attenuation, and it should be considered as a reasonable alternative to the standard parameter estimation approaches applied to the element-wise scalar linear regression (4).

## VI. CONCLUSION

This work has established that the DREM procedure provides a direct parameter estimation by transforming the initial vector linear regression into one with unknown parameters measured with noise. We have also proved that Kreisselmeier's filters preserve the initial excitation of the regressor in the DREM method. Noise filters can be effectively coupled with DREM. In the future, we shall investigate other estimation algorithms with accelerated convergence rates in discrete-time, especially when it is impossible to obtain a sign-definite regressor after DREM or to improve robustness to measurement noise.

## REFERENCES

[1] L. Ljung, *System Identification: Theory for the User*. Upper Saddle River, NJ: Prentice-Hall, 1987.

[2] X. Yan, *Linear Regression Analysis: Theory and Computing*. Singapore: World Scientific, 2009.

[3] I. D. Landau, R. Lozano, M. M'Saad, and A. Karimi, *Adaptive control: algorithms, analysis and applications*. Springer Science & Business Media, 2011.

[4] G. Chowdhary, T. Yucelen, M. Mühlegg, and E. N. Johnson, "Concurrent learning adaptive control of linear systems with exponentially convergent bounds," *International Journal of Adaptive Control and Signal Processing*, vol. 27, no. 4, pp. 280–301, 2012.

[5] Y. Pan, S. Aranovskiy, A. Bobtsov, and H. Yu, "Efficient learning from adaptive control under sufficient excitation," *International Journal of Robust and Nonlinear Control*, vol. 29, no. 10, pp. 3111–3124, 2019.

[6] A. Goel, A. L. Bruce, and D. S. Bernstein, "Recursive least squares with variable-direction forgetting: Compensating for the loss of persistency [lecture notes]," *IEEE Control Systems Magazine*, vol. 40, no. 4, pp. 80–102, 2020.

[7] J. Wang, D. Efimov, S. Aranovskiy, and A. Bobtsov, "Fixed-time estimation of parameters for non-persistent excitation," *European Journal of Control*, 2019.

[8] R. Marino and P. Tomei, "On exponentially convergent parameter estimation with lack of persistency of excitation," *Systems & Control Letters*, vol. 159, p. 105080, 2022.

[9] M. Bin, "Generalized recursive least squares: Stability, robustness, and excitation," *Systems & Control Letters*, vol. 161, p. 105144, 2022.
[10] K. S. Narendra and A. M. Annaswamy, "Persistent excitation in adaptive systems," *International Journal of Control*, vol. 45, no. 1, pp. 127–160, 1987.

[11] D. Efimov and A. Fradkov, "Design of impulsive adaptive observers for improvement of persistency of excitation," *Int. J. Adaptive Control and Signal Processing*, vol. 29, no. 6, pp. 765–782, 2015.

[12] D. Efimov, N. Barabanov, and R. Ortega, "Robust stability under relaxed persistent excitation conditions," *International Journal of Adaptive Control and Signal Processing*, 2019.

[13] S. Aranovskiy, A. Bobtsov, R. Ortega, and A. Pyrkin, "Performance enhancement of parameter estimators via dynamic regressor extension and mixing," *IEEE Transactions on Automatic Control*, vol. 62, no. 7, pp. 3546–3550, 2017.

[14] N. Barroso, R. Ushirobira, D. Efimov, M. Sow, and J.-C. Massabuau, "Model-based adaptive filtering of harmonic perturbations applied to high-frequency noninvasive valvometry," in *IFAC World Congress*, Berlin, Germany, 2020.

[15] X. Shao, Q. Hu, Y. Shi, and B. Yi, "Data-driven immersion and invariance adaptive attitude control for rigid bodies with double-level state constraints," *IEEE Transactions on Control Systems Technology*, 2021.

[16] A. Vediakova, A. Vedyakov, A. Pyrkin, A. Bobtsov, and V. Gromov, "Finite time frequency estimation for multi-sinusoidal signals," *European Journal of Control*, vol. 59, pp. 38–46, 2021.

[17] J. Schiffer, P. Aristidou, and R. Ortega, "Online estimation of power system inertia using dynamic regressor extension and mixing," *IEEE Transactions on Power Systems*, vol. 34, no. 6, pp. 4993–5001, 2019.

[18] G. Kreisselmeier, "Adaptive observers with exponential rate of convergence," *IEEE Transactions on Automatic Control*, vol. 22, no. 1, pp. 2–8, feb 1977.

[19] D. N. Gerasimov and V. O. Nikiforov, "On key properties of the lion's and kreisselmeier's adaptation algorithms," in *Proc. 21st IFAC World Congress*, Berlin, Germany, 2020.

[20] P. A. Ioannou and J. Sun, *Robust Adaptive Control*. New Jersey: Prentice-Hall, 1996.

[21] S. Aranovskiy, R. Ushirobira, M. Korotina, and A. Vedyakov, "On preserving-excitation properties of kreisselmeiers regressor extension scheme," *IEEE Transactions on Automatic Control*, 2022.

[22] S. Aranovskiy, A. Belov, R. Ortega, N. Barabanov, and A. Bobtsov, "Parameter identification of linear time-invariant systems using dynamic regressor extension and mixing," *International Journal of Adaptive Control and Signal Processing*, vol. 33, no. 6, pp. 1016–1030, 2019.

[23] R. Ortega, V. Nikiforov, and D. Gerasimov, "On modified parameter estimators for identification and adaptive control. a unified framework and some new schemes," *Annual Reviews in Control*, vol. 50, pp. 278–293, 2020.