prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: What is the advantage of decorrelating the data before running the PLS algorithm?

Read the following text and answer briefly.

Context 1: Paper Info

Title: Efficient nonparametric estimation of Toeplitz covariance matrices
Publish Date: March 20, 2023
Author List: Karolina Klockmann (from Department of Statistics and Operations Research, Universität Wien), Tatyana Krivobokova (from Department of Statistics and Operations Research, Universität Wien)

Figure

Figure 1: Spectral density functions (first row) and autocovariance functions (second row) for examples 1, 2, 3 
Figure 2: Distance between the first atom and the first center of mass of aquaporin (left) and the opening diameter y t over time t (right) 
black line in the left plot) confirms that the covariance matrix estimated with our VST-DCT method almost completely decorrelates the channel diameter Y on the training data set Next, we estimated the regression coefficients β with the usual PLS algorithm, ignoring the dependence in the data Finally, we estimated β with PLS that takes into account dependence using our covariance estimator Σ Based on these regression coefficient estimators, the prediction on the test set was calculated The plot on the right side of Figure 2 shows the Pearson correlation between the true channel diameter on the test set and the prediction on the same test set based on raw (grey) and decorrelated data (black) 
Figure 3: On the left, the auto-correlation function of Y (grey) and of Σ−1/2 Y (black), where Σ is estimated with the VST-DCT method; On the right, correlation between the true values on the test data set and prediction based on partial least squares (in grey) and corrected partial least squares (black) 
Uniform distributionThe observations follow a uniform distribution with covariance matrices Σ 1 , Σ 2 , Σ 3 of examples 1, 2, 3, i e , Y i = Σ 1/2 j X i , j = 1, 3, with X 1 ,    the parameter innov of the R function arima sim is used to pass the innovations X 1 ,    , X n i i d Table4, 5 and 6 show respectively the results for (A) p = 5000, n = 1, (B) p = 1000, n = 50 and (C) p = 5000, n = 10 
(A) p = 5000, n = 1: Errors of the Toeplitz covariance matrix and the spectral density estimators with respect to the spectral and L 2 norm, respectively, as well as the average computation time of the covariance estimators in seconds for one Monte Carlo sample (last column) 
(C) p = 5000, n = 10: Errors of the Toeplitz covariance matrix and the spectral density estimators with respect to the spectral and L 2 norm, respectively, as well as the average computation time of the covariance estimators in seconds for one Monte Carlo sample (last column) 
(A) p = 5000, n = 1: Errors of the Toeplitz covariance matrix and the spectral density estimators with respect to the spectral norm and the L 2 norm, respectively Average computation time of the covariance estimators in seconds for one Monte Carlo sample (last column) 
(B) p = 1000, n = 50: Errors of the Toeplitz covariance matrix and the spectral density estimators with respect to the spectral norm and the L 2 norm, respectively Average computation time of the covariance estimators in seconds for one Monte Carlo sample (last column) 
(C) p = 5000, n = 10: Errors of the Toeplitz covariance matrix and the spectral density estimators with respect to the spectral norm and the L 2 norm, respectively Average computation time of the covariance estimators in seconds for one Monte Carlo sample (last column) 

abstract

A new nonparametric estimator for Toeplitz covariance matrices is proposed  This estimator is based on a data transformation that translates the problem of Toeplitz covariance matrix estimation to the problem of mean estimation in an approximate Gaussian regression  The resulting Toeplitz covariance matrix estimator is positive definite by construction, fully data-driven and computationally very fast 
Moreover, this estimator is shown to be minimax optimal under the spectral norm for a large class of Toeplitz matrices  These results are readily extended to estimation of inverses of Toeplitz covariance matrices  Also, an alternative version of the Whittle likelihood for the spectral density based on the Discrete Cosine Transform (DCT) is proposed 
The method is implemented in the R package vstdct that accompanies the paper 

Introduction

Estimation of covariance and precision matrices is a fundamental problem in statistical data analysis with countless applications in the natural and social sciences  Covariance matrices with a Toeplitz structure arise in the study of stationary stochastic n = 1, to the best of our knowledge, there is no fully data-driven approach for selecting the banding/tapering/thresholding parameter 
suggested first to split the time series into non-overlapping subseries and then apply the cross-validation criterion of   However, it turns out that the right choice of the subseries length is crucial for this approach, but there is no data-based method available for this  In this work, an alternative way to estimate a Toeplitz covariance matrix and its inverse is chosen 
Our approach exploits the one-to-one correspondence between Toeplitz covariance matrices and their spectral densities  First, the given data are transformed into approximate Gaussian random variables whose mean equals to the logarithm of the spectral density  Then, the log-spectral density is estimated by a periodic smoothing spline with a data-driven smoothing parameter 
Finally, the resulting spectral density estimator is transformed into an estimator for Σ or its inverse  It is shown that this procedure leads to an estimator that is fully data-driven, automatically positive definite and achieves the minimax optimal convergence rate under the spectral norm over a large class of Toeplitz covariance matrices 
In particular, this class includes Toeplitz covariance matrices that correspond to long-memory processes with bounded spectral densities  Moreover, the computation is very efficient, does not require iterative or resampling schemes and allows to apply any inference and adaptive estimation procedures developed in the context of nonparametric Gaussian regression 
Estimation of the spectral density from a stationary time series is a research topic with a long history  Earlier nonparametric methods are based on smoothing of the (log-)periodogram, which itself is not a consistent estimator   Another line of nonparametric methods for estimating the spectral density is based on the Whittle likelihood, which is an ap-proximation to the exact likelihood of the time series in the frequency domain 
For example, estimated the spectral density from a penalized Whittle likelihood, while used polynomial splines to estimate the log-spectral density function maximizing the Whittle likelihood  Recently, Bayesian methods for spectral density estimation have been proposed (see , but these may become very computationally intensive in large samples due to posterior sampling 
The minimax optimal convergence rate for nonparametric estimators of Hölder continuous spectral densities from Gaussian stationary time series was obtained by under the L p , 1 ≤ p ≤ ∞, norm  Only few works on spectral density estimation show the optimality of the corresponding estimators  In particular, and derived convergence rates of their estimators for the log-spectral density under the L 2 norm, while neglecting the Whittle likelihood approximation error 
In general, most works on spectral density estimation do not exploit further the close connection to the corresponding Toeplitz covariance matrix estimation  In particular, an upper bound for the L ∞ risk of a spectral density estimator automatically provides an upper bound for the risk of the corresponding Toeplitz covariance matrix estimator under the spectral norm 
This fact is used to establish the minimax optimality of our nonparametric estimator for the Toeplitz covariance matrices  The main contribution of this work is to show that our proposed spectral density estimator is not only numerically very efficient, but also achieves the minimax optimal rate in the L ∞ norm, which in turn ensures the minimax optimality of the corresponding Toeplitz covariance matrix estimator 
The paper is structured as follows  In Section 2, the model is introduced and ap-proximate diagonalization of Toeplitz covariance matrices with the discrete cosine transform is discussed  Moreover, an alternative version of the Whittle's likelihood is proposed  In Section 3, new estimators for the Toeplitz covariance matrix and the precision matrix are derived, while in Section 4 their theoretical properties are presented 
Section 5 contains simulation results, Section 6 presents a real data example, and Section 7 closes the paper with a discussion  The proofs are given in the appendix to the paper 

Set up and diagonalization of Toeplitz matrices

Let Y 1 ,       , Y n i i d  ∼ N p (0 p , Σ), where Σ is a (p × p)-dimensional positive definite covariance matrix with a Toeplitz structure, that is, Σ = {σ |i−j| } p i,j=1 0  The sample size n may tend to infinity or to be a constant  The case n = 1 corresponds to a single observation of a stationary time series, and in this case the data are simply denoted by Y ∼N p (0 p , Σ) 
The dimension p is assumed to grow  The spectral density function f , corresponding to a Toeplitz covariance matrix Σ, is given by so that for f ∈ L 2 (−π, π) the inverse Fourier transform implies Hence, Σ is completely characterized by f , and the non-negativity of the spectral density function implies the positive definiteness of the covariance matrix 
Moreover, the decay of the autocovariance σ k is directly connected to the smoothness of f   Finally, the convergence rate of a Toeplitz covariance estimator and that of the corresponding spectral density estimator are directly related via Σ ≤ f ∞ := sup x∈ |f (x)|, where • denotes the spectral norm (see  
As in , we introduce a class of positive definite Toeplitz covariance matrices with Hölder continuous spectral densities  For β = γ + α > 0, where The optimal convergence rate for estimating Toeplitz covariance matrices over P β (M 0 , M 1 ) depends crucially on β  It is well known that the k-th Fourier coefficient of a function whose γ-th derivative is α-Hölder continuous decays at least with order O(k −β ) (see  
Hence, β determines the decay rate of the autocovariances σ k , which are the Fourier coefficients of the spectral density f , as k → ∞  In particular, this implies that for β ∈ (0, 1], the class P β (M 0 , M 1 ) includes Toeplitz covariance matrices corresponding to long-memory processes with bounded spectral densities, since the sequence of corresponding autocovariances is not summable 
A connection between Toeplitz covariance matrices and their spectral densities is further exploited in the following lemma  Lemma 1  Let Σ ∈ P β (M 0 , M 1 ) and let x j = (j − 1)/(p − 1), j = 1,    , p, then where δ i,j is the Kroneker delta, O(•) terms are uniform over i, j = 1,       , p and divided by √ 2 when i, j ∈ {1, p} is the Discrete Cosine Transform I (DCT-I) matrix 
The proof can be found in Appendix A 1  This result shows that the DCT-I matrix approximately diagonalizes Toeplitz covariance matrices and that the diagonalization error depends to some extent on the smoothness of the corresponding spectral density  In the spectral density literature the discrete Fourier transform (DFT) matrix
, where i is the imaginary unit, is typically employed to approximately diagonalize Toeplitz covariance matrices  Using the fact that introduced an approximation for the likelihood of a single Gaussian stationary time series (case n = 1), the so-called Whittle likelihood (1) The quantity , where F j denotes the j-th column of F , is known as the periodogram at the j-th Fourier frequency 
Note that due to periodogram symmetry, only p/2 data points I 1 ,    , I p/2 are available for estimating the mean f (2πj/p), j = 1,       , p/2 , where x denotes the largest integer strictly smaller than x  The Whittle likelihood has become a popular tool for parameter estimation of stationary time series, e g , for nonparametric and parametric spectral density estimation or for estimation of the Hurst exponent, see e g , ;  
Lemma 1 yields the following alternative version of the Whittle likelihood where W j = (D t j Y ) 2   Note that this likelihood approximation is based on twice as many data points W j as the standard Whittle likelihood  Thus, it allows for a more efficient use of the data Y to estimate the parameter of interest, such as the spectral density or the Hurst parameter 
Equations ( ) or (2) invite for the estimation of f by maximizing the (penalized) likelihood over certain linear spaces (e g , spline spaces), as suggested e g , in or   However, such an approach requires well-designed numerical methods to solve the corresponding optimization problem, since the spectral density in the second term of (1) or ( ) is in the denominator, which does not allow to obtain a closed-form expression for the estimator and often leads to numerical instabilities 
Also, the choice of the smoothing parameter becomes challenging  Therefore, we suggest an alternative approach that allows the spectral density to be estimated as a mean in an approximate Gaussian regression  Such estimators have a closed-form expression, do not require an iterative optimization algorithm and a smoothing parameter can be easily obtained with any conventional criterion 
First Hence, for W j = (D t j Y ) 2 , j = 1,       , p it follows with Lemma 1 that where Γ(a, b) denotes a gamma distribution with a shape parameter a and a scale parameter b  Note that the random variables W 1 ,       , W p are only asymptotically independent  Obviously, E(W j ) = f (πx j ) + O(1), j = 1,      
, p  To estimate f from W 1 ,       , W p , one could use a generalized nonparametric regression framework with a gamma distributed response, see e g , the classical monograph by   However, this approach requires an iterative procedure for estimation, e g , a Newton-Raphson algorithm, with a suitable choice for the smoothing parameter at each iteration step 
Deriving the L ∞ rate for the resulting estimator is also not a trivial task  Instead, we suggest to employ a variance stabilizing transform of that converts the gamma regression into an approximate Gaussian regression  In the next section we present the methodology in more detail for a general setting with n ≥ 1 

Methodology

For Y i ∼ N p (0 p , Σ), i = 1,       , n, it was shown in the previous section that with Lemma 1 the data can be transformed into gamma distributed random variables     , n, j = 1,       , p, where for each fixed i the random variable W i,j has the same distribution as W j given in (3)  Now the approach of Cai et al  ( ) is adapted to the setting n ≥ 1 
First, the transformed data points W i,j are binned, that is, fewer new variables     , T   Note that the number of observations in a bin is m = np/T   In Theorem 1 in Section 4, we show that setting T = p υ for any υ ∈ ((4 − 2 min{β, 1})/3, 1) leads to the minimax optimal rate for the spectral density estimator 
To simplify the notation, m is handled as an integer (otherwise, one can discard several observations in the last bin)  Next, applying the variance stabilizing transform (VST) ∼ where H(y) = {φ(m/2) + log (2y/m)} / √ 2 and φ is the digamma function (see   Now, the scaled and shifted log-spectral density H(f ) can be estimated with a periodic smoothing spline
where h > 0 denotes a smoothing parameter, q ∈ N is the penalty order and S per (2q − 1) a space of periodic splines of degree 2q − 1  The smoothing parameter h can be chosen either with generalized cross-validation (GCV) as derived in or with the restricted maximum likelihood, see   Once an estimator H(f ) is obtained, application of the inverse transform function H −1 (y) = m exp √ 2y − φ (m/2) /2 yields the spectral density estimator f = H −1 H(f )  
Finally, using the inverse Fourier transform leads to the fol- The precision matrix Ω is estimated by the inverse Fourier transform of the reciprocal of the spectral density estimator, i e , Ω = (ω |i−j| ) p i,j=1 with ωk = The estimation procedure for Σ and Ω can be summarised as follows  1  Data Transformation:
where D is the (p × p)-dimensional DCT-I matrix as given in Lemma 1 and D j is its j-th column  2  Binning: Set T = p υ for any υ ∈ ((4 − 2 min{β, 1})/3, 1) and calculate W i,j , k = 1,       , T 

VST:

where k are asymptotically i i d  Gaussian variables  Inverse VST: Estimate the spectral density f with f = H −1 H(f ) , where Note that Σ and Ω are positive definite matrices by construction, since their spectral density functions f and f −1 are non-negative, respectively  Unlike the banding and tapering estimators, the autocovariance estimators σk are controlled by a single smoothing parameter h, which can be estimated fully data-driven with several available automatic methods, which are numerically efficient and well-studied 
In addition, one can also use methods for adaptive mean estimation, see e g , , which in turn leads to adaptive Toeplitz covariance matrix estimation  All inferential procedures developed in the Gaussian regression context can also be adopted accordingly 

Theoretical Properties

In this section, we study the asymptotic properties of the estimators f , Σ and Ω  The results are established under the asymptotic scenario where p → ∞ and p/n → c ∈ (0, ∞], that is, the dimension p grows, while the sample size n either remains fixed or also grows but not faster than p  This corresponds to the asymptotic scenario when the sample covariance matrix is inconsistent 
Let f be the spectral density estimator defined in Section 3, i e , f = m exp{ √ 2 H(f ) − φ(m/2)}/2, where H(f ) is given in (4), m = np/T and φ is the digamma function  Furthermore, let Σ be the Toeplitz covariance matrix estimator and Ω the corresponding precision matrix defined in equations ( ) and (6), respectively 
The following theorem shows that both Σ and Ω attain the minimax optimal rate of convergence over the class and hT → ∞, then with T = p υ for any υ ∈ ((4 − 2 min{β, 1})/3, 1) and q = max{1, γ}, the spectral density estimator f , the corresponding covariance matrix estimator Σ and the precision matrix estimator Ω satisfy sup
For h {log(np)/(np)}   The proof of Theorem 1 can be found in the Appendix A 3 and is the main result of our work  The most important part of this proof is the derivation of the convergence rate for the spectral density estimator f under the L ∞ norm  In the original work, established an L 2 rate for a wavelet nonparametric mean estimator in a gamma regression where the data are assumed to be independent 
In our work, the spectral density estimator f is based on the gamma distributed data W i,1 ,       , W i,p , which are only asymptotically independent  Moreover, the mean of these data is not exactly f (πx 1 ),       , f (πx p ), but is corrupted by the diagonalization error given in Lemma 1  This error adds to the error that arises via binning and VST and that describes the deviation from a Gaussian distribution, as derived in  
Finally, we need to obtain an L ∞ rather than an L 2 rate for our spectral density estimator  Overall, the proof requires different tools than those used in   To get the L ∞ rate for f , we first derive that for the periodic smoothing spline estimator H(f ) of the log-spectral density  To do so, we use a closed-form expression of its effective kernel obtained in , thereby carefully treating various (dependent) errors that describe deviations from a Gaussian nonparametric regression with independent errors and mean f (πx i ) 
Note also that although the periodic smoothing spline estimator is obtained on T binned points, the rate is given in terms of the vector dimension p  Then, using the Cauchy-Schwarz inequality and a mean value argument, this rate is translated into the L ∞ rate for the spectral density estimator f   To obtain the rate for the Toeplitz covariance matrix estimator is enough to note that

Simulation Study

In this section, we compare the performance of the proposed Toeplitz covariance estimator, denoted as VST-DCT, with the tapering estimator of and with the sample covariance matrix  We consider Gaussian vectors Y 1 ,    , (3) such that the corresponding spectral density is Lipschitz continuous but not differentiable: f (x) = 1 44{| sin(x + 0 5π)| 1 7 + 0 45} 
In particular, var(Y i ) = 1 44 in all three examples  Figure shows the spectral densities and the corresponding autocorrelation functions for the three examples  A Monte Carlo simulation with 100 iterations is performed using R (version 4 1 2, seed 42)  For our VST-DCT estimator, we use a cubic periodic spline, i e , q = 2 is set in (4) 
The binning parameters are set to T = 500 bins with m = 10 points for (A) and T = 500 bins with m = 100 points for both (B) and (C)  To select the regularisation parameter for our estimator, we implemented the restricted maximum likelihood (ML) method, generalized cross validation (GCV) and the corresponding oracle versions, i e , as if Σ were known The tapering parameter
where T ap k (Σ ν 2 ) is the tapering estimator of with parameter k  If n = 1, that is, under scenario (A), suggest to split the time series Y into l non-overlapping subseries of length p/l and then proceed as before to select the tuning parameter k  To the best of our knowledge, there is no data-driven method for selecting this parameter l 
Using the true covariance matrix Σ, we selected l = 30 subseries for the example 1 and l = 15 subseries for the exam-ples 2 and 3  The parameter k can then be chosen by cross-validation as above  We employ this approach under scenario (A) instead of an unavailable fully data-driven criterion and name it semi-oracle 
Finally, for all three scenarios (A), (B) and (C), the oracle tapering parameter is computed using grid search for each Monte Carlo sample as kor = arg min k=2,3,   ,p/2 T ap k ( Σ) − Σ , where Σ is the sample covariance matrix  To speed up the computation, one can replace the spectral norm by the 1 norm, as suggested by  
In Tables , the errors of the Toeplitz covariance estimators with respect to the spectral norm and the computation time for one Monte Carlo iteration are given for scenarios (A), (B) and (C), respectively  To illustrate the goodness-of-fit of the spectral density, the L 2 norm f − f 2 is also computed 
The results show that the tapering and VST-DCT estimator perform overall similar in terms of the spectral norm risk  This is not surprising as both estimators are proved to be rate-optimal  Moreover, both the tapering and VST-DCT estimators are clearly superior to the inconsistent sample Toeplitz covariance matrix 
A closer look at the numbers shows that the VST-DCT method has better constants, i e , VST-DCT estimators have somewhat smaller errors in the spectral norm than the tapering estimators across all examples, but especially under scenario (C)  The oracle estimators show similar behaviour, but are slightly less variable compared to the data-driven estimators 
In general, both the tapering and VST-DCT estimators perform best for example 1, second best for example 3 and worst for example 2, which traces back to functions complexity  In terms of computational time, both methods are similarly fast for scenarios (A) and (B)  For scenario (C), the tapering method is much slower due to the multiple high-dimensional matrix multiplications in the cross-validation method 
It is expected that for larger p the tapering estimator is much more computationally intensive than the corresponding VST-DCT estimator  (1) polynomial σ ( (  To test how robust our approach is to deviations from the Gaussian assumption, we simulated the data from gamma and uniform distributions and conducted a simulation study for the same scenarios and examples 
The results are very similar to those of the Gaussian distribution, see supplementary materials for the details 

Application to Protein Dynamics

We revisit the data analysis of protein dynamics performed in Krivobokova et al  (2012) and   We consider data generated by the molecular dynamics (MD) simulations for the yeast aquaporin (Aqy1) -the gated water channel of the yeast Pichi pastoris  MD simulations are an established tool for studying biological systems at the atomic level on timescales of nano-to microseconds 
The data are given as Euclidean coordinates of all 783 atoms of Aqy1 observed in a 100 nanosecond time frame, split into 20 000 equidistant observations  Additionally, the diameter of the channel y t at time t is given, measured by the distance between two centers of mass of certain residues of the protein 
The aim of the analysis is to identify the collective motions of the atoms responsible for the channel opening  In order to model the response variable y t , which is a distance, based on the motions of the protein atoms, we chose to represent the protein structure by distances between atoms and certain fixed base points instead of Euclidean coordinates 
That is, we calculated where A t,i ∈ R 3 , i = 1,       , 783 denotes the i-th atom of the protein at time t, B j ∈ R 3 , j = 1, 2, 3, 4, is the j-th base point and d(•, •) is the Euclidean distance  Figure shows the diameter y t and the distance between the first atom and the first center of mass  It can therefore be concluded that a linear model Y = Xβ + holds, where
  This linear model has two specific features which are intrinsic to the problem: first, the observations are not independent over time and second, X t is high-dimensional at each t and only few columns of X are relevant for Y   have shown that the partial least squares (PLS) algorithm performs exceptionally well on this type of data, leading to a small-dimensional and robust representation of proteins, which is able to identify the atomic dynamics relevant for Y  
Singer et al  ( ) studied the convergence rates of the PLS algorithm for dependent observations and showed that decorrelating the data before running the PLS algorithm improves its performance  Since Y is a linear combination of columns of X, it can be assumed that Y and all columns of X have the same correlation structure 
Hence, it is sufficient to estimate Σ = cov(Y ) to decorrelate the data for the PLS algorithm, i e , Σ −1/2 Y = Σ −1/2 Xβ + Σ −1/2 results in a standard linear regression with independent errors  Our goal now is to estimate Σ and compare the performance of the PLS algorithm on original and decorrelated data 
For this purpose, we divided the data set into a training and a test set (each with p = 10 000 observations)  First, we tested whether the data are stationary  The augmented Dickey-Fuller test confirmed stationarity for Y with a p-value< 0 01  The Hurst exponent of Y is 0 85, indicating moderate long-range dependence supported by a rather slow decay of the sample autocovariances (see grey line in the left plot of Figure ) 
Therefore, we set q = 1 for the VST-DCT estimator to match the low smoothness of the corresponding spectral density  Moreover, the smoothing parameter is selected with the restricted maximum likelihood method and T = 550 bins are used  Obviously, the performance of the PLS algorithm on the decorrelated data is significantly better for the small number of components 
In particular, with just one PLS component, the correlation between the true opening diameter on the test set and its prediction that takes into account the dependence in the data is already 0 54, while it is close to zero for PLS that ignores the dependence in the data  showed that the estimator of β based on one PLS component is exactly the ensemble-weighted maximally correlated mode (ewMCM), which is defined as the collective mode of atoms that has the highest probability to achieve a specific alteration of the response Y  
Therefore, an accurate estimator of this quantity is crucial for the interpretation of the results and can only be achieved if the dependence in the data is taken into account  Estimating Σ with a tapered covariance estimator has two practical problems  First, since we only have a single realization of a time series Y (n = 1), there is no datadriven method for selecting the tapering parameter 
Second, the tapering estimator turned out not to be positive definite for the data at hand  To solve the second problem, we truncated the corresponding spectral density estimator ftap to a small positive value, i e , f + tap = max{ ftap , 1/ log(p)} (see   To select the tapering parameter with cross-validation, we experimented with different subseries lengths and found that the tapering estimator is very sensitive to this choice 
For example, estimating the tapered covariance matrix based on subseries of length 8/15/30 yields a correlation of 0 42/0 53/0 34 between the true diameter and the first PLS component, respectively  Altogether, our proposed estimator is fully data-driven, fast even for large sample sizes, automatically positive definite and can handle certain long-memory processes 
In contrast, the tapering estimator is not data-driven and must be manipulated to become positive definite  Our method is implemented in the R package vstdct 

Discussion

In this paper, we proposed a simple, fast, fully data-driven, automatically positive definite and minimax optimal estimator of Toeplitz covariance matrices from a large class that also includes covariance matrices of certain long-memory processes  Our estimator is derived under the assumption that the data are Gaussian 
However, simulations show that the suggested approach yields robust estimators even when the data are not normally distributed  In the context of spectral density estima- , for mixing processes (see Theorem 5 3 of Rosenblatt, 2012), as well as for non-linear processes (see   Since DFT and DCT matrices are closely related, we expect that equation (3) also holds asymptotically for these non-Gaussian time series, but consider a rigorous analysis to be beyond the scope of this paper 
In fact, our numerical experiments have even shown that if the spectral density is estimated from W j = f (πx j ) + j , that is, as if W j were Gaussian instead of gamma distributed, then the resulting spectral density estimator has almost the same L ∞ risk (and hence the corresponding covariance matrix has almost the same spectral norm) 
Of course, such an estimator would lead to a wrong inference about f (πx j ), since the growing variance of W j would be ignored  Since our approach translates Toeplitz covariance matrix estimation into a mean estimation in an approximate Gaussian nonparametric regression, all approaches developed in the context of Gaussian nonparametric regression, such as (locally)
adaptive estimation, as well as the corresponding (simultaneous) inference, can be directly applied  Bayesian tools for adaptive estimation and inference in Gaussian nonparametric regression as proposed in can also be employed 

Appendix

Throughout the appendix, we denote by c, c 1 , C, C 1 ,       etc  generic constants, that are independent of n and p  To simplify the notation, the constants are sometimes skipped and we write for less than or equal to up to constants  We embed the p-dimensional Toeplitz matrix Σ = toep(σ 0 ,       , σ p−1 ) in a (2p − 2)dimensional circulant matrix Σ = toep(σ 0 ,      
, σ p−1 , σ p−2 ,       , σ 1 )  Then, Σ = with the conjugate transpose U * , and Λ is a diagonal matrix with the k-th diagonal value for k = 1,    , p given by Furthermore, Σ = V * ΛV , where V ∈ C (2p−2)×p contains the first p columns of U   In particular, b(j, r) = b(j, 2p−r) and c(j, r) = −c(j, 2p−r) for r = p+1,      
, 2p−2  Together, we have (A 1) Some calculations show that for r = 1,       p Using the Taylor expansion of cot(x) for 0 < |x| < π one obtains for r = 1,       p where the O term does not depend on j and the hidden constant does not depend on r, p  If i = j, equations (A 1) -(A 3) imply where the O terms do not depend on j 
Since the complex exponential function is Lipschitz continuous with constant L = 1, it holds λ r = λ j + L r,j |r − j|p −1 where −1 ≤ L r,j ≤ 1 is a constant depending on r, j  Then, , it is sufficient to consider j = 1,    , p − 1  We begin with first sum  For a shorter notation, we use k := r − 1 and l := j − 1 in the following 
Then, summing the squares of the first term in (A 4) for l = 0,    , p−2 on sums of reciprocal powers  If p is even, then the residual terms are given by where φ and φ (1) denote the digamma function and its derivative  If p is odd, similar remainder terms can be derived  To see that R i (l, p) = O(p −1 ) for i = 1, 2, 3 and uniformly in l we use that asymptotically φ(x)∼ log(x)−1/(2x) and
The mixed term are both of the order p −1   Furthermore, since the harmonic sum diverges at a rate of log(p)  Finally, λ j = f (x j )+O{log(p)p −β } by the uniform approximation properties of the discrete Fourier series for Hölder continuous functions (see   All together, we have shown that (DΣD) j,j = where the O terms are uniform over j = 1,    , p 
Case i = j and |i − j| is even In this case, (DΣD) i,j = a i a j uniformly in i, j  To show that a i a j 2p−2 r=1 λ r c(i, r)c(j, r) = O(p −1 ), we proceed similarly as before  Setting k=r−1, l=j−1, m=i−1 and using that l =m and |l−m| is even, one obtains where for even p the residual terms are given by If p is odd, analogous residual terms can be derived 
Using similar techniques as before, one can show that the two residual terms and the remaining mixed and square terms vanish at a rate of the order O(p −1 ) and uniformly in i, j  Case i = j and |i − j| is odd |r − i| and |r − j| are either odd and even, or even and odd  Without loss of generality, assume that |r − i| is even 
Then, (DΣD) i,j = a i a j 2p−2 r=1 λ r b(i, r)c(j, r)  Since b(i, •) is an even function, c(j, •) is an odd function and λ r = λ 2p−r , it follows (DΣD) i,j = 0  The structure of the proof is as follows  First, we derive the L ∞ rate of the periodic smoothing spline estimator H(f )  Then, using the Cauchy-Schwarz inequality and a mean value argument, the convergence rate of the spectral density estimator f is
∞ the first claim of the theorem follows  Finally, we prove the second statement on the precision matrices  For the sake of clarity, some technical lemmas used in the proof are listed separately in A 4  hT → ∞, then with T = p υ for any υ ∈ ((4 − 2 min{1, β})/3, 1), the estimator H(f ) described in Section 3 with q = max{1, γ} satisfies
Proof : Application of the triangle inequality yields a bias-variance decomposition Set T = 2T − 2 and x k = (k − 1)/ T for k = 1,    , T   Using Lemma 4, we can write where Mirroring and renumerating ζ k , η k , k is similar as for Y * k , k = 1,    , T   Using the above representation, one can write First we reduce the supremum to a maximum over a finite number of points 
If q > 1, then W (•, x k ) is Lipschitz continuous with constant L > 0  In this case, it holds almost surely that sup ) is a piecewise linear function with knots at x j = j/ T   The factor (ζ k + ξ k ) can be considered as stochastic weights that do not affect the piecewise linear property  Thus, the supremum is attained at one of the knots x j = j/ T , j = 1,    , T , and (A 7) is also valid for q = 1 
Again with (a + b) 2 ≤ 2a 2 + 2b 2 we obtain We start with bounding   This requires a bound on 1 • ψ 2 denotes the sub-Gaussian norm  In case of a Gaussian random variable the norm equals to the variance  Thus with Lemma 2 and Lemma 4, we obtain Lemma 1 6 of ) then yields Recall that T = p υ for some fixed υ ∈ ((4 − 2 min{1, β})/3, 1) 
Using the inequality log(x) ≤ x a /a one can find constants x υ , C υ > 0 depending on υ but not on n, p such that log(2 T ) log(p) Next, we derive a bound for the second term The exponential decay property of the kernel K stated in Lemma 2 yields The first term in (A 9) can be bounded again with Lemma 1 6 of  
We use the fact that for not necessarily independent random variables X 1 ,    , X N R and R > 0 are constants  This is a consequence of Lemma 1 of which yields , it follows that N i=1 a i X i has a subGaussain distribution and the subGaussian norm is bounded by 2R( N i=1 a 2 i ) 1/2   See for further details on the subgaussian distribution 
T h   For the second inequality Lemma 2(ii) is used  Applying Lemma 1 6 of then yields To bound the second term in (A 9), we use the moment bounds for ξ k derived in Lemma 4  Then, for all integers > 1 Combining the error bounds (A 10) and (A 11) and choosing R=m −1/2 gives By assumption T = p υ and m = np (1−υ) for some fixed υ ∈ ((4 − 2 min{1, β})/3, 1) 
If is an integer such that ≥ 1/(1 − υ), then where we used log(x) ≤ x a /a with a = 1/(4 )  Consider 1/2 < β ≤ 1 and let 0 < χ < 1 be a constant  Applying log(x) ≤ x a /a twice with a = χ/(2 ) yields For any fixed υ∈((4 − 2 min{1, β})/3, 1) one can find an integer which is independent of n, p such that the right side of (A 12) holds 
Since p/n → c ∈ (0, ∞] and thus n/p = O(1) and p −1 = O(n −1 ), it follows for satisfying (A 12) that In total, choosing an integer Using the representation in Lemma 4 once more gives for each x ∈ [0, 1] The bounds on k in Lemma 4 imply Consider the case that β ≥ 1  In particular, q = γ and f (q) is α-Hölder continuous 
Since f is a periodic function with f (x) ∈ [δ, M 0 ] and H(y) ∝ φ(m/2)+ log (2y/m), it follows that {H(f )} (q) is also α-Hölder continuous  Extending g := H(f ) to the entire real line, we get Expanding g(t) in a Taylor series around x and using that h −1 K h is a kernel of order 2q, see Lemma 2(iii), it follows that for any x ∈ [0, 1]
where ξ x,t is a point between x and t  Using the fact that the kernel K h decays exponentially and that g (q) is α-Hölder continuous on [δ, M 0 ] with some constant L, the logarithm is Lipschitz continuous on a compact interval, it follows g = H(f ) is β-Hölder continuous  Expanding g to the entire line and using Lemma 2(iii) with
In a similar way as before, one obtains Note that T −β =o(h β ) as β > 1/2, T h → ∞ and h → 0 by assumption  Since the derived bounds are uniform for x ∈ [0, 1] it holds Putting the bounds A 13 and A 14 together gives If h > 0 such that h → 0 and hT → ∞, then with T = p υ for any υ ∈ ((4 − 2 min{1, β})/3, 1), the estimator f described in Section 3 with q = max{1, γ} satisfies
Proof : By the mean value theorem, it holds for some function g between H(f ) and To show that the second term on the right hand side of (A 15) is negligible we use the moment generating function of H(f ) ∞   In the next paragraph, we derive the asymptotic order of E[exp{λ H(f ) ∞ }] for n, p → ∞, where λ > 0 may depend on n, p or not 
By the exponential decay property of the kernel K stated in Lemma 2 holds First, H(f ) ∞ is bounded with the maximum over a finite number of points  Calculating the derivative of s : Since δ δx s(x) > 0 almost surely for x = x k , the extrema occur at x k , k = 1,    , T   Thus, for λ > 0 the moment generating function of H(f ) ∞ is bounded by
Let M j = ( T h) −1 T k=1 γ h (x j , x k ), which by Lemma 2 is bounded uniformly in j by some global constant M > 0  By the convexity of the exponential function we obtain √ 2 and by assumption 0 ≤ δ ≤ f ≤ M 0   Using Lemma 3, Q k can be written as a sum of m = np/T independent gamma random variables, i e 
The moment generating function of | log(X)| when X follows a Γ(a, b)-distribution is given by where Γ(a) is the gamma function and γ(a, b) is the lower incomplete gamma function  In particular, To derive the asymptotic order of E[exp{λ H(f ) ∞ }] for n, p → ∞ we first establish the asymptotic order of the ratio Γ(a + t)/Γ(a) for a → ∞ 
We distinguish the two cases where t is independent of a and where t linearly depends on a  Thus, for 0 < t < a and t independent of a, equation (A 17) implies for a → ∞ that Γ(a + t)/Γ(a) = O(a t )  Similarly, it can be seen that Γ(a − t)/Γ(a) = O(a −t )  If 0 < t < a and t linearly depends on a, i e  t = ca for some constant c ∈ (0, 1), then we get Γ(a ± t)/Γ(a) = O(a ±t exp{a}) for a → ∞ 
Hence, for a fixed λ not depending on n, p and such that 0 < λ < m/( √ 2M j ) we get for sufficiently large n, p If λ = cm such that 0 < λ < m/( √ 2M j ), then for sufficiently large n, p b∈{cδ/m,cM 0 /m} (bm/2) for some constant L > 1  Set K = min j=1,   , T 1/( √ 2M j ) which is a constant independent of n, p  Altogether, we showed that for 0 < λ < Km and n, p → ∞
Bounding the right hand side of (A 15) for some constants c 0 , c 1 > 0 and n, p → ∞ Since g lies between H(f ) and H(f ), and f almost surely pointwise  Thus, for C > f ∞ = M 0 it holds where c 1 := H(C − M 0 )  Applying Markov inequality for t = cm with c ∈ (0, K) and C = 2L 4/c + M 0 where c, K, L are the constants in gives
Together with Proposition 1 follows Using the fact that the spectral norm of a Toeplitz matrix is upper bounded by the sup norm of its spectral density we get sup According to the mean value theorem, for a function g between H(f ) and H(f ), it holds that some constant c 1 > 0 not depending on n, p  Chosing the same constant C as in section A 3 2 it follows
Noting that 1/f ∞ ≤ 1/δ and 2/m exp {φ(m/2)} ∈ [0 25, 1] for m ≥ 1, (A 18) implies for some constants c 2 , c 3 > 0 and n, p → ∞ Since the derived bounds hold for each Σ(f ) ∈ F β , we get all together sup This section states some technical lemmata needed for the proof of Theorem 1  The proofs can be found in the supplementary material 
The first lemma lists some properties of the kernel K h and its extension K h on the real line  The proof is based on   Lemma 2  Let h > 0 be the bandwith parameter depending on N   (i) There are constants 0 < C < ∞ and 0 < γ < 1 such that for all for x, t ∈ [0, 1] Lemma 3 states that the sum of the correlated gamma random variables in each bin can be rewritten as a sum of independent gamma random variables 
for i = 1,    , n and j = (k − 1)m + 1,    , km, and x j = (j − 1)/(2p − 2)  Finally, Lemma 4 gives explicit bounds for the stochastic and deterministic errors of the variance stabilizing transform  Thus, it quantifies the difference to an exact Gaussian regression setting  This result is a generalization of Theorem 1 of Cai et al 
(2010) adapted to our setting with n ≥ 1 observations and correlated observations  √ 2 can be written as where for the proof of the first statement  Furthermore, for x, t ∈ [0, 1] holds In particular, for some constants C 1 , C 2 > 0 depending on γ ∈ (0, 1) but not on h and x, it holds h (iii) See Lemma 15 of with p = 2q − 1 
It is sufficient to show the statement for n = 1 by independence of the Y i   Then, the number of points per bin is m = p/T   For simplicity, the index i is skipped in the following  First, we write Q k as a matrix-vector product and refactor it so that it corresponds to a sum of independent scaled χ 2 random variables 
In the second step, we calculate the scaling factors  Let E (km) be a diagonal matrix with ones on the (k − 1)m + 1,    , km-th entries and otherwise zero diagonal elements  Then, By Theorem 1 of for the gamma distribution it follows where Wi,j iid  ∼ Γ(1/2, 2 f (x * k )) and such that Cov( Wi,j , Wi,h ) = Cov(W i,j , W i,h ) for j = (k − 1)p/T + 1,    , kp/T and h ∈ {1,    , p} \ {(k − 1)p/T + 1,    , kp/T } 
Let θ be the maximum difference of the observations' means in each bin  Then, θ = max are defined via quantile coupling, it holds Z k = Φ −1 {F Q( Qk )} (see   Furthermore, define the uniform random variables Let ρ = Cov(Z k , Z l )  Then, the identity implies F Z,Z (x, y) − Φ(x)Φ(y) ≥ 0 for all x, y ∈ R ⇐⇒ ρ ≥ 0, (see  
Since Cov( Qk , Ql ) ≥ 0 and the ratio of two densities is non-negative, x = − 2/m, it follows that f Q(x) is monotone decreasing for x ≥ − 2/m  Furthermore, F Q(− m/2) ≤ 0 5 for all m ∈ N as f Q(x) is right-skewed  In particular, − m/2 ≤ F −1 Q (1/2) for all m ∈ N  Finally, since f Q(− 2/m) → φ(0) for m → ∞ there is a constant c > 0 not depending on m such that
The simulation study in Section 5 is performed in the same way, but with the uniform and the gamma distribution instead of the Gaussian distribution 
Context 3: \section{Model equations} \label{sec:equations}

In drift-fluid models the continuity equation
\begin{align}
 \frac{\partial n}{\partial t} + \nabla\cdot\left( n \vec u_E  \right) &= 0 \label{eq:generala} 
\end{align}
describes the dynamics of the electron density $n$  Here
$\vec u_E := (\hat{\vec b} \times \nabla \phi)/B$ gives the electric drift
velocity in a magnetic field $\vec B := B \hat{\vec b}$ and an electric
potential $\phi$  We neglect contributions of the diamagnetic drift~\cite{Kube2016} 




Equation~\eqref{eq:generala} is closed by invoking quasineutrality, i e  the divergence of the ion polarization, 
the electron diamagnetic and the gravitational drift currents must vanish
\begin{align}
  \nabla\cdot\left( \frac{n}{\Omega} \left( \frac{\partial}{\partial t} 
  + \vec u_E \cdot\nabla  \right)\frac{\nabla_\perp \phi}{B}  + n\vec u_d - n\vec u_g\right) &=0
    
 
 
  \label{eq:generalb}
\end{align}
Here we denote 
$\nabla_\perp\phi/B := - \hat{\vec b} \times \vec u_E$, 
the electron diamagnetic drift
$\vec u_d := - T_e(\hat{\vec b} \times\nabla n ) /enB$
with the electron temperature $T_e$,
the ion gravitational drift velocity  
$\vec u_g := m_i \hat{\vec b} \times \vec g /B$
with ion mass $m_i$, and the ion gyro-frequency
$\Omega := eB/m_i$ 

Combining Eq ~\eqref{eq:generalb} with Eq ~\eqref{eq:generala} yields
\begin{align}
 \frac{\partial \rho}{\partial t} + \nabla\cdot\left( \rho\vec u_E \right) + \nabla \cdot\left( n(\vec u_\psi + \vec u_d + \vec u_g) \right) &= 0\label{eq:vorticity}
\end{align}
with the polarization charge density 
$\rho = \nabla\cdot( n\nabla_\perp \phi / \Omega B)$ 
and
$\vec u_\psi := \hat{\vec b}\times \nabla\psi /B$ 
with 
$\psi:= m_i\vec u_E^2 /2e$ 
We exploit this form of Eq ~\eqref{eq:generalb} in our numerical simulations 

Equations~\eqref{eq:generala} and \eqref{eq:generalb} respectively \eqref{eq:vorticity} have several invariants 
First, in Eq ~\eqref{eq:generala} the relative particle number 
$M(t) := \int \mathrm{dA}\, (n-n_0)$ is conserved over time
$\d M(t)/\d t = 0$  
Furthermore, we integrate 
$( T_e(1+\ln n) -T_e \ln B)\partial_t n$
as well as
$-e\phi \partial_t\rho - (m_i\vec u_E^2/2+gm_ix - T_e\ln B)\partial_t n$ 
over the domain to get, disregarding boundary contributions,
\begin{align}
  \frac{\d}{\d t}\left[T_eS(t) + H(t) \right] = 0, \label{eq:energya}\\ 
    \frac{\d}{\d t} \left[ E(t) - G(t) - H(t)\right] =  0,
    \label{eq:energyb}
\end{align}
where we define 
the entropy
$S(t):=\int \mathrm{dA}\, [n\ln(n/n_0) - (n-n_0)]$,  
the kinetic energy 
$E(t):=m_i \int \mathrm{dA}\, n\vec u_E^2/2$ 
and the potential energies
$G(t) := m_i g\int \mathrm{dA}\, x(n-n_0)$
and
$H(t) := T_e\int \mathrm{dA}\, (n-n_0) \ln (B^{-1})$ 
Note that $n\ln( n/n_0) - n + n_0 \approx (n-n_0)^2/2$ for $|(n-n_0)/n_0| \ll 1$ and $S(t)$ thus reduces to the 
local entropy form in Reference~\cite{Kube2016}  

We now set up a gravitational field $\vec g = g\hat x$ and a constant homogeneous background
magnetic field $\vec B = B_0 \hat z$ in a Cartesian coordinate system 
Then the divergences of the electric and gravitational drift velocities $\nabla\cdot\vec u_E$ and $\nabla\cdot\vec u_g$
and the diamagnetic current $\nabla\cdot(n\vec u_d)$ vanish, which makes the 
flow incompressible  Furthermore, the magnetic potential energy vanishes $H(t) = 0$ 

In a second system we model the inhomogeneous magnetic field present in tokamaks as
$\vec B := B_0 (1+ x/R_0)^{-1}\hat z$ and neglect the gravitational drift $\vec u_g = 0$ 
Then, the potential energy $G(t) = 0$  
Note that 
$H(t) = m_i \ensuremath{C_\mathrm{s}}^2/R_0\int\mathrm{dA}\, x(n-n_0) +\mathcal O(R_0^{-2}) $
reduces to $G(t)$ with the effective gravity $g_\text{eff}:= \ensuremath{C_\mathrm{s}}^2/R_0$ with $\ensuremath{C_\mathrm{s}}^2 := T_e/m_i$  
For the rest of this letter we treat $g$ and $g_\text{eff}$ as well as $G(t)$ and $H(t)$ on the same footing 
The magnetic field inhomogeneity thus entails compressible flows, which is 
the only difference to the model describing dynamics in a homogeneous magnetic field introduced above  
Since both $S(t)\geq 0$ and $E(t)\geq 0$ we further derive from Eq ~\eqref{eq:energya} and Eq ~\eqref{eq:energyb} that the kinetic energy
is bounded by $E(t) \leq T_eS(t) + E(t) = T_e S(0)$; a feature absent from the gravitational system with 
incompressible flows, where $S(t) = S(0)$  

We now show that the invariants Eqs ~\eqref{eq:energya} and \eqref{eq:energyb} present restrictions on the velocity and
acceleration of plasma blobs  
First, we define the blobs' center of mass (COM) via $X(t):= \int\mathrm{dA}\, x(n-n_0)/M$ and 
its COM velocity as $V(t):=\d X(t)/\d t$  
The latter is proportional to the total radial particle flux~\cite{Garcia_Bian_Fundamensky_POP_2006, Held2016a} 
We assume
that $n>n_0$ and $(n-n_0)^2/2 \leq [ n\ln (n/n_0) - (n-n_0)]n $ to show for both systems 
\begin{align}
  (MV)^2 &= \left( \int \mathrm{dA}\, n{\phi_y}/{B} \right)^2
  = \left( \int \mathrm{dA}\, (n-n_0){\phi_y}/{B} \right)^2\nonumber\\
 
&\leq 2 \left( \int \mathrm{dA}\, \left[n\ln (n/n_0) -(n-n_0)\right]^{1/2}\sqrt{n}{\phi_y}/{B}\right)^2\nonumber\\
 
  &\leq 4 S(0) E(t)/m_i 
 
  \label{eq:inequality}
\end{align}
Here we use the Cauchy-Schwartz inequality and 
$\phi_y:=\partial\phi/\partial y$  
Note that although we derive the inequality Eq ~\eqref{eq:inequality} only for amplitudes $\triangle n >0$  we assume that the results also hold for depletions  This is justified by our numerical results later in this letter  
If we initialize our density field with a seeded blob of radius $\ell$ and amplitude $\triangle n$ as 
\begin{align}
  n(\vec x, 0) &= n_0 + \triangle n \exp\left( -\frac{\vec x^2}{2\ell^2} \right), \label{eq:inita}
 
 
\end{align}
and  
$\phi(\vec x, 0 ) = 0$,
we immediately have $M := M(0) = 2\pi \ell^2 \triangle n$, $E(0) = G(0) = 0$ and 
$S(0) = 2\pi \ell^2 f(\triangle n)$, where $f(\triangle n)$ captures the amplitude dependence of 
the integral for $S(0)$  

The acceleration for both incompressible and compressible flows can be estimated
by assuming a linear acceleration $V=A_0t$ and $X=A_0t^2/2$~\cite{Held2016a} and using 
$E(t) = G(t) = m_igMX(t)$ in Eq ~\eqref{eq:inequality}
\begin{align}
  \frac{A_0}{g} =  \mathcal Q\frac{2S(0)}{M} \approx \frac{\mathcal Q}{2} \frac{\triangle n }{n_0+2\triangle n/9} 
  \label{eq:acceleration}
\end{align}
Here, we use the Pad\'e approximation of order $(1/1)$ of $2S(0)/M $
and define a model parameter $\mathcal Q$ with $0<\mathcal Q\leq1$ to be determined by numerical simulations 
Note that the Pad\'e approximation is a better approximation than a simple 
truncated Taylor expansion especially for large relative amplitudes of order unity 
Eq ~\eqref{eq:acceleration} predicts that $A_0/g\sim \triangle n/n_0$ for small 
amplitudes $|\triangle n/n_0| < 1$ and $A_0 \sim g $ for very large amplitudes $\triangle n /n_0 \gg 1$, 
which confirms the predictions in~\cite{Pecseli2016} and reproduces the limits discussed in~\cite{Angus2014} 

As pointed out earlier for compressible flows Eq ~\eqref{eq:inequality} can be further estimated
\begin{align}
  (MV)^2  \leq 4 T_eS(0)^2/m_i  
  \label{}
\end{align}
We therefore have a restriction on the maximum COM velocity for compressible flows, which is absent for incompressible flows
\begin{align}
  \frac{\max |V|}{\ensuremath{C_\mathrm{s}}} = {\mathcal Q}\frac{2S(0)}{M} \approx \frac{\mathcal Q}{2} \frac{|\triangle n| }{n_0+2/9 \triangle n } \approx \frac{\mathcal Q}{2} \frac{|\triangle n|}{n_0} 
  \label{eq:linear}
\end{align}
For $|\triangle n /n_0|< 1$ Eq ~\eqref{eq:linear} reduces to the linear scaling derived in~\cite{Kube2016}  
Finally, a scale analysis of Eq ~\eqref{eq:vorticity} shows that~\cite{Ott1978, Garcia2005, Held2016a}
\begin{align}
  \frac{\max |V|}{\ensuremath{C_\mathrm{s}}} = \mathcal R \left( \frac{\ell}{R_0}\frac{|\triangle n|}{n_0} \right)^{1/2} 
  \label{eq:sqrt}
\end{align}
This equation predicts a square root dependence of the center of mass velocity 
on amplitude and size  





We now propose a simple phenomenological model that captures the essential dynamics
of blobs and depletions in the previously stated systems  More specifically 
the model reproduces the acceleration Eq ~\eqref{eq:acceleration} with and without
Boussinesq approximation, the square root scaling for the COM velocity 
Eq ~\eqref{eq:sqrt} for incompressible flows as well as the relation between the 
square root scaling Eq ~\eqref{eq:sqrt} and the linear scaling 
Eq ~\eqref{eq:linear} for compressible flows  
The basic idea is that the COM of blobs behaves like 
the one of an infinitely long plasma column immersed in an ambient plasma  
The dynamics of this column reduces to the one of a two-dimensional ball 
This idea is similar to the analytical ``top hat'' density solution for
blob dynamics recently studied in~\cite{Pecseli2016} 
The ball is subject to buoyancy as well as linear and nonlinear friction
\begin{align}
  M_{\text{i}} \frac{d V}{d t} = (M_{\text{g}} - M_\text{p}) g - c_1 V  - \mathrm{sgn}(V ) \frac{1}{2}c_2 V^2 
  \label{eq:ball}
\end{align}
The gravity $g$ has a positive sign in the coordinate system; sgn$(f)$ is the sign function  
The first term on the right hand side is the buoyancy, where 
$M_{\text{g}} := \pi \ell^2 (n_0 + \mathcal Q \triangle n/2)$ 
is the gravitational mass of the ball with radius $\ell$ and 
$M_\mathrm{p} := n_0 \pi \ell^2 $ 
is the mass of the displaced ambient plasma 
Note that if $\triangle n<0$ the ball represents a depletion and the buoyancy term has a negative sign, i e  the depletion will rise  
We introduce an inertial mass 
$M_{\text{i}} := \pi\ell^2 (n_0 +2\triangle n/9)$ 
different from the gravitational mass $M_{\text{g}}$ in order to 
recover the initial acceleration in Eq ~\eqref{eq:acceleration}  
We interpret the parameters $\mathcal Q$ and $2/9$ as geometrical factors 
that capture the difference of the actual blob form from the idealized
``top hat'' solution  
Also note that the Boussinesq approximation appears in the model as a neglect of inertia, $M_{\text{i}} = \pi\ell^2n_0$ 

The second term is the linear friction term with coefficient $c_1(\ell)$, which
depends on the size of the ball 
If we disregard the nonlinear friction, $c_2=0$, Eq ~\eqref{eq:ball} directly yields a 
maximum velocity $c_1V^*=\pi \ell^2 n g \mathcal Q\triangle n/2$ 
From our previous considerations $\max V/\ensuremath{C_\mathrm{s}}=\mathcal Q \triangle n /2n_0$, we thus identify 
\begin{align}
  c_1 = \pi\ell^2 n_0 g/\ensuremath{C_\mathrm{s}}  
  \label{}
\end{align}
The linear friction coefficient thus depends on the gravity and the size of the
ball  

The last term in \eqref{eq:ball} is the nonlinear friction  The sign of the force depends on whether
the ball rises or falls in the ambient plasma  
If we disregard linear friction $c_1=0$, we have the maximum velocity 
$V^*= \sigma(\triangle n)\sqrt{\pi \ell^2|\triangle n| g\mathcal Q/c_2}$, 
which must equal 
$\max V= \sigma(\triangle n) \mathcal R \sqrt{g \ell |\triangle n/n_0|}$ 
and thus
\begin{align}
  c_2 = {\mathcal Q\pi n_0\ell }/{\mathcal R^2} 
  \label{}
\end{align}
Inserting $c_1$ and $c_2$ into Eq ~\eqref{eq:ball}
we can derive the maximum absolute velocity in the form 
\begin{align}
  \frac{\max |V|}{\ensuremath{C_\mathrm{s}}} = 
        \left(\frac{\mathcal R^2}{\mathcal Q}\right) \frac{\ell}{R_0} \left( 
        \left({1+\left( \frac{\mathcal Q}{\mathcal R} \right)^{2} \frac{|\triangle n|/n_0 }{\ell/R_0}}\right)^{1/2}-1 \right)
  \label{eq:vmax_theo}
\end{align}
and thus have a concise expression for $\max |V|$ that captures both the linear
scaling \eqref{eq:linear} as well as the square root scaling \eqref{eq:sqrt} 
With Eq ~\eqref{eq:acceleration} and Eq ~\eqref{eq:sqrt} respectively Eq ~\eqref{eq:vmax_theo} we 
finally arrive at an analytical expression for the time at which the maximum velocity is reached via 
$t_{\max V} \sim \max V/A_0$  Its inverse $\gamma:=t_{\max V}^{-1}$ gives the
global interchange growth rate, for which an empirical expression was
presented in Reference~\cite{Held2016a} 


We use the open source library FELTOR 
to simulate 
Eqs ~\eqref{eq:generala} and \eqref{eq:vorticity} with and without 
drift compression 
For numerical stabilty we added small diffusive terms on the right hand 
sides of the equations 
The discontinuous Galerkin methods employ three polynomial coefficients and a minimum of $N_x=N_y=768$ grid cells  The box size is $50\ell$ in order to mitigate 
influences of the finite box size on the blob dynamics  
Moreover, we used the invariants in Eqs  \eqref{eq:energya} and \eqref{eq:energyb} as consistency tests to verify the code and repeated simulations 
also in a gyrofluid model



ground_truth: Decorrelating the data before running the PLS algorithm improves the performance of the algorithm.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: What types of data did the authors use in their experiments?

Read the following text and answer briefly.

Context 2: \section{Introduction}
\label{sec:introduction}

Probabilistic models have proven to be very useful in a lot of applications in signal processing where signal estimation is needed \cite{rabiner1989tutorial,arulampalam2002tutorial,ji2008bayesian}  Some of their advantages are that 1) they force the designer to specify all the assumptions of the model, 2) they provide a clear separation between the model and the algorithm used to solve it, and 3) they usually provide some measure of uncertainty about the estimation 

On the other hand, adaptive filtering is a standard approach in estimation problems when the input is received as a stream of data that is potentially non-stationary  This approach is widely understood and applied to several problems such as echo cancellation \cite{gilloire1992adaptive}, noise cancellation \cite{nelson1991active}, and channel equalization \cite{falconer2002frequency} 

Although these two approaches share some underlying relations, there are very few connections in the literature  The first important attempt in the signal processing community to relate these two fields was the connection between a linear Gaussian state-space model (i e  Kalman filter) and the RLS filter, by Sayed and Kailath \cite{sayed1994state} and then by Haykin \emph{et al } \cite{haykin1997adaptive}  The RLS adaptive filtering algorithm emerges naturally when one defines a particular state-space model (SSM) and then performs exact inference in that model  This approach was later exploited in \cite{van2012kernel} to design a kernel RLS algorithm based on Gaussian processes 

A first attempt to approximate the LMS filter from a probabilistic perspective was presented in \cite{park2014probabilistic}, focusing on a kernel-based implementation  The algorithm of \cite{park2014probabilistic} makes use of a Maximum a Posteriori (MAP) estimate as an approximation for the predictive step  However, this approximation does not preserve the estimate of the uncertainty in each step, therefore degrading the performance of the algorithm 

In this work, we provide a similar connection between state-space models and least-mean-squares (LMS)  Our approach is based on approximating the posterior distribution with an isotropic Gaussian distribution  We show how the computation of this approximated posterior leads to a linear-complexity algorithm, comparable to the standard LMS  Similar approaches have already been developed for a variety of problems such as channel equalization using recurrent RBF neural networks \cite{cid1994recurrent}, or Bayesian forecasting \cite{harrison1999bayesian}  Here, we show the usefulness of this probabilistic approach for adaptive filtering 

The probabilistic perspective we adopt throughout this work presents two main advantages  Firstly, a novel LMS algorithm with adaptable step size emerges naturally with this approach, making it suitable for both stationary and non-stationary environments  The proposed algorithm has less free parameters than previous LMS algorithms with variable step size \cite{kwong1992variable,aboulnasr1997robust,shin2004variable}, and its parameters are easier to be tuned w r t  these algorithms and standard LMS  Secondly, the use of a probabilistic model provides us with an estimate of the error variance, which is useful in many applications 

Experiments with simulated and real data show the advantages of the presented approach with respect to previous works  However, we remark that the main contribution of this paper is that it opens the door to introduce more Bayesian machine learning techniques, such as variational inference and Monte Carlo sampling methods \cite{barber2012bayesian}, to adaptive filtering \\


\section{Probabilistic Model}

Throughout this work, we assume the observation model to be linear-Gaussian with the following distribution,

\begin{equation}
p(y_k|{\bf w}_k) = \mathcal{N}(y_k;{\bf x}_k^T {\bf w}_k , \sigma_n^2),
\label{eq:mess_eq}
\end{equation}
where  $\sigma_n^2$ is the variance of the observation noise, ${\bf x}_k$ is the regression vector and ${\bf w}_k$ is the parameter vector to be sequentially estimated, both $M$-dimensional column vectors 


In a non-stationary scenario, ${\bf w}_k$ follows a dynamic process  In particular, we consider a diffusion process (random-walk model) with variance $\sigma_d^2$ for this parameter vector:


\begin{equation}
p({\bf w}_k|{\bf w}_{k-1})= \mathcal{N}({\bf w}_k;{\bf w}_{k-1}, \sigma_d^2 {\bf I}),
\label{eq:trans_eq}
\end{equation}
where $\bf I$ denotes the identity matrix  In order to initiate the recursion, we assume the following prior distribution on ${\bf w}_k$

\begin{equation}
p({\bf w}_0)= \mathcal{N}({\bf w}_0;0, \sigma_d^2{\bf I}) \nonumber
\end{equation}

\section{Exact inference in this model: Revisiting the RLS filter}

Given the described probabilistic SSM, we would like to infer the posterior probability distribution $p({\bf w}_k|y_{1:k})$ 
Since all involved distributions are Gaussian, one can perform exact inference, leveraging the probability rules in a straightforward manner  The resulting probability distribution is
\begin{equation}
p({\bf w}_k|y_{1:k}) =  \mathcal{N}({\bf w}_k;{\bf\boldsymbol\mu}_{k}, \boldsymbol\Sigma_{k}), \nonumber
\end{equation}
in which the mean vector ${\bf\boldsymbol\mu}_{k}$ is given by
\begin{equation}
{\bf\boldsymbol\mu}_k = {\bf\boldsymbol\mu}_{k-1} + {\bf K}_k (y_k - {\bf x}_k^T {\bf\boldsymbol\mu}_{k-1}){\bf x}_k, \nonumber
\end{equation}
where we have introduced the auxiliary variable
\begin{equation}
{\bf K}_k = \frac{ \left(\boldsymbol\Sigma_{k-1} + \sigma_d^2 {\bf I}\right)}{{\bf x}_k^T  \left(\boldsymbol\Sigma_{k-1} + \sigma_d^2 {\bf I}\right)  {\bf x}_k + \sigma_n^2}, \nonumber
\end{equation}
and the covariance matrix $\boldsymbol\Sigma_k$ is obtained as
\begin{equation}
\boldsymbol\Sigma_k = \left( {\bf I} -  {\bf K}_k{\bf x}_k {\bf x}_k^T \right) ( \boldsymbol\Sigma_{k-1} +\sigma_d^2), \nonumber
\end{equation}
Note that the mode of $p({\bf w}_k|y_{1:k})$, i e  the maximum-a-posteriori estimate (MAP), coincides with the RLS adaptive rule
\begin{equation}
{{\bf w}}_k^{(RLS)} = {{\bf w}}_{k-1}^{(RLS)} + {\bf K}_k (y_k - {\bf x}_k^T {{\bf w}}_{k-1}^{(RLS)}){\bf x}_k  
\label{eq:prob_rls}
\end{equation}
This rule is similar to the one introduced in \cite{haykin1997adaptive} 

Finally, note that the covariance matrix $\boldsymbol\Sigma_k$ is a measure of the uncertainty of the estimate ${\bf w}_k$ conditioned on the observed data $y_{1:k}$  Nevertheless, for many applications a single scalar summarizing the variance of the estimate could prove to be sufficiently useful  In the next section, we show how such a scalar is obtained naturally when $p({\bf w}_k|y_{1:k})$ is approximated with an isotropic Gaussian distribution  We also show that this approximation leads to an LMS-like estimation 
 


\section{Approximating the posterior distribution: LMS filter }

The proposed approach consists in approximating the posterior distribution $p({\bf w}_k|y_{1:k})$, in general a multivariate Gaussian distribution with a full covariance matrix, by an isotropic spherical Gaussian distribution 

\begin{equation}
\label{eq:aprox_post}
\hat{p}({\bf w}_{k}|y_{1:k})=\mathcal{N}({\bf w}_{k};{\bf \hat{\boldsymbol\mu}}_{k}, \hat{\sigma}_{k}^2 {\bf I} ) 
\end{equation}

In order to estimate the mean and covariance of the approximate distribution $\hat{p}({\bf w}_{k}|y_{1:k})$, we propose to select those that minimize the Kullback-Leibler divergence with respect to the original distribution, i e , 

\begin{equation}
\{\hat{\boldsymbol\mu}_k,\hat{\sigma}_k\}=\arg \displaystyle{  \min_{\hat{\boldsymbol\mu}_k,\hat{\sigma}_k}} \{ D_{KL}\left(p({\bf w}_{k}|y_{1:k}))\| \hat{p}({\bf w}_{k}|y_{1:k})\right) \}  \nonumber
\end{equation}

The derivation of the corresponding minimization problem can be found in Appendix A  In particular, the optimal mean and the covariance are found as
\begin{equation}
{\hat{\boldsymbol\mu}}_{k} = {\boldsymbol\mu}_{k};~~~~~~ \hat{\sigma}_{k}^2 = \frac{{\sf Tr}\{ \boldsymbol\Sigma_k\} }{M} 
\label{eq:sigma_hat}
\end{equation}


We now show that by using \eqref{eq:aprox_post} in the recursive predictive and filtering expressions we obtain an LMS-like adaptive rule  First, let us assume that we have an approximate posterior distribution at $k-1$, $\hat{p}({\bf w}_{k-1}|y_{1:k-1}) =  \mathcal{N}({\bf w}_{k-1};\hat{\bf\boldsymbol\mu}_{k-1}, \hat{\sigma}_{k-1}^2 {\bf I} )$  Since all involved distributions are Gaussian, the predictive distribution
is obtained as %
\begin{eqnarray}
\hat{p}({\bf w}_k|y_{1:k-1}) &=& \int p({\bf w}_k|{\bf w}_{k-1}) \hat{p}({\bf w}_{k-1}|y_{1:k-1}) d{\bf w}_{k-1} \nonumber\\
&=& \mathcal{N}({\bf w}_k;{\bf\boldsymbol\mu}_{k|k-1}, \boldsymbol\Sigma_{k|k-1}), 
\label{eq:approx_pred}
\end{eqnarray}
where the mean vector and covariance matrix are given by
\begin{eqnarray}
\hat{\bf\boldsymbol\mu}_{k|k-1} &=& \hat{\bf\boldsymbol\mu}_{k-1} \nonumber \\
\hat{\boldsymbol\Sigma}_{k|k-1} &=& (\hat{\sigma}_{k-1}^2 + \sigma_d^2 ){\bf I}\nonumber 
\end{eqnarray}

From \eqref{eq:approx_pred}, the posterior distribution at time $k$ can be computed using Bayes' Theorem and standard Gaussian manipulations (see for instance \cite[Ch  4]{murphy2012machine})  Then, we approximate the posterior $p({\bf w}_k|y_{1:k})$ with an isotropic Gaussian,
\begin{equation}
\hat{p}({\bf w}_k|y_{1:k}) =  \mathcal{N}({\bf w}_k ; {\hat{\boldsymbol\mu}}_{k}, \hat{\sigma}_k^2 {\bf I} ),\nonumber
\end{equation}
where 
\begin{eqnarray}
{\hat{\boldsymbol\mu}}_{k} &= & {\hat{\boldsymbol\mu}}_{k-1}+ \frac{ (\hat{\sigma}_{k-1}^2+ \sigma_d^2)  }{(\hat{\sigma}_{k-1}^2+ \sigma_d^2)  \|{\bf x}_k\|^2 + \sigma_n^2} (y_k - {\bf x}_k^T {\hat{\boldsymbol\mu}}_{k-1}){\bf x}_k  \nonumber  \\
&=& {\hat{\boldsymbol\mu}}_{k-1}+ \eta_k (y_k - {\bf x}_k^T {\hat{\boldsymbol\mu}}_{k-1}){\bf x}_k   
\label{eq:prob_lms}
\end{eqnarray}
Note that, instead of a gain matrix ${\bf K}_k$ as in Eq ~\eqref{eq:prob_rls}, we now have a scalar gain $\eta_k$ that operates as a variable step size 


Finally, to obtain the posterior variance, which is our measure of uncertainty, we apply \eqref{eq:sigma_hat} and the trick ${\sf Tr}\{{\bf x}_k{\bf x}_k^T\}= {\bf x}_k^T{\bf x}_k= \|{\bf x}_k \|^2$,

\begin{eqnarray}
\hat{\sigma}_k^2 &=& \frac{{\sf Tr}(\boldsymbol\Sigma_k)}{M} \\
&=& \frac{1}{M}{\sf Tr}\left\{ \left( {\bf I} -  \eta_k {\bf x}_k {\bf x}_k^T \right) (\hat{\sigma}_{k-1}^2 +\sigma_d^2)\right\} \\
&=& \left(1 - \frac{\eta_k \|{\bf x}_k\|^2}{M}\right)(\hat{\sigma}_{k-1}^2 +\sigma_d^2) 
\label{eq:sig_k}
\end{eqnarray}
If MAP estimation is performed, we obtain  an adaptable step-size LMS estimation

\begin{equation}
{\bf w}_{k}^{(LMS)} = {\bf w}_{k-1}^{(LMS)} + \eta_k (y_k - {\bf x}_k^T {\bf w}_{k-1}^{(LMS)}){\bf x}_k, 	
\label{eq:lms}
\end{equation}
with
\begin{equation}
\eta_k = \frac{ (\hat{\sigma}_{k-1}^2+ \sigma_d^2)  }{(\hat{\sigma}_{k-1}^2+ \sigma_d^2)  \|{\bf x}_k\|^2 + \sigma_n^2} \nonumber
\end{equation}
At this point, several interesting remarks can be made:

\begin{itemize}

\item The adaptive rule \eqref{eq:lms} has linear complexity since it does not require us to compute the full matrix $\boldsymbol\Sigma_k$ 

\item For a stationary model, we have $\sigma_d^2=0$ in \eqref{eq:prob_lms} and \eqref{eq:sig_k}  In this case, the algorithm remains valid and both the step size and the error variance, $\hat{\sigma}_{k}$, vanish over time $k$  

\item Finally, the proposed adaptable step-size LMS has only two parameters, $\sigma_d^2$ and $\sigma_n^2$, (and only one, $\sigma_n^2$, in stationary scenarios) in contrast to other variable step-size algorithms \cite{kwong1992variable,aboulnasr1997robust,shin2004variable}  More interestingly, both $\sigma_d^2$ and $\sigma_n^2$ have a clear underlying physical meaning, and they can be estimated in many cases  We will comment more about this in the next section  
\end{itemize}



\section{Experiments}
\label{sec:experiments}

We evaluate the performance of the proposed algorithm in both stationary and tracking experiments  In the first experiment, we estimate a fixed vector ${\bf w}^{o}$ of dimension $M=50$  The entries of the vector are independently and uniformly chosen in the range $[-1,1]$  Then, the vector is normalized so that $\|{\bf w}^o\|=1$  Regressors $\boldsymbol{x}_{k}$ are zero-mean Gaussian vectors with identity covariance matrix  The additive noise variance is such that the SNR is $20$ dB  We compare our algorithm with standard RLS and three other LMS-based algorithms: LMS, NLMS \cite{sayed2008adaptive}, VSS-LMS \cite{shin2004variable} \footnote{The used parameters for each algorithm are: for RLS $\lambda=1$, $\epsilon^{-1}=0 01$; for LMS $\mu=0 01$; for NLMS $\mu=0 5$; and for VSS-LMS $\mu_{max}=1$, $\alpha=0 95$, $C=1e-4$ } The probabilistic LMS algorithm in \cite{park2014probabilistic} is not simulated because it is not suitable for stationary environments 

In stationary environments, the proposed algorithm has only one parameter, $\sigma^2_n$  We simulate both the scenario where we have perfectly knowledge of the amount of noise (probLMS1) and the case where the value $\sigma^2_n$ is $100$ times smaller than the actual value (probLMS2)  The Mean-Square Deviation (${\sf MSD} = {\mathbb E} \| {\bf w}_0 - {\bf w}_k \|^2$), averaged out over $50$ independent simulations, is presented in Fig  \ref{fig:msd_statationary} 



\begin{figure}[htb]
\centering
\begin{minipage}[b]{\linewidth}
  \centering
  \centerline{\includegraphics[width=\textwidth]{results_stationary_MSD}}
\end{minipage}
\caption{Performance in terms of MSD of probabilistic LMS with both optimal (probLMS1) and suboptimal (probLMS2) compared to LMS, NLMS, VS-LMS, and RLS }
\label{fig:msd_statationary}
\end{figure}

The performance of probabilistic LMS is close to RLS (obviously at a much lower computational cost) and largely outperforms previous variable step-size LMS algorithms proposed in the literature  Note that, when the model is stationary, i e  $\sigma^2_d=0$ in \eqref{eq:trans_eq},  both the uncertainty $\hat{\sigma}^2_k$, and the adaptive step size $\eta_k$, vanish over time  This implies that the error tends to zero when $k$ goes to infinity  Fig  \ref{fig:msd_statationary} also shows that the proposed approach is not very sensitive to a bad choice of its only parameter, as demonstrated by the good results of probLMS2, which uses a $\sigma^2_n$ that is $100$ times smaller than the optimal value  


\begin{figure}[htb]
\centering
\begin{minipage}[b]{\linewidth}
  \centering
  \centerline{\includegraphics[width=\textwidth]{fig2_final}}
\end{minipage}
\caption{Real part of one coefficient of the measured and estimated channel in experiment two  The shaded area represents two standard deviations from the prediction {(the mean of the posterior distribution)} }
\label{fig_2}
\end{figure}


\begin{table}[ht]
\begin{footnotesize}
\setlength{\tabcolsep}{2pt}
\def1 5mm{1 5mm}
\begin{center}
\begin{tabular}{|l@{\hspace{1 5mm}}|c@{\hspace{1 5mm}}|c@{\hspace{1 5mm}}|c@{\hspace{1 5mm}}|c@{\hspace{1 5mm}}|c@{\hspace{1 5mm}}|c@{\hspace{1 5mm}}|}
\hline
Method &  LMS &  NLMS & LMS-2013 & VSSNLMS & probLMS & RLS \\
\hline
\hline
MSD (dB) &-28 45 &-21 07 &-14 36 &-26 90 &-28 36 &-25 97\\
\hline                                                                     
\end{tabular}
\end{center}
\caption{Steady-state MSD of the different algorithms for the tracking of a real MISO channel }
\label{tab:table_MSD}
\end{footnotesize}

\end{table}
\newpage
In a second experiment, we test the tracking capabilities of the proposed algorithm with {real} data of a wireless MISO channel acquired in a realistic indoor scenario  More details on the setup can be found in \cite{gutierrez2011frequency}  Fig  \ref{fig_2} shows the real part of one of the channels, and the estimate of the proposed algorithm  The shaded area represents the estimated uncertainty for each prediction, i e  $\hat{\mu}_k\pm2\hat{\sigma}_k$  Since the experimental setup does not allow us to obtain the optimal values for the parameters, we fix these parameters to their values that optimize the steady-state mean square deviation (MSD)  \hbox{Table \ref{tab:table_MSD}} shows this steady-state MSD of the estimate of the MISO channel with different methods  As can be seen, the best tracking performance is obtained by standard LMS and the proposed method  





\section{Conclusions and Opened Extensions}
\label{sec:conclusions}

{We have presented a probabilistic interpretation of the least-mean-square filter  The resulting algorithm is an adaptable step-size LMS that performs well both in stationary and tracking scenarios  Moreover, it has fewer free parameters than previous approaches and these parameters have a clear physical meaning  Finally, as stated in the introduction, one of the advantages of having a probabilistic model is that it is easily extensible:}

\begin{itemize}
\item If, instead of using an isotropic Gaussian distribution in the approximation, we used a Gaussian with diagonal covariance matrix, we would obtain a similar algorithm with different step sizes and measures of uncertainty, for each component of ${\bf w}_k$  Although this model can be more descriptive, it needs more parameters to be tuned, and the parallelism with LMS vanishes 
\item Similarly, if we substitute the transition model of \eqref{eq:trans_eq} by an Ornstein-Uhlenbeck process, 

\begin{equation}
p({\bf w}_k|{\bf w}_{k-1})= \mathcal{N}({\bf w}_k;\lambda {\bf w}_{k-1}, \sigma_d^2), \nonumber
\label{eq:trans_eq_lambda}
\end{equation}
a similar algorithm is obtained but with a forgetting factor $\lambda$ multiplying ${\bf w}_{k-1}^{(LMS)}$ in \eqref{eq:lms}  This algorithm may have improved performance under such a kind of autoregresive dynamics of ${\bf w}_{k}$, though, again, the connection with standard LMS becomes dimmer 

\item As in \cite{park2014probabilistic}, the measurement model \eqref{eq:mess_eq} can be changed to obtain similar adaptive algorithms for classification, ordinal regression, and Dirichlet regression for compositional data  

\item A similar approximation technique could be applied to more complex dynamical models, i e  switching dynamical models \cite{barber2010graphical}  The derivation of efficient adaptive algorithms that explicitly take into account a switch in the dynamics of the parameters of interest is a non-trivial and open problem, though the proposed approach could be useful 

\item Finally, like standard LMS, this algorithm can be kernelized for its application in estimation under non-linear scenarios 

\end{itemize}


\begin{appendices}

\section{KL divergence between a general gaussian distribution and an isotropic gaussian}
\label{sec:kl}

 We want to approximate  $p_{{\bf x}_1}(x) = \mathcal{N}({\bf x}; \boldsymbol\mu_1,\boldsymbol\Sigma_1)$ by $p_{{\bf x}_2}({\bf x}) = \mathcal{N}({\bf x}; \boldsymbol\mu_2,\sigma_2^2 {\bf I})$  In order to do so, we have to compute the parameters of $p_{{\bf x}_2}({\bf x})$, $\boldsymbol\mu_2$ and $\sigma_2^2$, that minimize the following Kullback-Leibler divergence,

\begin{eqnarray}
D_{KL}(p_{{\bf x}_1}\| p_{{\bf x}_2}) &=&\int_{-\infty}^{\infty} p_{{\bf x}_1}({\bf x}) \ln{\frac{p_{{\bf x}_1}({\bf x})}{p_{{\bf x}_2}({\bf x})}}d{\bf x} \nonumber  \\
&= &  \frac{1}{2} \{ -M + {\sf Tr}(\sigma_2^{-2} {\bf I}\cdot \boldsymbol\Sigma_1^{-1})  \nonumber \\
  & &  + (\boldsymbol\mu_2 - \boldsymbol\mu_1 )^T \sigma^{-2}_2{\bf I} (\boldsymbol\mu_2 - \boldsymbol\mu_1 )  \nonumber \\
 & &   +  \ln \frac{{\sigma_2^2}^M}{\det\boldsymbol\Sigma_1} \}   
\label{eq:divergence}
\end{eqnarray}
Using symmetry arguments, we obtain 
\begin{equation}
\boldsymbol\mu_2^{*} =\arg \displaystyle{  \min_{\boldsymbol\mu_2}} \{ D_{KL}(p_{{\bf x}_1}\| p_{{\bf x}_2}) \} = \boldsymbol\mu_1 
\end{equation}
Then, \eqref{eq:divergence} gets simplified into 

\begin{eqnarray}
D_{KL}(p_{{\bf x}_1}\| p_{{\bf x}_2}) = \frac{1}{2}\lbrace { -M + {\sf Tr}(\frac{\boldsymbol\Sigma_1}{\sigma_2^{2}}) + \ln \frac{\sigma_2^{2M}}{\det\boldsymbol\Sigma_1}}\rbrace 
\end{eqnarray}
The variance $\sigma_2^2$ is computed in order to minimize this Kullback-Leibler divergence as

\begin{eqnarray}
\sigma_2^{2*} &=& \arg\min_{\sigma_2^2} D_{KL}(P_{x_1}\| P_{x_2}) \nonumber \\
 &=& \arg\min_{\sigma_2^2}\{ \sigma_2^{-2}{\sf Tr}\{\boldsymbol\Sigma_1\} + M\ln \sigma_2^{2} \}  
\end{eqnarray}
Deriving and making it equal zero leads to

\begin{equation}
\frac{\partial}{\partial \sigma_2^2} \left[ \frac{{\sf Tr}\{\boldsymbol\Sigma_1\}}{\sigma_2^{2}} + M \ln \sigma_2^{2} \right] = \left  {\frac{M}{\sigma_2^{2}}-\frac{{\sf Tr}\{\boldsymbol\Sigma_1\}}{(\sigma_2^{2})^2}}\right|_{\sigma_2^{2}=\sigma_2^{2*}}\left  =0 \right   
\nonumber
\end{equation}
Finally, since the divergence has a single extremum in $R_+$,
\begin{equation}
\sigma_2^{2*} = \frac{{\sf Tr}\{\boldsymbol\Sigma_1\}}{M} 
\end{equation}




\end{appendices}

\vfill
\clearpage

\bibliographystyle{IEEEbib}

Context 3:  
No differences to the results presented here were found  
Initial perturbations on the particle density field are given by Eq ~\eqref{eq:inita},
where the perturbation amplitude $\triangle n/n_0$ was chosen between $10^{-3}$ and $20$ for blobs and $-10^0$ and $ -10^{-3}$ for depletions  
Due to computational reasons we show results only for $\triangle n/n_0\leq 20$  


For compressible flows we consider two different cases $\ell/R_0 = 10^{-2}$ and
$\ell /R_0 = 10^{-3}$  
 For incompressible flows Eq ~\eqref{eq:generala} and \eqref{eq:vorticity}
 can be normalized such that the blob radius is absent from the equations~\cite{Ott1978, Kube2012}  
 The simulations of incompressible flows can thus be used for both sizes  
The numerical code as well as input parameters and output data can be found 
in the supplemental dataset to this contribution~\cite{Data2017} 

\begin{figure}[htb]
    \includegraphics[width=\columnwidth]{com_blobs}
    \caption{
      The maximum radial COM velocities of blobs for compressible and incompressible flows are shown  
      The continuous lines show Eq ~\eqref{eq:vmax_theo} while the 
      dashed line shows the square root scaling Eq ~\eqref{eq:sqrt} with 
      $\mathcal Q = 0 32$ and $\mathcal R=0 85$ 
    }
    \label{fig:com_blobs}
\end{figure}
In Fig ~\ref{fig:com_blobs} we plot the maximum COM velocity for blobs 
with and without drift compression 
For incompressible flows blobs follow the square root scaling almost 
perfectly  Only at very large amplitudes velocities are slightly below
the predicted values  
For small amplitudes we observe that the compressible blobs follow
a linear scaling  When the amplitudes increase there is a transition to the
square root scaling at around $\triangle n/n_0 \simeq 0 5$ for 
$\ell/R_0=10^{-2}$ and $\triangle n/n_0 \simeq 0 05$ for $\ell/R_0=10^{-3}$, which is consistent with Eq ~\eqref{eq:vmax_theo} and Reference~\cite{Kube2016}  
In the transition regions the simulated velocities are slightly larger than the predicted ones from Eq ~\eqref{eq:vmax_theo} 
Beyond these amplitudes
the velocities of compressible and incompressible blobs align  

\begin{figure}[htb]
    \includegraphics[width=\columnwidth]{com_holes}
    \caption{
      The maximum radial COM velocities of depletions for compressible and incompressible flows are shown  
      The continuous lines show Eq ~\eqref{eq:vmax_theo} while the 
      dashed line shows the square root scaling Eq ~\eqref{eq:sqrt} with 
      $\mathcal Q = 0 32$ and $\mathcal R=0 85$ 
      Note that small amplitudes are on the right and amplitudes close to unity are on the left side 
  }
    \label{fig:com_depletions}
\end{figure}
In Fig ~\ref{fig:com_depletions} we show the maximum radial COM velocity 
for depletions instead of blobs 
For relative amplitudes below $|\triangle n|/n_0 \simeq 0 5$ (right of unity in the plot) the velocities
coincide with the corresponding blob velocities in Fig ~\ref{fig:com_blobs}  
 For amplitudes larger than $|\triangle n|/n_0\simeq 0 5$ the 
velocities follow the square root scaling 
We observe that for plasma depletions beyond $90$ percent the velocities 
in both systems reach a constant value that is very well predicted by the
square root scaling  

\begin{figure}[htb]
    \includegraphics[width=\columnwidth]{acc_blobs}
    \caption{
      Average acceleration of blobs for compressible and incompressible flows are shown 
      The continuous line shows the acceleration in Eq ~\eqref{eq:acceleration} 
      with $\mathcal Q=0 32$
      while the dashed line is a linear reference line, which corresponds to the Boussinesq approximation  
  }
    \label{fig:acc_blobs}
\end{figure}
In Fig ~\ref{fig:acc_blobs} we show the average acceleration of blobs 
for compressible and incompressible flows computed
by dividing the maximum velocity $\max V$ by the time  
to reach this velocity $t_{\max V}$  
We compare the simulation results
to the theoretical predictions Eq ~\eqref{eq:acceleration} of our model with and without inertia  
The results of the compressible and incompressible systems coincide and fit very
well to our theoretical values  
For amplitudes larger than unity the acceleration deviates significantly from the prediction with Boussinesq approximation 

\begin{figure}[htb]
    \includegraphics[width=\columnwidth]{acc_holes}
    \caption{
      Average acceleration of depletions for compressible and incompressible flows are shown 
      The continuous line shows the acceleration in Eq ~\eqref{eq:acceleration} 
      with $\mathcal Q=0 32$
      while the dashed line is a linear reference line, which corresponds to the Boussinesq approximation  
    }
    \label{fig:acc_depletions}
\end{figure}
In Fig ~\ref{fig:acc_depletions} we show the simulated acceleration of depletions in the
compressible and the incompressible systems  We compare the simulation results
to the theoretical predictions Eq ~\eqref{eq:acceleration} of our model with and without inertia 
Deviations from our theoretical prediction Eq ~\eqref{eq:acceleration} are visible for amplitudes smaller than $\triangle n/n_0 \simeq -0 5$ (left of unity in the plot)  The relative deviations are small at around $20$ percent  
As in Fig ~\ref{fig:com_depletions} the acceleration reaches a constant values
for plasma depletions of more than $90$ percent 
Comparing Fig ~\ref{fig:acc_depletions} to Fig ~\ref{fig:acc_blobs} the asymmetry between blobs and depletions becomes 
apparent  While the acceleration of blobs is reduced for large 
amplitudes compared to a linear dependence the acceleration 
of depletions is increased  In the language of our simple buoyancy 
model the inertia of depletions is reduced but increased for blobs  



In conclusion  
  we discuss the dynamics of seeded blobs and depletions in a 
  compressible and an incompressible system 
  With only two fit parameters our theoretical results reproduce the 
  numerical COM velocities and accelerations over five orders of magnitude 
  We derive the amplitude dependence of the acceleration of blobs and depletions from 
  the conservation laws of our systems in Eq ~\eqref{eq:acceleration}  
  From the same inequality a linear regime is derived in the compressible system for 
  ratios of amplitudes to sizes smaller than a critical value 
   In this regime 
  the blob and depletion velocity depends linearly on the initial amplitude and 
  is independent of size  The regime is absent from the system with incompressible flows 
  Our theoretical results are verified by numerical simulations for all 
  amplitudes that are relevant in magnetic fusion devices 
  Finally, we suggest a new empirical blob model that captures the detailed dynamics of more complicated models  
  The Boussinesq approximation is clarified as the absence of inertia and a thus altered acceleration of blobs and depletions 
  The maximum blob velocity is not altered by the Boussinesq approximation 

The authors were supported with financial subvention from the Research Council of Norway under grant
240510/F20  M W  and M H  were supported by the Austrian Science Fund (FWF) Y398   The computational
results presented have been achieved in part using the Vienna Scientific Cluster (VSC)  Part of this work was performed on the Abel Cluster, owned by the University of Oslo and the Norwegian metacenter
for High Performance Computing (NOTUR), and operated by the Department for Research Computing at USIT,
the University of Oslo IT-department 
This work has been carried out within the framework of the EUROfusion Consortium and has received funding from the Euratom research and training programme 2014-2018 under grant agreement No 633053  The views and opinions expressed herein do not necessarily reflect those of the European Commission 



ground_truth: The authors used simulated data and real data from a wireless MISO channel.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: What is the relationship between the maximum velocity and the amplitude of the blob or depletion?

Read the following text and answer briefly.

Context 4: 
An electric circuit is an interconnection of electric components such that electric charge is made to flow along a closed path (a circuit), usually to perform some useful task 
Electric power is the rate at which electric energy is transferred by an electric circuit  The SI unit of power is the watt, one joule per second 
Electricity generation is often done with electric generators, but can also be supplied by chemical sources such as electric batteries or by other means from a wide variety of sources of energy  Electric power is generally supplied to businesses and homes by the electric power industry  Electricity is usually sold by the kilowatt hour (3 6 MJ) which is the product of power in kilowatts multiplied by running time in hours  Electric utilities measure power using electricity meters, which keep a running total of the electric energy delivered to a customer  Unlike fossil fuels, electricity is a low entropy form of energy and can be converted into motion or many other forms of energy with high efficiency 
Electronics deals with electrical circuits that involve active electrical components such as vacuum tubes, transistors, diodes, optoelectronics, sensors and integrated circuits, and associated passive interconnection technologies  The nonlinear behaviour of active components and their ability to control electron flows makes amplification of weak signals possible and electronics is widely used in information processing, telecommunications, and signal processing  The ability of electronic devices to act as switches makes digital information processing possible  Interconnection technologies such as circuit boards, electronics packaging technology, and other varied forms of communication infrastructure complete circuit functionality and transform the mixed components into a regular working system 
Today, most electronic devices use semiconductor components to perform electron control  The study of semiconductor devices and related technology is considered a branch of solid state physics, whereas the design and construction of electronic circuits to solve practical problems come under electronics engineering 
Thus, the work of many researchers enabled the use of electronics to convert signals into high frequency oscillating currents, and via suitably shaped conductors, electricity permits the transmission and reception of these signals via radio waves over very long distances 
Early 20th-century alternator made in Budapest, Hungary, in the power generating hall of a hydroelectric station (photograph by Prokudin-Gorsky, 1905–1915) 
In the 6th century BC, the Greek philosopher Thales of Miletus experimented with amber rods and these experiments were the first studies into the production of electrical energy  While this method, now known as the triboelectric effect, can lift light objects and generate sparks, it is extremely inefficient  It was not until the invention of the voltaic pile in the eighteenth century that a viable source of electricity became available  The voltaic pile, and its modern descendant, the electrical battery, store energy chemically and make it available on demand in the form of electrical energy  The battery is a versatile and very common power source which is ideally suited to many applications, but its energy storage is finite, and once discharged it must be disposed of or recharged  For large electrical demands electrical energy must be generated and transmitted continuously over conductive transmission lines 
Electrical power is usually generated by electro-mechanical generators driven by steam produced from fossil fuel combustion, or the heat released from nuclear reactions; or from other sources such as kinetic energy extracted from wind or flowing water  The modern steam turbine invented by Sir Charles Parsons in 1884 today generates about 80 percent of the electric power in the world using a variety of heat sources  Such generators bear no resemblance to Faraday's homopolar disc generator of 1831, but they still rely on his electromagnetic principle that a conductor linking a changing magnetic field induces a potential difference across its ends  The invention in the late nineteenth century of the transformer meant that electrical power could be transmitted more efficiently at a higher voltage but lower current  Efficient electrical transmission meant in turn that electricity could be generated at centralised power stations, where it benefited from economies of scale, and then be despatched relatively long distances to where it was needed 
Since electrical energy cannot easily be stored in quantities large enough to meet demands on a national scale, at all times exactly as much must be produced as is required  This requires electricity utilities to make careful predictions of their electrical loads, and maintain constant co-ordination with their power stations  A certain amount of generation must always be held in reserve to cushion an electrical grid against inevitable disturbances and losses 
Electricity is a very convenient way to transfer energy, and it has been adapted to a huge, and growing, number of uses  The invention of a practical incandescent light bulb in the 1870s led to lighting becoming one of the first publicly available applications of electrical power  Although electrification brought with it its own dangers, replacing the naked flames of gas lighting greatly reduced fire hazards within homes and factories  Public utilities were set up in many cities targeting the burgeoning market for electrical lighting  In the late 20th century and in modern times, the trend has started to flow in the direction of deregulation in the electrical power sector 
The resistive Joule heating effect employed in filament light bulbs also sees more direct use in electric heating  While this is versatile and controllable, it can be seen as wasteful, since most electrical generation has already required the production of heat at a power station  A number of countries, such as Denmark, have issued legislation restricting or banning the use of resistive electric heating in new buildings  Electricity is however still a highly practical energy source for heating and refrigeration, with air conditioning/heat pumps representing a growing sector for electricity demand for heating and cooling, the effects of which electricity utilities are increasingly obliged to accommodate 
Electricity is used within telecommunications, and indeed the electrical telegraph, demonstrated commercially in 1837 by Cooke and Wheatstone, was one of its earliest applications  With the construction of first intercontinental, and then transatlantic, telegraph systems in the 1860s, electricity had enabled communications in minutes across the globe  Optical fibre and satellite communication have taken a share of the market for communications systems, but electricity can be expected to remain an essential part of the process 
The effects of electromagnetism are most visibly employed in the electric motor, which provides a clean and efficient means of motive power  A stationary motor such as a winch is easily provided with a supply of power, but a motor that moves with its application, such as an electric vehicle, is obliged to either carry along a power source such as a battery, or to collect current from a sliding contact such as a pantograph  Electrically powered vehicles are used in public transportation, such as electric buses and trains, and an increasing number of battery-powered electric cars in private ownership 
Electronic devices make use of the transistor, perhaps one of the most important inventions of the twentieth century, and a fundamental building block of all modern circuitry  A modern integrated circuit may contain several billion miniaturised transistors in a region only a few centimetres square 
A voltage applied to a human body causes an electric current through the tissues, and although the relationship is non-linear, the greater the voltage, the greater the current  The threshold for perception varies with the supply frequency and with the path of the current, but is about 0 1 mA to 1 mA for mains-frequency electricity, though a current as low as a microamp can be detected as an electrovibration effect under certain conditions  If the current is sufficiently high, it will cause muscle contraction, fibrillation of the heart, and tissue burns  The lack of any visible sign that a conductor is electrified makes electricity a particular hazard  The pain caused by an electric shock can be intense, leading electricity at times to be employed as a method of torture  Death caused by an electric shock is referred to as electrocution  Electrocution is still the means of judicial execution in some jurisdictions, though its use has become rarer in recent times 
Electricity is not a human invention, and may be observed in several forms in nature, a prominent manifestation of which is lightning  Many interactions familiar at the macroscopic level, such as touch, friction or chemical bonding, are due to interactions between electric fields on the atomic scale  The Earth's magnetic field is thought to arise from a natural dynamo of circulating currents in the planet's core  Certain crystals, such as quartz, or even sugar, generate a potential difference across their faces when subjected to external pressure  This phenomenon is known as piezoelectricity, from the Greek piezein (πιέζειν), meaning to press, and was discovered in 1880 by Pierre and Jacques Curie  The effect is reciprocal, and when a piezoelectric material is subjected to an electric field, a small change in physical dimensions takes place 
§Bioelectrogenesis in microbial life is a prominent phenomenon in soils and sediment ecology resulting from anaerobic respiration  The microbial fuel cell mimics this ubiquitous natural phenomenon 
Some organisms, such as sharks, are able to detect and respond to changes in electric fields, an ability known as electroreception, while others, termed electrogenic, are able to generate voltages themselves to serve as a predatory or defensive weapon  The order Gymnotiformes, of which the best known example is the electric eel, detect or stun their prey via high voltages generated from modified muscle cells called electrocytes  All animals transmit information along their cell membranes with voltage pulses called action potentials, whose functions include communication by the nervous system between neurons and muscles  An electric shock stimulates this system, and causes muscles to contract  Action potentials are also responsible for coordinating activities in certain plants 
In the 19th and early 20th century, electricity was not part of the everyday life of many people, even in the industrialised Western world  The popular culture of the time accordingly often depicted it as a mysterious, quasi-magical force that can slay the living, revive the dead or otherwise bend the laws of nature  This attitude began with the 1771 experiments of Luigi Galvani in which the legs of dead frogs were shown to twitch on application of animal electricity  "Revitalization" or resuscitation of apparently dead or drowned persons was reported in the medical literature shortly after Galvani's work  These results were known to Mary Shelley when she authored Frankenstein (1819), although she does not name the method of revitalization of the monster  The revitalization of monsters with electricity later became a stock theme in horror films 
As the public familiarity with electricity as the lifeblood of the Second Industrial Revolution grew, its wielders were more often cast in a positive light, such as the workers who "finger death at their gloves' end as they piece and repiece the living wires" in Rudyard Kipling's 1907 poem Sons of Martha  Electrically powered vehicles of every sort featured large in adventure stories such as those of Jules Verne and the Tom Swift books  The masters of electricity, whether fictional or real—including scientists such as Thomas Edison, Charles Steinmetz or Nikola Tesla—were popularly conceived of as having wizard-like powers 
With electricity ceasing to be a novelty and becoming a necessity of everyday life in the later half of the 20th century, it required particular attention by popular culture only when it stops flowing, an event that usually signals disaster  The people who keep it flowing, such as the nameless hero of Jimmy Webb’s song "Wichita Lineman" (1968), are still often cast as heroic, wizard-like figures 
Ampère's circuital law, connects the direction of an electric current and its associated magnetic currents 
^ Diogenes Laertius  R D  Hicks (ed )  "Lives of Eminent Philosophers, Book 1 Chapter 1 "  Perseus Digital Library  Tufts University  Retrieved 5 February 2017  Aristotle and Hippias affirm that, arguing from the magnet and from amber, he attributed a soul or life even to inanimate objects 
^ Aristotle  Daniel C  Stevenson (ed )  "De Animus (On the Soul) Book 1 Part 2 (B4 verso)"  The Internet Classics Archive  Translated by J A  Smith  Retrieved 5 February 2017  Thales, too, to judge from what is recorded about him, seems to have held soul to be a motive force, since he said that the magnet has a soul in it because it moves the iron 
^ a b c Guarnieri, M  (2014)  "Electricity in the age of Enlightenment"  IEEE Industrial Electronics Magazine  8 (3): 60–63  doi:10 1109/MIE 2014 2335431 
^ Srodes, James (2002), Franklin: The Essential Founding Father, Regnery Publishing, pp  92–94, ISBN 0-89526-163-4 It is uncertain if Franklin personally carried out this experiment, but it is popularly attributed to him 
^ a b Guarnieri, M  (2014)  "The Big Jump from the Legs of a Frog"  IEEE Industrial Electronics Magazine  8 (4): 59–61, 69  doi:10 1109/MIE 2014 2361237 
^ Hertz, Heinrich (1887)  "Ueber den Einfluss des ultravioletten Lichtes auf die electrische Entladung"  Annalen der Physik  267 (8): S  983–1000  Bibcode:1887AnP   267  983H  doi:10 1002/andp 18872670827 
^ "The Nobel Prize in Physics 1921"  Nobel Foundation  Retrieved 2013-03-16 
^ John Sydney Blakemore, Solid state physics, pp  1–3, Cambridge University Press, 1985 ISBN 0-521-31391-0 
^ Richard C  Jaeger, Travis N  Blalock, Microelectronic circuit design, pp  46–47, McGraw-Hill Professional, 2003 ISBN 0-07-250503-6 
^ "The repulsive force between two small spheres charged with the same type of electricity is inversely proportional to the square of the distance between the centres of the two spheres " Charles-Augustin de Coulomb, Histoire de l'Academie Royal des Sciences, Paris 1785 
^ Sewell, Tyson (1902), The Elements of Electrical Engineering, Lockwood, p  18   The Q originally stood for 'quantity of electricity', the term 'electricity' now more commonly expressed as 'charge' 
^ a b Berkson, William (1974), Fields of Force: The Development of a World View from Faraday to Einstein, Routledge, p  370, ISBN 0-7100-7626-6 Accounts differ as to whether this was before, during, or after a lecture 
^ "Lab Note #105 EMI Reduction – Unsuppressed vs  Suppressed"  Arc Suppression Technologies  April 2011  Retrieved March 7, 2012 
^ Almost all electric fields vary in space  An exception is the electric field surrounding a planar conductor of infinite extent, the field of which is uniform 
^ Paul J  Nahin (9 October 2002)  Oliver Heaviside: The Life, Work, and Times of an Electrical Genius of the Victorian Age  JHU Press  ISBN 978-0-8018-6909-9 
^ "The Bumpy Road to Energy Deregulation"  EnPowered  2016-03-28 
^ a b c d e f g h Van Riper, op cit , p  71 
Look up electricity in Wiktionary, the free dictionary 
Basic Concepts of Electricity chapter from Lessons In Electric Circuits Vol 1 DC book and series 



ground_truth: The maximum velocity scales with the square root of the amplitude.