# Classification of Event-Related Potentials with Regularized Spatiotemporal LCMV Beamforming
## 2 Materials and Methods
### Covariance Matrix Regularization
#### 2.3.3 Spatiotemporal Beamforming with Kronecker-Toeplitz-Structured Covariance  
Exploiting prior knowledge about the spatiotemporal structure of the EEG signal leads to a more regularized estimator of the covariance. When viewing the example of empirical spatiotemporal EEG covariance in Figure 1a, it becomes clear that this matrix consists  
Figure 1: Different estimators of the covariance and inverse covariance of 100 epochs of data from _Subject 01_ for channels _F2_, _C2_, _P2_, and _Oz_ and time samples between 0.1 s and 0.6 s. Regularized estimators of the inverse covariance exhibit less extreme values and have a sparser structure. **(A,B)** Empirical covariance and inverse covariance matrices. **(C,D)** Shrunk covariance and inverse covariance matrices with \(\alpha=0.14\) as determined by the closed-form leave-one-out cross-validation (LOOCV) method. **(E,F)** Kronecker–Toeplitz-structured covariance and inverse covariance matrices. **(G,H)** Spatial Kronecker factor of the Kronecker–Toeplitz-structured shrunk estimator and its inverse. **(I,J)** Temporal Kronecker factor of the Kronecker–Toeplitz-structured shrunk estimator and its inverse.  
of a block pattern of repeated, similar matrices. Due to the multi-channel nature of the signal, we assume that the covariance of spatiotemporal EEG epochs is a Kronecker product of two smaller matrices [35; 36; 37], as expressed by:  
\[\hat{C}_{\text{struct}}=\hat{S}\otimes\hat{T} \tag{11}\]  
with \(\otimes\) denoting the Kronecker product operator. \(\hat{S}\in\mathbb{R}^{c\times c}\) and \(\hat{T}\in\mathbb{R}^{s\times s}\) correspond to estimators of the spatial and temporal covariance of the data, respectively. Furthermore, because the temporal covariance of the EEG-signal is stationary (i.e., it is only dependent on interval length between covarying time samples) [38], it is assumed to have a Toeplitz-matrix structure:  
\[\hat{t}_{i,j}=\hat{t}_{i+1,j+1} \tag{12}\]  
Property 1 then leads to Equation (13) to estimate the inverse covariance.  
**Property 1**.: \((U\otimes V)^{+}=U^{+}\otimes V^{+}\) _for any non-singular matrices U and \(V\)[39]._  
\[\widehat{C^{-1}}_{\text{struct}}=\hat{S}^{+}\otimes\hat{T}^{+} \tag{13}\]  
Finally, based on Property 2, Equation (4) can be reformulated more efficiently as Equation (14).  
**Property 2**.: \((U\otimes V)\cdot vec(W)=vec(VWU^{\intercal})\) _for any matrices \(U\in\mathbb{R}^{p\times p}\), \(V\in\mathbb{R}^{q\times q}\), and \(W\in\mathbb{R}^{p\times q}\)[40]._  
\[\hat{\mathbf{w}}_{\text{struct}}=\frac{\hat{S}^{+}A^{\intercal}\hat{T}^{+}}{ \mathbf{a}\cdot\text{vec}(\hat{S}^{+}A^{\intercal}\hat{T}^{+})} \tag{14}\]  
Using Equation (14) removes the need to calculate the full, high-dimensional Kronecker product \(\hat{S}^{+}\otimes\hat{T}^{+}\). Figure 1e,f show examples of the structured covariance and inverse covariance estimators, respectively, consisting of a spatial Kronecker factor (Figure 1g,h) and a temporal component (Figure 1i,j).  
The Kronecker approach has shown significant performance yields in different linear spatiotemporal EEG and MEG applications [24; 37; 41; 42; 43]. [PERSON] and [PERSON] [25] applied a Kronecker-structured covariance estimator to ERP classification with linear models in a post hoc fashion. Our work goes further by embedding the Kronecker structure in the spatiotemporal beamformer training process, using a data-adaptive shrinkage method, and regularizing the covariance further by imposing a Toeplitz structure on the temporal covariance.