# Enhancing Motor Imagery BCI classification with Block-Toeplitz Augmented Covariance Matrices and Siegel metric

Igor Carrara, Théodore Papadopoulos

► To cite this version:

Igor Carrara, Théodore Papadopoulos. Enhancing Motor Imagery BCI classification with Block-Toeplitz Augmented Covariance Matrices and Siegel metric. CORTICO 2024 - Journée Jeunes Chercheurs en Interfaces Cerveau-Ordinateur et Neurofeedback, May 2024, Nancy, France. hal-04590774

HAL Id: hal-04590774

https://inria.hal.science/hal-04590774

Submitted on 28 May 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Block-Toeplitz Augmented Covariance
Matrices and Siegel metric for BCI

<figure><img src="image_1.png" /></figure>

<figure><img src="image_2.png" /></figure>

## I. Carrara¹*, T. Papadopoulo¹

¹ Université Côte d'Azur, INRIA, Cronos Team, France

<figure><img src="image_3.png" /></figure>

Brain Computer Interfaces or **BCI** can be defined as a technology that measures brain activity and translates that activity into instructions for a digital system. We will focus on **non-invasive (EEG)-based BCI (BCI-EEG)**.

Let us consider a typical EEG signal after preprocessing. In the figure on the left, we show an EEG signal measured with 4 different electrodes. Our work will be focused on **Motor Imagery (MI) BCI**, so we want to understand if the subject, for example, is thinking about moving the left or the right hand.

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /><figcaption>Phase Space Reconstruction</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Tangent Space Projection</figcaption></figure>

Can we leverage the Block-Toeplitz structure
of the Augmented Covariance Matrix (ACM)
to improve the results of the ACM?

## METHODS: BT-ACM+TS+SVM

### Stardard Riemann approach [1]:

Extract sample spatial covariances which are Symmetric Positive Definite (SPD) matrices and classify them using Minimum Distance to Mean (MDM) or classical machine learning procedure on the tangent space (SVM, LDA, ...)

$$ \mathrm{SPD}_d = \{ \mathbf{M} \in \mathbb{R}^{d \times d} \mid x^T \mathbf{M} x > 0 \ \forall x \in \mathbb{R}^d \setminus \{0\} \} $$

<figure><img src="image_13.png" /><figcaption>Figure extracted from [1]</figcaption></figure>

### Augmented Covariance Method [2]:

ACM has shown to enhance the performance for Motor Imagery (MI) BCI-EEG classification. This approach combines the **Phase Space Reconstruction (PSR) method** derived from non linear dynamics

with the **Riemannian approach**. This method is based on the ACM matrix, that depends on a **embedding dimension** $D_E$ and on a **delay** $\tau$. This matrix, when properly regularized, belongs to $\mathrm{SPD}_{dD_E}$.

$$ \Gamma_{Aug} = \begin{bmatrix} \Gamma_0 & \Gamma_{-1} & \Gamma_{-2} & \cdots \\ \Gamma_1 & \Gamma_0 & \Gamma_{-1} & \cdots \\ \Gamma_2 & \Gamma_1 & \Gamma_0 & \cdots \\ \vdots & \vdots & \vdots & \ddots \\ \Gamma_{p-1} & \Gamma_{p-2} & \Gamma_{p-3} & \cdots \end{bmatrix} $$

### Block-Toeplitz ACM:

With some care, ACM also has a specific **Block-Toeplitz structure**, with blocks of dimension $d \times d$, i.e. the **ACM belongs to the SPD matrices that possess also a Block-Toeplitz structure**

$$ \mathrm{BTSPD}_{d \times p} = \{ \mathbf{M} \in \mathbb{S}_d \cap \mathrm{BT}_{d \times p} \mid x^T \mathbf{M} x > 0 \ \forall x \in \mathbb{R}^{d \times p} \setminus \{0\} \} $$

The **off-diagonal block are then forced to have a well specific structure to respect these constraint**. To make this more evident, it is possible to use the **Verblunsky transformation** [3] to create an homeomorphism to make this relation self-evident.

$$ \mathrm{BTSPD}_{d \times p} \rightarrow \mathrm{SPD}_d \times \mathrm{SD}_d^{p-1} \\ \Gamma_{Aug} \mapsto (\Gamma_0, \Omega_1, \dots, \Omega_{p-1}). $$

$$ \Omega_i \in \mathbb{S}_d \quad \mathrm{SD}_d = \{ \mathbf{M} \in \mathbb{C}^{d \times d} ; \mathbf{I} - \mathbf{M}\mathbf{M}^H > 0 \} $$

This transformation allows us to treat the $\mathrm{BTSPD}_{d \times p}$ as a **Kähler manifold** [4], on which is possible to derive the **metric on this manifold** as the Hessian of the Kähler potential.

$$ ds^2 = p \operatorname{trace} (\mathbf{P}_0^{-1} d\mathbf{P}_0 \mathbf{P}_0^{-1} d\mathbf{P}_0) + \sum_{l=1}^{p-1} (p-l) \operatorname{trace} \left( (\mathbf{I} - \Omega_l \Omega_l^H)^{-1} d\Omega_l (\mathbf{I} - \Omega_l^H \Omega_l)^{-1} d\Omega_l^H \right) $$

The first term of equation is identifiable as the metric for the $\mathrm{SPD}_d$ space. The other terms represents the metric for the Siegel disk space $\mathrm{SD}_d$ repeated $p-1$ times [5], i.e. the **space of Block-Toeplitz SPD matrices is equipped with a product Riemannian metric**. This allows us to treat each component into their proper geometrical space, resulting in the new approach **BT-ACM+TS+SVM**: Block-Toeplitz Augmented Covariance Matrix (BT-ACM) with Tangent Space projection (TS) and SVM classifier.

## RESULTS

To validate our approach and assess its replicability, we applied it on several open datasets with the **MOABB framework** [6]. We compared the obtained results with the **state-of-the-art ML** and **DL algorithms**. Results are tested on task **right hand vs left hand** in a **within-session evaluation**.

<table><thead><tr><th>Pipeline</th><th>BNCI2014001</th><th>BNCI2014004</th><th>Cho2017</th></tr></thead><tbody><tr><td>BT-ACM+TS+SVM</td><td>0.89 ± 0.11</td><td><strong>0.83 ± 0.14</strong></td><td><strong>0.76 ± 0.14</strong></td></tr><tr><td>CSP+LDA</td><td>0.82 ± 0.17</td><td>0.80 ± 0.15</td><td>0.71 ± 0.15</td></tr><tr><td>MDM</td><td>0.82 ± 0.15</td><td>0.78 ± 0.16</td><td>0.63 ± 0.14</td></tr><tr><td>FgMDM</td><td>0.87 ± 0.12</td><td>0.79 ± 0.15</td><td>0.73 ± 0.13</td></tr><tr><td>TS+EL</td><td>0.86 ± 0.13</td><td>0.80 ± 0.15</td><td><strong>0.76 ± 0.14</strong></td></tr><tr><td>TS+SVM</td><td>0.87 ± 0.14</td><td>0.79 ± 0.15</td><td>0.75 ± 0.14</td></tr><tr><td>ACM+TS+SVM</td><td><strong>0.92 ± 0.10</strong></td><td><strong>0.83 ± 0.15</strong></td><td>0.74 ± 0.15</td></tr><tr><td>ShallowConvNet</td><td>0.86 ± 0.14</td><td>0.72 ± 0.18</td><td>0.74 ± 0.15</td></tr><tr><td>DeepConvNet</td><td>0.82 ± 0.16</td><td>0.72 ± 0.19</td><td>0.72 ± 0.13</td></tr><tr><td>EEGNet</td><td>0.77 ± 0.19</td><td>0.70 ± 0.20</td><td>0.67 ± 0.16</td></tr></tbody></table>

BT-ACM+TS+SVM is **statistically significantly better** than the state of the art pipelines (Except for TS+EL), moreover our method produce **less carbon emission** and computational time respect to basic ACM.

<figure><img src="image_14.png" /></figure>

## TAKE HOME MESSAGE

Leveraging the **Block-Toeplitz property of the ACM** improves the performance, or is at worst comparable, in several dataset. Moreover, since the decomposition in product of Riemannian manifold, the **computational power**, both in term of time and in carbon emission is **improved with respect to the standard ACM**.

<figure><img src="image_15.png" /></figure>

SCAN ME

**Acknowledgements:** This work has been partially financed by a EUR DS4H/Neuromod fellowship. The authors are grateful to the OPAL infrastructure from Université Côte d'Azur for providing resources and support.

**References:**
[1] Alexandre Barachant et al. "Riemannian geometry applied to BCI classification". In: International conference on latent variable analysis and signal separation. Springer. 2010, pp. 629-636.
[2] Igor Carrara et al. "Classification of BCI-EEG based on augmented covariance matrix" arXiv preprint arXiv:2302.04508 (2023).
[3] Dette, Holger et al. "Matrix measures on the unit circle, moment spaces, orthogonal polynomials and the Geronimus relations." Linear algebra and its applications 432.7 (2010): 1609-1626.
[4] Jeuris, Ben et al. "The Kahler mean of block-Toeplitz matrices with Toeplitz structured blocks." SIAM Journal on matrix analysis and applications 37.3 (2016): 1151-1175.
[5] Cabanes, Yann. Multidimensional complex stationary centered Gaussian autoregressive time series machine learning in Poincaré and Siegel disks: application for audio and radar clutter classification. Diss. Université de Bordeaux, 2022.
[6] Aristimunha Bruno et al. Mother of all bci benchmarks. Zenodo. 2023.
# Block-Toeplitz Augmented Covariance
Matrices and Siegel metric for BCI

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

## I. Carrara¹*, T. Papadopoulo¹

¹ Université Côte d'Azur, INRIA, Cronos Team, France

<figure><img src="image_4.png" /></figure>

Brain Computer Interfaces or **BCI** can be defined as a technology that measures brain activity and translates that activity into instructions for a digital system. We will focus on **non-invasive (EEG)-based BCI (BCI-EEG)**.

Let us consider a typical EEG signal after preprocessing. In the figure on the left, we show an EEG signal measured with 4 different electrodes. Our work will be focused on **Motor Imagery (MI) BCI**, so we want to understand if the subject, for example, is thinking about moving the left or the right hand.

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /><figcaption>Phase Space Reconstruction</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Tangent Space Projection</figcaption></figure>

Can we leverage the Block-Toeplitz structure
of the Augmented Covariance Matrix (ACM)
to improve the results of the ACM?

## METHODS: BT-ACM+TS+SVM

### Stardard Riemann approach [1]:

Extract sample spatial covariances which are Symmetric Positive Definite (SPD) matrices and classify them using Minimum Distance to Mean (MDM) or classical machine learning procedure on the tangent space (SVM, LDA, ...)

$$ \mathrm{SPD}_d = \{ \mathbf{M} \in \mathbb{R}^{d \times d} \mid x^T \mathbf{M} x > 0 \ \forall x \in \mathbb{R}^d \setminus \{0\} \} $$

<figure><img src="image_14.png" /><figcaption>Figure extracted from [1]</figcaption></figure>

### Augmented Covariance Method [2]:

ACM has shown to enhance the performance for Motor Imagery (MI) BCI-EEG classification. This approach combines the **Phase Space Reconstruction (PSR) method** derived from non linear dynamics

with the **Riemannian approach**. This method is based on the ACM matrix, that depends on a **embedding dimension** $D_E$ and on a **delay** $\tau$. This matrix, when properly regularized, belongs to $\mathrm{SPD}_{dD_E}$.

$$ \Gamma_{Aug} = \begin{bmatrix} \Gamma_0 & \Gamma_{-1} & \Gamma_{-2} & \dots \\ \Gamma_1 & \Gamma_0 & \Gamma_{-1} & \dots \\ \Gamma_2 & \Gamma_1 & \Gamma_0 & \dots \\ \vdots & \vdots & \vdots & \ddots \\ \Gamma_{p-1} & \Gamma_{p-2} & \Gamma_{p-3} & \dots \end{bmatrix} $$

### Block-Toeplitz ACM:

With some care, ACM also has a specific **Block-Toeplitz structure**, with blocks of dimension $d \times d$, i.e. the **ACM belongs to the SPD matrices that possess also a Block-Toeplitz structure**

$$ \mathrm{BTSPD}_{d \times p} = \{ \mathbf{M} \in \mathbb{S}_d \cap \mathrm{BT}_{d \times p} \mid x^T \mathbf{M} x > 0 \ \forall x \in \mathbb{R}^{d \times p} \setminus \{0\} \} $$

The **off-diagonal block are then forced to have a well specific structure to respect these constraint**. To make this more evident, it is possible to use the **Verblunsky transformation** [3] to create an homeomorphism to make this relation self-evident.

$$ \mathrm{BTSPD}_{d \times p} \rightarrow \mathrm{SPD}_d \times \mathrm{SD}_d^{p-1} \\ \Gamma_{Aug} \mapsto (\Gamma_0, \Omega_1, \dots, \Omega_{p-1}) $$

$$ \Omega_i \in \mathbb{S}_d \quad \mathrm{SD}_d = \{ \mathbf{M} \in \mathbb{C}^{d \times d} ; \mathbf{I} - \mathbf{M}\mathbf{M}^H > 0 \} $$

This transformation allows us to treat the $\mathrm{BTSPD}_{d \times p}$ as a **Kähler manifold** [4], on which is possible to derive the **metric on this manifold** as the Hessian of the Kähler potential.

$$ ds^2 = p \operatorname{trace} (\mathbf{P}_0^{-1} d\mathbf{P}_0 \mathbf{P}_0^{-1} d\mathbf{P}_0) + \sum_{l=1}^{p-1} (p-l) \operatorname{trace} \left( (\mathbf{I} - \Omega_l \Omega_l^H)^{-1} d\Omega_l (\mathbf{I} - \Omega_l^H \Omega_l)^{-1} d\Omega_l^H \right) $$

The first term of equation is identifiable as the metric for the $\mathrm{SPD}_d$ space. The other terms represents the metric for the Siegel disk space $\mathrm{SD}_d$ repeated $p-1$ times [5], i.e. the **space of Block-Toeplitz SPD matrices is equipped with a product Riemannian metric**. This allows us to treat each component into their proper geometrical space, resulting in the new approach **BT-ACM+TS+SVM**: Block-Toeplitz Augmented Covariance Matrix (BT-ACM) with Tangent Space projection (TS) and SVM classifier.

## RESULTS

To validate our approach and assess its replicability, we applied it on several open datasets with the **MOABB framework** [6]. We compared the obtained results with the **state-of-the-art ML** and **DL algorithms**. Results are tested on task **right hand vs left hand** in a **within-session evaluation**.

<table><thead><tr><th>Pipeline</th><th>BNCI2014001</th><th>BNCI2014004</th><th>Cho2017</th></tr></thead><tbody><tr><td>BT-ACM+TS+SVM</td><td>0.89 ± 0.11</td><td><b>0.83 ± 0.14</b></td><td><b>0.76 ± 0.14</b></td></tr><tr><td>CSP+LDA</td><td>0.82 ± 0.17</td><td>0.80 ± 0.15</td><td>0.71 ± 0.15</td></tr><tr><td>MDM</td><td>0.82 ± 0.15</td><td>0.78 ± 0.16</td><td>0.63 ± 0.14</td></tr><tr><td>FgMDM</td><td>0.87 ± 0.12</td><td>0.79 ± 0.15</td><td>0.73 ± 0.13</td></tr><tr><td>TS+EL</td><td>0.86 ± 0.13</td><td>0.80 ± 0.15</td><td><b>0.76 ± 0.14</b></td></tr><tr><td>TS+SVM</td><td>0.87 ± 0.14</td><td>0.79 ± 0.15</td><td>0.75 ± 0.14</td></tr><tr><td>ACM+TS+SVM</td><td><b>0.92 ± 0.10</b></td><td><b>0.83 ± 0.15</b></td><td>0.74 ± 0.15</td></tr><tr><td>ShallowConvNet</td><td>0.86 ± 0.14</td><td>0.72 ± 0.18</td><td>0.74 ± 0.15</td></tr><tr><td>DeepConvNet</td><td>0.82 ± 0.16</td><td>0.72 ± 0.19</td><td>0.72 ± 0.13</td></tr><tr><td>EEGNet</td><td>0.77 ± 0.19</td><td>0.70 ± 0.20</td><td>0.67 ± 0.16</td></tr></tbody></table>

BT-ACM+TS+SVM is **statistically significantly better** than the state of the art pipelines (Except for TS+EL), moreover our method produce **less carbon emission** and computational time respect to basic ACM.

<figure><img src="image_15.png" /></figure>

## TAKE HOME MESSAGE

Leveraging the **Block-Toeplitz property of the ACM** improves the performance, or is at worst comparable, in several dataset. Moreover, since the decomposition in product of Riemannian manifold, the **computational power**, both in term of time and in carbon emission is **improved with respect to the standard ACM**.

<figure><img src="image_16.png" /></figure>

SCAN ME

**Acknowledgements:** This work has been partially financed by a EUR DS4H/Neuromod fellowship. The authors are grateful to the OPAL infrastructure from Université Côte d'Azur for providing resources and support.

**References:**
[1] Alexandre Barachant et al. "Riemannian geometry applied to BCI classification". In: International conference on latent variable analysis and signal separation. Springer. 2010, pp. 629-636.
[2] Igor Carrara et al. "Classification of BCI-EEG based on augmented covariance matrix" arXiv preprint arXiv:2302.04508 (2023).
[3] Dette, Holger et al. "Matrix measures on the unit circle, moment spaces, orthogonal polynomials and the Geronimus relations." Linear algebra and its applications 432.7 (2010): 1609-1626.
[4] Jeuris, Ben et al. "The Kahler mean of block-Toeplitz matrices with Toeplitz structured blocks." SIAM Journal on matrix analysis and applications 37.3 (2016): 1151-1175.
[5] Cabanes, Yann. Multidimensional complex stationary centered Gaussian autoregressive time series machine learning in Poincaré and Siegel disks: application for audio and radar clutter classification. Diss. Université de Bordeaux, 2022.
[6] Aristimunha Bruno et al. Mother of all bci benchmarks. Zenodo. 2023.