# The contribution of sensory-motor interactions to imagined speech

Sophie Bouton, Timothée Proix, Stéphanie Martin, Anne-Lise Giraud Mamessier

<figure><img src="image_1.png" /></figure>

## To cite this version:

Sophie Bouton, Timothée Proix, Stéphanie Martin, Anne-Lise Giraud Mamessier. The contribution of sensory-motor interactions to imagined speech. FENS Forum 2022, Jul 2022, Paris, France. .
hal-04579676

HAL Id: hal-04579676

https://cnrs.hal.science/hal-04579676

Submitted on 21 May 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_1.png" /></figure>

# THE CONTRIBUTION OF SENSORY-MOTOR INTERACTIONS TO IMAGINED SPEECH

Sophie BOUTON¹,², Timothée PROIX³, Stephanie MARTIN⁴, Anne-Lise GIRAUD²

sophie.bouton@cnrs.fr

1. Laboratoire Dynamique du Langage, Université Lyon 2, Lyon, FRANCE

2. Institut de l'Audition, Institut Pasteur, Paris, FRANCE

3. Department of Basic Neurosciences, Faculty of Medicine, University of Geneva, Geneva, SWITZERLAND

4. NextSense Inc., Mountain View, California, USA

## RECONSTRUCTING SPEECH USING A
BRAIN-MACHINE INTERFACE

Reconstructing speech from neural activity using brain-computer interfaces could make it easier for people with severe speech production deficits to communicate.

The decoding of **imagined speech** has met with only limited success, mainly because the **neural features of covert speech** are not precisely described.

Would sensory-motor interactions be a feature to exploit?

## COMPARING SILENCE READING
AND IMAGINED SPEECH

Data from 21 participants were analysed

26 participants took part in the experiment. No triggers N = 2 participants, metallic object in the mouth N = 2, task not completed N = 1

The MEG experiment consisted of a 1 h-long
session.

whole-head acquisition
recording sampling rate of 2,035 Hz
3 EEG channels synchronized (2 for EOG, 1 for ECG)
Polhemus

<figure><img src="image_2.png" /></figure>

MEG
MARSEILLE

<figure><img src="image_3.png" /></figure>

Paradigm

<figure><img src="image_4.png" /></figure>

Syllabic stimulation sampling rate: 1.25 Hz

Hypothesis

The coupling between the **sensory-motor systems** is at the origin of the activation of syllabic and phonemic representations of **silent speech**.

## ERP – Analyses

<figure><img src="image_5.png" /></figure>

Regions of interest

Left

<figure><img src="image_6.png" /></figure>

Subcentral Gyrus/Sulcus

<figure><img src="image_7.png" /></figure>

Superior Temporal Gyrus

<figure><img src="image_8.png" /></figure>

Right

<figure><img src="image_9.png" /></figure>

Subcentral Gyrus/Sulcus

<figure><img src="image_10.png" /></figure>

Superior temporal Gyrus

<figure><img src="image_11.png" /></figure>

Silent Reading
Imaging Saying

Connectivity – granger analysis

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

**Temporal** and **motor** areas show an earlier evoked response, and a stronger **early** information transfer in the **Imagine Saying** condition compared to the **Silent Reading** condition

## TF – Analyses

Pairwise comparison

Beta (13-25 Hz)

<figure><img src="image_14.png" /></figure>

Left

<figure><img src="image_15.png" /></figure>

Subcentral Gyrus/Sulcus

<figure><img src="image_16.png" /></figure>

Right

<figure><img src="image_17.png" /></figure>

Low-gamma (25-50 Hz)

<figure><img src="image_18.png" /></figure>

Superior temporal Gyrus

<figure><img src="image_19.png" /></figure>

time (ms)

<figure><img src="image_20.png" /></figure>

The production of **imagined speech** is associated with bursts of **beta** (13-25 Hz) neural oscillations in the **sensory and motor** areas.

Granger Causality

<figure><img src="image_21.png" /></figure>

SubCentral *R* to SubCentral *L*

Sup Temporal *R* to SubCentral *L*

<figure><img src="image_22.png" /></figure>

In the **imagined speech** condition the propagation of information between motor and sensory areas is based on a **cross-frequency coupling** between beta and low gamma frequencies.

**Acknowledgments.** This work was funded by EU FET-BrainCom project (A-L. G.), and is supported by grants RD-2016-R (the Fondation pour l'Audition to S.B.) and 193542 (Swiss National Science Foundation career grant to T.P.).

<figure><img src="image_23.png" /></figure>

Institut
de
l'Audition

<figure><img src="image_24.png" /></figure>

INSTITUT
PASTEUR

<figure><img src="image_25.png" /></figure>

<figure><img src="image_26.png" /></figure>
<figure><img src="image_3.png" /></figure>

# THE CONTRIBUTION OF SENSORY-MOTOR INTERACTIONS TO IMAGINED SPEECH

Sophie BOUTON¹,², Timothée PROIX³, Stephanie MARTIN⁴, Anne-Lise GIRAUD²

sophie.bouton@cnrs.fr

1. Laboratoire Dynamique du Langage, Université Lyon 2, Lyon, FRANCE

2. Institut de l'Audition, Institut Pasteur, Paris, FRANCE

3. Department of Basic Neurosciences, Faculty of Medicine, University of Geneva, Geneva, SWITZERLAND

4. NextSense Inc., Mountain View, California, USA

## RECONSTRUCTING SPEECH USING A
BRAIN-MACHINE INTERFACE

Reconstructing speech from neural activity using brain-computer interfaces could make it easier for people with severe speech production deficits to communicate.

The decoding of **imagined speech** has met with only limited success, mainly because the **neural features of covert speech** are not precisely described.

Would sensory-motor interactions be a feature to exploit?

## COMPARING SILENCE READING
AND IMAGINED SPEECH

Data from 21 participants were analysed

26 participants took part in the experiment. No triggers N = 2 participants, metallic object in the mouth N = 2, task not completed N = 1

The MEG experiment consisted of a 1 h-long
session.

whole-head acquisition
recording sampling rate of 2,035 Hz
3 EEG channels synchronized (2 for EOG, 1 for ECG)
Polhemus

4D Neuroimaging magnes 3600

<figure><img src="image_4.png" /></figure>

MEG
MARSEILLE

<figure><img src="image_5.png" /></figure>

Paradigm

<figure><img src="image_6.png" /></figure>

Syllabic stimulation sampling rate: 1.25 Hz

Hypothesis

The coupling between the **sensory-motor systems** is at the origin of the activation of syllabic and phonemic representations of **silent speech**.

## ERP – Analyses

<figure><img src="image_7.png" /></figure>

Regions of interest

Left

<figure><img src="image_8.png" /></figure>

Subcentral Gyrus/Sulcus

<figure><img src="image_9.png" /></figure>

Superior Temporal Gyrus

<figure><img src="image_10.png" /></figure>

Right

<figure><img src="image_11.png" /></figure>

Subcentral Gyrus/Sulcus

<figure><img src="image_12.png" /></figure>

Superior temporal Gyrus

<figure><img src="image_13.png" /></figure>

Silent Reading
Imaging Saying

Connectivity – granger analysis

<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>

**Temporal** and **motor** areas show an earlier evoked response, and a stronger **early** information transfer in the **Imagine Saying** condition compared to the **Silent Reading** condition

## TF – Analyses

Pairwise comparison

Beta (13-25 Hz)

<figure><img src="image_16.png" /></figure>

Left

<figure><img src="image_17.png" /></figure>

Subcentral Gyrus/Sulcus

<figure><img src="image_18.png" /></figure>

Right

<figure><img src="image_19.png" /></figure>

Low-gamma (25-50 Hz)

<figure><img src="image_20.png" /></figure>

Superior temporal Gyrus

<figure><img src="image_21.png" /></figure>

time (ms)

<figure><img src="image_22.png" /></figure>

The production of **imagined speech** is associated with bursts of **beta** (13-25 Hz) neural oscillations in the **sensory and motor** areas.

Granger Causality

<figure><img src="image_23.png" /></figure>

SubCentral *R* to SubCentral *L*

Sup Temporal *R* to SubCentral *L*

<figure><img src="image_24.png" /></figure>

In the **imagined speech** condition the propagation of information between motor and sensory areas is based on a **cross-frequency coupling** between beta and low gamma frequencies.

**Acknowledgments.** This work was funded by EU FET-BrainCom project (A-L. G.), and is supported by grants RD-2016-R (the Fondation pour l'Audition to S.B.) and 193542 (Swiss National Science Foundation career grant to T.P.).

<figure><img src="image_25.png" /></figure>

DD

LABORATOIRE
DYNAMIQUE
DU L'AUDITION

Institut
de
l'Audition

<figure><img src="image_26.png" /></figure>

INSTITUT
PASTEUR

<figure><img src="image_27.png" /></figure>

FONDATION
POUR
L'AUDITION

A L'ECOUTE DE LA VIE

<figure><img src="image_28.png" /></figure>

HORIZON 2020