Speech artifact removal from EEG recordings of spoken
word production with tensor decomposition
1,3

1,2

1,2

3

Holy Lovenia , Hiroki Tanaka , Sakriani Sakti , Ayu Purwarianti , and Satoshi Nakamura1,21) Nara Institute of Science and Technology, Japan
2) RIKEN, Center for Advanced Intelligence Project AIP, Japan
3) Department of Informatics, Bandung Institute of Technology, Indonesia
Email: holy.lovenia@gmail.com

B. Overview

1. Introduction
A. Background

3. ExperimentResearch pipeline

Electrode placement(picture-naming experiment)

Lip EMG

Speech artifactcontamination

SAR-ICA > BSS-CCA,Porcaro et al, 2015

Eye artifact removal

C. Research Objectives

SAR-ICA

EEG
dataBSS-CCA

Recordings: 27 (+ 1 reference) EEG channels, 2
EMG channels, & 2 EOG channels
Picture stimuli: 45 line-drawings of commonobjects (Nishimoto et al, 2005)

Channel≈Frequency

EEG
data

DIFFIT

Number ofcomponents(by Timmerman etal, 2000)

Analysis
Statistical
calculation

Spectralcalculation(entropy,kurtosis, globalkurtosis)(PSD
correlation withlip EMG)

To compare performances of all methods-

Number of components range: 5-26
Estimated number of components (across subjects)- Lowest : 8- Highest : 24
- Average : 15

Component features used in analysis
1.
2.
3.
4.
5.

Analysis
Features

Manual artifact detection

Manual artifact
identification

Evaluation

Control cycle (to ensure only artifact is removed)(correlation with lip EMG)

Clustering

Visualization of channel mode in space
Visualization of frequency mode
Frequency mode correlation with lip EMG
Visualization of time mode
Time mode correlation with lip EMG

Time mode correlation calculated between

Predictions
whereClustering

Speech artifact cluster (SAR-ICA)
Cleaned data cluster (SAR-ICA)
Raw data without EOG cluster
1.
2.
3.

Raw data without EOG cluster
Speech artifact cluster (CPD)
Cleaned data cluster (CPD)

BSS-CCA decomposes EEG signals into sourcesin decreasing order of autocorrelation- EMG activity is weakly autocorrelated overtime- Brain activity is more autocorrelated

is averaged speech onset of the subject, andis end of trial

Manual observation of the difference
between before and after removing
the artifactual component

BSS-CCA (Vos et al, 2010)

Assumption: EEG → lower power at high freq,
EMG → higher power at high freq

3s

B. Performance Measure
Detailed evaluation results: the absolute correlation
between speech artifacts and cleaned data with lip
EMG in the time domain for 0-1350 ms (p < 0.01).

Grand-average cluster

R (0-1350 ms)

SAR-ICA’s Speech Artifact

0.875

CPD’s Speech Artifact

0.985

SAR-ICA’s Cleaned Data

0.351

BSS-CCA’s Cleaned Data

0.413

CPD’s Cleaned Data

0.101

Speech artifact by the proposed method
Comparison of the normalized grand-average lip EMG anddecomposed speech artifacts by the proposed method

Calculate the Pearson correlation (R) between
grand-average clusters and the lip EMG during0-1350 ms in the time domain

Interpretation
The higher the correlation, the stronger theassociation with lip EMG. Lip EMG is assumed
to represent the original speech artifacts.
- Speech artifact: the higher, the better- Cleaned data: the lower, the better

Advanced validation
AimTo ensure the preservation of brain signals
TechniqueCalculate R between the cleaned data cluster
and raw data without EOG cluster during:- 0-700 ms: before the earliest speech onset- 0-900 ms: before the grand-average speech
onsetsThe proposed method (CPD) outperforms
SAR-ICA and BSS-CCA, both in detecting speech
artifact (0.985) and cleaning data (0.101)- Almost identical at most time points to lip EMG- Slight difference → possibly caused by the inability of lip
EMG channels to pick up every single speech movement

Advanced validation results
Grand-average cluster

R (0-700 ms)

R (0-900 ms)

CPD’s Cleaned Data

0.927

0.942

Post-removal validation

Eye artifact removal
Performed with SAR-ICA, using PSD correlation with EOG
channel as the spectral calculation

where‐
is average power in EMG band (approx.
by 15-30 Hz)‐
is average power in EEG band (approx.
by 0.1-15 Hz)‐
is constant (default: 7)0s

The proposed method is written as CPD.

Aim

The difference of fit (DIFFIT)(time, spectral, & spatial)

It is an EMG activity ifRest

Block design- Consists of 45 trials- Displays each picture
exactly once randomly- Only EEG data fromexperiment block will
be processedTechnique

CPD tensor
decomposition

PSD analysis (using quantitative criterion)

Picture stimulus onset

Evaluation results

A. Assessments(grand-average correlation with lip EMG during 0-1350 ms)

Proposed method (with tensor decomposition)

Independent component analysis (ICA)

1.
2.
3.

Fixation cross onset

Time

A. Pipelines

SAR-ICA (Porcaro et al, 2015)(topographicaldistribution,single-trial,averagedtrials)

Total trial length: 4 seconds
At the beginning → fixation cross appears and remains on display for 1 second
Afterwards, it is replaced with a picture stimulus
Subject needs to name it as quick and as accurate as possible

Practice block

4. Results

Evaluation

Visualinspection

Instructions- Blink only after naming- Reduce movements
during experiment-+Evaluation

2. Methods

Picture-naming
experiment

Experiment block

Why propose a tensor-based method?
- Besides space and time, the spectral
aspect of EEG data is also important- The multi-way nature of EEG data

1. Propose a new method for removing speech artifacts withtensor decomposition for sources reconstruction
2. Evaluate the performance of the proposed method
against existing methods (SAR-ICA and BSS-CCA)

Preparation- Read a booklet ofpicture-name pairs-1 sPrevious work methods

Proposed method(tensor-based)

toMean age 23.3, SD 2.6
7 males, 2 females
8 right-handed, 1 corrected-to-right1 excluded because of many errors

Tensor decomposition

Speech artifact removal

Unable to process
the EEG dataaccording-Trial design

Pre-experimentParticipants: 9 native Japanese speakers

Both proposed a speech artifact removal
method with matrix decomposition.

Preprocessing
to monitor speech
artifacts

Task: picture-naming- SAR-ICA by Porcaro et al, 2015
- BSS-CCA by Vos et al, 2010

to monitor eye
artifacts

Which part of the signal
is a speech artifact?
- Unknown
characteristics- Difficult to remove

Experiment design

Related works

Data collection
Vertical EOG

EEG Data Collection

Proves to be the significant
differentiator in method performance

5. Closing

B. Key Differences

Blind source separation (BSS)
using canonical correlation analysis (CCA)
as ground truth

Overview comparison of the existing methods and the proposed method (CPD tensor decomposition)

Speech artifact removal method
PSD analysis

AspectSAR-ICA

BSS-CCA

Proposedmethod (CPD)

ICABSS

CPD

Artifact marking
Automatic muscle-related
artifact removal

Decomposition
methodEMG channels
usage

Yes

Need of visualinspection

Yes

No

Yes

Data reconstruction
Cleaned data cluster (BSS-CCA)

No

Yes

Interpretation
Preserved brain signals → high correlation

Matrix decomposition vstensor decomposition assources reconstruction method
Matrix decomposition (ICA &BSS) → 2 dimensional
Tensor decomposition (CPD) →multiple modes

The proposed method takes
spectral features of speech
artifact into account duringdecomposition, while theprevious methods do it afterdecomposition

Sum Up and Future Direction

Conclusion

Future works- We proposed a speech artifact removal
method using CPD tensor decomposition

Present study relied on manual visualinspection and EMG channels usage, sofuture research should...

- The proposed method (CPD) surpasses
former methods (SAR-ICA and BSS-CCA), bothin identifying speech artifacts (0.985) andproducing cleaned data (0.101)- Automate all of the steps needed forspeech artifact identification- Use other means to determine thespeech artifacts

The high validation results (0.92-0.94) during thepre-speech onset indicate the quality of the cleaned
data is enough for subsequent EEG processing.

6. References

C. Porcaro, M. T. Medaglia,and A. Krott, “Removing
speech artifacts fromelectroencephalographic
recordings during overt
picture naming,” NeuroImage,
vol. 105, pp. 171 – 180, 2015.

F. Cong, Q.-H. Lin, L.-D. Kuang, X.-F. Gong, P. Astikainen, and T. Ristaniemi, “Tensordecomposition of eeg signals: A brief review,” Journal of Neuroscience Methods, vol. 248, pp.
59 – 69, 2015. [Online].
M. de Vos, D. Maarten Vos, S. Ries, K. Vanderperren, B. Vanrumste, F.-X. Alario, S. Huffel, S.
Huffel, and B. Burle, “Removal of muscle artifacts from EEG recordings of spoken language
production,” Neuroinformatics, vol. 8, pp. 135–150, 06 2010.
M. E. Timmerman and H. A. L. Kiers, “Threemode principal components analysis: Choosing
the numbers of components and sensitivity to local optima,” British Journal of Mathematicaland Statistical Psychology, vol. 53, no. 1, pp. 1–16, 2000.
T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,” SIAM Review, vol. 51,
no. 3, pp. 455–500, September 2009.
T. Nishimoto, K. Miyawaki, T. Ueda, Y. Une, and M. Takahashi, “Japanese normative set of 359
pictures.” Behavior research methods, vol. 37 3, pp. 398–416, 2005.