Temporal Decomposition of EEG into Stimulusand Response-locked Components
刺激及び反応に時間同期した脳波信号の分離手法の開発

byYusuke Takeda
武田祐輔

Department of Physical and Health Education
Graduate School of Education
The University of TokyoAcknowledgements
I would like to express my sincere appreciation of Professor Yoshiharu Yamamoto for supervisingme throughout my graduate studies. He constantly and patiently encouraged me to complete mystudies, made many constructive comments and provided much theoretical input on a wide range.
I greatly appreciate all the help from him. I would also like to thank him for carefully reading
this thesis and making many valuable comments on writing in English, as well as on research.
I would like to thank Associate Professor Gentaro Taga for supervising me during my graduate
studies. He made many valuable comments and suggestions about my studies.
I would like to thank Associate Professor Daichi Nozaki for supervising me during my graduate
studies. He gave me with much theoretical advice, and improved my studies considerably.
I would also like to thank Assistant Professor Kentaro Yamanaka for supervising me throughout my graduate studies. He always gave me much valuable advice, encouraged me to study
diligently, and taught me many things about research: how to make a research plan, conduct
experiments, analyse experimental data, write a paper, and submit it. I greatly appreciate all
the help from him. I would also like to thank him for carefully reading this thesis and making
many valuable comments.
I would like to thank all the other staff at the laboratory, Professor Yoshiteru Muto, Professor
Shigeru Kitazawa, Associate Professor Mitsuaki Shibawaka, Associate Professor Zbigniew R.
Struzik, and Assistant Professor Rika Soma, for their support in my graduate studies.
I would like to thank Dr. Ken Kiyono of Nihon University and Dr. Ryoya Saji of Tamagawa
University for teaching me physics and mathematics, and making many valuable comments on1

my studies.
Thanks also to my colleagues at the laboratory, especially Mr. Takatsugu Aihara and Mr.
Tsuyoshi Ikegami, for discussing various problems of academic life with me.
Finally, I thank my family for all of the long-term support they have given to me, visible andinvisible.

2

Contents
Acknowledgements

1

1 Introduction

7

1.1

Motivation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7

1.2

EEG as Target Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

8

1.3

Proposed Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.4

Organization of Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.5

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2 EEG during Tasks with Overt Responses

14

2.1

Brain Activity during Tasks with Overt Responses . . . . . . . . . . . . . . . . . . 14

2.2

Mathematical Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.3

Previous Methods

2.4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.3.1

Stimulus-/Overt Response-triggered Averaging Procedure . . . . . . . . . . 20

2.3.2

Spatial Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3 Proposed Method for Overt Responses

25

3.1

Decomposition Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.2

Simulation with Artificial Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.1

Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3

3.2.2
3.3

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Simulation with EEG Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3.1

Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.3.2

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.4

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.5

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4 Application to EEG during a Simple Reaction Time Task
4.1

4.2

4.3

4.4

39

Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.1.1

Experimental Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.1.2

Data Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.2.1

Decomposition of Observed EEG (1–40 Hz) . . . . . . . . . . . . . . . . . . 43

4.2.2

Decomposition of Observed EEG (2–40 Hz) . . . . . . . . . . . . . . . . . . 48

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.3.1

Stimulus-locked EEG Component . . . . . . . . . . . . . . . . . . . . . . . . 51

4.3.2

Overt Response-locked EEG Component . . . . . . . . . . . . . . . . . . . . 52

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

5 EEG during Tasks with Covert Responses

55

5.1

Brain Activity during Tasks with Covert Responses . . . . . . . . . . . . . . . . . . 55

5.2

Mathematical Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

5.3

Previous Methods

5.4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

5.3.1

Time-frequency Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

5.3.2

Peak Picking, Woody’s Method, and Pham et al.’s Method . . . . . . . . . 58

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

6 Proposed Method for Covert Responses
6.1

61

Decomposition Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

4

6.2

6.3

Simulation Test with Artificial Data . . . . . . . . . . . . . . . . . . . . . . . . . . 66
6.2.1

Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

6.2.2

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Simulation Test with EEG Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.3.1

Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

6.3.2

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

6.4

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

6.5

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

7 Application to EEG during NoGo Trials of a Go/NoGo Task
7.1

7.2

7.3

7.4

74

Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
7.1.1

Experimental Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

7.1.2

Data Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
7.2.1

Estimation of RTs in Go Trials . . . . . . . . . . . . . . . . . . . . . . . . . 80

7.2.2

Testing Assumption for EEG during NoGo Trials . . . . . . . . . . . . . . . 80

7.2.3

Decomposing EEG during NoGo Trials. . . . . . . . . . . . . . . . . . . . 80

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
7.3.1

Estimated RTs from EEG during Go Trials . . . . . . . . . . . . . . . . . . 84

7.3.2

Extracted Components from EEG during NoGo Trials . . . . . . . . . . . . 85

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

8 General Discussion

87

8.1

Method for Overt Responses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

8.2

Method for Covert Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

8.3

Further Development of Proposed Methods . . . . . . . . . . . . . . . . . . . . . . 90

8.4

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

A Averages of Extracted Components Depend on c

598

B Parametric Method for Overt Responses

100

C Scalp Distributions of Potentials Earlier than P300 and Later than P160

103

D Accuracy of Estimation Depends on Number of Trials

104

6

Chapter 1

Introduction
1.1

Motivation

In our daily life, we make two types of responses to sensory stimuli. One is an externally observable“overt” response, which is executed by muscular activity. The other is an externally unobservable“covert” response, which occurs only within the brain. For example, let us imagine that you are
driving a car. You approach some traffic lights, and the light turns red. Then, you push down
the brake pedal. In this case, it is considered that you make an “overt” response to the visual
stimulus of the traffic light by pushing the brake pedal. On the other hand, let us imagine thatyou are climbing a mountain. You reach the top and view a beautiful landscape. Then, you feel“Great!”. In this case, it is considered that you make a “covert” response to the visual stimulusof the beautiful landscape by feeling “Great!”.
It is considered that the responses, pushing the brake pedal and feeling “Great!” occur withvariable delays after the onset of sensory stimuli. In this thesis, therefore, I define a “response” asan event which involves a fixed brain activity and of which delays after stimulus onsets fluctuatewith every response. Note that not all events driven by a stimulus are referred to as “responses” inthis thesis, although they are in a broad sense. For example, stimulus-locked changes in the brain
activity are not referred to as “responses”. Further, I divide responses into “overt responses”
and “covert responses” according to whether the responses are observable or not. Pushing thebrake pedal is categorized as an “overt response”, because the response, pushing the pedal, is7

observable. Feeling “Great!” is categorized as a “covert response”, because the response, thefeeling, covertly occurs in the brain and is not observable.
In brain research, what should be examined in such stimulus-response situations is how thebrain operates during the time period from a stimulus onset to a response onset. Because there
are two events, receiving a stimulus and making a response, it is considered that the two kinds ofbrain processes occur, related to the two events respectively. For example, in the case of pushing
the brake pedal, the brain process related to looking at the traffic light (stimulus-related activity)
and that related to pushing the pedal (response-related activity) occur. To examine such brain
activity in detail, we need to separate the stimulus- and response-related activity and examine
the two activities separately. By separating the brain activities, we can identify which activity is
responsible for different response patterns, e.g., reaction times (RTs), in different situations. To
separate the brain activity, we need data analysis methods to decompose brain signals, such as
an electroencephalography (EEG) signal, into stimulus- and response-related components.
This thesis deals with data analysis methods which decompose brain signals into the stimulusand response-related components.

1.2

EEG as Target Signal

The target signals of my analysis are human scalp EEG signals, and I propose methods fordecomposing EEG data.
Electroencephalography (EEG) is a non-invasive neurophysiologic measurement of the electrical activity of the brain recorded from electrodes placed on the scalp. Electroencephalography
represents a summation of post-synaptic potentials from a large number of neurons under theelectrodes. Electroencephalography has the limitation of poor spatial resolution. This is because
the meninges, cerebrospinal fluid and the skull smear the EEG signal and obscure its intracranial
source. On the other hand, the advantage of EEG is its high time resolution (on the level of asingle millisecond). This advantage is due to the fact that EEG measures the brain’s electricalactivity directly. Other methods of looking at brain activity, such as positron emission tomogra-

8phy (PET) and functional magnetic resonance imaging (fMRI), have lower time resolutions (onthe level of seconds and minutes), because these methods record changes in metabolic activity(e.g., PET) or blood flow (e.g., fMRI), which are indirect markers of brain electrical activity. The
high time resolution of EEG makes it possible to investigate rapid changes in the brain activity.
Because the interval between a stimulus and a response is usually shorter than 1 s, EEG is asuitable measure for investigating the brain activity during a stimulus-response situation.
In order to propose methods for decomposing EEG into the stimulus- and response-related
components, we need to know how the stimulus- and response-related brain activities appear in
the EEG. Figure 1.1 shows a color-coded representation of the EEG potentials at the midlinecentral electrode (Cz) during a simple reaction time task in which human subjects were instructedto press a button as soon as possible after beeps (described in Chapter 4). In this figure, theEEG epochs are sorted in the order of RTs, and the stimulus and response onsets are represented
by solid and dotted lines respectively1 . Two types of fluctuations appear in this image (see also

Sorted trials

10
1000

2000

0

3000

-10
[ V]

200

0

200

400

600

800

Time [ms]Figure 1.1: Time-trial image of EEG during a simple reaction time task. The solid line represents
stimulus onsets, and the dotted line represents the response onsets.
Fig. 2.6, A). One is fluctuations parallel to the stimulus onsets (stimulus-locked fluctuations),
1 In this thesis, I refer to this kind of image as a time-trial image.

9

and the other is fluctuations parallel to the response onsets (response-locked fluctuations). This
indicates that the stimulus- and response-related brain activity respectively appear in stimulusand response-locked ways. Further, the stimulus- and response-locked fluctuations are overlappingin the trials with fast RTs. This indicates that the stimulus- and response-related brain processes
can be temporally overlapping.
To summarize these observations, stimulus- and response-related EEG activities are respectively stimulus- and response-locked, and these two EEG activities can be temporally overlapping.
Therefore, to separate the stimulus- and response-related EEG activities, we need to propose
methods for temporally decomposing EEG into the stimulus- and response-locked components.
The schematic representation of the EEG activity in a stimulus-response situation is shown in
Fig. 1.2.

1.3

Proposed Methods

In this thesis, I propose two methods for decomposing EEG into stimulus- and response-lockedcomponents, and the two methods are used when responses are overt and covert respectively.
When responses are overt, the stimulus- and overt response-locked components have conventionally been extracted by averaging EEG epochs with respect to either stimulus or overt response
onset to increase the signal to noise ratio. However, when the two components are temporally
overlapping, the conventional stimulus- or overt response-triggered average EEG does not reflect
exactly pure stimulus- or overt response-locked brain activity, because, by the averaging procedure, each component is contaminated by the other component. Here, I propose a method to
extract the uncontaminated stimulus- and overt response-locked components. In this method,
the two components are extracted from single-channel EEG epochs and RTs using a discrete
Fourier transform.
When responses are covert, conventionally, only the stimulus-locked component has been extracted by the averaging procedure. This is because the onset of the covert response-locked component is unknown, and the response-triggered averaging procedure, or even the above-mentioned

10

Stim.
onset

Stim.-locked comp.
Resp.
onset

Resp.-locked comp.

EEG
Temporaldecomposition

Resp.-locked comp.

Stim.-locked comp.

Figure 1.2: Schematic representation of the EEG including the stimulus- and response-lockedcomponents. I propose methods for temporally decomposing the EEG into the two components.

11method for overt responses, cannot be used. Here, I propose a method to obtain the stimuluslocked component, the covert response-locked component and its delays of individual trials only
from single-channel EEG epochs. In this method, I estimate the delays of the covert responselocked component, and extract the two components simultaneously using the above method forovert responses.

1.4

Organization of Thesis

The organization of this thesis is as follows.
In Chapters 2–4, I deal with the decomposition of EEG into the stimulus- and overt responselocked components. In Chapter 2, I introduce previous reports on the brain activity during tasks
which require overt responses, and discuss the previous methods for extracting the stimulus- andovert response-locked components. In Chapter 3, I propose a method for decomposing EEG into
stimulus- and overt response-locked components by using RTs, and examine the performance ofthe proposed method by two types of simulation tests with artificial and EEG data. In Chapter
4, I apply the proposed method to the EEG during a simple reaction time task, and examine thecontamination by the conventional averaging procedure.
In Chapters 5–7, I deal with the decomposition of EEG into the stimulus- and covert responselocked components. In Chapter 5, I introduce previous reports on the brain activity during the
cognitive tasks which induce covert responses, and discuss the previous methods for extracting
the covert response-locked component. In Chapter 6, I propose a method for estimating thedelays of the covert response-locked component and decomposing EEG into the stimulus- andcovert response-locked components, and examine the performance of the method by two types ofsimulation tests with artificial and EEG data. In Chapter 7, I apply the proposed method to theEEG during NoGo trials of a Go/NoGo task.
In Chapter 8, I summarize the proposed methods and the presented results, and discuss thefurther development of the proposed methods.

12

1.5

Glossary

Stimulus-locked component: The EEG component time-locked to stimulus onsets.
Overt response-locked component: The EEG component time-locked to overt response onsets.
Covert response-locked component: The EEG component which has a fixed waveform andof which the delay fluctuates from trial to trial.
Method for overt responses: The proposed method for obtaining the stimulus- and overt
response-locked components from EEG epochs and RTs.
Method for covert responses: The proposed method for obtaining the stimulus-locked component, the covert response-locked component and its delays of individual trials from EEG
epochs.
Simple reaction time task: A reaction time task in which a specified overt response is requiredon the presentation of a stimulus.
Go/NoGo task: A choice reaction time task in which a specified overt response is required on
the presentation of a “Go” signal (Go trial) but is not to a “NoGo” signal (NoGo trial).

Some of the studies in this thesis are based on papers, published or in press, as below:
1. Takeda, Y., Yamanaka, K., Yamamoto, Y., 2008. Temporal decomposition of EEG during
a simple reaction time task into stimulus- and response-locked components. NeuroImage39, 742-754.
2. Takeda, Y., Yamanaka, K., Nozaki, D., Yamamoto, Y., 2008. Extracting a stimulusunlocked component from EEG during NoGo trials of a Go/NoGo task. NeuroImage (inpress).
13

Chapter 2

EEG during Tasks with Overt
Responses
In this chapter, I firstly introduce reports on the brain activity during tasks which require overt
responses (i.e., reaction time tasks). Then, based on the reports, I mathematically formulate theEEG during the reaction time tasks. Finally, I discuss the previous methods for extracting thestimulus- and overt response-locked components from the EEG, and discuss the necessity of anovel method for extracting the two components.

2.1

Brain Activity during Tasks with Overt Responses

The brain activity during the period from a stimulus presentation to an overt response was
examined in cats, monkeys and humans by using various reaction time tasks, and the stimuluslocked activity and/or the overt response-locked activity was observed [1–5].
As expected, the stimulus-locked activity was observed from the sensory cortex. For example,
Nelson [2] investigated single unit activity in the somatosensory cortex in monkeys during aparadigm with wrist movements in response to vibration of their hands, and showed that someneurons exhibit the stimulus-locked activities (Fig. 2.1).
The overt response-locked activity was observed from the motor cortex. For example, Tanji
and Kurata [5] investigated single unit activity in the precentral motor cortex in monkeys during
a paradigm with wrist extension in response to three types of stimuli: visual, auditory and tactile
14

Figure 2.1: Activity of an area 3b QA neuron (A) and an area 1 QA neuron (B) showing thestimulus-locked activity. The activity of the neurons is presented as histograms (top) and rasterplots (bottom). Discharges are aligned at the onset of the vibratory cue and ordered by reaction
times (dark marks in the raster plots signify movement onset). (Adapted from [2])15

stimuli. They showed that some neurons exhibit similar response-locked activities regardless ofthe modality of the sensory stimuli (Fig. 2.2).

Figure 2.2: Activity of a neuron showing response-locked activities. In the left and middle
columns of this figure, the activities of the neuron are presented as a raster histogram. Leftcolumn: Discharges are aligned at the onset of visual, auditory, and tactile stimuli (indicatedwith a dotted line labeled S). Middle column: Discharges are aligned at the onset of the wrist
extension movement (indicated with a dotted line labeled M). Right column: EMG activity
recorded from m. extensor carpi radialis. (Adapted from [5])On the other hand, there are reports showing that the same neurons exhibit both stimulusand overt response-locked activities. For example, Pefiliev [4] investigated single unit activity inthe forelimb area of the motor cortex (area 4γ) in cats during target-reaching with the contralateral limb for a morsel of food, and showed that some neurons exhibit both stimulus- and overt
response-locked activities (Fig. 2.3).
These reports indicate that the stimulus- and overt response-locked brain activities occur
during tasks which require overt responses, and further that, in some cases, the same neuron
16

B

A

C

Figure 2.3: Activity of a neuron clearly showing two components in relation to target-reaching.
A: Histogram which is aligned with the stimulus onset indicated by a triangle. The horizontal
line below the histogram indicates the range of the response onsets. B: Histogram which isaligned with the response onset indicated by a triangle. The activity of the cell is presented asa raster histogram (C), where each dot indicates one spike and each line one trial. The trials
are arranged in the order of increasing RTs indicated by short vertical lines and aligned with thestimulus onset (triangle). (Adapted from [4])17

exhibits both of the two activities.
By using EEG or magnetoencephalography (MEG), human brain activities during reaction
time tasks have been examined non-invasively, and the stimulus- and overt response-locked components have been observed [6–9]. For example, Jung et al. [7] spatially decomposed the multichannel EEG during a choice reaction time task using independent component analysis (ICA) [10],and showed some stimulus- and overt response-locked independent components (Fig. 2.4).
From the above reports, it is considered that the human scalp EEG during tasks with overt
responses includes the stimulus- and overt response-locked components, and that both of thecomponents are spatially overlapping. Because of the spatial overlapping, the two components
are temporally overlapping when intervals between stimulus and response onsets are short.

2.2

Mathematical Formulation

Here, let me mathematically formulate the EEG during tasks which require overt responses. By
formulating the EEG, we can consider the problem in its simplest form. As a result, I can clearly
describe the limitations of the previous methods for extracting the stimulus- and overt responselocked components, and can propose a novel method for the extraction by simple mathematicaloperations as shown in Chapter 3.
It is reported that brain activity during a reaction time task consists of stimulus-locked activity, overt response-locked activity shifted by the RT of an individual trial, and noise. Therefore,
observed single-channel EEG data during a reaction time task can be expressed by:yn (t) = s(t) + or(t − τn ) + vn (t) (t = 0, · · · , T − 1; n = 1, · · · , N ),(2.1)

where yn (t): observed EEG data in trial n; s(t): stimulus-locked component; or(t): overt
response-locked component; τn : RT in trial n; vn (t): noise in trial n. This implies that ongoing EEG activity and trial-to-trial variability of stimulus- and overt response-locked activity
are included in vn (t).

18

Figure 2.4: Stimulus-locked independent components and overt response-locked independent components obtained by ICA. A: Time-trial images of stimulus-locked independent components. B:
Time-trial images of overt response-locked independent components. Thin vertical lines represent
stimulus onsets, and thick oblique lines represent response onsets. (Adapted from [7])

19

2.3

Previous Methods

2.3.1

Stimulus-/Overt Response-triggered Averaging Procedure

Conventionally, the stimulus- and overt response-locked components have been extracted byaveraging EEG epochs with respect to either stimulus or overt response onset to increase thesignal to noise ratio. Indeed, when the EEG includes either the stimulus- or overt response-lockedcomponent, the averaging procedure is a suitable means. However, when the EEG includes boththe two components and these are temporally overlapping, the conventional stimulus- or overt
response-triggered average EEG is not appropriate. This is because, by the averaging procedure,
the stimulus-/overt response-locked component is contaminated by the other component. With
the contamination, the peak amplitudes and latencies in the stimulus-/overt response-triggered
average EEG become more or less different from the original stimulus-/overt response-lockedcomponent, leading to wrong conclusions about the stimulus/overt response-locked brain activity.
Figure 2.5 is a simulation result showing how the contamination occurs in the stimulus-/overt
response-triggered average EEG. The waveform of the stimulus-triggered average in the late partis different from that of the stimulus-locked component, indicating that the contamination islarge in the late part of the stimulus-triggered average (Fig. 2.5, C, indicated by the arrow). The
waveform of the overt response-triggered average in the early part is different from that of theovert response-locked component, indicating that the contamination is large in the early part ofthe overt response-triggered average (Fig. 2.5, D, indicated by the arrow).
The contamination effects in the stimulus-/overt response-triggered average EEG are quantitatively described as follows. Under the assumption of Eq. (2.1), the stimulus-triggered average
EEG ȳs (t) is expressed by:ȳs (t) = s(t) +

t
N
1 !
1 !
h(τ )or(t − τ ) +vn (t),
N τ =0
N n=1

(2.2)

where h(τ ) represents the distribution of RTs. In this equation, the contamination by the overt"tresponse-locked component is expressed by 1/N τ =0 h(τ )or(t−τ ). Similarly, the overt response20

A

B

Resp.
onset

1
0

0
0

C

1000

RT
1

0

0

0
-1
1

0
Resp.
onset

1000

D

Stim.
onset

0
-1

1000

0
0

Time

1000

1000
Time

0
0

Time

1000

Figure 2.5: Simulation with artificial data. A: Original stimulus-locked component (top),
overt response-locked component (middle), and noise (bottom). B: Simulated data obtainedby Eq. (2.1) using the real RTs of a simple reaction time task (described in Chapter 4). C:
Stimulus-triggered average of the simulated data (solid line) and the original stimulus-lockedcomponent (dotted line). D: Overt response-triggered average of the simulated data (solid line)
and the original overt response-locked component (dotted line). Arrows in C and D indicate theparts around which the contamination occurs.

21triggered average EEG ȳor (t) is expressed by:ȳor (t) = or(t) +T −t−1N
1 !
1 !
h(τ )s(t + τ ) +vn (t + τn ),N τ =0
N n=1

(2.3)

and in this equation the contamination by the stimulus-locked component is expressed by"T −t−1
1/N τ =0 h(τ )s(t+τ ). Obviously, the terms for the contamination include stimulus- and overt

response-locked components, indicating that we need these components to identify the level ofthe contamination. In other words, whether the contamination is large or small should be unclear
only from the contaminated average EEG.
When the effect of the contamination is unclear, we cannot distinguish whether different
waveforms of the stimulus/overt response-triggered average EEG in different task conditions are
attributable to the difference in the level of the contamination due to a change in the RTs or to
the difference in the stimulus/overt response-locked EEG activity itself. Therefore, we cannot
discuss the brain activity in detail from the stimulus/overt response-triggered average EEG.

2.3.2

Spatial Decomposition

Other methods for extracting the stimulus- and overt response-locked components have been spatial decompositions by principal component analysis (PCA) [11] and ICA [7,12]. For example, as
mentioned above, Jung et al. [7] spatially decomposed the EEG during a choice reaction time task
into the stimulus- and the overt response-locked independent components by ICA (Fig. 2.4). The
PCA and ICA are computational methods for decomposing multi-channel data into orthogonalcomponents or independent components respectively [13,14]. Therefore, to decompose EEG data
into the two components by PCA or ICA, the EEG needs to satisfy the following two conditions.
1. Time series of stimulus- and overt response-locked components are orthogonal (for PCA)
or independent (for ICA).
2. Stimulus- and overt response-locked components have different scalp distributions.
Unless the condition 1 is satisfied, obviously, PCA/ICA should not separate the stimulusand overt response-locked components. Considering that RTs are not distributed uniformly but
22within a fixed range after stimulus onsets, the overt response-locked components tend to occur
at a certain phase of the stimulus-locked components. This indicates that the occurrences of thetwo components tend to be correlated and the time series of the two components also tend to becorrelated; that is, condition 1 is rarely satisfied.
Further, unless condition 2 is satisfied, PCA/ICA should not separate the stimulus- andovert response-locked components. This is because PCA/ICA does not distinguish components
attributable to the same sources. In animal studies, as mentioned above, the same neurons inthe sensory and motor cortices exhibit both the stimulus- and overt response-locked activities,
indicating that the sources of the stimulus- and overt response-locked activities are the samein some cases. Of course, these reports do not directly lead to the same scalp distributions ofhuman scalp EEG signals. Nevertheless, these reports suggest that EEG does not always satisfy
condition 2.
To sum up, we cannot guarantee that EEG always satisfies the two conditions, indicatingthat PCA/ICA does not always separate the stimulus- and overt response-locked components.
In fact, I applied ICA to the EEG during the auditory simple reaction time task (describedin Chapter 4), and found that some independent components exhibited both the stimulus- andovert response-locked fluctuations in their time-trial images (Fig. 2.6), indicating that ICA was
not able to separate the two components completely.

A

B

IC1

Trials

1
1000
2000
0

800

400
Time [ms]

0

5

-1

-5

Figure 2.6: An independent component (IC1) exhibiting both the stimulus- and overt responselocked fluctuations. A: Time-trial image of IC1. Solid line represents stimulus onsets, and dottedline represents overt response onsets. B: Scalp distribution of IC1.

23

2.4

Conclusions

From previous reports, it is shown that the EEG during tasks which require overt responses consists of the stimulus- and overt response-locked components. To extract the two components, thestimulus-/overt response-triggered averaging procedure and the spatial decomposition methods(PCA and ICA) have been used. However, these methods are not adequate. Therefore, it is necessary to propose a novel method which can always be used for extracting the uncontaminatedstimulus- and overt response-locked components. In the next chapter, I propose such a method
for the extraction.

24

Chapter 3

Proposed Method for Overt
Responses
In the previous chapter, I describe the necessity of a novel method for temporally decomposing
EEG into the stimulus- and overt response-locked components. In this chapter, therefore, I
propose a method for the temporal decomposition. We can consider two types of methods for thedecomposition; parametric methods and non-parametric methods. With parametric methods,
parametric models for the stimulus- and overt response-locked components are assumed andidentified. With non-parametric methods, the stimulus- and overt response-locked components
are extracted without the assumption of parametric models for these components. In this thesis,
I propose a non-parametric method, because the non-parametric method does not need a priori
knowledge of parametric models themselves, in contrast to the parametric method, and is better
than the parametric method.

3.1

Decomposition Method

I assume that the EEG activity consists of the stimulus- and overt response-locked components,
and is expressed by Eq. (2.1). Here, the objective is to obtain s(t) and or(t) from yn (t) and τn(n = 1, · · · , N ) (Fig. 3.1).

25

Resp.
onset

Stim.
onset

EEG in trial 1
Resp.
onset

Stim.
onset

EEG in trial 2

Resp.
onset

Stim.
onset

EEG in trial N
Temporaldecomposition

Overt resp.-locked comp.

Stim.-locked comp.

Figure 3.1: Schematic representation of the objective26

By taking the discrete Fourier transform of Eq. (2.1), I obtain:
Yn (ω) = S(ω) + exp(−i2πωτn /T )OR(ω) + Vn (ω) ω = 0, · · · , T − 1 ,(3.1)

where Yn (ω) is the discrete Fourier transform of yn (t); S(ω) is the discrete Fourier transform ofs(t); OR(ω) is the discrete Fourier transform of or(t); Vn (ω) is the discrete Fourier transformof vn (t). Note that the last τn points in or(t) should be zero, because the phase shift in the
Fourier domain is equal to the circular shift in the time domain. By averaging Eq. (3.1) acrossn, I obtain:Ȳ (ω) = S(ω) + Ē(ω)OR(ω) + V̄ (ω) ,
where

(3.2)

N

Ȳ (ω) =1 !
Yn (ω);
N n=1

(3.3)

N

Ē(ω) =1 !
exp(−i2πωτn /T );
N n=1

(3.4)

N

V̄ (ω) =1 !
Vn (ω).
N n=1

(3.5)

By solving Eqs. (3.1) and (3.2) simultaneously for S(ω) and OR(ω), I obtain:exp(−i2πωτn /T )V̄ (ω) − Ē(ω)Vn (ω)
exp(−i2πωτn /T )Ȳ (ω) − Ē(ω)Yn (ω)= S(ω) +;exp(−i2πωτn /T ) − Ē(ω)
exp(−i2πωτn /T ) − Ē(ω)
Yn (ω) − Ȳ (ω)
Vn (ω) − V̄ (ω)= OR(ω) +.
exp(−i2πωτn /T ) − Ē(ω)
exp(−i2πωτn /T ) − Ē(ω)

(3.6)
(3.7)

To prevent the denominators of Eqs. (3.6) and (3.7) from being zero, I use a function D(n, ω)instead of exp (−i2πωτn /T ) − Ē(ω),
#
c
ω=0
,
D(n, ω) =exp (−i2πωτn /T ) − Ē(ω) ω "= 0

(3.8)

where c represents a constant number. By calculating Eq. (3.9) or (3.10) (see below) and fromsimulation results (see Appendix A), I find that only the average of extracted components depends
on the parameter c. So we can set c arbitrarily. In this study, I set c to unity.

27

Then, by averaging across n and taking the inverse discrete Fourier transform (IDFT), I
obtain:
IDFT

$

%
N
1 ! exp (−i2πωτn /T ) Ȳ (ω) − Ē(ω)Yn (ω)
N n=1
D(n, ω)%$N
1 ! exp (−i2πωτn /T ) V̄ (ω) − Ē(ω)Vn (ω)+ cs ; (3.9)= s (t) + IDFT
N n=1
D(n, ω)

IDFT

$

N

1 ! Yn (ω) − Ȳ (ω)
N n=1
D(n, ω)%= or (t) + IDFT$N

1 ! Vn (ω) − V̄ (ω)
N n=1
D(n, ω)%+ cr ,(3.10)

where cs and cr represent the shifts of the averages by D(n, ω).

Equations (3.9) and (3.10) show that s(t) or or(t) plus a noise term can be obtained by
calculating the left-hand side of Eq. (3.9) or (3.10). The noise terms converge to zero as N
increases, because the average of Vn (ω) converges to zero as N increases, whereas D(n, ω) varies
depending on τn regardless of N . Let me refer to the real parts of the left-hand sides of Eqs. (3.9)
and (3.10) as the extracted stimulus-locked component and the extracted overt response-lockedcomponent respectively.
Before applying the decomposition method, we need to set the parameters l0 , the number ofdata points before stimulus and overt response onset, and l1 , the number of data points after
stimulus and overt response onset. We need to set l1 to be adequately long so that the last data
points in or(t), which must be longer than the maximum τn , will be the baseline. In this study,
I set l0 to 100 and l1 to 300, which correspond with 500 ms and 1500 ms respectively if thesampling rate is 200 Hz.
The Matlab codes for this decomposition method are available at: “www.p.u-tokyo.ac.jp/
˜takeda/decomp/”.

3.2

Simulation with Artificial Data

In order to verify the performance of the proposed method, a numerical experiment was performed
on a set of known artificial signals.

28

3.2.1

Methods

In this simulation, the original stimulus- and overt response-locked components were generated
by the exponential and the cosine functions respectively. The RTs, which were randomly selectedfrom all the subjects’ RTs of the simple reaction time task (described in Chapter 4), were usedas τn (n = 1, · · · , N = 100), and white noise [standard deviation (SD) = 0.5] was used as vn (t)(Fig. 3.2, A). The simulated data yn (t) was generated from s(t), or(t), τn and vn (t) according to
Eq. (2.1) (Fig. 3.2, B). Then, I extracted the stimulus- and overt response-locked component by
Eqs. (3.9) and (3.10) respectively. The similarity of the extracted and original components wasquantified by calculating the correlation coefficient between these two waveforms.
Further, to evaluate the residual errors between the extracted and original stimulus-/overt
response-locked components, I repeated the above procedure 500 times using different sets ofwhite noise and τn . The averages of the residual errors across time were adjusted to zero. To
examine whether the residual errors fluctuate randomly around zero, the time courses of the mean
and SD of the 500 residual errors were plotted (Fig. 3.3, B1, C1). To examine the frequency
characteristics of the residual errors (Fig. 3.3, B1, C1), I calculated the amplitude spectrum(Fig. 3.3, B2, C2):500

AP (ω) =1 !
|REp (ω)|,
500 p=1(3.11)

where REp (ω) represents the Fourier transform of the p-th residual error rep of the 500 repetitions. Finally, I examined how the noise level in the extracted stimulus-/overt response-lockedcomponents decreased as the number of trials increased (Fig. 3.3, B3, C3). The noise level N L(N )was obtained by calculating the variance of the residual error as follows:
&
'
500
T −1
1 !
1 !
N L(N ) =rep,N (t)2 ,500 p=1 T − 1 t=0(3.12)

where rep,N (t) represents the p-th residual error obtained from the simulation data consisting ofN trials. The variance was fitted by a function y = a/x by the least square method. Further, inorder to compare the noise level in the component obtained by the proposed method with that by
the averaging procedure, the noise used in the above repeated simulations was simply averaged29

across trials, and examined in the same manner as the residual errors (Fig. 3.3, A1–3).
By comparing the parameter a of the fitting function of the extracted components with thatof the averaged noise, we can estimate the number of trials the proposed method needs in orderto achieve the same noise level as the averaging procedure. Let me refer to the coefficient aobtained from the extracted component as ac , and that obtained from the averaged noise as an .
The equality of the variance of the residual errors to that of the averaged noise leads to thefollowing equation:
ac
an
=
,
xc
xn

(3.13)

where xc and xn represent the number of trials used for the decomposition and the averaging
respectively. Eq. (3.13) can be rewritten as:
xc =

ac
xn .
an

(3.14)

Eq. (3.14) means that the decomposition method requires ac /an times as many trials as theaveraging procedure in order to achieve the same noise level.

3.2.2

Results

I extracted the original components used for the simulation with the artificial data from thesimulated data (Fig. 3.2). The extracted and original stimulus-locked components are highly
correlated [rs = 0.93 (rs : the correlation coefficient between the extracted and original stimuluslocked components)], as are the extracted and original overt response-locked components [rr =0.90 (rr : the correlation coefficient between the extracted and original overt response-lockedcomponents)] (Fig. 3.2, C).
Figure 3.3 shows the detailed property of the residual errors between the extracted and originalcomponents. The time courses of the mean and SD of the residual errors are almost constant(Fig. 3.3, B1, C1), indicating that the residual errors have no temporal modulation patterns andfluctuate randomly. Note that the SD of the residual errors [0.091 ± 0.0025 (mean ±SD)] is larger
than that of the averaged noise (0.050 ± 0.0016), indicating that the noise level of the components
extracted by the proposed method is larger than that obtained by the averaging procedure. The
30

A

B

Resp.
onset

1

C

0

0
0

1000

RT
1

0

0

0
-1
1

0
Resp.
onset

1000

Stim.
onset

0
-1

0
0

Time

1000

1000

Time

1000

0
0

Time

1000

Figure 3.2: Simulations with artificial data. A: The original stimulus-locked component (top),
overt response-locked component (middle), and noise (bottom). B: The simulated data obtainedby Eq. (2.1) and the RTs from which the stimulus- and overt response-locked components are extracted. C: The extracted (solid line) and original (dotted line) stimulus-locked component (top);
the extracted (solid line) and original (dotted line) overt response-locked component (bottom).

31

large amplitude spectra of the residual errors at low frequencies (Fig. 3.3, B2, C2) indicate thatthe larger residual errors are attributable to amplified slow waves in the noise. The varianceof the residual errors is inversely proportional to the number of trials N , in the same way as
the averaged noise (Fig. 3.3, A–C3), indicating that the noise level of the extracted components
decreases as the number of trials increases. The coefficient is an = 0.25 for the averaged noise,
while ac = 0.80 for the extracted stimulus-locked component and ac = 0.83 for the extracted
overt response-locked component. Thus, the ratios ac /an in Eq. (3.14) are 3.2 and 3.3 for the
stimulus- and overt response-locked components respectively. This indicates that the proposed
method needs about 3.3 times as many trials as the averaging procedure in order to achieve thesame noise level, when the noise is white.

3.3

Simulation with EEG Data

In order to test the performance of the method for more EEG-like data, an additional numericalexperiment was performed using real EEG data (Fig. 3.4).

3.3.1

Methods

In this simulation, an auditory evoked potential (AEP)1 and a movement-related potential(MRP)2 were used as the original stimulus- and overt response-locked components respectively,
and the RTs, which were randomly selected from all the subjects’ RTs of a simple reaction timetask (described in Chapter 4), were used as τn (n = 1, · · · , N = 400). The EEG data related toneither the stimulus nor the overt response were used as the noise; I randomly selected them from
EEG in the interval of 2500–500 ms before the stimulus onset during the simple reaction timetask of all the subjects. As the simulation with the artificial data suggested the need to apply ahigh-pass filter before decomposing real EEG involving non-negligible slow waves, I applied threekinds of filters to the noise EEG data: no filtering, bandpass of 1–40 Hz and bandpass of 2–40 Hz(described in Chapter 4). By using the AEP, MRP, RTs and noise EEG data, I generated the
1 The average EEG during the passive hearing of an auditory stimulus (described in Chapter 4).
2 The average EEG during self-paced voluntary movements (described in Chapter 4).

32

Averaged noise

Residual error

Potential [ V]

A1

0.1

0.1

0

0

0

-0.1

-0.1

Amplitude [ V]

A2

B2

10
0

0

4
1
3
2
Frequency [Hz]52
Variance [ V ]

0

C2

0

100

Number of trials

1000

0

B3

0

1000
Time [ms]

10

4
1
3
2
Frequency [Hz]

5

0.01

0.01

0

1000
Time [ms]10

A3
Variance ofresidual error

-0.1
0

1000
Time [ms]Resp.-lockedcomponent

C1

0.1

0

Amplitude
spectrum of
residual error

Stim.-lockedcomponent

B1

0

0

C3

4
1
3
2
Frequency [Hz]

5

0.01

100

1000
Number of trials

0

100

1000
Number of trials

Figure 3.3: Residual errors between the original and extracted stimulus-/overt response-lockedcomponents for simulated artificial data. A–C1: Means (solid lines) and means ±SD (dottedlines) of the averaged noise (A1) and of the residual errors across 500 repeated simulations as
in Fig. 3.2 (B1, C1). A–C2: Amplitude spectra of the averaged noise (A2) and of the residual
errors (B2, C2). A–C3: Variance of the averaged noise (A3) and of the residual errors (B3, C3)
as a function of the total number of trials N (circle). The solid lines represent fitting curves in
the form of y = a/x. A1–3: Results for the averaged noise across trials. B1–3: Results forthe residual errors for the extracted stimulus-locked component. C1–3: Results for the residual
errors for the extracted overt response-locked component.

33

simulated EEG data according to Eq. (2.1).
Then, I extracted the stimulus- and overt response-locked components from the simulated
EEG data (Fig. 3.4, A–C1, A–C2). To examine the residual errors between the extracted andoriginal components, I repeated the above procedure 100 times using different sets of the noiseand τn . The averages of the residual errors across time were adjusted to zero. To examine whether
the residual errors fluctuate randomly around zero, the time course of the mean and SD of theresidual errors across the repeated procedures were plotted (Fig. 3.4, A–C3). Then, I examined
how the noise level decreased as the number of trials N increased (Fig. 3.4, A–C4, diamond).
The noise level N L(N ) was obtained by calculating the variance of the residual errors as follows:
&
'
100
T −1
T −1
!
1
1 !
1 !
2
2
N L(N ) =rep,N,s (t) +rep,N,r (t) ,(3.15)
100 × 2 p=1 T − 1 t=0
T − 1 t=0
where rep,N,s (t) and rep,N,r (t) respectively represent the p-th residual errors of the extracted
stimulus- and overt response-locked components obtained from the simulation data consisting ofN trials. The variance was fitted by a function y = a/x by the least square method. Further, inorder to compare the noise level in the components extracted by the proposed method with thatobtained by the averaging procedure, the noise EEG data used in the above repeated simulations
were simply averaged across trials, and examined in the same manner as the residual errors(Fig. 3.4, A–C4, circle).

3.3.2

Results

I extracted the original stimulus-/overt response-locked components used for this simulation fromthe simulated data (Fig. 3.4). The baseline fluctuations of the components extracted from the simulated EEG without filtering are large (Fig. 3.4, A1, A2), whereas those obtained from the filtered
EEG are well suppressed (Fig. 3.4, B1, B2, C1, C2). Indeed, the correlation coefficient between
the extracted and original components is low for the EEG without filtering (rs = 0.52, rr = 0.30),
whereas it is much higher for the filtered EEG (rs = 0.79, rr = 0.51 for the EEG filtered with
1–40 Hz; rs = 0.85, rr = 0.54 for the EEG filtered with 2–40 Hz). This is because the remaining
slow waves in the EEG without filtering are amplified by the decomposition. The time courses
34

ac
an
ac /anEEG(no filter)

EEG(1–40 Hz)EEG
(2–40 Hz)?@

6241
125
50

407
98
4.2

127
80
1.6

Table 3.1: Coefficients a of a fitting function y = a/x for an error level y as a function of thenumber of trials x. ac : Coefficients obtained from the components extracted by the proposed
method. an : Coefficients obtained from the averaged noise.
of the mean and SD of the residual errors are almost constant, indicating that the residual errors
have no temporal modulation patterns and fluctuate randomly regardless of the filter properties.
The variance of the residual errors is inversely proportional to the number of trials, in the same
way as the averaged noise (Fig. 3.4, A–C4), but the coefficients ac for the extracted components
are consistently greater than the an for the averaged noise. These coefficients are presented
in Table 3.1, and it is shown that the proposed method needs 1.6 times as many trials as theaveraging procedure in order to achieve the same noise level when slow waves in the EEG lower
than 2 Hz are eliminated.

3.4

Discussion

In the proposed method, the stimulus- and overt response-locked components are extracted by
calculating the left-hand side of Eqs. (3.9) and (3.10) respectively. The equations are obtained bysolving Eqs. (3.1) and (3.2) simultaneously. On the other hand, we are able to define other sets ofsimultaneous equations to obtain the extracted stimulus- and overt response-locked components.
For example, we are able to use the equation:
Yn! (ω) = S(ω) + exp(−i2πωτn! /T )R(ω) + Vn! (ω) τn "= τn! ,(3.16)

instead of Eq. (3.2). Like Eq. (3.2), Eq. (3.16) is different from Eq. (3.1) in that the RTs
are different (τn "= τn! ), and we can solve Eqs. (3.1) and (3.16) for s(t) and or(t). However,in that case the denominator of the solved equations frequently tends to be zero, making thedecomposition difficult. I also tried some other simultaneous equations instead of Eqs. (3.1) and35

Stim.-lockedcomponent

Potential [ V]No filtering
A1

Potential [ V]
Potential [ V]Residual error

Variance

5

0

0

0

-5

-5

-5

0

1000
Time [ms]

0

B2

1000
Time [ms]

3

3

0

0

0

-3

0

A3

1000
Time [ms]

5

0

B3

1000
Time [ms]01000Time [ms]1000Time [ms]

4

40
0

500
1000
Number of trials

0

-3

0

1000
Time [ms]01000Time [ms]

C3
0

0

B4

1000
Time [ms]

1

1
0
-1

0
-5

0

-3

0

C2

3

A4
Variance ofresidual error

2-40 Hz bandpass
C1

5

5

A2
Resp.-lockedcomponent

1-40 Hz bandpass
B1

-1

C4
1

0

500
1000
Number of trials

0

0

500
1000
Number of trials

Figure 3.4: Simulations with EEG data. A–C1: Extracted stimulus-locked components (solidlines) and the original AEP data used for the simulation (dotted lines). A–C2: Extracted
overt response-locked components (solid lines) and the original MRP data used for the simulation (dotted lines). A–C3: Residual errors between the extracted and original stimulus-/overt
response-locked components. Thick and thin solid lines represent the means of the residual errors
across the repeated simulations for the stimulus- and overt response-locked components respectively. The thick and thin dotted lines represent the means ±SD of the residual errors across therepeated simulations for the stimulus- and overt response-locked components respectively. A–
C4: Variance of the residual errors between the extracted and original components (diamonds)
and that of the averaged noise across trials (circles) as a function of the total number of trials N .
The solid lines represent fitting curves in the form of y = a/x. A1–4: Results for the simulations
in which pre-stimulus EEG without filtering was used as the noise. B1–4: Results for the simulations in which pre-stimulus EEG filtered with 1–40 Hz was used as the noise. C1–4: Results
for the simulations in which pre-stimulus EEG filtered with 2–40 Hz was used as the noise.

36

(3.2). Among these trials, Eqs. (3.1) and (3.2) are the best in that the denominators of the solved
equations rarely become zero, and the calculation for the decomposition is simple. Even if thedenominators of Eqs. (3.9) and (3.10) do become zero, we have a solution: adjust T (= l0 + l1 )so that the denominator does not become zero.
The proposed method non-parametrically decomposes single-channel EEG into the stimulusand overt response-locked components using a discrete Fourier transform. An alternative would
be to identify parametric models of evoked potentials (for example, [15–17]). In my preliminary
study, indeed, I was also able to identify these components parametrically using a Finite Impulse
Response model (see Appendix B). Another alternative would be applying Woldorff’s method [18],called the Adjacent Response (Adjar) technique. In the time domain, the Adjar removes the
distortion due to the overlap of the consecutive event-related potentials in short inter-stimulusintervals. Although originally developed to extract only the stimulus-locked component, theAdjar could also be used to extract not only the stimulus-locked component but also the overt
response-locked component [9]. As far as comparable performance is obtained, I believe theproposed method is better because it does not require a priori knowledge of parametric models
themselves, and its rationale and procedure are quite simple and clear.
The simulation results show that the noise level of the components extracted by the proposed
method is larger than that of the stimulus/overt response-triggered average. The amplitude
spectra of the residual errors (Fig. 3.3, B2, C2) indicate that this is because the absolute valueof 1/D(n, ω) is higher at low frequencies (∼ 1 Hz) if the observed RTs are used as τn ; thatis, slow waves (∼ 1 Hz) in the noise are amplified by the decomposition. In other words, thelarger noise in the extracted stimulus-/overt response-locked components is attributable to thedecomposition itself rather than to the other data operations, such as the data segmentation
strategy. On the other hand, the simulation results also show that the noise level of the extracted
components decreases as the number of trials increases. These results lead to solutions for thismethodological limitation: increasing the number of trials and/or applying a high-pass filter tothe EEG before the decomposition. The number of trials required for each filter property is37

shown in Table 3.1.
Generally speaking, a higher cut-off frequency of EEG leads to a smaller number of trials,
but that would also result in the attenuation of slower potentials of interest. To prevent this,
employing a higher number of trials with minimally filtered EEG would be a choice, but the noiselevel for this is substantially greater than that for the simple averaging. The averaging procedure,
then, is not capable of perfectly separating temporally overlapping components when a stimulus
and an overt response are temporally closer, as shown in Chapter 2. Thus, one would need tobalance one choice against the others, depending on the phenomenon of interest and practicallimitations of the research settings.

3.5

Conclusions

In this chapter, I propose a method temporally to decompose EEG into the stimulus- and overt
response-locked components. The simulation results indicate the feasibility of the proposed
method for artificial and EEG data. In the next chapter, I apply the method to the EEG during
a simple reaction time task and extract the stimulus- and overt response-locked components.

38

Chapter 4

Application to EEG during a
Simple Reaction Time Task
In this chapter, I apply the method proposed in the previous chapter to human surface EEG
data during the simple reaction time task with an auditory stimulus (SR-task). To examine
the effect of the temporal smearing by the averaging procedure, the extracted stimulus-/overt
response-locked components are compared with the stimulus-/overt response-triggered average
EEG during the simple reaction time task. Furthermore, I compare the extracted stimulus-lockedcomponent with an auditory evoked potential (AEP) obtained by averaging the EEG during thepassive hearing of an auditory stimulus (S-task), and also compare the overt response-lockedcomponent with a movement-related potential (MRP) obtained by averaging the EEG during
self-paced voluntary movement (M-task).

4.1

Methods

4.1.1

Experimental Procedures

Thirteen healthy adults aged between 20 and 31 years old constituted the experimental population. All the subjects gave their informed consent, and the local ethics committee approved theexperimental procedure.
The subjects were seated comfortably in a chair with their eyes closed and right index finger
placed on a button. The behavioral experiment consisted of 6 sessions. In each of the sessions, the39

subjects were instructed to perform three kinds of tasks in the following order: a simple reaction
time task (SR-task), a stimulus task (S-task) and a movement task (M-task). In the SR-task, thesubjects were instructed to press the button as soon as possible after hearing an auditory stimulus(75 dB SPL, 2 ms duration, 3500 Hz). The auditory stimulus was presented via earphones. Theinter-stimulus interval was randomized from 4 to 7 s. In the S-task, the subjects were instructedpassively to hear the same auditory stimulus. The inter-stimulus interval was randomized from 2
to 3 s. In the M-task, the subjects were instructed to press the button repeatedly at an intervalof about 4 s in the same manner as the SR-task. In total, about 300 trials were collected for eachtask from each of the subjects.
During the tasks, surface EEG was recorded from 19-ch tin electrodes, mounted in a cap(Electro-Cap International, Inc., Eaton, Ohio, USA) according to the International 10-20 system,
referred to a tin electrode placed on AFz. The EEG was amplified on a Nihon Kohden EEG1100 with a time constant of 0.3 s. Because I expected that large EEG activity related to thetask execution would not appear around the earlobes [19], I placed Ag/AgCl electrodes on both
earlobes and recorded their potentials separately. Their averaged potentials were subtracted fromthe EEG data off-line. For monitoring eye movements, an electrooculogram (EOG) was recordedwith a pair of Ag/AgCl electrodes placed above and below the left eye. The sampling rate of the
EEG and EOG was 1000 Hz.

4.1.2

Data Analyses

In an offline analysis, I resampled the EEG data at a rate of 200 Hz. The simulation results(Figs. 3.3 and 3.4) gave rise to the need to reduce slow waves in the EEG by a digital filter before
the decomposition (see Chapter 3). Therefore, the EEG data was filtered with the bandpass of
1–40 Hz and of 2–40 Hz by using three kinds of finite impulse (FIR) filters: a high-pass of 1 Hz
(600-point, −20 dB at 0.5 Hz), a high-pass of 2 Hz (300-point, −26 dB at 1 Hz) and a low-passof 40 Hz (15-point, −45 dB at 50 Hz). Then, I segmented the filtered EEG data into 2 s epochs
from -500 to 1500 ms after the stimulus onset for the SR- and S-task, or after the button pushonset for the M-task.
40

Reaction times were defined as the intervals between the stimulus onset and the button pushsignal onset. In the following analysis, I used the 19-ch EEG epochs of the trials in which RTs
were within 100–400 ms and EOG were within ±100 µV. In addition, an artifact criterion of±100 µV was used for each of the channels to reject trials with excess electromyographic activity
or measurement noise. Because huge measurement noise was included in one subject’s EEG data,
I excluded his/her data from the analysis.
The stimulus- and overt response-locked components were obtained by Eqs. (3.9) and (3.10)
respectively from each set of the EEG epochs and the RTs during the SR-task. The extracted
stimulus-locked components were then compared with the stimulus-triggered average EEG during
the SR-task and with the AEP obtained by averaging the EEG epochs during the S-task triggering
the stimulus onset. I set the baseline at the interval from 100 to 0 ms before the stimulus
onset, and subtracted the average potentials during the interval from these three potentials
for individual channels. The extracted overt response-locked components were compared withthe overt response-triggered average EEG during the SR-task and with the MRP obtained byaveraging the EEG epochs during the M-task triggering the response onset. I set the baseline at
the interval from 500 to 400 ms before the response onset, and subtracted the average potentials
during the interval from these three potentials for individual channels.
In the extracted stimulus-locked components (Fig. 4.1, A), negative peaks around 100 ms(N100), positive peaks around 160 ms (P200), positive peaks around 315 ms (P300) and negative
peaks around 410 ms (N400) were observed. In the extracted overt response-locked components(Fig. 4.1, B), negative peaks around −40 ms (N-40), positive peaks around 160 ms (P160) andnegative peaks around 500 ms (N500) were observed. Since I expected that the effects of thetemporal smearing would be large in the late parts of the stimulus-triggered average EEG andin the early parts of the overt response-triggered average EEG (see Fig. 2.5, C), in this study, I
only examined the scalp distributions of the P300, N400, N-40 and P160. Indeed, I confirmedthat the peak amplitudes, latencies, and scalp distributions of the other potentials earlier than
the P300 or later than the P160 were not substantially different depending on the methods used41

(see Appendix C).
For the stimulus-locked component and the stimulus-triggered average EEG during the SRtask, the peak latencies of the P300 and N400 were measured from the extracted stimulus-lockedcomponents at Cz as the time points of the largest positive and negative peaks within 250–350
and 300–500 ms. For the AEP, its peak latencies were measured from the AEP at Cz in the samemanner. For the overt response-locked component and the overt response-triggered average EEG
during the SR-task, the peak latencies of the N-40 and P160 were measured from the extracted
overt response-locked components at Cz as the time points of the largest negative and positive
peaks within −100–100 and 50–250 ms. For the MRP, its peak latencies were measured from
the MRP at Cz in the same manner. Then, scalp distributions were obtained from the 19-ch
potentials at these latencies and compared with each other.
To confirm that the differences of the scalp distributions between the extracted stimulus/overt response-locked components and the conventional stimulus-/overt response-triggered average EEG during the SR-task are attributable to the temporal overlapping with each other,I reconstructed stimulus-/overt response-triggered average EEG by overlapping both of the extracted stimulus- and overt response-locked components with the delays of the individual RTs
according to Eq. (2.1).
The similarities of the scalp distributions by different methods were quantified by a dot
product (dp). I normalized the 19-dimensional EEG vectors to unit vectors with a magnitudeof 1, and calculated a dot product between the vectors, where the dot product of 1 represents
perfect similarity of the distributions. A two-tailed Wilcoxon signed-rank test was conducted
to test the null hypothesis that the dot products obtained from each of the subjects did not
differ from zero. Also, peak amplitudes in different waveforms were compared by means of the
two-tailed Wilcoxon signed-rank test. An alpha level of 0.05 was used for the statistical test.

42

4.2

Results

4.2.1

Decomposition of Observed EEG (1–40 Hz)

I obtained 256.8±34.7 trials per subject for the SR-task. The mean and SD of all the RTs are
221.0 ms and 57.8 ms respectively. I first decomposed the EEG filtered with 1–40 Hz during theSR-task. Figure 4.1 shows the stimulus- and overt response-locked components extracted fromeach/all of the subjects’ EEG at Cz, and the RTs. The extracted stimulus-locked component at
Cz exhibits the N100, P200, P300 and N400 (Fig. 4.1, A). The extracted overt response-lockedcomponent at Cz exhibits the N-40, P160 and N500 (Fig. 4.1, B).

Potential [ V]

A

B

Stim.
onset

10

10

0

0

-10

-10
-400

0

400
Time [ms]800Overt resp.
onset

0
400
Time [ms]Figure 4.1: Extracted stimulus- and overt response-locked components at Cz. A: Extracted
stimulus-locked components. The arrows indicate, from left to right, the N100, P200, P300
and N400. B: Extracted overt response-locked components. The arrows indicate, from left toright, the N-40, P160 and N500. The blue lines represent the components extracted from all thesubjects’ EEG and RTs; the black lines represent the components extracted for each subject. In
this figure, the average of each wave is adjusted to zero.

Extracted Stimulus-locked Component
Figure 4.2 shows the extracted stimulus-locked components, the stimulus-triggered average EEG
during the SR-task and the AEP obtained from all the subjects’ EEG data. The AEP was
obtained from 269.8±36.4 trials per subject during the S-task. Figure 4.4, A shows the scalp
distributions of the P300 and N400 in each of the waveforms.

43

At around the P300, the waveforms of the extracted stimulus-locked components shift frontallyas compared to those of the stimulus-triggered average EEG during the SR-task; the amplitudesof the P300 in the extracted stimulus-locked components are large at around Fp1 and Fp2,
whereas those in the stimulus-triggered average EEG during the SR-task are large at around Cz(Fig. 4.2). However, the dp of the P300 between both waves are significantly greater than zero
(dp = 0.74 ± 0.26, p < 0.05) (Fig. 4.4, A). This result indicates that the differences between theextracted stimulus-locked component and the stimulus-triggered average EEG appear to exist ataround the P300, but is not reflected in the (global) dp measure.
In the AEP, clear P300 waveforms are not observed (Fig. 4.2), although the dp of the P300
between the AEP and the extracted stimulus-locked components are significantly greater thanzero (dp = 0.69 ± 0.22, p < 0.05) (Fig. 4.4, A).
At around the N400, the waveforms of the extracted stimulus-locked components are clearly
different from those of the stimulus-triggered average EEG during the SR-task; the amplitudesof the N400 in the extracted stimulus-locked components are large at around Cz, whereas thosein the stimulus-triggered average EEG during the SR-task are large at around Fp1 and Fp2(Fig. 4.2). The dp of the N400 between both waves are not significantly greater than zero
(dp = 0.13 ± 0.38, p > 0.05) (Fig. 4.4, A). It is of note that the amplitudes of the N400 in thereconstructed stimulus-triggered average EEG are large at around Fp1 and Fp2, in the same
way as the observed stimulus-triggered average EEG, and the dp between them approach 1.0(dp > 0.99 for all the subjects) (Fig. 4.4, A). This result indicates that the differences in the
N400 amplitudes between the extracted stimulus-locked components and the stimulus-triggered
average EEG disappear by overlapping the extracted overt response-locked components on theextracted stimulus-locked components, suggesting that the differences are attributable to theoverlap of the overt response-locked components.
In the AEP, the amplitudes of the N400 are large at around Cz, as are those in the extracted
stimulus-locked components (Fig. 4.2), and the dp of the N400 between them are significantly
greater than zero (dp = 0.37 ± 0.44, p < 0.05) (Fig. 4.4, A).

44

Fp1

Fp2

F3

Fz

F4

F8

T3

C3

Cz

C4

T4

T5

P3

Pz

P4

T6

O1

Stim.
onset

O2

5 V

F7

500 msExtracted stim.-locked component
Average on stim. onset
Auditory evoked potential
Figure 4.2: Extracted stimulus-locked components, stimulus-triggered average EEG during theSR-task and the AEP obtained from all the subjects’ EEG.

45

Extracted Overt Response-locked Component
Figure 4.3 shows the extracted overt response-locked components, the overt response-triggered
average EEG during the SR-task and the MRP obtained from all the subjects’ EEG data. The
MRP was obtained from 236.8±32.3 trials per subject during the M-task. Figure 4.4, B shows
the scalp distributions of the N-40 and P160 in each of the waveforms.
The amplitudes of the N-40 in the extracted overt response-locked components are large ataround Cz, as is the overt response-triggered average EEG during the SR-task (Fig. 4.3), and the
dp of the N-40 between both waves are significantly greater than zero (dp = 0.72 ± 0.18, p < 0.05)(Fig. 4.4, B).
In the MRP, clear N-40 waveforms are not observed, in contrast to the extracted overt
response-locked component (Fig. 4.3), although the dp of the N-40 between them are significantly greater than zero (dp = 0.54 ± 0.43, p < 0.05) (Fig. 4.4, B).
The amplitudes of the P160 in the extracted overt response-locked components are large ataround Cz, as is the overt response-triggered average EEG during the SR-task (Fig. 4.3), and the
dp of the P160 between both waves are significantly greater than zero (dp = 0.75 ± 0.20, p < 0.05)(Fig. 4.4, B).
In the MRP, the amplitudes of the P160 are large at around Cz, as are those in the extracted overt response-locked components (Fig. 4.3), and the dp of the P160 between them aresignificantly greater than zero (dp = 0.60 ± 0.41, p < 0.05) (Fig. 4.4, B).
On the other hand, the N-40 and P160 amplitudes of the extracted overt response-lockedcomponents do not exhibit significant differences between C3 and C4 (−4.1 ± 4.6 and −4.0 ± 3.4,
respectively, p > 0.05 for the N-40; 4.8 ± 2.5 and 3.5 ± 2.8, respectively, p > 0.05 for the P160),
although the overt response-triggered average EEG does exhibit significant differences (−4.4±2.4
and −3.7 ± 1.9, respectively, p < 0.05 for the N-40; 5.1 ± 2.2 and 4.2 ± 1.9, respectively, p < 0.05
for the P160).

46

Fp1

Fp2

F3

Fz

F4

F8

T3

C3

Cz

C4

T4

T5

P3

Pz

P4

T6

O1

Resp.
onset

O2

5 V

F7

500 msExtracted overt resp.-locked component
Average on overt resp. onset
Movement-related potential
Figure 4.3: Extracted overt response-locked components, overt response-triggered average EEG
during the SR-task and the MRP obtained from all the subjects’ EEG.

47

4.2.2

Decomposition of Observed EEG (2–40 Hz)

I also decomposed the EEG filtered with 2–40 Hz during the SR-task. Figure 4.5 shows theextracted stimulus-/overt response-locked components and the stimulus-/overt response-triggered
average EEG during the SR-task obtained from all the subjects’ data.
The extracted stimulus-locked component at Cz exhibits the P300 and N400 in the same
way as that extracted from the EEG filtered with 1–40 Hz (Fig. 4.5, A). In contrast to the
EEG filtered with 1–40 Hz, however, the stimulus-triggered average EEG at Cz has the P300
and N400 peaks in the same way as the extracted stimulus-locked component (Fig. 4.5, A). The
amplitudes of the P300 and N400 in the stimulus-triggered average EEG are large at around Fz
in the same way as the extracted stimulus-locked components (Fig. 4.5, B), and the dp of the
P300 and N400 between them are significantly greater than zero (dp = 0.91 ± 0.14, p < 0.05 for
the P300; dp = 0.91 ± 0.12, p < 0.05 for the N400). These results indicate that, in the case of
the EEG filtered with 2–40 Hz, the effect of the temporal smearing by the overt response-lockedcomponents is not so large as to change these scalp distributions significantly.
Similar to the EEG filtered with 1–40 Hz, both the extracted overt response-locked component
and the overt response-triggered average EEG at Cz exhibit the N-40 and P160 (Fig. 4.5, C). The
amplitudes of the N-40 and P160 in the overt response-triggered average EEG are large at aroundCz in the same way as the extracted overt response-locked components (Fig. 4.5, D), and the dpof the N-40 and P160 between them are significantly greater than zero (dp = 0.87 ± 0.11, p < 0.05
for the N-40; dp = 0.92 ± 0.067, p < 0.05 for the P160). On the other hand, the amplitudes of
the N-40 and P160 at Cz in the overt response-locked components are significantly smaller thanthose in the overt response-locked components extracted from the EEG filtered with 1–40 Hz(−2.63 ± 1.07 and −4.82 ± 3.44, respectively, p < 0.05 for the N-40; 4.03 ± 1.07 and 5.31 ± 2.27,
respectively, p < 0.05 for the P160). This result suggests that the above-mentioned small effect of
the temporal smearing by the overt response-locked components is due to the attenuated peaksin the overt response-locked components.

48

A

Stim.-locked
comp.

Average on
stim. onset

Reconstructedaverage EEG

AEP

P300

4

N400

0

B

Overt resp.
Average on
Reconstructed-locked comp. overt resp. onset average EEG

[ V]

MRP

N-40

4

P160

0

[ V]

Figure 4.4: Scalp distributions of the P300, N400, N-40 and P160. A: The P300 and N400 in theextracted stimulus-locked components, the stimulus-triggered average EEG, the reconstructed
stimulus-triggered average EEG and the AEP obtained from all the subjects’ EEG. B: The N-40
and P160 in the extracted overt response-locked components, the overt response-triggered average
EEG, the reconstructed overt response-triggered average EEG and the MRP obtained from all
the subjects’ EEG.

49

Potential [ V]

A

C

Stim.
onset
5

5

0

0

-5
0

400
Time [ms]

-5
-400

800

B

0
Time [ms]400D
Stim.-lockedcomponent

Overt resp.-lockedcomponent

Average on
stim. onset

P300

N400

Overt resp.
onset

Average on
overt resp. onset

N-40

2
0
-2

P160
[ V]

2
0
-2

[ V]

Figure 4.5: Decomposition of EEG (2–40 Hz) during the SR-task. A: Extracted stimulus-lockedcomponent at Cz (solid blue line) and stimulus-triggered average EEG during the SR-task at
Cz (dotted blue line). For comparison, the stimulus-locked component at Cz and the stimulustriggered average EEG at Cz obtained from the EEG filtered with 1–40 Hz are superimposed(solid red and dotted red lines respectively). B: Scalp distributions of the P300 and N400 in theextracted stimulus-locked components and the stimulus-triggered average EEG during the SRtask. C: Extracted overt response-locked component at Cz (solid blue line) and overt responsetriggered average EEG during the SR-task at Cz (dotted blue line). For comparison, the overt
response-locked component at Cz and the overt response-triggered average EEG at Cz obtained
from the EEG filtered with 1–40 Hz are superimposed (solid red and dotted red lines respectively).
D: Scalp distributions of the N-40 and P160 in the extracted overt response-locked components
and the overt response-triggered average EEG during the SR-task.

50

4.3

Discussion

4.3.1

Stimulus-locked EEG Component

I extract the stimulus-locked components by Eq. (3.9) from the EEG data during the SR-task.
The extracted stimulus-locked components exhibit the N100, P200, P300 and N400. The existenceof the N100 and P200 is in agreement with previous studies which have examined EEG duringsimple reaction time tasks [20,21]. In this study, I evaluate the effect of the temporal overlappingat the late peaks: the P300 and N400.
As for the P300, the scalp distributions between the extracted stimulus-locked components
and the stimulus-triggered average EEG appear to be different in the case of the EEG filtered with
1–40 Hz. However, the effect of the temporal smearing by the overt response-locked components
is not so large as to change the global scalp distributions (measured by the dp) significantly. The
local difference between the stimulus-locked P300 and the stimulus-triggered average P300, theformer showing more frontal topology, should be studied further.
On the contrary, at the N400, the significant difference of the scalp distributions between theextracted stimulus-locked components and the stimulus-triggered average EEG is observed.
As for the EEG filtered with 1–40 Hz, the N400 amplitudes in the extracted stimulus-lockedcomponents are large in the central region. The validity of the central N400 in the extracted
stimulus-locked components is supported by a previous study which reports a stimulus-locked,
centro-parietal N400 during audio-visual cognitive tasks with overt responses [22]. The result that
the AEP, which is thought to reflect only a stimulus-related brain process, also has a centrally
distributed N400 could also be evidence supporting the validity of the N400 in the extracted
stimulus-locked components.
The scalp distributions of the N400 in the extracted stimulus-locked components are different
from those in the stimulus-triggered average EEG. However, when the extracted overt responselocked components are overlapped on the stimulus-locked components, the difference in the N400
between them disappears. This suggests that the difference is attributable to the overlapping of
the overt response-locked components.

51

As for the EEG filtered with 2–40 Hz, the N400 appears at around Cz even in the stimulustriggered average EEG in the same way as the extracted stimulus-locked components. This is
likely because, in the case of the EEG filtered with 2–40 Hz, the overt response-locked components
are attenuated by the high-pass filter, and the resultant small effect of the temporal smearing
discloses the original N400 in the stimulus-triggered average EEG. Its scalp distributions are
similar across subjects; the dot product between a subject’s vector and the other subjects’ average
vector is calculated for each subject and compared with zero by the Wilcoxon signed-rank test
(dp = 0.83±0.20, p < 0.05). Then again, the N400 which appears in the EEG filtered with 2–40 Hz
additionally confirms the existence of the N400 in the stimulus-locked components extracted fromthe EEG filtered with 1–40 Hz.
The above results indicate that the differences between the stimulus-locked components andthe stimulus-triggered average EEG can be attributed to the temporal overlapping of the overt
response-locked components, and the effects of the temporal overlapping are different between
the filter properties used, because the waveforms of the overlapping components vary dependingon the filter properties.
On the other hand, although the auditory stimuli are the same during the SR-task and the
S-task, the waveforms of the late parts in the extracted stimulus-locked components and the
AEP are different; the AEP does not exhibit the clear P300 in contrast to the stimulus-lockedcomponents (Fig. 4.2). This indicates that even the “pure” stimulus-related activity is modulated
depending on whether the overt response is required or not. In another task requiring no motor
responses, different P300 evoked by the same stimuli in different instructions have also beenreported [23]. These results might reflect the context-dependent modulation of the stimulusrelated EEG activity.

4.3.2

Overt Response-locked EEG Component

In many studies which examine movement-related potentials, slow components of averaged EEG
have been discussed (for example, [24–27]). However, in this study, such slow components are
cut by the high-pass filter of 1 or 2 Hz because of the methodological limitation (see Chapter 3).
52

Therefore, in the present study I cannot easily deal with classical slow components, such as theBereitshaftspotential [25, 27].
I extract the overt response-locked components by Eq. (3.10) from the EEG data during theSR-task. The extracted overt response-locked components exhibit the three distinct peaks: theN-40, P160 and N500. The existence of these peaks is in agreement with a previous study which
examined EEG during cued movements [26]. In this study, I evaluate the effect of the temporal
overlapping at the earlier peaks: the N-40 and P160.
In the EEG filtered with both 1–40 and 2–40 Hz, the scalp distributions of the N-40 and P160
in the extracted overt response-locked components are similar to those in the overt responsetriggered average EEG during the SR-task. This suggests that the distributions of the overt
response-locked components are less smeared by the temporal overlapping of the stimulus-lockedcomponents. This is because the stimulus-locked components do not have large slow waves incomparison with the overt response-locked components and, by the averaging procedure, areeasily diminished as a result of phase cancellations of fast waves.
The MRP do not exhibit the clear N-40, in contrast to the overt response-locked components
and the overt response-triggered average EEG. By the averaging procedure, some studies [24, 26]
have also revealed different EEG activity during self-paced movements and cued movements. The
similarity of the overt response-locked component and the overt response-triggered average EEG
suggests that the differences in the overt response-triggered average EEG between the tasks observed in this and previous studies are due not to the temporal overlapping of the stimulus-lockedcomponents but to the context-dependent modulation of overt response-related EEG activity.
Although the subjects responded with their right index fingers, significant asymmetry is notobserved for the N-40 and P160 in the extracted overt response-locked components, in contrast
to the overt response-triggered average EEG and to previous studies (for example, [25]). This isbecause the noise amplified by the decomposition makes the inter-subject variability of the N-40
and P160 distributions large and the p-values become higher.

53

4.4

Conclusions

In conclusion, the results presented in this chapter confirm the feasibility and usefulness of thefrequency domain decomposition of EEG data into stimulus- and overt response-locked components during a reaction time task, and show that the contamination by the temporal overlappingis large especially in the late part of the stimulus-triggered average EEG during the SR-task whenslow waves remain in the EEG data.
It should be noted that the effect of the temporal smearing shown in this study may not holdtrue for every reaction time task. This is because the effect of the temporal smearing is determinedby the distribution of RTs and the waveform of the stimulus- or overt response-locked components[see Eqs. (2.2) and (2.3)]; that is, the contamination is task-dependent and cannot be examined
directly without the decomposition. Therefore, to examine the brain activity during reaction
time tasks in detail, the decomposition is required eventually, regardless of which components we
use and whether they are obtained by the averaging procedure or by the proposed method.

54

Chapter 5

EEG during Tasks with Covert
Responses
In this chapter, I firstly introduce reports on the brain activity during tasks which require covert
responses. Then, based on the reports, I mathematically formulate the EEG during the tasks
with covert responses. Finally, I discuss the previous methods for extracting the stimulus- andcovert response-locked components from the EEG, and discuss the necessity of a novel method
for extracting the two components.

5.1

Brain Activity during Tasks with Covert Responses

As stated in Chapter 2, when a subject makes an overt response to a stimulus, stimulus- andovert response-locked brain activities occur. Similarly, when a subject makes a covert response
to a stimulus, such as the perception of ambiguous figures or the solving of complex problems, it
is considered that stimulus- and covert response-locked brain activities occur [28–32].
Indeed, studies by Tallon-Baudry et al. [31, 32] suggest the presence of the covert responselocked component in addition to the stimulus-locked component in EEG during cognitive tasks.
They examined the EEG during the viewing of illusory (Kanizsa) triangles (Fig. 5.1, A) by timefrequency analysis, showed that the gamma activity which is not time-locked to the stimulus onset
was induced when the subject perceived the triangles, and suggested the role of the induced

55

gamma activity1 in the construction of a coherent representation of objects and the rehearsalof the representation in the memory. This report indicates that some brain activities related
to perception and memory recall are not time-locked to the stimulus onsets but to the covert
response onsets.

5.2

Mathematical Formulation

Like the EEG during tasks with overt responses, the EEG during tasks with covert responses isassumed to consist of a stimulus-locked component, a covert response-locked component shiftedby the delay of an individual trial, and noise. Therefore, the EEG is expressed by:yn (t) = s(t) + cr(t − τn ) + vn (t) (t = 0, · · · , T − 1; n = 1, · · · , N ),(5.1)

where yn (t): observed EEG data in trial n; s(t): stimulus-locked component; cr(t): covert
response-locked component; τn : delay of cr(t) in trial n; vn (t): noise in trial n. This implies
that ongoing EEG activity and trial-to-trial variability of stimulus- and covert response-locked
activity are included in vn (t). Note that, in this case, τn is unknown, in contrast to tasks withovert responses.

5.3

Previous Methods

While the stimulus-locked component has been extracted by the stimulus-triggered averaging
procedure, the covert response-locked component cannot be extracted by the averaging procedure,or even by the method proposed in Chapter 3, because its delays in individual trials are of
unknown length.

5.3.1

Time-frequency Analyses

As mentioned above, using time-frequency analyses, some studies have examined the induced
EEG activity, or covert response-locked EEG activity [28,29,31–33]. A time-frequency analysis is
1 They call EEG activity not time-locked to stimulus onset “induced” activity. In contrast, I define the covert
response-locked component as being the EEG component not time-locked to stimulus onset (see Chapter 1).
Therefore, their term “induced” corresponds to the term “covert response-locked” used in this thesis.

56

Figure 5.1: Induced gamma activity. A: The subjects were presented with four different stimuli.
Both the illusory (Kanizsa) and the real triangles were coherent stimuli, leading to a coherent
percept through a bottom-up feature binding process. The “no-triangle” stimulus served as acontrol. Subjects had silently to count the occurrences of the target stimulus, a curved illusory
triangle, and to report this number at the end of each recording block. This task, when correctly
performed, ensured that the subjects perceived correctly the illusory triangles and remained
attentive throughout the whole recording session. B: Time-frequency power of the EEG atelectrode Cz, in response to the illusory triangle (top) and to the no-triangle stimulus (bottom).
Two successive bursts of oscillatory activities were observed. A first burst occurred at about
100 ms and 40 Hz; it was a stimulus-locked response peaking at electrode Cz. It showed nodifference between stimulus types, and thus cannot reflect the spatial feature-binding process
required to perceive the triangles. The second burst, around 280 ms and between 30 and 60
Hz, was not stimulus-locked, and most prominent in response to coherent stimuli. There was no
statistical difference in the gamma range between the responses to the illusory and real triangles.
Induced gamma could thus reflect the spatial binding of the elementary features of the picture
into a coherent representation of the triangle. It should be noted that no component of thestimulus-locked response discriminated between coherent and non-coherent stimuli. (Adaptedfrom [31])57a method for mapping a one-dimensional signal of time into a two-dimensional function of timeand frequency by a time-frequency transform, such as the short-time Fourier transform and thecontinuous wavelet transform [34]. Because the stimulus-triggered averaging procedure canceledout the induced EEG activity (Fig. 5.2, B), in these studies [28,29,31–33], time-frequency analyses
were used before applying the averaging procedure as follows. First, time-frequency analyses
were applied to individual single-trial EEG epochs, and time-frequency powers were estimated(Fig. 5.2, D). Then, the time-frequency powers were averaged across trials (Fig. 5.2, E). TallonBaudry and Bertrand [32] stated that this approach can identify non-phase-locked activity as longas the signal-to-noise ratio is high enough and the jitter does not exceed the wavelet duration
used in the time-frequency transform.
However, the limitation of the time-frequency analyses is that this method only identifies
the power and average delay of the covert response-locked component. To obtain more detailed
information on the covert response-locked components, their waveforms and delays of individual
trials, we need to use another method rather than the time-frequency analyses.

5.3.2

Peak Picking, Woody’s Method, and Pham et al.’s Method

Hitherto, the waveforms and delays of the covert response-locked components have been obtainedby peak picking, Woody’s method and Pham et al.’s method [35–43]. In peak picking, EEG is
low-pass filtered, and the individual delay of the covert response-locked component is estimatedby detecting the largest positive/negative deflection in a predefined time window. In Woody’s
method [43], cross-correlation is calculated between a template that resembles the waveform of
the covert response-locked component and the low-pass filtered EEG. The individual delay isdefined as the time lag by which the template should be shifted to obtain the maximal crosscorrelation. In Pham et al.’s method [40], the individual delays are estimated by the maximum
likelihood approach employing iterative Fisher-scoring.
A critical problem of these methods is their underlying assumption. These methods implicitly
or explicitly assume that only one component, the covert response-locked component, appears

58

Figure 5.2: Identification of induced activity (covert response-locked activity) by time-frequency
analyses. (Adapted from [32])59after stimulus onsets; that is, these methods model the EEG by:yn (t) = cr(t − τn ) + vn (t),(5.2)

being obviously different from Eq. (5.1). In my assumption of Eq. (5.1), there are two components
and the two components can be temporally overlapping. The temporal overlapping of the two
unknown components makes the problem quite difficult because the two components contaminate
each other, and the appearance of the two components varies from trial to trial (see Fig. 2.5, B).
For example, it is possible that a negative peak in the stimulus-locked component averages out apositive peak in the covert response-locked component and both of the components appear not
to have the peaks. Because methods which assume only one component do not consider such acontamination effect, they cannot be used when two components are temporally overlapping.

5.4

Conclusions

From the previous studies on the EEG during cognitive tasks, it is suggested that, even whena subject does not respond overtly to a stimulus, EEG activity can exhibit two components:
the stimulus- and covert response-locked components. To extract the two components, timefrequency analyses, peak picking, Woody’s Method, and Pham et al.’s Method have been used.
However, these methods are not adequate. Therefore, it is necessary to propose a novel method forextracting the stimulus- and covert response-locked components. In the next chapter, I propose
such a method.

60

Chapter 6

Proposed Method for Covert
Responses
In the previous chapter, I describe the necessity of a novel method for temporally decomposing
EEG into the stimulus- and covert response-locked components without knowledge of the individual delays of the covert response-locked component. In this chapter, I propose a non-parametricmethod for the temporal decomposition.

6.1

Decomposition Method

I assume that the EEG activity consists of the stimulus- and covert response-locked components
and is expressed by Eq. (5.1). Here, the objective is to obtain s(t), cr(t) and τn only from yn (t)(n = 1, · · · , N ) (Fig. 6.1).
Once a set of delays τ = [τ1 , τ2 , · · · , τN ] is determined, the waveforms of s(t) and cr(t) aredetermined by the proposed method for overt responses (see Chapter 3). In the method for overt
responses, s(t) and cr(t) extracted by using τ are respectively expressed by:&'N
1 ! exp (−i2πωτn /T ) Ȳ (ω) − Ē(ω)Yn (ω)
sτ (t) = IDFT,
N n=1
D(n, ω)'N
1 ! Yn (ω) − Ȳ (ω).
crτ (t) = IDFT
N n=1
D(n, ω)
&

61

(6.1)

(6.2)

Stim.
onset

EEG in trial 1
Stim.
onset

EEG in trial 2

Stim.
onset

EEG in trial N

Stim.-locked comp.
Covert resp.-locked comp.

Delay of covert
resp.-locked comp.

Temporaldecomposition

Figure 6.1: Schematic representation of the objective.

62

Trial

I estimate true τ under a certain assumption. Figure 6.2 shows the simulation results on
which the assumption is based. If τ is true, the waveforms of sτ (t) and crτ (t) are close to thoseof s(t) and cr(t) (Fig. 6.2, C), and the waveform of sτ (t) + crτ (t − τn ) becomes close to yn (t)(Fig. 6.2, E). In contrast, if τ is wrong, the waveforms of sτ (t) and crτ (t) are not close to thoseof s(t) and cr(t) (Fig. 6.2, D), and the waveform of sτ (t) + crτ (t − τn ) does not become close
to yn (t) (Fig. 6.2, F). Based on these simulation results, I assume that sets of nearly true delays
provide better approximations of the observed EEG data than sets of wrong delays. Under thisassumption, I estimate the true τ by solving an optimization problem of:
Minimize

oτ =

t2 !
N
!

{yn (t) − sτ (t) − crτ (t − τn )}2

t=t1 n=1

Subject toτ ⊂ N.

In this study, t1 and t2 were respectively set at 0 and 1000 ms after the stimulus onset.
To select algorithms for solving this optimization problem, by using simulated data, I checked
the structure of the objective function by changing τk (k was fixed) one step at a time (an example
is shown in Fig. 6.3). Consequently, as the objective function seemed to have many local minima(Fig. 6.3), hill-climbing methods [44], such as the steepest descent method [44], are not suitablebecause these methods are easily trapped in a local minimum. Stochastic algorithms, such as
simulated annealing [44–46], are thus considered suitable because these methods can escape froma local minimum and find a global minimum.
Among the algorithms I tested, a random search [44, 46, 47] is the best in the point that
this algorithm achieves speedy convergence to a global minimum with a high probability. The
procedure of the random search is as follows:
1. Generate a set of delays τ by random numbers, and set the index of the delays k to 1.
2. Obtain oτ .
3. Make τ " by changing τk in τ randomly.
4. Obtain oτ ! .
63

A

B

Stim.
onset

C

E

D

F

Delay

Value of objective function

Figure 6.2: Simulation results on which the proposed method is based. A: Original stimuluslocked component (top), covert response-locked component (middle), and noise (bottom). B:
Simulated data obtained by Eq. (5.1). C: Stimulus-locked component (top) and covert responselocked component (bottom) extracted using the true τ by the method for overt responses. The
waveforms of these components are similar to those of the original ones. D: Stimulus-lockedcomponent (top) and covert response-locked component (bottom) extracted using a wrong τ bythe method for overt responses. The waveforms of these components are different from thoseof the original ones. E: Reconstructed data from the components extracted by the true τ . Its
waveform is similar to that of the simulated data (top row in B). F: Reconstructed data fromthe components extracted by the wrong τ . Its waveform is different from that of the simulated
data (top row in B).

2718

2716

0

100

200

300

400

k (k = 1)
Figure 6.3: Value of objective function for each value of τk (k = 1) obtained from simulated data.
The other τk (k "= 1) are fixed at random numbers.
64

5. Replace τ and oτ by τ " and oτ ! respectively if oτ ! is smaller than oτ .
6. Return to step 3 with increasing k by 1 (k returns to 1 if k becomes larger than N ).
This procedure is different from the so-called general random search [44, 46, 47] in the point that,at step 3 in each iteration, this procedure does not change all of the τn (n = 1, · · · , N ), but onlyone τk , where k circulates from 1 to N as the iteration number increases.
By using yn (n = 1, · · · , N ) shown in Fig. 6.2, B, I demonstrate the performance of the random
search in Fig. 6.4. Figure 6.4, A, shows how the values of the objective function decrease within
the same computational time (10 s) by the following four algorithms: the random search, a moregeneral random search [47] where the procedure is the same as one I used except for making
τ " by random numbers at step 3, a genetic algorithm [48] and simulated annealing [45]. The
convergence of the random search is the fastest among these (Fig. 6.4, A). Figure 6.4, B, shows
the converged points by 2000 [= 20 × N (N = 100)] iterations of the random search, taking about
13 s by a personal computer [Dell XPS M1210 Intel(R) Core(TM)2 CPU T7200 @ 2.00GHz].
The 2000 iterations were repeated 100 times with different initial τ at step 1. Among the 100
converged points, 54 points (encircled points in Fig. 6.4, B) reached values smaller than 200, andthe correlation coefficients between the estimated and original delays were higher than 0.94. This
indicates that the random search achieves convergence to a global minimum with a probabilityof about 0.5, being adequately high for practical use.
In the optimization, I firstly repeated 20 × N iterations of the random search 50 times, andobtained 50 sets of τ and oτ . Then, I started 20 × N iterations from the τ which had the smallestoτ among the 50 sets.
During the optimization, I monitored the variance across n of yn (t) − sτ (t) − crτ (t − τn ),which became smaller and smaller as the optimization proceeded, and stopped the optimization
if the variance became smaller than those of the prestimulus level. This is because yn (t) − sτ (t) −crτ (t − τn ) is considered as the extracted background noise and the level of original backgroundnoise is considered not to decrease after stimulus onsets.
After the optimization, I adjusted the average of the obtained τ . This adjustment is re65

A

B
1
Correlation coefficient

Value of objective function

500
400
300
200
0

5
Time [s]

0.5
0
-0.5
150
250
1420
Value of objective function

10

Figure 6.4: Performance of a random search. A: Decay of the values of the objective function
by the restricted random search proposed (thick solid line), complete random search (thin solidline), genetic algorithm (thick dotted line), and simulated annealing (thin dotted line). B: Scatterplots of the converged values of the objective function and the correlation coefficients between
the estimated and original delays by the restricted random search. Encircled dots indicate theconverged points whose values of the objective function are lower than 200.
quired because the average of τ varies depending on the time point defined as the onset of thecovert response-locked component and the onset is arbitrarily determined in the optimization.
For example, I adjusted the average of τ obtained from the EEG during NoGo trials so thatthe maximum peaks in the extracted covert response-locked components became their onsets(described in Chapter 7). I then called the adjusted τ the estimated delays.

6.2

Simulation Test with Artificial Data

In order to verify the performance of the proposed method, a numerical experiment was performed
on a set of known artificial signals.

6.2.1

Methods

In this simulation, original s(t) and cr(t) were generated by the exponential and the cosine
functions respectively (Fig. 6.2, A). Gaussian random numbers [mean = 270 ms, standard
deviation (SD) = 50 ms] were used as τ , and white noise was used as vn (t). To examine66

the relation between the accuracy of the estimated τ and noise level, I generated 11 sets ofyn (t)(n = 1, · · · , 100) with different signal to noise ratio (SNR) of −10, −9, · · · , 0 by adjusting
SD of vn (t) to 0.56, 0.50, 0.45, 0.39, 0.36, 0.32, 0.28, 0.25, 0.22, 0.20, 0.18, respectively. The SNR
"T −1
"T −1
was defined as 10 log10 ( t=0 cr(t)2 / t=0 vn (t)2 ).
Because the proposed method uses a stochastic algorithm (the random search), the estimation

results should vary more or less in every estimation. To examine the robustness of the estimation
results, I applied the proposed method to each set of yn (t)(n = 1, · · · , 100) for 10 times, andobtained 10 sets of τ , sτ (t) and crτ (t) for each set. Because the averages of τ were arbitrarily
determined in the optimization, the averages of the estimated τ were adjusted to those of theoriginal τ . The accuracy of the estimated τ was quantified by the correlation coefficient between
the estimated and original τ (Figs. 6.5, D). Also, the similarity of the extracted component to
the original one was quantified by the correlation coefficient between them, corresponding to thecross correlation between them at the lag of zero.

6.2.2

Results

Figure 6.5 shows the results of the simulation test with artificial data. Examples of the extracted
components and estimated delays from the simulated data with a SNR of 0 are shown in Fig. 6.5,
A–C. The extracted stimulus-locked components are highly correlated with the original one (r =0.96) (Fig. 6.5, A), as are the extracted covert response-locked components (r = 0.93) (Fig. 6.5,
B). The estimated delays are significantly correlated with the original ones (r = 0.99, p < 0.05,
slope = 0.96) (Fig. 6.5, C). This indicates that the performance of the method is adequately
accurate when the SNR is 0. The correlation coefficients between the original and estimateddelays from the simulated data with a SNR of −10, · · · , 0 are shown in Fig. 6.5, D. For a SNR≥ −7 (SD of the noise ≤ 0.39), the correlation coefficients are high in all the repeated estimations(Fig. 6.5, D), indicating that the estimation is adequately accurate and robust when the SNR ishigh. In contrast, for a SNR < 7 (SD of the noise > 0.39), the correlation coefficients are low andvariable across the repeated estimations (Fig. 6.5, D), indicating that the estimation is neither
accurate nor robust when the SNR is low.
67

A

B

Stim.
onset

1

1

0

0

0

Time

1000

0

Time

1000

D
Correlation coefficientC
Estimated delay

Resp.
onset

350

150
150

Original delay

350

1

0

-10

-5
SNR

0

Figure 6.5: Simulation with artificial data. A–C: Examples of the extracted components and theestimated delays when the SNR is set to zero. A: Extracted stimulus-locked component (solidline) and original one (dotted line). B: Extracted covert response-locked component (solid line)
and original one (dotted line). The horizontal axis represents relative time to the defined onsetsof the covert response-locked component. C: Scatter plot of the estimated and original delays. D:
Correlation coefficients between the estimated and original delays for each SNR. The diamonds
and error bars respectively represent the means and SDs of the correlation coefficients.

68

6.3

Simulation Test with EEG Data

In order to test the performance of the method for more EEG-like data, an additional numericalexperiment was performed using real EEG data.

6.3.1

Methods

In this simulation, the stimulus- and overt response-locked components extracted by the method
for overt responses (see Chapter 3) from all the subjects’ EEG and RTs during Go trials in the
Go/NoGo task (described in Chapter 7) were used as s(t) and cr(t) respectively, and the RTs,
which were randomly selected from all the subjects’ RTs during Go trials, were used as τ . Noisevn (t)(n = 1, · · · , 100) was obtained from all the subjects’ EEG during a passive viewing task
(described in Chapter 7) in the following way. The averaged EEG across trials were subtracted
from each of the sets of EEG data, and the resultant EEG was discrete Fourier transformed andnew sweeps were constructed by randomizing the phase information of the transformed EEG,
while keeping their distributions constant (see [49], for a detailed description). From the sweeps
obtained, I randomly selected 100 sweeps, and used them as vn (t)(n = 1, · · · , 100). To examine
the relation between the accuracy of the estimated τ and the noise level, I generated 11 setsof yn (t)(n = 1, · · · , 100) with different SNR of −10, −9, · · · , 0 by adjusting the SD of vn (t) to
2.6, 2.3, 2.0, 1.8, 1.6, 1.4, 1.3, 1.2, 1.0, 0.91, 0.81, respectively. Without the adjustment, the SD ofvn (t) was 6.8 and the SNR was −18.
As in the simulation test with artificial data, I applied the proposed method to each set ofyn (t)(n = 1, · · · , 100) 10 times, and obtained 10 sets of τ , sτ (t) and crτ (t) for each set. Because
the averages of τ were arbitrarily determined in the optimization, the averages of the estimatedτ were adjusted to those of the original τ . The accuracy of the estimated τ was quantified by thecorrelation coefficient between the estimated and original τ (Figs. 6.6, D). Also, the similarity ofthe extracted component to the original one was quantified by the correlation coefficient between
them, corresponding to the cross correlation between them at a lag of zero.

69

6.3.2

Results

Figure 6.6 shows the results of the simulation test with EEG data. Examples of the extracted
components and estimated delays from the simulated data with a SNR of 0 are shown in Fig. 6.6,
A–C. The extracted stimulus-locked components are highly correlated with the original one (r =0.94) (Fig. 6.6, A), as are the extracted covert response-locked components (r = 0.89) (Fig. 6.6,
B). The estimated delays are significantly correlated with the original ones (r = 0.94, p < 0.05,
slope = 0.78) (Fig. 6.6, C). This indicates that the performance of the method is adequately
accurate when the SNR is 0. The correlation coefficients between the original and estimateddelays from the simulated data with a SNR of −10, · · · , 0 are shown in Fig. 6.6, D. For a SNR≥ −4 (SD of the noise ≤ 1.3), the correlation coefficients are high in all the repeated estimations(Fig. 6.6, D), indicating that the estimation is adequately accurate and robust when the SNR
is high. In contrast, for a SNR < −4 (SD of the noise > 1.3), the correlation coefficients are
low and variable across the repeated estimations (Fig. 6.6, D), indicating that the estimation isneither accurate nor robust when the SNR is low.

6.4

Discussion

The simulation tests show that the proposed method can obtain the delays of the covert responselocked component even from noisy data, the SNR of which is lower than zero but higher than
−6 (for artificial data) or −3 (for EEG data) (Figs. 6.5 and 6.6, D). Here, the SNR lower thanzero means that the level of the noise is larger than that of the signal. Generally speaking, when
the noise is larger than the signal, it is quite difficult, or impossible, to estimate the variabledelays of the signal from single-trial data. The proposed method deals with the difficulty in aparadoxical way. To estimate the delays of “individual” trials, the method uses data epochs of“all” the trials in calculating the objective function. That is, I use all the trials for individual
trials. This strategy is an essential point in the feasibility of the proposed method for noisy data,such as EEG data.
The simulation tests also show that the estimation results become inaccurate when the SNR

70

B

Stim.
onset

3

Potential [ V]

Potential [ V]

A

0
-3
0

Time [ms]

3
0
-3

1000

0

1000
Time [ms]D
Correlation coefficientC
Estimated delay [ms]

Resp.
onset

350

150
350
150
Original delay [ms]

1

0

-10

-5
SNR

0

Figure 6.6: Simulation with EEG data. A–C: Examples of the extracted components and theestimated delays when the SNR is set to zero. A: Extracted stimulus-locked component (solidline) and original one (dotted line). B: Extracted covert response-locked component (solid line)
and original one (dotted line). The horizontal axis represents relative time to the defined onsetsof the covert response-locked component. C: Scatter plot of the estimated and original delays. D:
Correlation coefficients between the estimated and original delays for each SNR. The diamonds
and error bars respectively represent the means and SDs of the correlation coefficients.

71

is lower than −7 (for artificial data) or −4 (for EEG data) (Figs. 6.5 and 6.6, D). It is of notethat these results hold true only when the number of trials N is 100, and the accuracy of theestimation depends on the number of trials. When the number of trials is smaller, the waveformsof the components extracted by using true delays become less accurate. This indicates that theestimated delays obtained by searching for the delays which minimize the value of the objectivefunction become less accurate. In contrast, when the number of trials is larger, the waveformsof the components extracted by using true delays become more accurate. As a result, the true
delays decrease the value of the objective function more adequately, and the estimated delays
obtained by searching for the delays which minimize the value of the objective function become
more accurate. In fact, in my simulation tests (see Appendix D), as the number of trials increases,
the correlation coefficients between the estimated and original delays become greater. Therefore,
as long as the optimization problem can be solved, the proposed method can overcome a higher
noise level by increasing the number of trials.
In the optimization, I search for the set of the delays which minimizes the objective function.
I adopt the random search for the optimization, because it achieves speedy convergence to a
global minimum with a high probability. Generally speaking, a complete random search is a very
slow algorithm when the number of trials N is large (see Fig. 6.4, A, thin solid line). This isbecause, as N increases, the size of the search space increases exponentially and the probabilityof finding an optimal set of delays decreases exponentially. I overcome this problem by searchingfor the delays sequentially. That is, I reduce the dimension of the search space from N to 1 bysearching for the delays with respect to individual trials, and increase the probability to find abetter set of delays. This is the reason why the convergence of the restricted random search
I used is fast (Fig. 6.4, A, thick solid line), with a high probability of convergence to a global
minimum (Fig. 6.4, B).
Because the random search is a stochastic algorithm, the estimated delays and the extracted
components should vary more or less in each repeated estimation. The repeated simulation tests
show that, when the SNR is high, the accuracy of the estimation is high and almost the same72

across the repetitions (Figs. 6.5 and 6.6, D), indicating that the estimation is robust when it isaccurate. When the SNR is low, the accuracy of the estimation is low and variable across therepetitions (Figs. 6.5 and 6.6, D), indicating that the estimation is not robust when it is notaccurate. From these results, it is suggested that the robustness can be an index of the accuracyof the estimation.
On the other hand, the proposed method has a limitation. In calculating the objectivefunction, this method uses the method for overt responses proposed in Chapter 3. This method
has the technical limitation that slow waves (∼ 1 Hz) in noise are amplified by the decomposition.
Because of this limitation, the proposed method cannot easily deal with slow components, such as
the contingent negative variation [50]. To extract slow components by the proposed method, weneed to increase the number of trials at the expense of the calculation time for the optimization.

6.5

Conclusions

In this chapter, I propose a method for obtaining the stimulus-locked component, the covert
response-locked component, and its delays of individual trials. The simulation results indicate
the feasibility of the proposed method for artificial and EEG data. In the next chapter, I apply
the proposed method to the EEG during NoGo trials of a Go/NoGo task.

73

Chapter 7

Application to EEG during NoGo
Trials of a Go/NoGo Task
In this chapter, I apply the method proposed in the previous chapter to the EEG during NoGo
trials of a Go/NoGo task, in which subjects are instructed to withhold a response. Although thesubjects do not response overtly in NoGo trials, it is expected that the subjects response covertly
in their brains, and the proposed method can extract the covert response-locked components
together with the stimulus-locked components.

7.1

Methods

7.1.1

Experimental Procedure

Nine healthy adults (age 28.4 ± 3.7 years) constituted the experimental population. All thesubjects gave their informed consent and the local ethics committee approved the experimental
procedure.
The subjects were comfortably seated on a chair in a dimly lit, electrically shielded room.
At about 50 cm in front of the subjects’ eyes, red and green light-emitting diodes (LEDs) for
imperative signals were vertically arrayed 1.5 cm apart on a black panel. The subjects were
instructed to perform two kinds of tasks in the following order: a Go/NoGo task, and a passive
viewing task. In the Go/NoGo task, the subjects undertook four experimental blocks, eachconsisting of 50 trials. The subjects were instructed to push a button immediately after a “Go”
74signal (green LED) or not to push it after a “NoGo” signal (red LED). The green or red LED was
illuminated in random order with almost equal probability. In two blocks, the subjects had torespond with their right index finger, and in the other two blocks with their left index finger. In
an off-line analysis, the data from the blocks of right and left fingers were mixed. In the passive
viewing task, the subjects undertook two experimental blocks, each consisting of 50 trials. The
subjects were instructed passively to view the same stimulus as in the Go/NoGo task. In bothtasks, each trial began with a warning signal (a beep), followed, after a variable delay of 1.8–2.2 s,by the imperative signals (duration: 500 ms). Inter-trial intervals were randomized from 3.5 to
7.5 s.
During the tasks, surface EEG was recorded from 19-ch Tin electrodes, mounted in a cap(Electro-Cap International, Inc., Eaton, Ohio, USA) according to the International 10-20 system,
referred to a Tin electrode placed on AFz. The EEG was amplified on a Nihon Kohden EEG-1100
with a time constant of 0.3 s. Because I expected that large EEG activity related to the task
execution would not appear around the earlobes, I placed Ag/AgCl electrodes on both earlobes
and recorded their potentials separately. Their averaged potentials were subtracted from theEEG data off-line. For monitoring eye movements, an electrooculogram (EOG) was recordedwith a pair of Ag/AgCl electrodes placed above and below the left eye. The sampling rate of the
EEG and EOG was 1000 Hz.

7.1.2

Data Analyses

In an off-line analysis, I resampled the EEG data at a rate of 100 Hz. Because the proposed
method has the technical limitation that slow waves (∼ 1 Hz) in the background EEG activity
are amplified by the decomposition (see Chapter 6), the EEG data were filtered with a bandpass
of 2–40 Hz by using two kinds of finite impulse response (FIR) filters: a high-pass of 2 Hz (300point, −26 dB at 1 Hz) and a low-pass of 40 Hz (15-point, −45 dB at 50 Hz). Then, the filtered
EEG data were segmented into 2 s epochs from −500 to 1500 ms after stimulus onsets.
Reaction times of Go trials were defined as the intervals between the stimulus onset and the
button push signal onset. I excluded Go trials with the RT either shorter than 100 ms or longer
75

than 400 ms, and excluded NoGo trials with responses. An artifact criterion of ±50 µV was used
for the EEG and EOG to reject trials with excess ocular artifacts or measurement noise. The
numbers of trials obtained per subject were 77 ± 12 for Go trials, 87 ± 13 for NoGo trials, and78 ± 25 for the passive viewing task.
Application to EEG during Go Trials
Because the EEG during Go trials consists of the stimulus- and overt response-locked components,
it is expected that the EEG during Go trials satisfies the assumption of the proposed method forcovert responses and the proposed method can estimate the RTs of Go trials, i.e., the delays of theovert response-locked component. To test this, I applied the proposed method to the individual
subject’s EEG data at Cz during Go trials for 10 times, and obtained 10 sets of estimated RTs.
In estimating the RTs, I set the range of the RTs at the width of 300 ms. The accuracy of theestimated RTs was quantified by the correlation coefficient between the estimated and real RTs.
Check for Existence of Covert Response-locked Component in EEG during NoGo
TrialsAlthough the proposed method assumes that EEG activity consists of the stimulus- and covert
response-locked components, the validity of this assumption, especially the existence of the covert
response-locked component, is not always guaranteed at least for the EEG during NoGo trials of
the Go/NoGo task. As an alternative assumption, we might also consider that the EEG during
NoGo trials includes only the stimulus-locked component. Therefore, in order to check whether
the EEG included the covert response-locked component or not, I examined the time course of
the EEG variance across trials.
Suppose that the background noise is a stationary process, from the assumption of Eq. (5.1),
the EEG variance across trials Var[y(t)] is approximately expressed by:
Var[y(t)] ≈

N
N
1 !
1 !
[cr(t − τn ) −cr(t − τn )]2 + Var[v],
N n=1
N n=1

(7.1)

where Var[v] represents the variance of noise across trials. From Eq. (7.1), if the covert response-76

locked component does not exist, the EEG variance is expressed by:
Var[y(t)] ≈ Var[v].

(7.2)

Equations (7.1) and (7.2) indicate that the time courses of the EEG variance are qualitatively
different depending on whether the covert response-locked component exists or not. If the covert
response-locked component exists, the variance should increase and decrease after stimulus onset
because of the trial-to-trial variability in its delays. In contrast, if the covert response-lockedcomponent does not exist, the variance should be constant. We can confirm this fact by thesimulation results shown in Fig. 7.1. When the simulated data includes the covert responselocked component (Fig. 7.1, A), the variance across trials of the simulated data shows a transient
increase after stimulus onset (Fig. 7.1, C). In contrast, when the simulated data does not include
the covert response-locked component (Fig. 7.1, B), the variance across trials of the simulated
data is almost constant (Fig. 7.1, D). Therefore, I checked the existence of the covert responselocked components by examining the time course of the EEG variance across trials (Fig. 7.2).
Application to EEG during NoGo Trials
Because the variance of the EEG during NoGo trials at C3, C4, Fz and Cz showed a transient
increase (Fig. 7.2), I applied the proposed method to these EEG data. I obtained the stimuluslocked components, the covert response-locked components and the delays for each subject andeach channel. In estimating the delays, I set their range at the width of 400 ms, because the EEG
variance was significantly greater than the pre-stimulus level for about 400 ms after the stimulus
onset (p < 0.05, Wilcoxon signed-rank test).
To examine the robustness of the estimated delays, I repeated the estimation of the delays
10 times for the same EEG data, and obtained 10 sets of delays. The robustness was checked in
the following way. First, I calculated the correlation coefficient between each set and the averageof the other sets, and obtained the 10 correlation coefficients. Then, I conducted a two-tailed
Wilcoxon signed-rank test to assess the null hypothesis that the correlation coefficients did not
differ from zero. If the null hypothesis was not rejected at the alpha level of 0.05, I regarded the

77

A

B

1

1

0

0

-1

0

-1

1000

C

D

0.05

0.05

0

0

Time

0

1000

0

0

1000

Time

1000

Figure 7.1: Time course of variance of simulated data. A: Simulated data, consisting of thestimulus- and covert response-locked components. B: Simulated data, consisting of only thestimulus-locked components. In A and B, the black lines represent the simulated data of individual trials, the blue solid lines represent the means of the simulated data, and the blue dottedlines represent the mean ±SD of the simulated data. C: Variance across trials of the simulated
data in A. D: Variance across trials of the simulated data in B.

78

SNR of the EEG not to be high enough to estimate the delays robustly and excluded the EEG
data from the following analyses. If the null hypothesis was rejected, I regarded the estimation
results to be robust and, in the following analyses, used the set of delays whose correlation
coefficient was the highest. As a result, I obtained the delays of 6 sets for C3, 6 sets for C4, 7
sets for Fz, and 6 sets for Cz, from the 9 subjects’ EEG data during NoGo trials.
I adjusted the averages of the estimated delays so that the estimated delays represented
latencies of the same peaks in the extracted covert response-locked components across subjects.
First, I set a subject’s covert response-locked components as references, and shifted the othersubjects’ delays and the covert response-locked components so that their values of the crosscorrelations with the references became maximum at the lag of zero. Then, I shifted all thesubjects’ delays so that the delays represented latencies of the maximum peaks in the averagecovert response-locked components of all the subjects.
After the estimation, to examine the validity of the estimated delays, I formed time-trial
images [7] of the EEG during NoGo trials sorted by the estimated delays (Fig. 7.3). In theseimages, all the subjects’ EEG epochs were smoothed vertically with a 15-trial moving average,
and the potential fluctuations were shown as color-coded horizontal lines. If the estimated delays
are valid, both the stimulus- and estimated delay-locked fluctuations should appear, whereas, if
the estimated delays are wrong, only the stimulus-locked fluctuations should appear, as shown in
the left panel in Fig. 7.3. Therefore, I examined whether the estimated delay-locked fluctuations
appeared or not in the time-trial images of the EEG sorted by the estimated delays.
The similarity of the extracted components across subjects was quantified by calculating thecorrelation coefficient between a subject’s component and the other subjects’ average component
for each subject, corresponding to the cross-correlation between the two waveforms at a lag ofzero. As for the stimulus-locked components, the similarity of the extracted component to thestimulus-triggered average EEG was also quantified by the correlation coefficient between them.
A two-tailed Wilcoxon signed-rank test was conducted to assess the null hypothesis that thecorrelation coefficients obtained from individual subjects did not differ from zero. An alpha level79

Table 7.1: Correlation coefficients (r) between the estimated and real RTs in Go trials.
Subj. 1

Subj. 2

Subj. 3

Subj. 4

Subj. 5r

−0.39 ± 0.061
Subj. 60.017 ± 0.092
Subj. 70.042 ± 0.15
Subj. 80.37 ± 0.16
Subj. 90.69 ± 0.043

r−0.097 ± 0.11

−0.22 ± −0.25

0.17 ± 0.23

−0.018 ± 0.11

Values are mean ± SD for 10 estimations.
of 0.05 was used for the statistical tests.

7.2

Results

7.2.1

Estimation of RTs in Go Trials

Table 7.1 shows the correlation coefficients between the real and estimated RTs from the individual subjects’ EEG at Cz during Go trials by the proposed method. For Subjs. 4 and 5,
the estimated RTs are highly correlated with the real RTs (Table 7.1). For the other subjects,
however, the estimated RTs are not correlated with the real RTs (Table 7.1).

7.2.2

Testing Assumption for EEG during NoGo Trials

Figure 7.2 shows the time courses of the EEG variance during NoGo trials of the Go/NoGo task
obtained from all the subjects’ EEG. The variance of the EEG at C3, C4, Fz and Cz during
NoGo trials shows transient increases during 300–500 ms after the stimulus onset (Fig. 7.2). In
contrast, the variance is almost constant for the EEG during the passive viewing task (Fig. 7.2).
This suggests that the EEG at these channels during NoGo trials includes the covert responselocked component, whereas the EEG during the passive viewing of the same stimulus does not.
Therefore, I apply the proposed method to the EEG at C3, C4, Fz and Cz during NoGo trials.

7.2.3

Decomposing EEG during NoGo Trials

Then, I obtain the stimulus-locked components, covert response-locked components and the delays
for individual EEG channels and individual subjects. The estimated delays, representing the

80

Fz

100
50

Variance [ V2 ]

0

C3

100

500

Cz

100
50

50
0

500

C4

100
50

0

500
Time [ms]0500

Figure 7.2: Time course of the EEG variance across trials at C3, C4, Fz and Cz obtained fromall the subjects’ EEG. The blue lines represent the EEG variance during NoGo trials; the redlines represent the EEG variance during the passive viewing task.
latencies of the positive peaks in the covert response-locked components, are 403 ± 76 ms for C3,
360 ± 66 ms for C4, 292 ± 78 ms for Fz, and 336 ± 69 ms for Cz, and these are much longer thanthe RTs of Go trials (269 ± 50 ms).
Figure 7.3 shows the time-trial images obtained from all the subjects’ EEG at Cz. When the
EEG epochs are randomly sorted, only the stimulus-locked fluctuations appear (Fig. 7.3, left).
However, when the EEG epochs are sorted by the estimated delays, not only the stimulus-locked
but also the estimated delay-locked fluctuations appear (Fig. 7.3, right), suggestive of the validityof the estimated delays.
Figure 7.4, A, shows the extracted stimulus-locked components for each EEG channel. The extracted components exhibit negative peaks at around 240 ms (N200) and positive peaks at around
370 ms (P300) after the stimulus onset for all the 4 channels. The correlation coefficients between
the extracted components of individual subjects and the averaged components of the other subjects are significantly larger than zero (r = 0.54 ± 0.26, p < 0.05 for C3; r = 0.42 ± 0.31, p < 0.05
for C4; r = 0.66 ± 0.15, p < 0.05 for Fz; r = 0.43 ± 0.31, p < 0.05 for Cz), indicating that the extracted components exhibit similar patterns across subjects. The correlation coefficients between

81

Stim.
onset

Stim.
onset

10

Trials 100

10
2

2

0

0
4

4

-10

-10
0

400
Time [ms]

0

800

400
Time [ms]

800

[ V]

Figure 7.3: Time-trial images of all the subjects’ EEG at Cz during NoGo trials. In the left
image, the EEG epochs are randomly sorted. In the right image, the EEG epochs are sorted by
the estimated delays. The solid lines represent the stimulus onsets, and the dotted line represents
the estimated delays.
the extracted components and the stimulus-triggered average EEG are significantly larger thanzero (r = 0.90 ± 0.049, p < 0.05 for C3; r = 0.87 ± 0.12, p < 0.05 for C4; r = 0.94 ± 0.050, p < 0.05
for Fz; r = 0.89±0.10, p < 0.05 for Cz), indicating that the extracted stimulus-locked components
exhibit similar patterns to the stimulus-triggered average EEG.
Conventionally, EEG activity related to a response inhibition is examined by using the
NoGo−Go subtracting waveforms of the stimulus-triggered average EEG (for example, [20, 51–53]). However, the stimulus-triggered average EEG is more or less contaminated by the temporal overlapping of the response-locked components (see Chapter 2). From the above-mentioned
similarity between the extracted stimulus-locked component and the stimulus-triggered average
EEG, it is expected that the difference in the stimulus-triggered average EEG between Go and
NoGo trials comes from that of the stimulus-locked components. To confirm this expectation, I
compare the NoGo−Go subtracting waveforms of the stimulus-triggered average EEG with thoseof the stimulus-locked components. The stimulus-locked components of Go trials are extracted
from the EEG and real RTs of Go trials by the method for overt responses. The NoGo−Go
subtracting waveforms of the stimulus-triggered average EEG appear to be almost the same as
those of the stimulus-locked components (Fig. 7.4, B), and the squared correlation coefficients
between them are high (r2 = 0.59 for C3; r2 = 0.70 for C4; r2 = 0.94 for Fz; r2 = 0.83 for Cz).

82

Therefore, it is suggested that, at around Fz and Cz, the differences of the stimulus-triggered
average EEG between Go and NoGo trials are mainly (about 83–94%) attributable to those of
the stimulus-locked components.

A

Fz
10
0
-10
0

400

Potential [ V]

C3

C4

10

10

10

0

0

0

-10

-10
0

B
Potential [ V]

800

Cz

400

800

-10
0

NoGo (Cz)

400

800

0

Go (Cz)

NoGo

5

5

5

0

0

0

-5

-5
0

400
Time [ms]

800

400

800

Go (Cz)-50400Time [ms]

800

0

400
Time [ms]800

Figure 7.4: Extracted stimulus-locked components at C3, C4, Fz and Cz. A: Stimulus-lockedcomponents extracted from the EEG during NoGo trials. The black lines represent the extracted
stimulus-locked components of individual subjects, and the blue lines represent the averaged
components across subjects. B: Differences of the extracted stimulus-locked components between
Go and NoGo trials at Cz. The left panel shows the stimulus-locked component (blue line) andthe stimulus-triggered average (red line) of NoGo trials. The middle panel shows the stimuluslocked component (blue line) and the stimulus-triggered average (red line) of Go trials. The rightpanel shows the NoGo−Go subtracting waveforms of the stimulus-locked component (blue line)
and that of the stimulus-triggered average EEG (red line).
Figure 7.5 shows the extracted covert response-locked components for each EEG channel.
The extracted covert response-locked components exhibit positive peaks, whose magnitudes arecomparable with those in the extracted stimulus-locked components. The correlation coefficients
between the extracted components of individual subjects and the averaged components of the83other subjects are significantly larger than zero (r = 0.67 ± 0.097, p < 0.05 for C3; r = 0.67 ±
0.24, p < 0.05 for C4; r = 0.56 ± 0.27, p < 0.05 for Fz; r = 0.67 ± 0.17, p < 0.05 for Cz), indicatingthat the extracted covert response-locked components exhibit similar patterns across subjects.

Fz
10
0
-10
-400

0

Cz

C3
Potential [ V]

400

C4

10

10

10

0

0

0

-10
-400

-10
0
400
Time [ms]

-400

-10
0
400
Time [ms]

-400

0
400
Time [ms]Figure 7.5: Extracted covert response-locked components from the EEG at C3, C4, Fz and Cz
during NoGo trials. The black lines represent the extracted covert response-locked components
of individual subjects, and the blue lines represent the averaged components across subjects. The
horizontal axes represent relative time to the time points of the maximum peaks in the averagecovert response-locked components.

7.3

Discussion

7.3.1

Estimated RTs from EEG during Go Trials

By applying the proposed method to the EEG during Go trials, the RTs of Go trials are successfully estimated for Subjs. 4 and 5, but are not for the other subjects (Table 7.1). The successesof the estimation support the feasibility of the method for real EEG data. On the other hand,the failures of the estimation are partially due to the low SNR of EEG during Go trials andto the fact that the stimulus-unlocked components are not necessarily time-locked to the motor
responses. As for the latter, Verleger et al. [54] examined the EEG during choice reaction time
tasks by the stimulus- and response-triggered averaging procedures, and reported that the P3b
is time-locked to neither stimulus nor motor response onsets. Considering that the Go/NoGo84

task is a choice reaction time task, it is possible that the EEG during Go trials also includes thecomponents time-locked to neither stimulus nor motor response onsets and the method extracts
such components instead of the overt response-locked components.

7.3.2

Extracted Components from EEG during NoGo Trials

The proposed method assumes that the EEG activity consists of a stimulus-locked component
and a covert response-locked component. However, the existence of a covert response-lockedcomponent is less evident than a stimulus-locked component, which appears with the use of thestimulus-triggered averaging procedure. Therefore, to check the validity of the assumption, I
examine whether the EEG variance across trials increases after the stimulus onset. Although
one might argue that the changes in the EEG variance are attributable to the variability of the
waveform of the stimulus-locked component rather than the covert response-locked component
and its variable delays, from the relatively constant EEG variance during the passive viewing
task, I believe that the variability of the waveform of the stimulus-locked component is not solarge as to generate a drastic change in the variance of the EEG during NoGo trials (Fig. 7.2).
Therefore, I apply the proposed method to the EEG during NoGo trials, and extract the stimulusand covert response-locked components.
The waveforms of the extracted stimulus-locked components are almost the same as those of
the stimulus-triggered average EEG. This indicates that the covert response-locked components
are almost cancelled out by the stimulus-triggered averaging procedure. The extracted stimuluslocked components exhibit the N200 and P300 as well as the stimulus-triggered average EEG. This
indicates that the N200 and P300, which are the classical peaks of the EEG during NoGo trials [20,51, 53, 55, 56], are stimulus-locked. We can visually confirm this fact by the stimulus-locked
fluctuations at around 240 and 370 ms in the time-trial images (Fig. 7.3). At around Fz and Cz,
the differences in the stimulus-locked components between Go and NoGo trials largely account for
those in the stimulus-triggered average EEG (Fig. 7.4, B). This result suggests that, in the frontocentral region, the conventional differences in the EEG between Go and NoGo trials are mainly
attributable to those in the stimulus-locked components rather than the contamination effects by85

the covert response-locked components. Further, the different waveforms of the stimulus-lockedcomponent between Go and NoGo trials indicate that the stimulus-related brain processes aredifferent between Go and NoGo trials.
As for the extracted covert response-locked components, its magnitudes are comparable withthose of the stimulus-locked components (Figs. 7.4 and 7.5). The presence of the extracted covert
response-locked components and the validity of the estimated delays are confirmed by the resultsthat the time-trial images show the fluctuations time-locked to the estimated delays and that the
waveforms of the extracted components are similar across subjects.
The average delays of the positive peaks in the covert response-locked components are almost
the same as the P300 latency in the stimulus-locked component (363 ± 26 ms), indicating thatthe positive peaks mainly overlap with the P300 in the stimulus-locked components (see Fig. 7.3,
right). From this result, I consider that the proposed method for the first time decomposes theconventional P300 of NoGo trials, which appears by the stimulus-triggered averaging procedure,
into the stimulus-locked P300 and the covert response-locked P300.

7.4

Conclusions

The proposed method successfully extracts a covert response-locked component and its delays
from the EEG during NoGo trials. In the brain, internal and subjective events related to cognitive
functions seem to occur, not with precisely constant delays, but with various delays from trial totrial after stimulus onset. Since the proposed method can extract such brain activity, the method
will provide a new tool to look into complex and implicit brain functions.

86

Chapter 8General Discussion
In this thesis, I propose two methods for decomposing EEG into the stimulus- and response-lockedcomponents: the method for overt responses and the method for covert responses.
The method for overt responses is used when responses are observable. This method decomposes EEG into the stimulus- and overt response-locked components using RTs by the discrete
Fourier transform (Chapter 3). I apply the method to the EEG during a simple reaction timetask, and show that the contamination by the temporal overlapping is large especially in the latepart of the stimulus-triggered average EEG when slow waves remain in the EEG data (Chapter
4).
The method for covert responses is used when responses are not observable. This method obtains the individual delays of the covert response-locked component together with the stimulusand covert response-locked components. In this method, I initially set random values for thedelays and extract uncontaminated stimulus- and covert response-locked components using thepreset delays by the method for overt responses. Then, I reconstruct the EEG by overlapping theextracted components with the preset delays, and calculate the residual errors between the reconstructed and original EEG. This procedure is repeated by updating the delays until the residual
errors become adequately small (Chapter 6). I apply the method to the EEG during NoGo trials
of a Go/NoGo task, and extract the covert response-locked components, the magnitudes of which
are comparable with those of the stimulus-locked components (Chapter 7).

87

In this chapter, I discuss the remaining problems and possible applicability of these methods,
and show the further development of the methods.

8.1

Method for Overt Responses

In Chapter 3, I propose the method for decomposing EEG into the stimulus- and overt responselocked components using RTs.
This method has the limitation that the slow waves (∼ 1 Hz) in the noise are amplified by
the decomposition. This is due to the fact that the absolute values of 1/D(n, ω) in Eqs. (3.9) and(3.10) tend to be high at low frequencies when τn is RTs. To overcome this limitation, I propose
two solutions: increase the total number of trials, and apply a highpass filter to the EEG before
the decomposition. At the same time, it is necessary to improve the method itself to eliminate
the limitation. Originally, the source of the limitation, 1/D(n, ω), arises from the simultaneous
equations (3.1) and (3.2). It is possible that another set of simultaneous equations may solve
the limitation. Therefore, we need to try various sets of simultaneous equations to improve themethod itself.
On the other hand, this method has the practical advantage of requiring only single-channel
EEG data. This advantage makes it possible to apply the method in clinical use, because thesingle-channel recording is more easily applied. To date, several EEG studies have attempted toinvestigate the relation between stimulus-/overt response-related brain activity and movement
disorders by the averaging procedure (for example, [57,58]). However, the average EEG does notreflect pure stimulus- or overt response-related brain activity due to the temporal smearing. By
using the proposed method, we are able to extract more detailed information about stimulusand overt response-related brain activities, and to identify which process, the stimulus- or theovert response-related process, is responsible for the disorders. The proposed method can be used
as an alternative to the averaging procedure to investigate stimulus- and overt response-related
brain activities during various cognitive tasks with stimulus-overt response paradigms.

88

8.2

Method for Covert Responses

In Chapter 6, I propose a method for estimating the individual delays of the covert responselocked component and extracting the stimulus- and covert response-locked components from only
single-channel EEG epochs.
The delays are estimated by solving the optimization problem. The limitation of this method
is the calculation cost for solving this optimization problem. Suppose the number of trials is
N and the delay is within M ms. We need to search for the optimal set of delays from M N
sets. When N and M are large, this is quite a difficult problem and it takes a lot of time tosolve it. There are two approaches to overcome this limitation: 1) to make the optimization
problem easy, and 2) to improve the optimization algorithm. For 1), I reduce N by decreasing
the number of trials and M by using a priori information about the range of the delays derived
from the time course of EEG variance, and make M N as small as possible. For 2), I test various
optimization algorithms, and adopt the best algorithm, the random search, which achieves speedy
convergence to the optimal delays. Although these attempts work well for the EEG during NoGo
trials of the Go/NoGo task, to apply this method to various noisy data, it is necessary to propose
more powerful solutions to the limitation. Especially, there is plenty of room for improving the
optimization algorithm, because the random search used is so simple and seems to be brute force.
On the other hand, the advantage of this method is, of course, its ability to extract the covert
response-locked component, which has hardly been extracted owing to its variable delays. Then,
what will be gained by applying this method to other EEG data (and possibly to other brain
signals)?
Tallon-Baudry et al. [31, 32] showed evidence for a role for induced gamma activity, whose
phase is not time-locked to stimulus onset, in the construction of a coherent representation of
objects and the rehearsal of the representation in memory. This suggests that some brain activity
related to perception and memory recall is not time-locked to stimulus onset. Also, the brain
activity related to solving problems, or the “Aha!” experience [59], is not time-locked to thepresentation of problems. Until now, the time points of solving problems have been estimated89

from the onset of the subject’s response claiming to have solved the problem (for example, [59]).
However, the relation between the onset of the solving of a problem and that of the subject’s
response is not clear. Considering the large variability of RTs even during “simple” reaction
time tasks, it is possible that the interval between the two onsets also has large variability. If
so, we cannot extract pure brain activity related to problem solving from the response onset.
By applying this method to EEG during these kinds of cognitive tasks, we can obtain moredetailed information, the pure EEG waveform and its onsets, on the brain activity involved insuch complex functions.
The advantages of this method are not only to extract the covert response-locked component
but also to extract the uncontaminated stimulus-locked component. Conventionally, the stimulustriggered average EEG has been thought to reflect the stimulus-locked EEG component in thehope that the covert response-locked component, even if it exists, would be cancelled out by thestimulus-triggered averaging procedure. However, whether this premise is well satisfied or notis unclear unless uncontaminated components are disclosed. This is because the effect of thecontamination is determined by the waveform of the covert response-locked component and its
delays, which were unknown before the application of this method. In Chapter 7, I reveal thatthe contamination is small in the fronto-central region by comparing the NoGo−Go subtracting
waveforms of the stimulus-triggered average EEG with those of the stimulus-locked components.
By applying the proposed method, we can obtain the uncontaminated stimulus-locked component
and examine the level of contamination by the covert response-locked component.

8.3

Further Development of Proposed Methods

The essential strategies on which the proposed two methods based are as follows.
1. Separate components by solving simultaneous equations in the Fourier domain.
2. Estimate delays by searching for the delays which give the optimal approximation of EEG.
The method for overt responses is based on the first strategy, and the method for covert responses
is based on the second strategy in addition to the first one. Based on these strategies, it is possible
90

easily to propose another method which is used for more complicated problems.
For example, I can propose a method for extracting the three components: the stimuluslocked component, the covert response-locked component and the overt response-locked component (Fig. 8.1). The three components may exist in daily complex stimulus-response situations.
For example, let us consider a situation in which you are presenting your research. Someone asks
you a question, and you have to answer it as soon as possible. In such a situation, it is possiblethat listening to the question evokes stimulus-locked activity, solving the question evokes covert
response-locked activity, and answering the question evokes overt response-locked activity in yourbrain.
The method for decomposing the three components is proposed as follows. The EEG including
the three components is expressed by:yn (t) = s(t) + cr(t − τn ) + or(t − rtn ) + vn (t) (t = 0, · · · , T − 1; n = 1, · · · , N ),(8.1)

where yn (t): observed EEG data in trial n; s(t): stimulus-locked component; cr(t): covert
response-locked component; or(t): overt response-locked component; τn : delay of cr(t) in trial n;
rtn : RT in trial n; vn (t): noise in trial n.
To separate s(t), cr(t) and or(t), according to strategy 1, I take the discrete Fourier transformof Eq. (8.1), generate two more equations, and solve the equations simultaneously for s(t), cr(t)and or(t) as follows. The discrete Fourier transform of Eq. (8.1) is expressed by:Yn (ω) = S(ω) + exp(−i2πωτn /T )CR(ω) + exp(−i2πωrtn /T )OR(ω) + Vn (ω)(ω = 0, · · · , T − 1), (8.2)
where Yn (ω) is the discrete Fourier transform of yn (t); S(ω) is the discrete Fourier transform ofs(t); CR(ω) is the discrete Fourier transform of cr(t); OR(ω) is the discrete Fourier transformof or(t); Vn (ω) is the discrete Fourier transform of vn (t). I make the following two equations by
averaging Eq. (8.2) across trials with fast RTs and across trials with slow RTs respectively.
¯ f (ω)CR(ω) + Eo
¯ f (ω)OR(ω) + V̄f (ω) ,Ȳf (ω) = S(ω) + Ec
91

(8.3)

Stim.
onset

Stim.-locked comp.
Covert resp.
onset

Covert resp.-locked comp.
Overt resp.
onset

Overt resp.-locked comp.

EEG
Temporaldecomposition
Stim.-locked comp.
Covert resp.-locked comp.
Overt resp.-locked comp.

Figure 8.1: Schematic representation of the EEG including the three components: the stimuluslocked component, the covert response-locked component and the overt response-locked component. I propose a method for temporally decomposing the EEG into the three components.

92¯ s (ω)CR(ω) + Eo
¯ s (ω)OR(ω) + V̄s (ω) ,Ȳs (ω) = S(ω) + Ec

(8.4)

where
Ȳf (ω) =1 !
Yn (ω);
2N n∈n

(8.5)

f

1 !
Ȳs (ω) =Yn (ω);
2N n∈n(8.6)

s!
¯ f (ω) = 1
Ecexp(−i2πωτn /T );2N n∈n(8.7)

f!
¯ s (ω) = 1
exp(−i2πωτn /T );Ec
2N n∈n(8.8)

s!
¯ f (ω) = 1
Eoexp(−i2πωrtn /T );2N n∈n(8.9)

f!
¯ s (ω) = 1
exp(−i2πωrtn /T );Eo
2N n∈n(8.10)sV̄f (ω) =1 !
Vn (ω);
2N n∈n(8.11)fV̄s (ω) =1 !
Vn (ω),
2N n∈n(8.12)sand nf is the trial with fast RTs, ns is the trial with slow RTs. Equations (8.2), (8.3) and (8.4)
are rewritten by:

where



 

Vn (ω)S(ω)
Yn (ω)
Ȳf (ω) = A CR(ω) + V̄f (ω) ,V̄s (ω)OR(ω)
Ȳs (ω)1
A = 11exp(−i2πωτn /T ) exp(−i2πωrtn /T )¯ f (ω)¯ f (ω).
EcEo¯¯ s (ω)Ecs (ω)Eo

By multiplying Eq (8.13) by A−1 , I obtain:



 

Vn (ω)S(ω)
Yn (ω)
A−1 Ȳf (ω) = CR(ω) + A−1 V̄f (ω) .
V̄s (ω)OR(ω)
Ȳs (ω)(8.13)(8.14)(8.15)

Then, by averaging across n and taking the inverse discrete Fourier transform (IDFT), I obtain:



 



N
N
s(t)
Yn (ω)Vn (ω)!
!
1
1
IDFT 
(8.16)
A−1 Ȳf (ω) = cr(t) + IDFT 
A−1 V̄f (ω) .
N n=1
N n=1
or(t)
Ȳs (ω)V̄s (ω)93

Equation (8.16) shows that, if both the τn and rtn are observable, we can extract s(t), cr(t) andor(t) by calculating the left-hand side of Eq. (8.16).
Then, according to strategy 2, I estimate the unknown delays of the covert response-lockedcomponent τn . Let me define sτ (t), crτ (t) and orτ (t) respectively as s(t), cr(t) and or(t) extractedusing a set of delays τ = [τ1 , τ2 , · · · , τN ] by calculating the left-hand side of Eq. (8.16). Under theassumption that sets of nearly true delays provide better approximations of the observed EEG
data than sets of wrong delays, I estimate the true τ by solving an optimization problem of:
Minimizet2 !
N!

{yn (t) − sτ (t) − crτ (t − τn ) − orτ (t − rtn )}2

t=t1 n=1

Subject toτ ⊂ N.

In this method, the stimulus-locked component, the covert response-locked component and theovert response-locked component are extracted together with the delays of the covert responselocked component.
The performance of this method is examined by a simulation with artificial data (Fig. 8.2).
In this simulation, the original stimulus-locked component, covert response-locked component
and overt response-locked component are generated by the exponential, rectangular and cosine
functions respectively (Fig. 8.2, A–C). The delays of the covert response-locked component and
RTs are generated by Gaussian random numbers (180 ± 50 ms). The simulated signal yn (t) is
generated from the three components according to Eq. (8.1) (Fig. 8.2, E). Then, I estimate thedelays of the covert response-locked components and extract the three components by the method.
The estimated delays are highly correlated with the original ones (r = 0.96, p < 0.05, slope =0.87) (Fig. 8.2, H). The correlation coefficients between the original and extracted components arehigh for all the three components (r = 0.99 for the stimulus-locked component; r = 0.95 for thecovert response-locked component; r = 0.98 for the overt response-locked component) (Fig. 8.2,
F, G, I). These results indicate that the estimated delays and the extracted components by thismethod are sufficiently accurate in this simulation, suggestive of the validity of this method.
As mentioned in the “Discussion” section of Chapter 7, it is expected that the EEG during
Go trials of the Go/NoGo task includes the three components: the stimulus-locked component,
94

A

Stim.
onset

E

F

Delay

G

B

H

RT

Delay

C

0

D

I

50

100

Trial

Overt resp.
onset

Figure 8.2: Simulation result with artificial data. A: Original stimulus-locked component. B:
Original covert response-locked component shifted by an unknown delay. C: Original overt
response-locked component shifted by a RT. D: Background noise. E: Simulated data generated
according to Eq. (8.1). F: Extracted stimulus-locked component. G: Extracted covert responselocked component. H: Estimated delays of the covert response-locked component. I: Extracted
overt response-locked component. The blue lines represent the extracted components and theestimated delays, and the black lines represent the original data.

95

the covert response-locked component, and the overt response-locked component. Therefore,
I apply this method to a subject’s EEG at Cz during Go trials of the Go/NoGo task. The
average of the estimated delays of the covert response-locked component is adjusted so that thedelays represent latencies of the largest positive peak in the extracted covert response-lockedcomponent. The estimated delays of the covert response-locked component are 311 ± 62 ms.
The validity of the estimated delays is visually confirmed by the fluctuations time-locked to thedelays in the time-trial image sorted by the delays (Fig. 8.3, A). The magnitude of the extracted
covert response-locked component is comparable with those of the extracted stimulus- and overt
response-locked components (Fig. 8.3, D). In the expectation that the covert response-lockedcomponent relates to decision making, its delays should correlate with the RTs. However, thecorrelation coefficient between the delays and RTs is not significantly high (r = 0.067, p > 0.05)(Fig. 8.3, B). Although I cannot identify the functional role of the extracted covert responselocked components at this time, these results indicate the feasibility of the method for EEG, andindicate the presence of the three components in the EEG during Go trials.
As shown above, based on the two strategies, it is possible to propose various decomposition
methods easily. It should be worthwhile developing a family of decomposition methods which areused for broad problems.

8.4

Conclusions

The original question of this thesis is how the human brain operates in daily stimulus-response
situations. As the tools to solve this question, in this thesis, I propose two data analysis methods.
One is used when responses are observable and the other is used when responses are not observable. These methods decompose EEG signals into the stimulus- and response-locked components.
By applying these methods, we can examine brain activity in detail, and, in the future, we will
be able to answer the original question, based on the decomposed components.

96

20

5

40

0

Estimated delay of covert
resp.-locked comp. [ms]

B

0

400

800

400

200

200

RT [ms]

10

400

C

Stim.
onset

0
-10

-5
[ V]
Potential [ V]60

Potential [ V]10

Potential [ V]

Sorted trials

A

10

0

D

400

800

Covert resp.
onset

0
-10
-400
10

E

0

400

Overt resp.
onset

0
-10
-400

0

Time [ms]400

Figure 8.3: Decomposition of a subject’s EEG at Cz during Go trials of Go/NoGo task into thestimulus-, covert response- and overt response-locked components. A: Time-trial image of the
EEG sorted by the estimated delays of the covert response-locked component. The solid line
represents the stimulus onsets, and the dotted line represents the estimated delays of the covert
response-locked component. B: Scatter plot of the RTs and the estimated delays. C: Extracted
stimulus-locked component. D: Extracted covert response-locked component. E: Extracted overt
response-locked component.

97

Appendix A

Averages of Extracted
Components Depend on c
Only the averages of extracted stimulus- and overt response-locked components vary dependingon the parameter c in Eq. (3.8).
To demonstrate this, I performed a simulation test. In this simulation, I generated simulated
data by the same procedure as the one in the simulation test with artificial data (Chapter 3),
except for setting vn (t) = 0. Then, I extracted the stimulus- and overt response-locked components using different c from the same simulated data, and obtained the residual errors between
the extracted and original stimulus-/overt response-locked components. The simulation results
show that only the average of the residual errors varies depending on c (Fig. A.1).

98

A

B

-12

10

c=10

-6

c=1

0

8

Residual error

-12

10

c=2 10-6-4

c=2 10

-8

c=10

-6

4

-6

c=1

0
0

Stim.
onset

Time

1000

0

Overt resp. Time
onset

1000

Figure A.1: Residual errors between the original and extracted components using different c. A:
Result for the extracted stimulus-locked components. B: Result for the extracted overt responselocked components.

99Appendix B

Parametric Method for Overt
Responses
Here, I show a parametric method for obtaining the stimulus- and overt response-locked components from EEG epochs and RTs. In this method, I firstly generate three sweeps z(t), us (t), ur (t)from observed data, then assume a parametric model relating the three sweeps, and finally identify the model.
I generate z(t) by concatenating EEG epochs, generate us (t) by:#
1 t = Stimulus onset
us (t) =,0 t "= Stimulus onset
and generate ur (t) by:

(B.1)

#
1 t = Overt response onsetur (t) =.
0 t=" Overt response onset(B.2)

The three sweeps generated from simulated data are shown in Fig. B.1, A.
Then, I assume that z(t) is expressed by:z(t) = a1 us (t − 1) + · · · + ans us (t − ns ) + b1 ur (t − 1) + · · · + bnr ur (t − nr ) + e(t)(t = 0, · · · , J), (B.3)
where e(t) represents noise. Equation (B.3) is considered as a Finite Impulse Response (FIR)
system [60,61] with double-input (us (t) and ur (t)) and single-output (z(t)) (Fig. B.1, A). Further,in Eq. (B.3), the stimulus- and overt response-locked components are respectively represented by100the impulse responses to the individual inputs. Therefore, we can obtain the stimulus- and overt
response-locked components by identifying this FIR system.
In the least square method, the parameters in this system are identified by calculatingθ̂ = [Ω T Ω]−1 Ωx,

(B.4)

where
θ = [a1 , · · · , ans , b1 , · · · , bnr ]T ,and


us (ns − 1) · · ·

..
..
Ω=
.
.
us (J − 1) · · ·us (0)..
.

us (J − ns )ur (nr − 1) · · ·
..
..
.
.
ur (J − 1) · · ·(B.5)
ur (0)
..
.




,
ur (J − nr )x = [z(ns + nr ), z(ns + nr + 1), · · · , z(J)]T .

(B.6)

(B.7)

In practice, we can use system identification software, such as the System Identification Toolbox
for MATLAB (The MathWorks, Inc.) [61].
Figure B.1, B and C show the simulation result obtained by identifying the system shownin Fig. B.1, A. The identified and original stimulus-locked components are highly correlated(r = 0.97) (Fig. B.1, B), as are the identified and original overt response-locked components(r = 0.98) (Fig. B.1, C). This result indicates that we can also obtain the stimulus- and overt
response-locked components by the parametric method.

101

A
e (t)
us (t)ur (t)B

FIR
system

C

Stim.
onset

1

z (t)

Overt resp.
onset

0.5
0
-0.5

0
0

Time

1000

0

Time

1000

Figure B.1: Parametric method for identifying stimulus- and overt response-locked components.
A: Schematic representation of the FIR system expressed by Eq. (B.3). B and C: Stimulus- andovert response-locked components obtained by identifying the FIR system in A. B: Identified(solid line) and original (dotted line) stimulus-locked components. C: Identified (solid line) andoriginal (dotted line) overt response-locked components.

102Appendix C

Scalp Distributions of Potentials
Earlier than P300 and Later thanP160
The scalp distributions of the potentials earlier than the P300 and later than the P160 were notsubstantially different depending on the methods used.
As examples, in Fig. C.1, I show the scalp distributions of the N100 in the extracted stimuluslocked components and the stimulus-triggered average EEG during the SR-task, and also show
the scalp distributions of the N500 in the extracted overt response-locked components and theovert response-triggered average EEG during the SR-task.
Stim.-lockedcomponent

Average on
stim. onset

Overt resp.-lockedcomponent

Average on
overt resp. onset

N100

6

N500

-6

V

Figure C.1: Scalp distributions of the N100 and N500.
103

Appendix D

Accuracy of Estimation Depends
on Number of Trials
The accuracy of the estimation depends on the number of trials.
To demonstrate this, I performed a simulation test. In this simulation, I generated simulated
data by the same procedure as the one in the simulation test with artificial data (Chapter 6). I
set the number of trials N to 10, 20, 30, 40, and the SD of noise vn (t) to 0.5. Then, I estimatedthe delays of the covert response-locked component from the simulated data, and obtain thecorrelation coefficients between the estimated and original delays. I repeated this simulation 20
times. The simulation results show that the correlation coefficients become greater as the number

Correlation coefficientof trials increases (Fig. D.1).
1
0.5
0
-0.5
10

20

30

40

50

Number of trials

Figure D.1: Correlation coefficients between the original and estimated delays for each number oftrials. The diamonds and error bars respectively represent the means and SDs of the correlation
coefficients.

104

Bibliography[1] Lamarre Y, Busby L, and Spidalieri G. Fast ballistic arm movements triggered by visual,auditory, and somesthetic stimuli in the monkey. i. activity of precentral cortical neurons.
J. Neurophysiol., Vol. 50, pp. 1343–1358, 1983.
[2] Nelson RJ. Activity of monkey primary somatosensory cortical neurons changes prior toactive movement. Brain Res., Vol. 406, pp. 402–407, 1987.
[3] Nelson RJ, Smith BN, and Douglas VD. Relationships between sensory responsiveness andpremovement activity of quickly adapting neurons in areas 3b and 1 of monkey primarysomatosensory cortex. Exp. Brain Res., Vol. 84, pp. 75–90, 1991.
[4] Perfiliev SN. Responses in the motor cortex time-locked to the sensory stimuli conditioning
target-reaching in the cat. Neurosci. Res., Vol. 32, pp. 273–279, 1998.
[5] Tanji J and Kurata K. Comparison of movement-related activity in two cortical motor areasof primates. J. Neurophysiol., Vol. 48, pp. 633–653, 1982.
[6] Endo H, Kizuka T, Masuda T, and Takeda T. Automatic activation in the human primary
motor cortex synchronized with movement preparation. Cogn. Brain Res., Vol. 3, pp. 229–
239, 1999.
[7] Jung TP, Makeig S, Westerfield M, Townsend J, Courchesne E, and Sejnowski TJ. Analysis
and visualization of single-trial event-related potentials. Hum. Brain Mapp., Vol. 14, pp.
166–185, 2001.

105

[8] Goodin DS, Aminoff MJ, and Mantle MM. Subclasses of event-related potentials: responselocked and stimulus-locked components. Ann. Neurol., Vol. 20, pp. 603–609, 1986.
[9] CMJ Braun, Villeneuve L, and Gruzelier JH. Topographical analysis of stimulus-related andresponse-related electrical scalp activity and interhemispheric dynamics in normal humans.
Int. J. Psychophysiol., Vol. 46, pp. 109–122, 2002.
[10] Jutten C and Herault J. Blind separation of sources, part i: An adaptive algorithm basedon neuromimetic architecture. Signal Process., Vol. 24, pp. 1–10, 1991.
[11] Kayser J and Tenke CE. Principal components analysis of laplacian waveforms as a genericmethod for identifying erp generator patterns: I. evaluation with auditory oddball tasks.
Clin. Neurophysiol., Vol. 117, pp. 348–368, 2006.
[12] Makeig S, Delorme A, Westerfield M, Jung TP, Townsend J, Courchesne E, and Sejinowski
TJ. Electroencephalographic brain dynamics following manually responded visual targets.
PLos Biol., Vol. 2, pp. 747–762, 2004.
[13] 村田昇. 入門 独立成分分析. 東京電機大学出版局, 2004.
[14] 甘利俊一, 村田昇. 独立成分分析 多変量データ解析の新しい手法. サイエンス社, 2002.
[15] Cerutti S, Baselli G, Liberati D, and Pavesi G. Single sweep analysis of visual evoked
potentials through a model of parametric identification. Biol. Cybern., Vol. 56, pp. 111–120,
1987.
[16] Jensen EW, Lindholm P, and Henneberg SW. Autoregressive modeling with exogenous
input of middle-latency auditory-evoked potentials to measure rapid changes in depth ofanesthesia. Meth. Inform. Med., Vol. 35, pp. 256–260, 1996.
[17] Litvan H, Jensen EW, Galan J, Lund J, Rodriguez BE, Henneberg SW, Caminal P, Villar,
and Landeira JM. Comparison of conventional averaged and rapid averaged, autoregressivebased extracted auditory evoked potentials for monitoring the hypnotic level during propofol
induction. Anesthesiology, Vol. 97, pp. 351–358, 2002.
106

[18] Woldorff MG. Distortion of erp averages due to overlap from temporally adjacent erps:
analysis and correction. Psychophysiology, Vol. 30, pp. 98–119, 1993.
[19] 柳澤信夫, 柴崎浩. 神経生理を学ぶ人のために. 医学書院, 2002.
[20] Falkenstein M, Hoormann J, and Hohnsbein J. Erp components in go/nogo tasks and theirrelation to inhibition. Acta Psychologica, Vol. 101, pp. 267–291, 1999.
[21] Hohnsbein J, Falkenstein M, Hoormann J, and Blanke L. Effects of crossmodal divided
attention on late erp components. i. simple and choice reaction tasks. Electroencephalogr.
Clin. Neurophysiol, Vol. 78, pp. 438–446, 1991.
[22] Cummings A, Čeponienė R, Koyama A, Saygin AP, Townsend J, and Dick F. Auditory
semantic networks for words and natural sounds. Brain Res., Vol. 1115, pp. 92–107, 2006.
[23] Verleger R and Berg P. The waltzing oddball. Psychophysiology, Vol. 28, pp. 468–477, 1991.
[24] Jankelowitz SK and Colebatch JG. Movement-related potentials associated with self-paced,cued and imagined arm movements. Exp. Brain Res., Vol. 147, pp. 98–107, 2002.
[25] Neshige R, Lüders H, and Shibasaki H. Recording of movement-related potentials from scalp
and cortex in man. Brain, Vol. 111, pp. 719–736, 1988.
[26] Papa SM, Artieda J, and Obeso JA. Cortical activity preceding self-initiated and externallytriggered voluntary movement. Mov. Disord., Vol. 6, pp. 217–224, 1991.
[27] Shibasaki H, Barrett G, Halliday E, and Halliday AM. Components of the movement-related
cortical potential and their scalp topography. Electroencephalogr. Clin. Neurophysiol., Vol.
111, pp. 719–736, 1980.
[28] David O, Kilner JM, and Friston KJ. Mechanisms of evoked and induced responses in
meg/eeg. NeuroImage, Vol. 31, pp. 1580–1591, 2006.

107

[29] Haenschel C, Baldeweg T, Croft RJ, Whittington M, and Gruzelier J. Gamma and beta
frequency oscillations in response to novel auditory stimuli: A comparison of human electroencephalogram (eeg) data with in vitro models. Proc. Natl. Acad. Sci. USA, Vol. 97, pp.
7645–7650, 2000.
[30] Schürmann M and Başar E. Functional aspects of alpha oscillations in the eeg. Int. J.
Psychophysiol., Vol. 39, pp. 151–158, 2001.
[31] Tallon-Baudry C, Bertrand O, Delpuech C, and Pernier J. Stimulus specificity of phaselocked and non-phase-locked 40 hz visual responses in human. J. Neurosci., Vol. 16, pp.
4240–4249, 1996.
[32] Tallon-Baudry C and Bertrand O. Oscillatory gamma activity in humans and its role inobject representation. Trends Cogn. Sci., Vol. 3, pp. 151–162, 1999.
[33] Kirmizi-Alsan E, Bayraktaroglu Z, Gurvit H, Keskin YH, Emre M, and Demiralp T. Comparative analysis of event-related potentials during go/nogo and cpt: Decomposition of electrophysiological markers of response inhibition and sustained attention. Brain Res., Vol.
1104, pp. 114–128, 2006.
[34] 電気学会ウェーブレット解析の産業応用に関する協同研究委員会（編）. ウェーブレット解析の産業応用. 朝倉書店, 2005.

[35] Biggins CA, MacKay S, Clark W, and Fein G. Event-related potential evidence for frontal
cortex effects of chronic cocaine dependence. Biol. Psychiatry., Vol. 42, pp. 472–485, 1997.
[36] Jaśkowski P and Verleger R. Amplitudes and latencies of single-trial erp’s estimated by amaximum-likelihood method. IEEE Trans. Biomed. Eng., Vol. 46, pp. 987–993, 1999.
[37] Jaśkowski P and Verleger R. An evaluation of methods for single-trial estimation of p3
latency. Psychophysiol., Vol. 37, pp. 153–162, 2000.
[38] McGillem CD, Aunon JI, and Pomalaza CA. Improved waveform estimation procedures forevent-related potentials. IEEE Trans. Biomed. Eng., Vol. 32, pp. 371–379, 1985.
108

[39] Möcks J, Köhler W, Gasser T, and Pham DT. Novel approaches to the problem of latency
jitter. Psychophysiol., Vol. 25, pp. 217–226, 1988.
[40] Pham DT, Möcks J, Köhler W, and Gasser T. Variable latencies of noisy signals: estimationand testing in brain potential data. Biometrika, Vol. 74, pp. 525–533, 1987.
[41] Puce A, Berkovic SF, Cadusch PJ, and Bladin PF. P3 latency jitter assessed using 2techniques. i. simulated data and surface recordings in normal subjects. Neurophysiol.,Vol. 92, pp. 352–364, 1994.
[42] Puce A, Berkovic SF, Cadusch PJ, and Bladin PF. P3 latency jitter assessed using 2techniques. ii. surface and sphenoidal recordings in subjects with focal epilepsy. Electroencephalogr. Clin. Neurophysiol., Vol. 92, pp. 555–567, 1994.
[43] Woody CD. Characterization of an adaptive filter for the analysis of variable latency neuroelectric signals. Med. Biol. Eng., Vol. 5, pp. 539–553, 1967.
[44] 長尾智晴. 最適化アルゴリズム. 昭晃堂, 2000.
[45] Kirkpatrick S, Gelatt Jr CD, and Vecchi MP. Optimization by simulated annealing. Science,Vol. 220, pp. 671–680, 1983.
[46] 茨木俊秀. 離散最適化法とアルゴリズム. 岩波書店, 1993.
[47] Zhigljavsky AA. Theory of global random search. Kluwer Academic Publishers, 1991.
[48] Holland JH. Adaptation in natural and artificial systems. The Univ. Michigan Press, 1975.
[49] 合原一幸（編）. カオス時系列解析の基礎と応用. 産業図書, 2000.
[50] Walter WG, Cooper R, Aldridge VJ, Mccallum WC, and Winter AL. Contingent negative
variation: an electric sign of sensori-motor association and expectancy in the human brain.
Nature, Vol. 203, pp. 380–384, 1964.
[51] Bokura H, Yamaguch S, and Kobayashi S. Electrophysiological correlates for response inhibition in a go/nogo task. Clin. Neurophysiol., Vol. 112, pp. 2224–2232, 2001.
109

[52] Kok A. Overlap between p300 and movement-related-potentials: a response to verleger.
Biol. Psychol., Vol. 27, pp. 51–58, 1988.
[53] Yamanaka K, Kimura T, Miyazaki M, Kawashima N, Nozaki D, Nakazawa K, Yano H, andYamamoto Y. Human cortical activities during go/nogo tasks with opposite motor control
paradigms. Exp Brain Res, Vol. 142, pp. 301–307, 2002.
[54] Verleger R, Jaśkowski P, and Wascher E. Evidence for an integrative role of p3b in linking
reaction to perception. J. Psychophysiol., Vol. 19, pp. 165–181, 2005.
[55] Verleger R. The true p3 is hard to see: some comments on kok’s (1986) paper on degradedstimuli. Biol. Psychol., Vol. 27, pp. 45–50, 1988.
[56] Verleger R, Paehge T, Kolev V, Yordanova J, and Jaśkowski P. On the relation of movementrelated potentials to the go/no-go effect on p3. Biol. Psychol., Vol. 73, pp. 298–313, 2006.
[57] Cunnington R, Iansek R, and Bradshaw JL. Relationships between movement initiation
times and movement-related cortical potentials in parkinson s disease. Hum Mov Sci,
Vol. 18, pp. 443–459, 2006.
[58] Golob EJ and Starr A. Effects of stimulus sequence on event-related potentials and reaction
time during target detection in alzheimer s disease. Clin Neurophysiol, Vol. 111, pp. 1438–
1449, 2000.
[59] Jung-Beeman M, Bowden EM, Haberman J, Frymiare JL, Arambel-Liu S, Greenblatt R,
Reber PJ, and Kounios J. Neural activity when people solve verbal problems with insight.
Plos Biol., Vol. 2, pp. 500–510, 2004.
[60] 相良節夫, 中溝高好, 秋月影雄, 片山徹. システム同定. 計測自動制御学会, 1987.
[61] 足立修一. MATLAB による制御のためのシステム同定. 東京電機大学出版局, 1996.

110