Electrical and magnetic brain waves of seven subjects under three experimental conditions were recorded for the purpose of recognizing which one of seven words was processed. The analysis consisted of averaging over trials to create prototypes and test samples, to both of which Fourier transforms were applied, followed by filtering and an inverse transformation to the time domain. The filters used were optimal predictive filters, selected for each subject and condition. Recognition rates, based on a least-squares criterion, varied widely, but all but one of 24 were significantly different from chance. The two best were above 90%. These results show that brain waves carry substantial information about the word being processed under experimental conditions of conscious awareness. In the last two decades new methods of imaging brain activity [positron emission tomography, functional magnetic resonance imaging, and magnetoencephalography (MEG)] have augmented older methods such as electroencephalography (EEG) to dramatically increase our knowledge, especially about where in the brain different kinds of activity occur (1-5). Increased knowledge of temporal sequence of location of activity, as in orally naming a visual object, also has been substantial (6). On the other hand, aside from some success in simple mental state classification (7, 8) and EEG-based human computer communications (9, 10), the detailed analyses of how or what information is processed by the brain, at almost all levels, are still mostly lacking (11). Early attempts to classify averaged EEG waveforms associated with speech production date back to 1967 (12). However, it was later discovered (13) that scalp-recorded potentials preceding and accompanying speech primarily represent volume-conducted activity from musculature involved in speech production. In the current study, we were careful to rule out contributions from muscle movement in the auditory comprehension and internal speech conditions. The research reported here is meant to be a definitely positive, even if limited, step toward giving such an analysis of brain-wave activity as imaged by EEG and MEG. The brief description of our approach is simple. We analyze brain waves to recognize the word being processed. The general methodological approach is similar to speech recognition, but almost all the details are different. In terms of performance, we are at a level comparable to that of speech recognition in its early days (14, 15). For subjects S1-S5, EEG and MEG recordings were performed simultaneously in a magnetically shielded room in the Magnetic Source Imaging Laboratory (Biomagnetic Technology, San Diego) housed in Scripps Institute of Research. Sixteen EEG sensors were used. Specifically, the sensors, referenced to the average of the left and the right mastoids, were attached to the scalp of a subject (F7, T3, T5, FP1, F3, C3, P3, Fz, Cz, FP2, F4, C4, P4, F8, T4, and T6); sensors FP1, Fz, and FP2 were not used with S1 and S2. Two electrooculogram sensors, referenced to each other, were used to monitor eye movement. The Magnes 2500 WH Magnetic Source Imaging System (Biomagnetic Technology) with 148 superconductive quantum interference device sensors was used to record magnetic field near the scalp. The sensor array is arranged like a helmet that covers the entire scalp of most of the subjects. The recording bandwidth was from 0.1 Hz to 200 Hz with a sampling rate of 678 Hz. For the first two subjects, 0.5-s prestimulus baseline was recorded, followed by 2.0-s recording after the onset of the stimulus. For the other three subjects, baseline recording was 0.3 s, followed by 1.2-s recording after the onset of the stimulus. An Amiga 2000 computer was used to present the auditory stimuli (digitized speech at 22 kHz) to the subject via airplane earphones with long plastic tube leads. The duration of the stimulus was about 300 ms. Stimulus onset asynchrony varied from 2.5 to 2.7 s for the first two subjects, and from 1.5 to 1.7 s for the other three subjects. To reduce the alpha wave in this condition, a scenery picture was placed in front of the subject. He was asked to look at the picture during the recording. The visual stimuli were generated on the Amiga computer outside of the magnetically shielded room and projected to the front visual field of the subject by using an optical mirror system. Each word was presented for 200 ms, preceded and followed by dynamic random noise masks. Stimulus onset asynchrony was the same as in the auditory comprehension condition. For subjects S6 and S7, data were collected by using a 64-channel NeuroScan EEG system at the Palo Alto (CA) Department of Veterans Affairs Health Care System. Subjects wore a 64-channel Tin electrode cap that covered the entire scalp during the experiment. The electrodes were connected to two 32-channel SynAmps amplifiers with the linked ears as the reference electrode. The recording bandwidth was set from DC to 200 Hz. with a sampling rate of 500 Hz. The stimulus conditions were similar to those for S1 and S2. Subjects S1-S5, five normal male native English speakers, aged 25 to 40 years, four right-handed and one left-handed, were run under two or three different conditions, with simultaneous 16-sensor EEG and 148-sensor MEG recordings of brain activity in each condition. The observations recorded were of electric (EEG) or magnetic (MEG) field amplitude every 1.47 ms for each sensor. Two additional male subjects, S6 and S7, the first a 30-year-old native speaker of Chinese but with excellent command of English and the second a 75-yearold native speaker of English, were run in one condition with 64-sensor EEG recordings every 2.0 ms.
In REF , it has been shown that isolated words can be recognized based on EEG and MEG recordings.
Brain wave recognition of words
Proceedings of the National Academy of Sciences of the United States of America
Proceedings of the National Academy of Sciences of the United States of America
Computer Science Medicine