# Research on High-Frequency Combination Coding-Based SSVEP-BCIs and Its Signal Processing Algorithms

Authors: Feng Zhang; Chengcheng Han; Lili Li; Xin Zhang; Jun Xie; Yeping Li
Journal: Shock and Vibration
 (2015)

---

## Abstract

This study presents a new steady-state visual evoked potential (SSVEP) paradigm for brain computer interface (BCI) systems. The new paradigm is High-Frequency Combination Coding-Based SSVEP (HFCC-SSVEP). The goal of this study is to increase the number of targets using fewer stimulation frequencies, with diminishing subject’s fatigue and reducing the risk of photosensitive epileptic seizures. This paper investigated the HFCC-SSVEP high-frequency response (beyond 25 Hz) for 3 frequencies (25 Hz, 33.33 Hz, and 40 Hz). HFCC-SSVEP producesn
n with n high stimulation frequencies through Time Series Combination Code. Furthermore, The Improved Hilbert-Huang Transform (IHHT) is adopted to extract time-frequency feature of the proposed SSVEP response. Lastly, the differentiation combination (DC) method is proposed to select the combination coding sequence in order to increase the recognition rate; as a result, IHHT algorithm and DC method for the proposed SSVEP paradigm in this study increase recognition efficiency so as to improve ITR and increase the stability of the BCI system. Furthermore, SSVEPs evoked by high-frequency stimuli (beyond 25 Hz) minimally diminish subject’s fatigue and prevent safety hazards linked to photo-induced epileptic seizures. This study tests five subjects in order to verify the feasibility of the proposed method.


---

## Body

## 1. Introduction

A brain computer interface (BCI) is a direct communication pathway between a human or animal brain and an external device. Nowadays, noninvasive scalp electroencephalogram (EEG) measurements have become a popular solution in BCI research. The most commonly used signals in EEG-based BCI systems are event-related synchronization of mu and beta bands, event-related potentials, and steady-state visual evoked potential (SSVEP) [1–3].The SSVEP is usually elicited by flickering stimulation frequency higher than 6 Hz, while the frequency for inducing FVEP (Flash Visual Evoked Potential) should be lower than 2 Hz [4]. The most common measure to assess the performance of a BCI system is Shannon’s information transfer rate (ITR) [5]. Steady-state visual evoked potential (SSVEP) has been regarded as an efficient approach to design BCI with high information transfer rate (ITR). SSVEP-based systems lead to transfer rates of 100 bits/min and beyond [2, 6], compared to about 10–25 bits/min of other BCI systems [5]. Furthermore, SSVEP-based BCI systems have the advantages of short responding time, minimum training, and high ITR [4, 7, 8].The visual stimulator plays an important role in an SSVEP BCI. Several visual stimulators have been used for evoking SSVEP, such as a cathode ray tube (CRT) monitor [9], liquid crystal display (LCD) monitor, and light-emitting diode (LED) array [10]. Considering stimulation parameters such as size, color, and position, LED arrays are not flexible and increase the overall system cost; presenting flickers on a computer monitor is more flexible than using stand-alone lights/LEDs [9].Current SSVEP-based BCI system utilizes single frequency to encode each target. Hence, a large number of targets require a large number of frequencies. Increasing the number of targets then decreases the frequency resolution which in turn makes classification more difficult. This is especially problematic on computer screens. Recently, some researchers began to study how to increase the number of targets with fewer frequencies, such as stimuli flickered at the same frequency and differed only in relative phase [11], right-and-left field stimulation with two frequencies [12], and using dual frequency stimulation [4]. However, SSVEPs are usually evoked by low-mid frequencies in the 7–30 Hz, but the low-mid frequency band has some disadvantages [8]. First, the low frequency band covers the alpha band (8–13 Hz) which can cause a considerable amount of false positives. Second, subjective evaluations showed that frequencies between 5 and 25 Hz are more annoying than higher ones; visual fatigue would easily occur. Third, flash and pattern reversal stimuli can provoke epileptic seizures especially in the 15–25 Hz range [13]. All of these disadvantages can be avoided by using the high frequency band [14–16].This study proposes High-Frequency Combination Coding-Based flickers for evoking SSVEP (HFCC-SSVEP). The HFCC-SSVEP is essentially based on frequency modulation. Each flicker comprises different high flickering frequencies through High-Frequency Combination Code. The High-Frequency Combination Coding-Based SSVEP (beyond 25 Hz) is induced by CRT (Cathode Ray Tube) and LED. This system uses only one Oz-A1 EEG channel for SSVEP recording. The proposed new paradigm producesn
n with n high stimulation frequencies through Time Series Combination Code. Furthermore, conventional spectral methods are not always suitable for detecting high-frequency SSVEPs. In this paper, an improved HHT- (Hilbert-Huang Transform-) based high-frequency-modulated SSVEP feature extraction method is proposed to extract time-frequency feature of the proposed SSVEP response. Furthermore, SSVEPs evoked by high-frequency stimuli (beyond 25 Hz) minimally diminish subject’s fatigue and prevent safety hazards linked to photo-induced epileptic seizures.


## 2. Subject and Experimental Condition

Three healthy volunteers (three males randomly selected from the students in the research institution), aged from 21 to 25 years old, participated in this study from Xi’an Jiaotong University. They were seated in a comfortable armchair in a dimly illuminated EEG signals testing lab, which is quiet without any distractions. All participants were 50 cm away from the stimulation unit (CRT display OR LED). EEG signals were measured from three EEG electrodes (g.USBamp, g.tec Guger Technologies, Austria) placed at Oz-A1 (Oz-unilateral earlobe) and Fpz (ground) in compliance with the international EEG 10–20 system. The unilateral (left or right) earlobe was used as the recording reference, and all electrode impedances were kept below 5 kOhm. The experimental data sampling frequency is 1200 Hz.


## 3. Experiment Paradigm

Figure1 shows the block diagram of a HFCC-SSVEP for BCI. HFCC-SSVEPs were induced by 27 stimulus series; these stimuli were displayed by cycle flickers at the center of CRT. The 27 stimulus series were generated from 3 fundamental frequency elements through Time Series Combination Code. The used fundamental frequency elements were only three high frequencies, that is, 25, 33.33, and 40 Hz; the three frequencies were labeled as 1, 2, and 3. The selection of fundamental frequency elements depended on the screen refresh rate and sampling rate. Each fundamental frequency element was generated from 20 pulses. The data lengths of 27 stimulus series were variably between 1.5 s and 2.4 s. The subjects gazed each stimulus sequence and the stimulus sequence was displayed by cycle flickers at the center of CRT.Figure 1
Block diagram of a TSCCBH-SSVEP for BCI.


## 4. Data Processing and Result

Figure2 shows EEG signals of SSVEP response presented through LED with 25 Hz, 30 Hz, 40 Hz, and 75 Hz. The length of EEG data is 8 s. The frequency spectrum of EEG signals was determined using the fast Fourier transform (FFT) technique in MATLAB software. Figure 3 shows the FFT spectrum of SSVEP EEG signals of LED. The spectrum frequency detection is usually the appropriate for frequency analysis of SSVEPs in BCI applications. Figure 4 shows FFT’s amplitude of EEG signals for LED. Figure 3 shows that SSVEP response of 25 Hz, 30 Hz, and 40 Hz is sensitive, but the 75 Hz spectral line cannot be distinguished in the FFT spectrum from Figure 3. Figure 4 shows the fitted curve of FFT spectrum energy.Figure 2
SSVEP signals of LED.
(a)
(b)
(c)
(d)Figure 3
FFT of signals of LED.
(a)
(b)
(c)
(d)Figure 4
The fitted curve of FFT energy.In this study, each Oz-A1 EEG signal was segmented into ten segments. Figure10 shows that SCCBH-SSVEP EEG signals 6 of 27 selections. The length of EEG data is unequal between 1.5 s and 2.4 s. The frequency spectrum of averaged EEG signals was determined using the fast Fourier transform (FFT) technique in MATLAB software. The spectrum frequency detection is usually the appropriate for frequency analysis of SSVEPs in BCI applications; but the FFT spectrum cannot contain temporal information. The 27 selections represented by three high frequencies elements cannot be distinguished from each other in the FFT spectrum.Spectrum analysis is usually used to extract the frequency information in traditional evoked SSVEPs responses. The underlying idea is always the same: a blinking or moving visual stimulus at a constant frequency (the stimulus frequency) elicits a response in the brain at the same frequency and its even harmonics (SSVEP frequency is equal to stimulus frequency plus its even harmonics). Note that this behavior denotes a nonlinearity of the visual system [5].The evoked SSVEP response of High-Frequency Combination Coding-Based stimulus not only contains frequency information but also contains temporal information. The frequency information consists of three frequencies through Time Series Combination Code; the length oft evoked SSVEP response data varies between 1.5 s and 2.4 s (see Figure 2); as a result, the evoked SSVEP responses are nonstationary and nonlinear. The traditional spectrum detection method is not suited for analyzing the time series combination coding-based high-frequency evoked SSVEP response. More sophisticated nonstationary and nonlinear signal processing techniques have recently been used to analyze the evoked SSVEP response [17–19].Hilbert-Huang transforms [20], consisting of empirical mode decomposition (EMD) and Hilbert spectral analysis, is a newly developed adaptive data analysis method. The HHT is designed specifically for analyzing nonlinear and nonstationary data. The key part of HHT is EMD with which any complicated data set can be decomposed into a finite and often small number of intrinsic mode functions (IMFs). The instantaneous frequency defined using the Hilbert transform denotes the physical meaning of local phase change better for IMFs than for any other non-IMF time series. As the decomposition is based on the local characteristics of the data, it has been proved quite versatile in a broad range of applications for extracting signals from data generated in noisy nonlinear and nonstationary processes [21]. It has been widely used to analyze EEG signals [22–26]. In order to extract the time-frequency characteristics of high-frequency time series combination coding-based SSVEPs, we proposed IHHT-based high-frequency time-modulated SSVEP method.Figure9 shows the steps of HHT-based high-frequency time-modulated SSVEP feature extraction, a local spectrum extreme target identification algorithm and differentiation combination method. The extraction method consists of synchronous averaging, band pass filtering, EMD, selection of IMF, instantaneous frequency, and Hilbert spectrum. In order to ensure that the time-frequency characteristics of high-frequency time series combination coding-based SSVEPs are efficiently extracted, we must choose and optimize the key algorithm. The specific method is as follows. The end effect and stopping criterion are the core problems of empirical mode decomposition (EMD); two methods are selected and optimized in order to overcome the shortage of end effect and stopping criterion of empirical mode decomposition (EMD) in the processing of variable frequency EEG data.(i) Optimization Selection of the EMD Endpoint Prediction. Upper and lower envelope averaging is one of the cores of the EMD algorithm; the upper and lower envelopes are attained through the extreme value point of the spline curve fitting. The EMD end effect is how to find the proper fitting curve between the last extreme value point and endpoint in the curve fitting. The boundary prediction method and traditional method are compared in order to optimize the problem for the end effect in the high-frequency time-modulated SSVEP feature extraction.Figure5 shows the boundary prediction method. The boundary prediction method is to use the 1 order approximate point as the endpoint. Through the theoretical and real data analysis [27], it is not only to conform to the conditions of the cubic spline curve fitting but also to ensure that the fitting curve fluctuation is minimal. In Figure 6, E-Natural curve is the upper envelope curve fitting based on cubic spline curve through the boundary prediction method, Natural curve is the upper envelope curve fitting based on cubic spline curve through the traditional method. Figure 6 shows that the fluctuation of the fitting curve based on boundary prediction in the end is minimal. So the boundary predictions are used to overcome the shortage of end effect of empirical mode decomposition (EMD) in the processing of variable frequency EEG data.Figure 5
The boundary prediction method. ■—endpoint; ●—1 order approximate point: the intersection of the vertical axis and the curve, that is, straight line continuation of two extreme value points near the end; ▲—the last extreme value that is obtained through the boundary prediction method; the formula of extreme value is as follows through boundary prediction method:The Extreme Point
=
MAX
⁡
{
endpoints
,
1 order approximate point
}.
(a)
(b)
(c)
(d)Figure 6
Different methods comparison results about the end effect.(ii) The Parameter Optimization of EMD Stopping Criterion. The EMD algorithm is essentially a process of screening; the screening mathematical formula is as follows:(1)
x
t
=
∑
j
=
1
n
c
j
=
x
t
-
∑
j
=
1
k
m
j
+
∑
j
=
1
k
m
j
-
∑
j
=
1
p
m
2
j
+
…
.In formula, x
(
t
) is the target signal, c
j is the IMF that is decomposed through the EMD algorithm, n is the number of IMF, m
j, m
2
j is the average of the upper and lower envelope curve in the process of screening, and k, p is the number of screening.The core problem of EMD is how to choose the end conditions in every process of screening, which is the problem of the EMD stopping criterion.The EMD algorithm is essentially binary half-band filter, which is actually binary wavelet. In order to ensure the screening results of IMF in the amplitude and that frequency has enough physical meaning, which essentially is to ensure that scale ratio of the adjacent two IMF values is close to 2, the number of screening (k
,
p) must be limited in the process of screening.The fixed sifting (iterating) 10 times are used to overcome the shortage of stopping criterion of empirical mode decomposition (EMD) in the processing of variable frequency EEG data, That isk
=
p
=
10, which can ensure that scale ratio of the adjacent two IMF values is close to 2. The advantage of this selection is as follows [28]:(a)
Ensure that the EMD algorithm is essentially binary half-band filter bank.
(b)
Ensure that the result has physical meaning.
(c)
The decomposition results number is limited <=log
⁡
2
(
N
).(iii) The Optimization of Calculation Method for Instantaneous Frequency. In order to ensure that the instantaneous frequency based on Hilbert Transform (HT) has physical meaning, the conditions are as follows:(a)
The result must be IMF.
(b)
The signal amplitude change is not too large (due to the Bedrosian theoretical limitation).
(c)
The phase of signal cannot be too complicated (due to the Nuttall theoretical limitation).The GZC (generalized zero-crossing) algorithm is proposed to calculate the instantaneous frequency of variable frequency EEG data, as shown in Figure7 and formula (2).Figure 7
Calculate the instantaneous frequency based on GZC algorithm.The computational formula of instantaneous frequencyf
g
z
c is as follows:(2)
f
g
z
c
=
4
f
1
+
2
f
21
+
f
22
+
f
41
+
f
42
+
f
43
+
f
44
12
,
f
1
=
1
4
T
1
,
f
2
i
=
1
2
T
2
i
,
i
=
1,2
,
f
4
i
=
1
T
4
i
,
i
=
1,2
,
3,4
.In the formula, (2) T is the time frame between zero and peak time.The GZC algorithm is based on the average concept in nature, so the GZC algorithm has the advantage of high stability and high precision of calculation. Therefore the GZC algorithm is very suitable for the fact that the frequency fluctuation of signal is not big. The variable frequency characteristics of high-frequency time series combination coding-based SSVEPs in this paper is segment-wise stationary; as a result, the generalized zero-crossing (GZC) is used to compute the instantaneous frequency of the proposed SSVEP respondent signals.(iv) Local Spectrum Extreme Target Identification Algorithm. The relevant IMFs are evenly divided into 3 sections, then to respectively calculate their spectrum extreme X. Then according to the following rules, X is encoded.(a)
20
<
x
<
=
29, x is 1.
(b)
29
<
x
<
=
37, x is 2.
(c)
37
<
x
<
=
45, x is 3.Finally, compare the coding result to the element in the encoded library; if the coding result is in the library, it is marked as nonzero; otherwise it is marked as zero. Then of nonzero number is the correct identification number; as a result, the recognition rate is quantitatively calculated through the local spectrum extreme target identification algorithm. The recognition accuracy p of computational formula is as follows:(3)
p
=
n
N
.(v) Differentiation Combination Method. In the experiment, the used fundamental frequency elements were only three high frequencies, that is, 25, 33.33, and 40 Hz, which are labeled as 1, 2, and 3. The data lengths of 27 stimulus series were variably between 1.5 s and 2.4 s. The 27 stimulus series are averaged by superposition of 1~10 times. The recognition accuracy p of the different superposition time’s average results is calculated through the local spectrum extreme target identification algorithm. Figure 8 shows the statistical result between the different superposition time’s average results, n, and the recognition accuracy, p.Figure 8
Statistical result between the different superposition times average resultsn and the recognition accuracy p.Figure 9
IHHT-based high-frequency time-modulated SSVEP method.Figure 10
TSCCBH-SSVEP EEG signals.
(a)
Data 123
(b)
Data 132
(c)
Data 213
(d)
Data 231
(e)
Data 312
(f)
Data 321From statistics results in Figure8, we can see that the recognition accuracy p is only 77.78% through 10 times superposition average. This is obviously not what we expected. The statistical result between the different superposition time’s average results n and the recognition accuracy p in Figure 8 shows that the recognition accuracy p is different for all of the combinations of three stimulation units. Different combinations make a difference in recognition rate. Through the comparison and analysis, the recognition accuracy p is larger through differentiation combination method of three stimulation units. The reason is that differentiation combination method can break the adaptability of the vision system [29], compared with single frequency, so the different frequency encoding (differentiation combination method) can improve the recognition rate. As a result, on the basis of the fundamental frequency elements, that is, 25, 33.33, and 40 Hz, which are labeled as 1, 2, and 3, the differentiation combination encoding of 27 combination encoding mode is 123, 132, 213, 231, 312, and 321. Figure 10 shows SCCBH-SSVEP EEG single response signals of differentiation combination method. Figure 12 shows that the Hilbert spectra of the selections signals are presented by using HHT-based high-frequency time-modulated SSVEP method.Each Oz-A1 EEG signal is firstly segmented into ten segments for synchronous averaging, which are to decrease the noise of EEG data. Next, the data in the frequency range of 25–45 Hz are chosen for EMD analysis, owing to the frequency range of three selected fundamental frequency elements, 25 Hz, 33.33 Hz, and 40 Hz. According to EMD results, the relevant IMF is selected empirically for analysis. Then, the instantaneous frequency of the selected relevant IMF is calculated by using generalized zero-crossing algorithm [30]. At last, according to the calculated instantaneous frequency, the Hilbert spectrum is presented. A local spectrum extreme target identification algorithm is proposed to calculate recognition rate; the experimental result shows that the recognition rate is not high for all combination coding sequences. So the differentiation combination method is proposed to select the combination coding sequence in order to increase the recognition rate; as a result, 6 combination coding sequences are selected; they are 123, 132, 213, 231, 312, and 321 (fundamental frequency: 25, 33.33, and 40 Hz, number resp.: 1, 2, and 3).Figure11 shows that the spectra of selected EEG signals, the frequency spectra of the selected EEG signals, were determined using the fast Fourier transform (FFT) technique in MATLAB software. In the spectrum in Figure 11, the lime lines represent 25 Hz, the red lines represent 33.33 Hz, and the springs green lines represent 40 Hz. The spectrum frequency detection is usually the appropriate for frequency analysis of SSVEPs in BCI applications; but the FFT spectrum cannot contain temporal information. The 6 selections represented by three high frequencies elements cannot be distinguished from each other in the FFT spectrum. Figure 12 shows that the Hilbert spectra of the selected signals are presented by using HHT-based high-frequency time-modulated SSVEP method. Obviously, the Hilbert spectrum presents the time-frequency characteristics of flashing sequences based on high-frequency time series combination code using three high frequencies. Compared to FFT spectrum, it not only provides the frequency information but also presents frequency change with time, which extracts the features of Time Series Combination Code of three high frequencies. According to the combination code time-frequency characteristics, the Hilbert spectra (see Figure 12) obviously identify the selections.Figure 11
The FFT of TSCCBH-SSVEP.
(a)
Data 123
(b)
Data 132
(c)
Data 213
(d)
Data 231
(e)
Data 312
(f)
Data 321Figure 12
The Hilbert spectra of HFCC-SSVEP.
(a)
Hilbert spectrum 123
(b)
Hilbert spectrum 132
(c)
Hilbert spectrum 213
(d)
Hilbert spectrum 231
(e)
Hilbert spectrum 312
(f)
Hilbert spectrum 321Finally, six stimulus targets are presented with three high frequencies through HFCC-SSVEP; in contrast, three stimulus targets are presented with three low frequencies through traditional SSVEP; the above two kinds of different contrast experiments in Figure13 are applied to intelligent wheelchair navigation control in order to verify the technical advantage of the proposed method and ensure the HFCC-SSVEP-based intelligent wheelchair navigation system efficiency and undamaging. Figure 13(a) shows the experimental paradigm of HFCC-SSVEP with 6 targets: 123, 132, 213, 231, 312, and 321 (fundamental frequency: 25, 33.33, and 40 Hz, number resp.: 1, 2, and 3); they are presented on the screen. Figure 13(b) shows the experimental paradigm of the traditional SSVEP; the frequencies of 3 targets are, respectively, 12, 12.5, and 15 Hz. Table 1 is the comparison of HFCC-SSVEP and traditional SSVEP; it is average experimental results of five subjects. In Table 1, t is stimulation time, p is the recognition rate, df is the degree of fatigue (fatigue time), and ITR is the information transmission rate of BCI system. Table 1 shows that HFCC-SSVEP has the advantage of shorter time, higher accuracy, not easy fatigue, and higher ITR, compared with the traditional SSVEP (Figure 14).Table 1
Comparison of HFCC-SSVEP and traditional SSVEP.
Paradigm
t/s
p
df/h
ITR (b/min)
CCH-SSVEP
1.9
92%
10
43
Traditional SSVEP
5.8
70%
2
13Figure 13
Experimental comparison of HFCC-SSVEP and traditional SSVEP.
(a)
HFCC-SSVEP
(b)
Traditional SSVEPFigure 14
Advantage of HFCC-SSVEP.


## 5. Discussion and Conclusion

We studied SSVEP high frequency (beyond 25 Hz) response of SSVEP whose paradigm is on the LED. Figure3 shows that the 75 Hz spectral line cannot be distinguished in the FFT spectrum. The SNR (signal to noise ratio) of high frequency (beyond 40 Hz) response is very low, which has been unable to be distinguished through the traditional analysis method. So we must study the weak signal feature extraction method for SSVEP high frequency (beyond 40) response. This part of the research will be conducted in the future research work. As a result, 25 Hz, 33.33 Hz, and 40 Hz in the high frequency area are selected as the fundamental frequency for HFCC-SSVEP paradigm.The proposed HFCC-SSVEP paradigm for a BCI system not only can increase the number of targets through Time Series Combination Code of fewer frequencies for stimulation but also can shorten the recognition time. Furthermore, it can diminish user’s fatigue and risk of photosensitive epileptic seizures. This study specifies twenty-seven (33) selections using only 3 frequencies with about 2 s EEG data. However, the targets are not distinguished from each other using traditional spectrum method. Consequently, in this case, IHT-based high-frequency time-modulated SSVEP feature extraction method, which is a nonlinear and nonstationary signal processing method, is proposed to extract the time-frequency characteristics of targets. As a result, the HFCC-SSVEP targets can be obviously distinguished from each other in the Hilbert spectrum. The proposed method helps increase the recognition efficiency of SSVEP for BCI systems. Results show that this High-Frequency Combination Code paradigm is suitable for the SSVEP BCI system.



---

*Source: 194230-2015-09-30.xml*