BRAIN-COMPUTER TARGET READING METHOD BASED ON DYNAMIC GRAPH REPRESENTATION NETWORK AND SYSTEM THEREOF

A brain-computer target reading method based on a dynamic graph representation network and a system thereof are provided. The system includes a dynamic temporal graph constructing module, a dual-branch graph pooling module and a dynamic temporal attention module. The dynamic temporal graph constructing module captures a time-varying connectivity relationship between Electroencephalography (EEG) signal channels. The dual-branch pooling module retains local structure information and global structure information in the process of purifying features, which reduces the loss of effective information. Finally, the dynamic temporal attention module allows a model to pay more attention to task-related representations, thus improving the overall classification performance of the model. Compared with the existing event-related potential identification method, the result of the brain-computer target reading method is better. The brain-computer target reading method overcomes the limitation of static graph network in terms of dynamically capturing the time-varying connectivity between the EEG signal channels.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application claims the benefit and priority of Chinese Patent Application No. 202311636395.X filed with the China National Intellectual Property Administration on Nov. 30, 2023, the disclosure of which is incorporated by reference herein in its entirety as part of the present application.

TECHNICAL FIELD

The present disclosure belongs to the technical field of brain-computer interfaces, and relates to a brain-computer target reading method based on a dynamic graph representation network, in particular to a method of capturing the time-varying connectivity between channels of Electroencephalography (EEG) signals using a dynamic temporal graph constructing module and extracting a task-related feature using a dual-branch graph pooling module and a dynamic temporal attention module, so as to identify event-related potentials.

BACKGROUND

The Brain-Computer Interface (BCI) technology realizes the direct communication between a human brain and an external device. In a BCI system, the commonly used non-invasive Electroencephalography (EEG) technology is widely used to record the brain activity. Rapid Serial Visual Presentation (RSVP) is a widely used BCI paradigm based on the EEG, which is often used to carry out a target detection task. Under this paradigm, a subject needs to identify a specific target image in a rapidly displayed image sequence. When the subject detects the target image, the EEG signals will induce corresponding Event-Related Potentials (ERP), such as P300, N100, N400, etc. By analyzing these ERP components, researchers can judge whether the subject has successfully observed the target image.

At present, many researches rely on a convolutional neural network or a cyclic neural network to extract features from EEG data. However, these methods usually ignore the connectivity relationships between different channels of EEG signals, which are critical for EEG analysis. In addition, a model based on the convolutional neural network or the cyclic neural network often limits feature extraction to the Euclidean domain. In order to solve these problems, researchers developed a graph neural network, which is a powerful tool for learning non-Euclidean data representation and is especially suitable for analysis of EEG signals.

The existing brain-computer target reading method based on the graph neural network can effectively model the complex interaction between brain areas and the connectivity relationship between electrodes in the EEG data, thus significantly improving the accuracy and efficiency of classifying EEG signals. Therefore, the method according to the present disclosure has attracted more and more attention in the research field. Specifically, these methods construct an adjacency matrix in a graph neural network by analyzing the physical distance between electrodes or the correlation between EEG channel signals. However, such methods based on the static graph neural network fail to fully consider the temporal dynamics of the connectivity between electrodes in EEG signals, thus failing to fully capture the dynamic changes of the brain activity. Such limitation may result in the insufficient performance of the model in the process of adapting to the changes of the brain network between different individuals or the same individual under different conditions, which limits the classification effect and the generalization ability.

SUMMARY

An object of the present disclosure is to provide a brain-computer target reading method of EEG signals based on a dynamic graph representation network.

In a first aspect, the present disclosure provides a brain-computer target reading method based on dynamic graph features, which specifically includes the following steps:

Preferably, in Step 1, the collected EEG data is preprocessed; and the pre-processing operation includes filtering, down-sampling and re-reference.

Preferably, in Step 3, the sequence graph Gi=<Vi, Ei, Ai>; where Vi is nodes in the sequence graph, which represents channels of EEG signals; Ei represents a set of edges for connecting nodes, and Ai represents an adjacency matrix of the sequence graph.

Preferably, an expression of the adjacency matrix Ai is:

Preferably, an expression of extracting a graph feature hi in Step 5 is:

Preferably, in Step 7, the global graph feature gi is extracted by a Softmax aggregation function, with an expression:

Preferably, in Step 9, aggregated features are classified by using a linear layer and a Softmax activation function to obtain the prediction result y.

In a second aspect, the present disclosure provides a brain-computer target reading system, including a dynamic temporal graph constructing module, a dual-branch graph pooling module and a dynamic temporal attention module;

In a third aspect, the present disclosure provides a computer device, including a memory, a processor and a computer program stored in the memory and operable on the processor, wherein the memory stores the computer program; and the processor executes the brain-computer target reading method.

In a fourth aspect, the present disclosure provides a readable storage medium, having a computer program is stored; wherein the computer program, when executed by a processor, is configured to implement the brain-computer target reading method.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The method of the present disclosure will be described in detail with reference to the attached drawings.

As shown in FIG. 1 and FIG. 2, a brain-computer target reading method based on dynamic graph features is used to judge whether the subject has observed the target object according to the collected EEG data. In this embodiment, the method is specifically used to judge whether a user sees pedestrians in a picture.

The brain-computer target reading method based on the dynamic graph features specifically includes the following Steps 1-9.

The tagged EEG data (namely EEG data) based on the RSVP target detection paradigm is acquired. The EEG data is pre-processed; and the pre-processing operation includes filtering, down-sampling and re-reference to obtain a plurality of EEG samples.

For each sequence graph Gi, the adjacency matrix Ai is constructed using two learnable one-dimensional vectors, and the construction method is as follows:

The most important k nodes are selected in each area, the features are pooled in this way, and the adjacency matrix is modified accordingly. The specific operation is as follows:

First, the feature hilocal_pool obtained by local pooling is added with the feature higlobal_pool obtained by global pooling. Thereafter, a two-dimensional convolution layer with a convolution kernel size of 1×1 is applied to fuse the features to obtain the final pooled feature hipool. The expression is as follows:

After acquiring the weights, the global graph feature gi of each time slice of one sample is weighted and summed to obtain the weighted global graph feature giattn, which are specifically expressed as follows:

It is judged whether the predictor has observed the target object according to the obtained prediction result y.

In this embodiment, the trained loss function uses the cross entropy loss function.

After the training is completed, the EEG data collected from the subjects is preprocessed and input to the model for classification to obtain the prediction results.

The present disclosure is compared with some brain-computer target reading methods with superior effects on a disclosed RSVP target detection data set. The data set includes 14 subjects. Each subject needs to detect the target picture including pedestrians in the picture sequence presented at a high speed. In the present disclosure, the method is tested using a leave-one-out method, that is, EEG data of a subject is selected as a test set, and the EEG data of the remaining subjects is selected as a training set. Furthermore, the performance of the method is evaluated using the Balanced Classification Accuracy (BCA). The accuracy calculation formula is as follows:

present

accuracy

As can be seen from the data in the table, the method of the present disclosure is greatly improved compared with the prior art, and is improved by at least 5% compared with the prior art. Therefore, the effectiveness of the method according to the present disclosure is proved.