Venue: NeurIPS 2021 Track Datasets_and_Benchmarks Round1

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: This paper presents a dataset of paired Electroencephalography (EEG) and video-infrared eye tracking (ET) recordings from 356 subjects for more than 47 hours in total. A benchmark consisting of 3 evaluation tasks with increasing difficulty is introduced alongside the dataset. Experiment results of both traditional machine learning methods and deep learning methods are reported as baseline performance.

strengths: (Data Acquisition) Compared to other existing datasets with paired EEG and ET data, the presented dataset contains way more subjects from a larger range of ages, i.e. 356 subjects range from 18 to 80 years old; besides, the data is collected in a more advance setup: a 128-channel EEG Geodesic Hydrocel system and an infrared video-based ET EyeLink 1000 Plus from SR Research recording at a sampling rate of 500 Hz.
(Benchmark) The paper presents three benchmark tasks with different level of difficulty as well as extensive experiments assessing performance of baseline methods of different complexity. Detailed analysis makes sense of the experiment results.
(Accessibility and accountability) Alongside the raw data, two different level of preprocessing with code is provided which makes it more flexible to the users. Project website, dataset, code and infrastructure are released to the public.

weaknesses: (No cross-dataset comparison) There is no baseline comparisons with other datasets in this field. Such kind of comparison helps to make clear in a quantitative sense the limitations of existing datasets and the virtues of the presented dataset. For examples, some experiments can be done to show that model trained using presented dataset outperform that trained using other existing dataset in term of accuracy and generalizability. Such experimental evidences are quite important to justify the contribution of this work.

(Lack figure demonstration) The paper explains dataset construction methodology and statistics in details, but miss description of the dataset using figures. For example, I would expect at least a figure showing the data acquisition setup and a figure demonstrating paired EEG and ET data with their dimensions for a single sample.

correctness: The submission is a dataset with benchmark. The dataset is constructed in a sound way align with existing works. The benchmark adopts an extensive list of both traditional machine learning methods as well as some baseline deep learning methods for evaluation in an appropriate way.

clarity: Yes, the paper is written in a well-organized way and easy to understand.

relation to prior work: Yes, this work extends previous work in terms of scale, diversity and accuracy.

additional feedback: -	I would include a table that characterize different datasets of this kind for comparison.
-	I would add a figure showing data capture setup.
-	I would add a figure showing statistics of the dataset.
-	I would add a figure demonstrating a single sample.
-	I would push more contents about data acquisition and experimental paradigms to appendix to make some room.
review_title: The paper presents the most large-scale and diverse dataset of its kind as well as benchmark with experiments evaluating baseline performance, which is promising and very beneficial for the community. However, it does not include any quantitative comparison with other existing dataset which make it less convincing.

final_score: rating: 10: Top 5% of accepted papers, seminal paper
review_content: summary and contributions: This paper presents a dataset of EEG and ET data from over 300 subjects under three different conditions. A benchmark for gaze prediction, consisting of multiple tasks, is included along with extensive experiments and complete code that appears quite usable for other researchers.
strengths: - This is a very well written paper that is clinically oriented, easy to read, and written with appropriate detail. The broader relevance to the research community will come through the adoption of this dataset by those seeking to assess and potentially diagnosis neurological disease, as the authors state. 
- Good level of detail for recording, preprocessing, data annotation sections
- The authors have demonstrated excellent thought and effort into obtaining a large scale data set, and have provided meaningful experimental procedures as well as thorough benchmarking and validation. 

Overall, this is precisely the type of manuscript I would have expected for this track. 
weaknesses: I would challenge the authors to consider moreover the ethical and social implications, particularly negative contributions, of this work.

I am curious regarding the inclusion and exclusion criteria of their participants, the demographics of the study sample, the prevalence of neurological diseases, and prevalence of individuals with risk factors (family history). Characteristics of the study sample may have influence on downstream applications. 


correctness: This submission contains both a dataset and a benchmark. The dataset is constructed in such a way that it is easy to follow and understand. It is very intuitive. The benchmarking provides excellent tasks and a good variety of approaches. I have no concerns in this area.
clarity: This paper is very well written. I found virtually no errors. 
relation to prior work: The authors have a minimal related works section, although this may be due to the limited number of data sets available. They appropriate differentiate between their own prior work in REF [31] as being from young participants. 
additional feedback: Major
- Has the Visual Symbol Search been validated in adult populations? Given the details provided in the WISC-IV, it's not clear to me if the VSS is valid for adults given the differences in neurophysiology for processing speed. 
- Could the authors elaborate on why there remains such a significant gap in the angle/amplitude task?  


Minor
- I would move the description of how the EEG and eye tracking setups were synchronized into the main text body, rather than in the appendix.

- Shouldn't the results for Task 3 be reported in physical millimeters, given that everything about the acquisition was controlled and the authors know the distance from face to screen + the screen scale?
review_title: An outstanding, well validated, and rigorous data set and benchmarking. 

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: In this work, the authors mainly focus on eye movement prediction using Electroencephalography signals. They build a large-scale dataset under three difficult level tasks. They also design three tasks and evaluate lots of machine learning and deep learning methods on the three tasks on their dataset. 
strengths: This paper is well written and easy to follow. 
They did wonderful and impressive work on the cognitive area research, to understand brain activity. 
The data collection process is detailed and the proposed three tasks are also reasonable. They also evaluate lots of methods on the tasks on their dataset. 

weaknesses: The authors might provide a table to compare their proposed dataset with the previous dataset, in the term of the number of subjects, the modality, the number of fixations. Besides, I suggest the authors could show some real collected images in the manuscript, in different gaze directions, to help the reader have a clear or direct understanding of the data. If possible, in the following data collection process, it would be much better to record the eye images. 
correctness: Yes
clarity: Yes
relation to prior work: Yes
additional feedback: See the weakness.
review_title: This work collects an electroencephalography and eye-tracking dataset and do the eye movement prediction using electroencephalography data. 

Meta-review: The reviewers all liked the paper. The authors' response clarified some important points.  In view of that, the authors are strongly invited to take the feedback on board for the final version. 
Decision: Accept