dataesr/scientific-paragraphs-categorization huggingface dataset:
Intra-subject analyses. Both the training and test data contained a subset of baseline and follow-up scans of the same subject. As this is common in clinical practice, it is vital that a measurement method should perform to a similar standard for both baseline and follow-up imaging, even though the two scans may differ in scanner type, acquisition protocol and quality. An accurate measure of the volume difference between follow-up and baseline scans is important to be able to detect growth of the UIA. To assess if the method could detect growth, the difference in volume between baseline and follow-up ground truth segmentations was determined (ground truth volume change). This was compared to the difference in volume of follow-up and baseline predicted segmentations by each method (predicted volume change).
This was performed for sensitivity, to assess detection performance, and DSC and volumetric similarity for segmentation performance. 2. 4. 2. 6. Train vs test performance. To assess performance differences between the training and test data, all methods were re-run on the training set and detection and segmentation metrics were determined. Performance should be similar to that of the test set and a large increase in performance indicates that the method may not be very generalisable to unseen data. A similar ranking of methods was made and this performance was compared to the performance of the methods on the test set. All data analyses were conducted using pandas ( McKinney, 2010 ), scipy  ( Virtanen et al. , 2020 ) , seaborn ( Michael Waskom and the seaborn development team, 2020 ) and pingouin  ( Vallat, 2018 )  toolboxes with Python 3.
Training and test data There were no statistically significant differences between the cases of the training and test datasets in age ( p = 0. 20), sex ( p = 1), maximum diameter of the UIA ( p = 0. 58), number of UIAs ( p = 0. 32), number of treated UIAs ( p = 0. 45), magnetic field strength of the scanner ( p = 0. 11), in-plane voxel spacing of the scan ( p = 0. 43), slice thickness of the scan ( p = 0. 78).
Challenge submission Over 250 users registered for the challenge on the website, and 11 teams submitted methods. Two teams submitted only under task 1, for the detection of UIAs, and nine teams submitted under task 2, for the segmentation of UIAs. Results, presentations, posters and a brief description of all submitted methods can be found on the challenge website ( http://adam. isi. uu. nl/results/resultsmiccai-2020/ ). The inference code submitted in Docker containers for the challenge is also available for most methods on DockerHub ( https://hub. docker. com/orgs/adamchallenge ).
Task 1 Submissions MiBaumgartner submitted a 3D neural network based on the Retina U-Net architecture  ( Jaeger et al. , 2018 ). The decoder was extended to incorporate semantic segmentation information and followed by a Path Aggregation Network  ( Liu et al. , 2018 )  to generate the features used for the detection prediction. ( Baumgartner et al. , 2020 )  Unil_chuv submitted a 3D U-Net  ( Ronneberger et al. , 2015 )  which was patch trained using patches selected based on landmark points from a registered vessel atlas  ( Mouches and Forkert, 2019 ). Both the ADAM dataset and an in-house dataset for training. On inference, patches were evaluated only if they were within a set distance from the registered landmark points and had a minimum intensity. A maximum number of four false positives were allowed based on the average brightness of the connected components.
Train vs test performance All the submitted methods were also evaluated on the training data. The results can be seen in Appendix F , Tables  1a  and 1b  ; which correspond to Tables  1a  and 1b  in the main text. As expected, the results on the training data are generally better than on the test data. For task 1, the overall ranking remains roughly similar, with some teams going up or down a few places. This could suggest that some methods generalise less well to the unseen test data, resulting in a lower performance on the test data as compared with the training data. For task 2, the top-4 ranking methods remained the same order of ranking as when assessed on the training data. All methods show a considerable drop in performance when assessed on the test set, relative to the training set.
As treated UIAs were masked on evaluation, this did not negatively affect the performance. Furthermore, mibaumgartner used both the structural MR images and the MRAs, which may have aided in the performance of the model by incorporating more information. Although mibaumgartner has the highest overall ranking, it does not achieve the highest sensitivity or lowest false positive count. For task 1 and task 2, the methods of junma and joker showed comparable performance, both ranking above the other methods. Both use a 3D U-Net architecture based on the no new net (nnUnet). The nnUnet is an "out-of-the box tool for state-of-the-art segmentation " which is an open-source deep learning segmentation framework that automatically adapts to new datasets. In December 2019, the nnUNet performed optimally or on par with the best methods in 19 different biomedical image analysis challenges, including the KiTS challenge ( https://kits19.
Kubiac and inteneural both included the boundary loss in their loss functions for training their models. By including boundary loss, the models are trained to minimise the distance between the predicted and ground truth segmentations. This reduces the problems associated with Sensitivity of all teams for each UIA as a function of maximum UIA diameter in mm, when separating UIA diameter into four quartiles. Each point included in the box plot is the mean sensitivity of all teams across each UIA. regional based metrics, such as Dice, for highly imbalanced data. Kubiac and inteneural have similar performance for task 2 (rankings 0. 24 and 0. 39 respectively) and this may be due to the similar architecture and loss function used. Many teams performed post-processing to only accept positive detections of a certain number of voxels, within a range that was common in the training dataset.
Instead, future developments should think out of the box. It was also noted that few methods use information from the structural images to aid in their methods. Perhaps some prior knowledge of, for example, the location, shape and size of the UIA would aid in the method performance. The dataset was a true clinical dataset, with a mixture of scan parameters, and although this makes it technically challenging, a method that performs well over the whole test set would be very convenient to have for clinical use. For larger aneurysms, the top-ranked detection methods had a performance that was on par with human visual detection suggesting that these methods could be used for the detection of larger UIAs. The method of junma showed promising segmentation performance on the true UIAs. This suggests that a semi-automatic workflow allowing a radiologist to identify the location of the UIA and then using the model of junma as an accurate method of UIA segmentation may already be of use in current clinical practice.
Data availability Training data and results are available at http://adam. isi. uu. nl/. Scripts for evaluation of methods can be found at: https://github. com/ hjkuijf/ADAMchallenge. The test set is not publicly available, as it is kept secret for evaluation purposes of the submitted methods. The inference code submitted in Docker containers for the challenge is also available for most methods, whose teams gave permission, on DockerHub ( https://hub. docker. com/orgs/adamchallenge ).
Acknowledgements We acknowledge the support from the  Netherlands Cardiovascular Research Initiative :  An  initiative with support of the  Dutch Heart Foundation ,  CVON2015-08 ERASE  and  CVON2018-02 ANEURYSM@RISK. We further acknowledge support from the focus area Applied Data Science,  Utrecht University. YMR has received funding from the  European Research Council (ERC)  under the European Union's  Horizon 2020 research and innovation programme (PRYSM , grant agreement No. 852173 ). Team inteneural wish to acknowledge members of the Inteneural Group:  Mikolaj A. Pawlak MD PhD ,  Marek Kraft PhD ,  Micha ł Miko ł ajczak MSc  and  Dominik Pieczy ń ski MSc  for their contributions. Team joker acknowledges support from the  Key-Area Research and Development Program of Guangdong Province, China  (No.
This work provides the community with high-density Electroencephalography (HD-EEG, 256 channels) datasets collected during task-free and task-related paradigms. It includes forty-three healthy participants performing visual naming and spelling tasks, visual and auditory naming tasks and a visual working memory task in addition to resting state. the HD-EEG data are furnished in the Brain Imaging Data Structure (BIDS) format. These datasets can be used to (i) track brain networks dynamics and their rapid reconfigurations at sub-second time scale in different conditions, (naming/ spelling/rest) and modalities, (auditory/visual) and compare them to each other, (ii) validate several parameters involved in the methods used to estimate cortical brain networks through scalp EEG, such as the open question of optimal number of channels and number of regions of interest and (iii) allow the reproducibility of results obtained so far using HD-EEG.
Background & Summary Emerging evidence shows that brain (dys)functions arise from communications between spatially distant brain regions  1, 2. Although functional MRI has revolutionized neuroscience in the last decades, its intrinsic poor time resolution (>sec) is a major drawback limiting its use in tracking fast brain network dynamics that underlies the execution of several brain (cognitive and perceptivo-motor) processes. Electro/Magneto-encephalography (EEG/MEG) are unique non-invasive techniques, which enable the tracking of brain dynamics on a millisecond time-scale. Several studies have been done to track cortical brain networks, using the EEG/MEG source connectivity method, during task-free  3, 4  or task-related paradigms  5. However, despite laudable efforts on the model of MEG dataset from Human Connectome Project (HCP)  6, 7  and several EEG datasets  8, 9  , only few are available for both rest and task and open-access high density EEG (HD-EEG, 256 channels) data during different task are still missing.
The HD-EEG combined with sophisticated signal processing algorithms is increasingly transforming EEG into a potential neuroimaging modality  10, 11. Recent EEG studies revealed the possibility to track fast dynamics of functional connectivity at rest  12  and during cognitive tasks  13, 14. Also, some studies reported the potential use of HD-EEG data (compared to low EEG channel density) in some pathological conditions such as the localization of epileptic networks  15  and the detection of cognitive decline in neurodegenerative diseases  16. In addition, emerging evidence show the possibility, to some extent, to capture sub-cortical structures using HD-EEG  17, 18. In this context, the availability of task-free and task-related open-access HD-EEG databases is becoming mandatory to (i) decipher the fast (sub-seconds) reconfiguration of functional brain networks during cognition, (ii) develop new signal processing methods to adequately estimate cortical brain networks and (iii) allow the reproducibility of results obtained so far using HD-EEG.
Here, we provide the first open-access HD-EEG (256 channels) datasets recorded during resting state and four different tasks (visual naming, auditory naming, visual spelling and working memory). Parts of the data have already been used for developing and analyzing various signal processing methods. In particular, our efforts were focused on the estimation of functional brain networks at rest and during picture naming  12, 14, [19] [20] [21] [22]. None of these studies, however, have described the database in details, and previous works have so far only used small parts of the data presented. In this work, we provide all the necessary details and an open-access to the database so that the international scientific community can freely generate greater understanding of brain functions during task-free and task-related paradigms.
Methods Data collection. Data were collected between 2012 and 2017 in Rennes (France) during two different experiments. The first dataset consists of naming and spelling the names of visually presented objects (Fig. 1 ). The second dataset includes resting state, visual/auditory naming and visual working memory tasks (Fig. 2 ). The same equipment was used in both datasets and recordings were performed in the same place (Rennes University Hospital Center). HD-EEG system (EGI, Electrical Geodesic Inc. , 256 electrodes) was used to record brain activity with a sampling rate equal to 1 KHz and electrodes impedances were kept below 50 kΩ. Involved participants were different for the two studies. They provided their written informed consent to participate and fulfilled some inclusion/exclusion criteria questionnaires (summarized in Table  1 ).
The participants were seated in a medical armchair linked to the faraday structure of the room. The room was lit by natural light attenuated by blinds. Heads of our participants were approximately located 1 m in front of the 17′screen. Images were presented centrally as black drawings on a white background without any size modification (10 cm × 10 cm). This setup corresponds to a viewing angle of 2. 86 degrees of maximum excenticity from fixation point so that the entire image falls within the participant's foveal vision. Sounds were displayed through 50 watts logitech speakers without any possibility of audio isolation. Dataset 1. Participants. Twenty-three right-handed healthy volunteers of whom 12 females, with an age range between 19-40 years (mean age 28 year), and 11 males with an age range between 19-33 years (mean age 23 years) participated.
Afterwards, the acquisition procedure is explained to the participant (see Fig. 1c ). The experimental paradigm includes two conditions, the first one corresponds to the picture naming task and the second one to the picture spelling task. The spelling task always follow the naming task and its instruction was not given before the naming task was completed to avoid any reminiscence of words orthographic structures Both naming and spelling tasks are divided into two runs of 74 stimuli to avoid tiredness of participants. Each run contains balanced numbers of animals and objects as well as long and short words. Pictures are presented on a screen using a computer and the experimental paradigm is diffused using E-prime ® Psychology Software Tools © 24. The responses produced by the participants are collected via a Logitech ® microphone and analyzed to detect onsets of speech using Praat v5.
3. 13 (University of Amsterdam, 1012VT Amsterdam, The Netherlands)  25. Task 1: Picture naming. Participants were asked to name at a normal speed 148 displayed pictures on a screen divided in two runs of about 8 min each. The images were selected from a database of 400 pictures standardized for French  26. The presented pictures represent two categories (animals (74) and tools (74)). Naming agreement of picture is very high leading to few mistakes while recovering words associated to images. The word length is controlled such that the paradigm includes 74 short words (37 animals and 37 tools) of 3 to 5 letters and 74 long words (37 animals and 37 tools) of 7 to 10 letters. Other psycholinguistic parameters were controlled to get equivalent datasets (name agreement, image agreement, age of acquisition as well as linguistic parameters like oral frequency, written frequency, letters/phonemes/syllables and morphemes numbers), see Table  2.
All pictures were shown as black drawings on a white background. The order of presentation within a run of 74 stimuli was fully randomized across participants. Naming latencies were determined as the time between picture onset and the beginning of vocalization recorded by the system. EEG Triggers of images that were not correctly recognized or not recognized at all are discarded so that they don't appear in the dataset. Task 2: Picture spelling. In this task, participants were asked to spell the same images used in the picture naming task. The instruction about the spelling task was given after the completion of the naming task to keep the participant naïve about our interest to the spelling. Also, this task always happened after the picture naming task to avoid the re-activation of the spelling of image's names.
Two sessions of visual spelling were performed for each participant and the duration of each session was about 9 to 10 min. This task is very similar to the previous one except that it involves the orthography of words that correspond to the named images. One can easily recognize the drawing and named it without going through activation of the orthography of the word. In spelling, recovering the exact orthographic structure with the sequence of letters is an additional step that closes spelling and writing. This task includes the same 148 images selected from the database of 400 pictures standardized for French  25  that were used for the naming task. These data have not been extensively analyzed, especially under the psycholinguistic angle. They have also never been compared together with the picture naming task.
Dataset 2. Participants. Twenty right-handed healthy volunteers (10 females, 10 males, mean age 23 years-See Fig. 2a ) participated in this experiment. Like for the dataset 1, all participants provided a written informed consent to participate in this study which was approved by an independent ethics committee and authorized by the IRB (CCPPRB-Ouest V)). Recorded study name was "Braingraph" and the study agreement number was 2014-A01461-46. Its promoter was still the Rennes University Hospital. Experimental procedure and design. The experiment is composed of four tasks: resting state, picture naming, auditory naming and working memory. The three first tasks are distributed in a counterbalanced way within the group. The memory task always happened at the end of the session because it involved 40 images from the previous naming task amongst its 80 displayed images.
All responses for each trial within the tasks are available with the data. Triggers corresponding to false responses or no responses are simply discarded from the dataset. Resting state EEG. The participants were asked to relax for 10 minutes with their eyes opened during the recordings. Participants were facing the computer screen which displayed a fixation cross. They were told not to fixate the cross but to keep their eyes in the vicinity of it so that the rest run could also be used as a control for visual or auditory naming. Task 1: Picture naming. The naming task of this second dataset contains 40 unrecognizable scrambled objects on top of 80 meaningful pictures taken from the Alario and Ferrand database  25  (see Fig. 3  for typical examples of the presented images). Scrambled pictures were generated from the Alario and Ferrand database by mixing drawings lines and participants were instructed to say nothing when viewing them.
Pictures were displayed on a screen as black drawings on a white background. Pictures were selected to get a high name agreement (avg = 96. 86%, min = 86%; max = 100%), see Table  3  for image's parameters of dataset 2. We also controlled our data set to have as many semantical representatives at each end of some principal components continua whose spatial distribution on the surface of the brain has been demonstrated  27  (mobile/ non-mobile: 40 vs. 40; animacy/non-animacy: 46 vs 34; social/non-social: 30 vs. 50; civilization/nature: 41 vs. 39; animal/not-animal: 41 vs. 39; biological/non-biological 48 vs. 32). Ones could also check these dimensionality effects against controlled semantical information that are supposed not to produce any distinct spatial clusters as for example: portable vs.
not portable (44 vs. 36). Some of these categories are adjustable depending on cultures. The E-Prime 2. 0 software (Psychology Software Tools, Pittsburgh, PA) was used to display the pictures. The typical trial starts with a fixation cross that lasts 1000 ms, then the image is shown during 2500 ms followed by a blank screen for 1000 or 2000 ms (randomly selected). The picture triggers are recorded when the image disappears from the screen (offset and not the onset as for dataset1). To get the onsets of a picture display, ones have to subtract 2500 ms. The time between the picture onset and the beginning of vocalization recorded by the system was considered as naming latencies. The voice onset times were then analyzed using Praat software  25. Task 2: Auditory naming. The participants were asked to name 80 different sounds from the NESSTI database  28  and displayed through headphones.
Data records All the data are available in BIDS format  29, 30  , and uploaded separately to OpenNeuro site (https://openneuro. org). The main folder of Dataset 1 (available at: https://doi. org/10. 18112/openneuro. ds003420. v1. 0. 2) 31 contains 23 folders, one for each participant, and two files: i) "data-description. json" that describes the dataset and contains information about where and when the data are registered and ii) "participants. tsv" that contains information about the participants such as sex, age and education level. Each participant's folder contains two subfolders for the naming and spelling tasks, the two tasks folders contain the EEG data, channels, events, etc … for two sessions 1 and 2 (See Fig. 4 ). The same folder's structure applies for the Dataset 2 (available at: https://doi.
technical Validation preprocessing/data quality. First, we provide an overview qualitative and quantitative evaluation of the EEG signals quality of both datasets. Results are illustrated in Fig. 5. A typical example of the event related potentials (ERPs) for a given participant (averaged over trials) is illustrated in Fig. 5a ,b (left) for both visual (from dataset 1) and auditory (from dataset 2), respectively. The global average ERPs (over trials and participants) are presented in Fig. 5a ,b (right). The channels used to perform the electrooculography (EOG) regression and the removed channels (from Automagic) are presented in Fig. 5c. In total, 195 EEG channels (over 256 channels from the EGI system) were kept. Figure  5d  provides four quantitative metrics used by Automagic as criteria to ensure the quality of the preprocessed signals.
First, the number of interpolated channels is presented in Fig. 5d  (up/left) showing that only few channels were interpolated (lower than the 15% of the channels). The figure shows that more channels were interpolated in dataset 2 compared to dataset 1 and that some trials can be removed (above the yellow line denoting the 15% threshold). All the other metrics show very low values below the threshold set by Automagic used to accept/reject trials. See https://github. com/methlabUZH/ automagic for more technical details about these metrics.
Brain dynamics analysis. Several analyses can be done on this data such as time, frequency, time-frequency or connectivity-based analysis, among others. Here we provide an example of the analysis that can be done using these datasets to explore the dynamics of brain activity at scalp, source and network level. To do so, we use only open-source tools, in order to ease the reproducibility of these examples. A tool for automatic preprocessing will be used called Automagic https://github. com/methlabUZH/automagic, and another tool for the automatic segmentation (into microstates) from EEGLAB https://archive. compute. dtu. dk/files/public/users/atpo/Microstate. Results are visualized using either EEGLAB for voltage topography and BrainNet viewer https://www. nitrc. org/ projects/bnv/ for the networks.