dataesr/scientific-paragraphs-categorization huggingface dataset:
Choice of the Classifier After selecting the most relevant features, the choice of the classifier was based on the comparison of total accuracy and AUC scores between four classifiers. Figure  2  presents the average values over 5 runs for the recordings acquired with the standard EEG system (Figure  2a ) and with the dry electrodes device (Figure  2b ) databases. At each run, all the EEG segments of each class were randomly shuffled. Results from databases with wet EEG electrodes are displayed in Figure  2a. We can observe that the Weighted kNN has the best accuracy (88. 38%), followed by the Linear SVM (85. 8%). AUC values indicate that differences among classifiers are negligible, with scores reaching 95% except for LDA that performed slightly less. Classifiers' performance on signals acquired with dry electrodes are in Figure  2b.
Although Weighted kNN accuracy (90. 22%) does not provide the highest performance, it is very close to the best value (91. 75%) obtained by the Linear SVM. As for wet sensors, AUC scores exceed 95% except for LDA. To study the impact of the number of nearest neighbors (k) used in Weighted kNN classification, we obtained the accuracy values corresponding to k from 1 to 20, averaged after applying a 5-fold cross validation on each database. The best result is obtained with k = 7 for artBA and artMM; and k = 10 for the artificially contaminated EEG segments from the public database (publicDB). A complementary criterion to compare the four classifiers, decisive for wearable devices working in a real-time setting, is the execution time to predict the quality of EEG segments. As it can be observed in Figure  3 , the Weighted kNN is the fastest algorithm, providing predictions 12 times faster than the slowest solution, the Linear SVM.
Assessment of Quality Checker's Performances The performance of our quality checker was evaluated on each database for which the ground truth is known (artBA, artMM and publicDB), by the method described in Section 2. 2. The total accuracy and the accuracies in each class, are then computed (see Table  2 ). Table  2. Detection accuracy values obtained for each class of contaminated segments (LOW-Q, MED-Q and HIGH-Q) in the three datasets (artBA, artMM, publicDB). For MED-MUSC segments, accuracy is computed on EEG segments classified as MED-Q by the Weighted kNN. Table  2  shows that the proposed method successfully classifies more than 90% of EEG segments in each database. The highest accuracies concerned LOW-Q segments, with 94. 11% for artBA, 96. 67% for artMM and 99. 67% for publicDB. Right after, HIGH-Q EEG is successfully detected in 92.
The good detection performance of low quality segments in publicDB database can be explained by the construction of this artificial contaminated data. Indeed, these EEG segments contain some extreme values whose amplitudes are higher than the fixed thresholds. The poor detection of the contaminated segments can be due to the features used by the algorithm, which probably cannot completely characterize the different artefacts, thus unable to discriminate them from clean EEG signal. These results can also be explained by the use of fixed thresholds. Indeed, although identified thresholds encompass the majority of the maxima extracted from the clean EEG, they do not include them all.
Quality Assessment of Unlabelled EEG Recordings To evaluate our algorithm in real unlabelled EEG recordings, two databases were collected on 10 subjects during a resting state condition with a standard EEG system (Acticap BrainProducts, GmbH, Gilching, Germany) and with a low-cost system (Melomind, myBrain Technologies, Paris, France). To determine the quality level of each segment in the new (unlabelled) databases, the classifiers were trained on the EEG signals contained in the labelled databases (see Section 2. 1). For the quality assessment of EEG segments, weighted kNN classifiers were used, with k set to 7 as previously used (see Section 3. 1. 2). The percentage of detected artefacts are indicated on Table  4. Results show that most of the recordings in both databases are detected as clean EEG data (HIGH-Q): 91.
50% for the standard EEG setting and 80. 58% for the dry electrodes device. Only a few segments are detected as LOW-Q quality level: 1. 25% of EEG acquired with wet electrodes and 0. 9% of signals recorded with dry sensors. Finally, 7. 25% and 18. 50% of segments are detected with MED-Q quality level for the standard EEG headset and mobile device, respectively. The assessment of quality suggests that both datasets are practically free of artefacts, as expected when subjects were asked to be at rest with eyes closed. The slightly higher proportion of contaminated segments detected in the recordings of the low-cost device can be explained by the fact that the contact between dry sensors and the skin is poorer than that obtained by the wet sensors of the standard EEG system.
Impact of the Contamination Level As introduced in Section 2. 1, the public database served to generate artefacted EEG by controlling the degree of contamination (artefacts) and hence to evaluate the sensitivity of our method in different SNRs defined earlier (see Figure  4 ). We therefore used Weighted kNN with k = 10 and repeated a 5-fold cross-validation procedure 10 times, each run with randomly selected training set. In this database, the averaged percentage of correctly detected clean segments is equal to 94. 8% whereas the most contaminated excerpts (SNR < 0 dB) reach in almost perfect detections (99. 8% mean accuracy). For moderate artefacts (0 ≤ SNR < 10 dB) correct predictions range from 80 to 85%, a good result if we consider that SNRs between 5 and 10 dB are hardly recognizable visually.
Execution Time The execution time of the proposed method was computed and averaged through 10 runs, using a 2. 5 GHz dual-core Intel Core i5 processor, of 8 GB memory. Using Matlab (version R2017b) our approach estimates the quality of 1 s EEG segment in 14. 3 ms on average. In comparison, a classification using Linear SVM is ten times slower than the proposed approach. Our algorithm was also implemented in C++ language to be used in embedded EEG systems, and it takes, on average, 3. 2 ms to assess the quality of 1 s EEG data.
Discussion and Conclusions The proposed approach is a classification-based method to evaluate the quality of EEG data that includes a spectral distance to discriminate muscular artefacts from the other types of artefacts. We propose a fast and efficient an approach for detecting and characterizing artefacts that can be applied in single-channel EEG configurations. The method was validated on different databases containing real artefacts generated in real conditions, and one database with artificially generated artefacts superimposed to clean EEG data. A comparison of performance in terms of accuracy and AUC was made to choose the best classifier among the LDA, Linear SVM, Euclidean and Weighted kNNs. Although the Linear SVM obtained slightly better accuracies for one of the tested databases, the Weighted kNN was selected as a good compromise regarding the artefact detection and the execution time.
Indeed, for each labelled database (for which the quality level of segments was known), the proposed approach with a Weighted kNN reached more than 90% of good detection in the quality assessment of EEG segments, taking less than 15 ms for each EEG segment. For artificially contaminated EEG signals, we show that our method may yield almost perfect detections in moderate to high artefactual conditions and very fair performance even with high signal-to-noise ratios. Indeed, contaminated signals with SNR between 0 and 10 dB were detected more than 80% of times although they are hardly recognizable visually. Currently, the algorithm can detect muscular artefacts but further investigations will be performed to automatically recognize specific patterns of other sources of artefact contamination (blinks, saccades, head movements,.
). When applied to the unlabelled databases, our algorithm detected similar amount of contaminated segments on EEG recordings from both the standard EEG system (with wet electrodes) and the dry sensor device. These results were in full agreement with the high quality of EEG recordings obtained during resting state, where the subjects were asked to be at rest with eyes closed. Finally, the results presented in this work suggest that our approach is a good EEG quality checker in off-line environments with either dry or wet EEG electrodes. The presented algorithm is not subject-driven and classifiers are trained with data collected from different subjects at different time periods. Although beyond the scope of our study, we notice that an optimization of subject-driven classifiers for longitudinal recordings (weeks or months) might increase classification performance.
Acknowledgments: The authors thank  L. Hugueville  for kindly helping in the preparation of the experiments, particularly for the recording data at  Centre de NeuroImagerie de Recherche -CENIR, Centre de Recherche de l'Institut du Cerveau et de la Moelle Epinière. The authors also thank the  Electronic Department of myBrain Technologies, Paris, France , in particular  N. Pourchier , who contributed to develop and improve the electronic part of the  Melomind device.
We present Visbrain, a Python open-source package that offers a comprehensive visualization suite for neuroimaging and electrophysiological brain data. Visbrain consists of two levels of abstraction: (1) objects which represent highly configurable neurooriented visual primitives (3D brain, sources connectivity, etc. ) and (2) graphical user interfaces for higher level interactions. The object level offers flexible and modular tools to produce and automate the production of figures using an approach similar to that of Matplotlib with subplots. The second level visually connects these objects by controlling properties and interactions through graphical interfaces. The current release of Visbrain (version 0. 4. 2) contains 14 different objects and three responsive graphical user interfaces, built with PyQt: Signal, for the inspection of time-series and spectral properties, Brain for any type of visualization involving a 3D brain and Sleep for polysomnographic data visualization and sleep analysis.
Each module has been developed in tight collaboration with end-users, i. e. , primarily neuroscientists and domain experts, who bring their experience to make Visbrain as transparent as possible to the recording modalities (e. g. , intracranial EEG, scalp-EEG, MEG, anatomical and functional MRI). Visbrain is developed on top of VisPy, a Python package providing high-performance 2D and 3D visualization by leveraging the computational power of the graphics card. Visbrain is available on Github and comes with a documentation, examples, and datasets (http://visbrain. org).
INTRODUCTION The aim of scientific visualization is to graphically illustrate datasets-which are can be highly complex-in order to provide a better understanding and facilitate the interpretation of the data. As scientific technologies continue to evolve, it becomes increasingly important to develop up-to-date and comprehensive visualization software capable of handling complex and large datasets. This is especially true in the field of neuroscience, which involves a myriad of neural recording types, and consequently, a wide and diverse range of possible data representations. To date, Matlab  (Mathworks, 2012)  is one of the most widelyused commercial programming language for brain data analysis and visualization, thanks to a large number of toolboxes such as SPM  (Penny et al. , 2011) , Brainstorm 1   (Tadel et al.
, 2011) , EEGlab 2   (Delorme and Makeig, 2004)  and Fieldtrip 3   (Oostenveld et al. , 2011). Alternative visualization solutions that run on noncommercial open-source programming environments, such as Python, are rare. These include high-quality packages such as MNE 4   (Gramfort et al. , 2013) , PySurfer 5  , Nilearn 6   (Abraham et al. , 2014)  or 3d slicer  (Fedorov et al. , 2012). Both MNE and Nilearn rely on Matplotlib for visualizations which is not suited for real-time interactions of brain imaging data involving thousands of data points. In addition, MNE also relies on PySurfer for 3D visualizations. PySurfer is built on top of Mayavi which contains a powerful rendering engine and allows smooth interactions. However, some issues have been reported when installing Mayavi, (which uses VTK), which may affect its userfriendliness.
In this context, we propose a Python open-source software called Visbrain, distributed under a Berkeley Software Distribution (BSD) license and dedicated to the visualization of neuroscientific data. Visbrain is built on top of VisPy  (Campagnola et al. , 2015) , a high-performance visualization library that leverages the Graphics Processing Units (GPU). As a result, Visbrain efficiently handles the visualization of large and complex multi-dimensional datasets. The purpose of Visbrain is two-fold:  (1)  To provide within a common framework several Python-based visualization tools for neuroscientific data, (2) To allow users, including those with little or no programming skills access to high-end visualization functions, through a comprehensive documentation 7  and a user-friendly API. Many scenarii for the use of Visbrain are possible.
Programming Language and Code Guidelines Although we initially considered Matlab and Julia  (Bezanson et al. , 2017)  as language for Visbrain given their high level of abstraction, we finally chose Python since this mature and easyto-learn language benefits from a large range of high-quality packages, a thriving and rapidly growing user community, and thorough documentation. Python software packages are portable, cross-platform, and easily distributed. More importantly, Python is free, open source, open access, and is thereby ideal for open science. From a programming perspective, we paid particular attention to avoid memory-intensive data copy and to enable loading and processing of large dataset. Visbrain is hosted on GitHub 12  , and is documented using NumPyDoc, a Sphinx extension to generate NumPy-like documentation.
Dependencies As Python 2. 7 will not be maintained past 2020, Visbrain is a pure Python package for Python 3 only. Here is the list of Visbrain's dependencies are listed in Table  1. In addition to the above-mentioned packages, the use of some specific functionalities will require a few more dependencies. These include: • Pandas  (McKinney, 2011) : for importing and exporting region of interest defined in the brain • MNE-Python  (Gramfort et al. , 2013) : alternative for loading sleep data files instead of using functions included in Visbrain • Nibabel: for supporting certain file formats • Tensorpac 13  for computing phase-amplitude coupling • Imageio: for Graphics Interchange Format (GIF) export Finally, the Visbrain package can be downloaded using the python package manager pip 14.
GPU-Powered High-Speed Graphics As the size, dimensionality and complexity of brain data continues to increase, data visualization tools have to be increasingly efficient, in particular if real-time interaction is needed. For example, high-density EEG or full-night sleep recordings can be associated with files of up to tens of gigabytes. Matplotlib, which is one of the most FIGURE 1 | Architecture of the Visbrain software. The left branch in blue illustrates the three included graphical user interfaces (Signal, Brain, and Sleep and). For advanced users that want to interact programmatically with Visbrain, the right branch in red shows 6 of the 14 implemented objects in Visbrain. These objects are presented in circles to emphasize the fact that each of them is independent. Then, using the scene (SceneObj) these objects can be superimposed or juxtaposed into subplots inside a unique figure.
famous Python plotting libraries  (Hunter, 2007) , is primarily designed to provide static publication-quality figures and is unfortunately currently not suited for handling large data and user interactions. Seaborn 15  , which is built on top of Matplotlib is also not a viable option for the same reasons. Among libraries with mature development and real-time interaction, we also considered PyQtGraph 16  and Glumpy 17   (Rougier, 2015). Both options could certainly have been excellent alternatives. We rather considered the VisPy package  (Campagnola et al. , 2015) , which is a high-performance interactive 2D/3D data visualization library leveraging the computational power of the GPU through OpenGL. The choice of VisPy was made mainly for the ease of installation and also because it is a combined effort by the authors of several visualization libraries (PyQtGraph, VisVis, Galry, and Glumpy) The use of VisPy library is a critical component of Visbrain.
Graphical Interface and User Interactions Scientific visualization software often come with easyto-use GUIs. Although most of the analyses can be performed in the command-line, such interfaces often greatly enhance the user experience. GUIs also allow users with no or little programming knowledge to use the software, making it more accessible to the scientific community. To embed VisPy graphics in full-featured widgets, we chose to use the cross-platform C++ GUI toolkit Qt 18 , for which Python FIGURE 3 | Illustration of the main features of the region of interest (ROI) object (RoiObj). Visbrain provides several default atlases that can be used to extract the mesh of specific regions (A,B,F). In addition, the source object (SourceObj) can interact with the ROI object. For example, sources' activity can be projected onto the mesh (C).
The RoiObj can also be used to identify in which region a source is contained. Here, sources are color-coded according to the MIST (D) but a table with all of the anatomical informations can also be exported. Finally, it is also possible to keep only the sources that fall into the volume formed by the mesh (E). bindings are available (i. e. , PyQt & PySide). Specifically, GUIs of the different Visbrain modules were built using the Qt designer tool and were then converted to Python code using PyQt.
Documentation and Examples Visbrain comes with a detailed step-by-step documentation built with Sphinx 19 and hosted on the Visbrain website 20. This documentation describes how to install Visbrain and use its modules. We also provide a description of GUI components and inputs for all class modules. Moreover, we provide a description of each graphical element using tooltips that appear when hovering corresponding widgets with the cursor. Lastly, we provide examples 21 and python scripts that can be downloaded from the website. Some examples requires additional data to be fully functionals. Those data are either generated or comes from other open-sources softwares (i. e. , MNE-Python, PySurfer, and Nilearn). 19 http://www. sphinx-doc. org/en/stable/ 20 http://visbrain. org 21 http://visbrain.
This settings panel contains PyQt widgets to control objects' properties and apply changes in real time. -The use of VisPy to exploit GPU capabilities. -A "File" menu to import and export files (such as datasets, annotations,. ). From this menu, it is also possible to save and load the GUI state, i. e. , the value of each PyQt graphical elements (checkboxes, comboboxes,. ). The configuration is saved into a text file with a JavaScript Object Notation (JSON) structure and can later be reloaded to retrieve the session. -A "Display" menu that controls which elements are displayed or hidden on the screen. -A "Help" menu to open an informative web page in a browser about the current module and features. This help can also be downloaded in PDF format. -The support of keyboard shortcuts and mouse events (left and right clicks, double clicks, mouse wheel scrolling,.
Signal: Time-Series Visualization and Spectral Properties A common first step when exploring electrophysiological data consists of inspecting time-series. This inspection phase is useful to get an idea of the shape of the signals, as well as quickly detecting artifacts, epileptic spikes, eye movement contamination, etc. Spectral properties such as power spectrum density (PSD) or time-frequency maps are complementary tools for such quality control and data exploration. Such data inspection can be complicated for multi-dimensional datasets (e. g. , number of trials x tasks x time points). To address this issue, we developed the Signal 24  module for the visualization of multidimensional signals. The GUI is divided into two layouts presented in Figure  7. On the left, the dataset overview. This consists of a grid where all of the time-series in the dataset are displayed.
Grid disposition The usefulness of this data exploration module is demonstrated by one of the VisPy examples 25  , in which thousands of signals, each having thousands of points, can be instantly plotted using the GPU graphics rendering. These signals are presented in a twodimensional grid and the user can zoom on each of them. Since this grid of signals can be useful for plotting electrophysiological data, this representation has been integrated into the Signal module (see Figure  7 ). The aim of this grid is to provide an overview of the entire dataset in a convenient way to visualize all the time-series at once. In order to take advantage of the width and height of the screen, the program tries to determine an optimal number of rows and columns for the grid. A title can also be added on top of each signal of the grid to facilitate the orientation of the user.
Sleep: Polysomnographic Data Visualization and Edition Sleep 27  is the Visbrain module dedicated to the visualization and analysis of sleep data and has been previously described  (Combrisson et al. , 2017b). It should be noted that new features have been added to the Sleep module since the publication of this article, such as the possibility to replace the default event detections with custom external algorithms. This allows different sleep research teams to use the same data visualization platform while still keeping their custom, lab-specific, algorithms for the detection of transient events during sleep.
This canvas allows fluid rotation, zoom and translation, but also mesh slicing along the (x, y, z) axes. Here, the colorbar of connectivity strength is displayed but it can also be hidden. on a distant server. In addition, the API provision also implies that other toolboxes that have intensive visualization needs (e. g. MNE-Python) can benefit from this API and the modularity of Visbrain objects. The full Visbrain's API can be found in the online documentation 28.
Summary The ever-growing complexity of neuroimaging recording techniques, relying on analyses in higher-dimensional space and on larger datasets, are gradually transforming brain data visualization into a real challenge for the existing body of neuroimaging software. This challenge is further complicated by the difficulty of meeting the specific needs from individual research teams and by license compatibility issues with proprietary software. With these problems in mind, we propose Visbrain, a versatile Python 3 package for multi-modal brain data visualization. As other softwares, Visbrain includes graphical user interfaces for higher level interactions with visual primitives. But the greatest novelty and added-value of Visbrain lies in its structure and especially the object level which, once configured properly, can offer a great modularity for designing figures and layout that 28 http://visbrain.