dataesr/scientific-paragraphs-categorization huggingface dataset:
NNX08AR22G  issued through the  Planetary Science Division of the NASA Science Mission Directorate , the  National Science Foundation  under Grant No. AST-1238877 , the  University of Maryland , and  Eotvos Lorand University (ELTE). This paper makes use of software developed for the  Large Synoptic Survey Telescope. We thank the  LSST  Project for making their code available as free software at http://dm. lsst. org. The following software and programming languages made this research possible: PYTHON (v2. 7);ASTROPY (v2. 0;Astropy Collaboration et al. 2013Collaboration et al. , 2018)), a community-developed core PYTHON package for Astronomy; Source Extraction and Photometry in Python (SEP v1. 0; Barbary et al. 2015); Halotools (v0. 7;Hearin et al. 2017); COsmology, haLO and large-Scale StrUcture toolS (COLOSSUS; Diemer 2018).
B. RESEARCH PROBLEM AND CONTRIBUTION There has been rapid contribution in the area of digital technology towards the fight against COVID-19, with comprehensive surveys being provided on the topic in general. However, there is a need for research into the contributions that IoT specifically has made in the fight, the challenges faced and the resultant evolution. The policies surrounding the need for social distancing on IoT data collection have affected the various IoT implementation strategies. The infectious nature of COVID-19 has led IoT managers to seek various effective and ethical means of sensor deployment. Secure IoT device communication protocols to protect user privacy and maintain system integrity are also expected. Therefore, there is a need to analyze these various contributions and identify potential research gaps.
B. H-IoT BASED SOFTWARE Various software and management tools are being developed to better manage sensor data collection and filtration. Major focus areas include energy efficiency, lightweight mechanisms, efficient and less intrusive data collection techniques among others. Recent software implementations tend to focus on making the most use of already existing hardware such as smartphones to form part of the H-IoT ecosystem. The first line of software solutions in tracking and tracing infections has been demonstrated by android and ios mobile phone operating systems. Both systems have integrated APIs to alert the user on a potential COVID-19 exposure risk  [27]. Using embedded smartphone proximity sensors the API provides an opportunity to alert a user if someone near them recently tested positive.
A. DATA COLLECTION To discuss the data collection aspect with enough clarity we need to identify the information required in managing the spread of a virus pandemic such as COVID-19. The following parameters have been identified as essential for first-line detection and tracking. These data parameters are collected by IoT devices in the field and fed into IoT monitoring platforms. 1) Temperature: High fever is a major early symptom of COVID-19. Therefore, temperature checks are critical in detecting a potential infection. Several mechanisms for temperature collection are being promoted for being non-touch and within acceptable social distancing measures. Examples include non-contact infrared thermometers  [47] , thermal imaging cameras, smartphone cameras, robot mounted temperature scanners  [48]  and wearable thermal sensors  [26].
5) Social media feeds: AI techniques such as COVID-Sens  [34]  can extract posts and sentiments from social media as input data and provide a simulation of the COVID-19 virus spread. Using feeds from platforms like Twitter and Facebook, information on social distancing, virus hotspots and individual tracking can be extracted. The nature of this data is such that it is possible for fake news to be posted and therefore extensive trend analysis and big data analytics need to be implemented. AI can improve the accuracy of the data being provided by social media feeds.
B. MONITORING SYSTEMS The data collected by IoT sensory systems can get very large and therefore a suitable framework is required to produce the expected outcomes of the technology. There is a need for intelligent systems to analyze and monitor the spread of a virus pandemic based on the provided data. Consequently, this brings in the aspect of big data analytics, AI and visual mapping for IoT infrastructure frameworks. 1) IoT big data and AI: Big data analytics and modeling can enable real-world simulation of the coronavirus spread. Several researchers are using artificial intelligence to better analyze COVID-19 big data and enable effective track and trace mechanisms  [34] ,  [46] ,  [54]. Naude  [55]  mentions in an early review that AI is a powerful tool in the fight against COVID-19 and discussed several use-case scenarios for pandemic detection, tracking, tracing and spread prediction.
Particularly, areas affected by COVID-19 are displayed on the system and strictly monitored using big data analytics  [71]. For instance, human movement patterns from previously infected locations are analyzed to identify, track and predict the outbreak of the disease in probable locations. Take the UK for example, data collected by the National Health Service (NHS) telephone networks were analyzed in partnership with Amazon, Microsoft, and Hancock to move available resources to tackle the pandemic. In China, local governments were permitted by the China Health Commission to use AI-assisted big data to model, estimate and to track the COVID-19 pandemic in real-time  [70]. • Patient diagnosis: Biomedical big data collected from several COVID-19 big data sources have been used to establish a cloud-based system for the diagnosis and treatment of COVID-19 patients.
5) CLOUD ASSISTED IoT With advancements in wireless and digital technologies, computer system resources e. g. database, networking, servers, and intelligence can today be delivered over the internet  [20]. Cloud computing provides faster and flexible resources and innovation as well as the cost-effective and efficient running of infrastructure. During the COVID-19 pandemic for instance, while most people were isolated from their normal lives they have been able to continue their digital lives thanks to applications such as Zoom video, Google Meet, Google Cloud, Slack, Amazon Web Wervices, Netflix, and Microsoft Azure. Similarly, healthcare workers have been able to manage a large volume of requests arising from the COVID-19 pandemic with healthcare-specific apps like Salesforce Care solution  [20].
In this table, the first two columns refer to the method, the third to the database used for the evaluation, and the last ones to the reported metrics, when available, for evaluating the algorithm: classification and sensitivity in the range [0, 100], latency in seconds and false positive rate per hour (FPR/h). In the following paragraphs we describe more in detail some of these studies. Recent contributions to the application of deep learning methods for seizure detection have been developed on the dataset provided by the University of Bonn  26. This database allows the study of a variety of tasks since it comprises intracranial EEG signals of interictal and ictal episodes from epileptic patients, as well as normal scalp recordings of healthy subjects. In particular, in  23  , a single Recurrent Neural Network (RNN) was proposed to exploit the temporal dependencies in EEG signals, which was evaluated in a binary (normal EEG vs.
ictal) and a multi-class classification (normal vs. interictal vs. ictal) schemes. In  27  , a 13-layer Convolutional Neural Network (CNN) was evaluated on a variant of the multi-class problem, which directly included the pre-ictal category. In the binary task, the authors of  21  defined an augmentation scheme with overlapping windows to alleviate the few available data to train deep learning models. In addition, they introduced a pyramidal one-dimensional CNN that leverages local information to generate a final prediction. Although the previous methods successfully detected seizures segments, they require additional adjustments to process multi-channel data, and more importantly, they need to be validated on long-term EEG signals since the very short duration of recordings in this database (23.
6s) prevents an evaluation of their robustness and generalization capabilities. In parallel to the Bonn dataset, several other studies tested their algorithms for seizure detection using the CHB-MIT Scalp EEG database  3, 28. The advantage of this collection of signals is that it contains continuous longterm recordings that can last several hours, which, combined with detailed clinical annotations about seizures times, makes it very suitable for comparisons between different methods. In particular, in  29  , the authors proposed a deep learning approach in a supervised learning framework to automatically learn more robust features. They used a RNN architecture in a leave-one-out scheme that captures spectral, spatial and temporal patterns representing a seizure. In addition, a novel input representation was proposed in which an image representation of EEG signals was created combining spatial information (electrodes montage) with the magnitude of different frequency bands as the pixel attribute.
In 20 , a CNN model based on multilayer perceptrons was proposed in which time domain signals were transformed to frequency domain features by calculating the power spectrum density for 10-s windows. In addition to Sensitivity, they report more reliable metrics, such as Precision and F-measure, obtaining a performance of around 95% for this last one. Nevertheless, the authors chose only 8 h of known non-seizure data for each patient, which does not correspond to the actual class imbalance present in the database. In a similar approach  30  , EEG signals were transformed to power spectrum density energy diagrams which are fed into a deep CNN in a transfer learning strategy to classify four categories of epileptic states. Using a subset of 11 patients and only 116 min for the inter-ictal and pre-ictal classes from this dataset, they obtained an average classification accuracy of 90% for seizure episodes.
Alternatively, using raw EEG signals in the time domain, the authors in  31  trained a CNN model to detect inter-ictal epileptic discharges as well as seizure onsets. They obtained an overall seizure detection sensitivity of around 90% but, as in other studies, only a subsample of all available data limited to few hours was used to train and evaluate their model. In another study  32  , the authors employed a CNN to compare the seizure detection performance from time and frequency domains through the use of raw signals and their corresponding spectral representation respectively. In addition to scalp EEGs, the algorithms were also tested in intracranial recordings from a different dataset. Surprisingly, while they obtained high performances for the frequency domain scheme, the metrics for the time domain strategy in scalp recordings were very low compared to other approaches (see Table  1 ).
Using different datasets, other approaches have evaluated alternative input modalities for CNNs in seizure detection. For instance, in  18  , in addition to 5-s raw time-series as input, the authors explored a transformed black and white image of the raw waveforms, a periodogram between 0 and 99 Hz, and a gray scale image of the shorttime Fourier transform. These data representations were extracted from intracranial EEGs (single channel) from a mouse model of epilepsy, and further validated on a human intracranial EEG dataset. For each input modality, different architectures were evaluated depending on the dimensionality of the data (1D or 2D). In another study, the authors proposed the use of 3D inputs from multi-channel EEG recordings  22  ; the results over 13 patients with 159 seizures demonstrated that 3D models surpassed 2D models in metrics such as accuracy, specificity and sensitivity in about two points.
However, 3D models have a larger number of parameters to be learned, and thus require much more training samples. An alternative to reduce the number of parameters was proposed in  19  using Fully Convolutional Networks (FCN) for neonatal seizure detection. The FCN architecture had a Global Average Pooling layers that replaced the fully connected ones. The experiments were conducted on a large dataset of continuous multi-channel neonatal EEG totaling 834 h and 1389 seizures from 18 newborns, and tested on a public dataset of neonatal EEG waveforms collected at Helsinki University Hospital. The area under the receiveroperating curve (AUC) for specificities over 90% was 86. 9% for the 18 test patients. In this work, we address the seizure detection problem following a deep learning strategy derived from robust methods for object recognition tasks in the computer vision field.
Our intention is to mimic the visual inspection performed by clinical specialists when reading EEG recordings. For this, we propose a strategy to generate image representations of raw EEG signals that we use as inputs to our seizure detection algorithms. We tested our strategies in two different datasets, one with only scalp electrodes and the other with combined scalp and intracranial recordings. Our lightweight model with only 314 thousand parameters and minimal pre-processing steps reached high seizure detection performance, comparable or even improving previous studies in which signal transformations are needed for its implementation. Furthermore, in addition to the evaluation on a standard database for seizure detection, we had the special opportunity to compared our seizure detection algorithm on scalp and intracranial recordings in the same subject and to evaluate transfer learning strategies between these modalities, as we have simultaneous signals from the EPILEPSIAE dataset.
EPILEPSIAE. To compare the results between the EPILEPSIAE and CHB-MIT Scalp datasets, we first carried out experiments using only scalp electrodes from the former database in the First Seizures setup. We departed from the parameter exploration conducted in the CHB-MIT dataset and used the base FCN with Dropout (DO) and weight decay. For this model configuration, we explored a finetuning strategy using the First Seizures model in CHB-MIT patients as the weight initialization. Besides, we evaluated models trained from scratch with electrodes in monopolar and bipolar configuration. For a complete comparison of the experiments in EPILEPSIAE patients with scalp recordings refer to Fig. 1  in the Supplementary Material. We summarize the best results achieved with the monopolar model in Fig. 6. Among all patients, we obtained an average F-measure of 20.
In addition, due to the large class imbalance between ictal and inter-ictal episodes, we report more appropriate metrics such as the precision, the F-measure, the AP, and recall, for which we reached high results for some of the analyzed patients. The comparison of our results with previous studies is not straightforward since there is not a standard evaluation for the seizure detection problem; each author reports its own metrics and the definition of a positive or negative sample depends on the window length being analyzed. To provide the possibility to fully reproduce our results and allow future comparisons, we make publicly available our source code and trained models in GitHub. Despite this, we still compared our method with the metrics of a recent report in the CHB-MIT dataset  32. In this work, the authors trained and evaluated their algorithm for each patient individually, but they did not specify how data was split into train and test sets.
Comparing the average accuracy and specificity of our best model in the First Seizures configuration ( 99. 3 ± 1. 16 and 99. 6 ± 0. 59 respectively), we obtained a performance that is superior to the one reported by the authors in the inter-ictal vs. ictal task (accuracy of 62. 3 and specificity of 63. 3), but their recall (61. 2) was greater by 3 points. However, as mentioned above, due to the large class imbalance of ictal to inter-ictal episodes, the accuracy and specificity might be presented in conjunction with more robust metrics to imbalance tasks, such as the precision or the FPR/h. In another recent work using the same CHB-MIT Scalp EEG database  29  , the average recall or sensitivity was 85% for cross-patients models in a LOO configuration, but the authors do not report precision.
Methods Dataset description. CHB-MIT scalp EEG database. This epilepsy dataset consists of scalp EEG signals from 23 pediatric subjects with intractable seizures admitted at the Boston Children's Hospital 3 , which is publicly available at PhysioNet's website  28. The recordings were acquired using the International 10-20 System for elec-trode positioning in bipolar configuration, sampled at 256 Hz with 16-bit depth. Each recording was annotated by experts indicating seizure beginning and ending times in seconds. From the 24 cases, we analyzed a total of 941. 6 h of inter-ictal activity and 3 h of ictal acitivity from 181 annotated seizures (see Table  1  in the supplementary materia for a complete description of cases).
EPILEPSIAE. The European Epilepsy Database contains high-quality EEG recordings of 275 epilepsy patients  37. Raw EEG data comprise at least 5 days of continuous recording for each subject, with several scalp and/or intracranial channels. The dataset contains metadata information about recordings and patients, including the EEG onset and offset times for all of the seizures (in a date-hour format), the electrodes involved in the seizure activity (primary and secondary propagations), additional classification of the seizures, among others, all annotated by specialist neurologists. From the 275 patients, only 50 have intracranial recordings, and 225 have additional scalp EEG. Intracranial channels consisted of implanted grids, strips and/or depth electrodes, with a variable number of invasise contacts depending on each patient.
Surface recordings consisted of 10-20 or 10-10 standard EEG electrode placement. From these patients, we selected the ones that fulfilled the requirement of simultaneously having intracranial and scalp recordings (n = 10 patients), the latter with the same channels of the CHB-MIT patients for comparison purposes. All signals were originally sampled at 1024 Hz with a 16-bit resolution and then downsampled to 256 Hz for the analyses. From this database, we analyzed a total of 1302. 2 h of inter-ictal activity, and 2. 5 h of ictal activity from 109 annotated seizures (see Table  1  in the Supplementary material for a complete description). Seizures segments shorter than 9s were discarded.
Data representation. Signal representation. To change from signals to images domain, we took the raw amplitude at each sample and channel as the pixel attribute of a 2D image, where the rows corresponded to the different EEG channels aligned vertically, and the columns to the temporal dimension. For the CHB-MIT dataset, we standardized the order of electrodes across all patients by choosing the 23 common bipolar channels among all the registers. We then split the time course of EEG signals into segments (windows) of 4-s of duration, so each resulting image had dimensions of N × (4 × SamplingRate) , where N is the number of channels (re- corded electrodes), and SamplingRate the number of samples per second depending on the dataset. We wanted to design seizure detection methods requiring a minimum of pre-processing steps, as these last require several subjective considerations that we wanted to avoid.
Due to the spontaneous nature of epileptic seizures and their short duration compared to the extensive recording of inter-ictal activity, we augmented the number of training examples for the ictal class. To do so, we created overlapped images during the complete seizure duration with a time-shift between consecutive windows of 1/8 and 1/16 s for the CHB-MIT and EPILEPSIAE datasets, respectively. Even after data augmentation, the class imbalance remained. Thus, the inter-ictal instances were then randomly sub- sampled such that a fraction of them was close to the seizures, by defining a temporal closeness parameter as 150s before and after, and the other fraction was sampled far from the seizures. The inter-ictal instances close to the seizures were augmented with the same strategy mentioned before, but with a time-shift between consecutive windows of 1/4 and 1/8 s for the CHB-MIT and EPILEPSIAE datasets, respectively.
The kernels in the last convolutional layers, which replace the fully connected ones, were defined such that the network produced a class label every window of a fixed size. In this way, the input could be of variable length and the number of predictions depended on the length of the input (recording) to be evaluated. We further introduced two regularization strategies for Deep Neural Networks. The first one, referred as Dropout (DO), prevents that the units of the model from co-adapting to the training data by randomly dropping out units and their connections  40. The second one, L2 regularization (weight decay) helps the network to generalize better. Due to the limited number of patients, seizures and their short duration, we used a transfer learning strategy, called finetuning, to train a model in a database that does not have sufficient size  41.
In brief, once a model has been trained on a base dataset for a specific task, the learned features are transferred to a second network that is trained with samples from a different dataset for a new task. The weights of the pretrained model are said to be finetuned by continuing backpropagation. All the layers from the network can be finetuned, or only a fraction by freezing the weights of the layers that want to be preserved. This process will work if the learned features could be generalized, and if the nature of both the tasks and data is similar.
Models and evaluation. CHB-MIT Scalp EEG. In the CHB-MIT Scalp EEG dataset, we defined crosspatient models by splitting the recordings from the patients in two ways. First, in the 3 fold Cross-Validation (3FCV), we divided patients into three sets by training in two sets and evaluating in the remaining one, such that we covered the three possible combinations. Second, in the Leave-One-Patient-Out (LOO), 24 models were trained, each one using data from k -1 patients (where k is the total number of patients), and evaluated in all the recordings of the patient left out. In both configurations, we sampled 4s windows of the ictal and inter-ictal classes from the recordings of the patients in the train set, and evaluated our model in the complete recordings of the patients in the test set. In addition, we followed an approach to train the models based on chronological information of the signals  42  , which we called First Seizures model.
In particular, we defined a criterion to train the models with the initial 80% of the total seizures from each patient, and we evaluated in the remaining percentage of seizures and the associated inter-ictal recordings. Thus, the model can learn a representation from the information of the first seizures from all patients. Based on the original architecture and the pooling operations over the temporal dimension, the model made predictions every 64 sample points. Transfer learning in EPILEPSIAE. For the EPILEPSIAE dataset, we evaluated a cross-patient model in the First Seizures setup for the scalp recordings. For the intracranial electrodes, we evaluated patient-specific models in the First Seizures setup. To define the models for EPILEPSIAE patients using scalp electrodes, we departed from the parameter exploration conducted in the CHB-MIT dataset and used the base FCN architecture with DO and weight decay.