dataesr/scientific-paragraphs-categorization huggingface dataset:
Cohorts of patients with similar characteristics will be informative for algorithm development applicable to increasingly specific patient profiles. The system will contain clinical information (past medical history, family history, medications, dose, side effects, and other interventions), with specifics on these conditions at the time the EEG was obtained and how they evolved in response to the interventions. Such cohort information would allow for improvement in the seizure detection algorithm, as well as clinical management and medication selection based on increasingly large patient data sets.
Patient observing MAS (POMAS) The architecture and implementation of the Patient Observing MAS are discussed below. The prototype system has been implemented and experimented. In particular, we have implemented PO-MAS in DALI, a well-known publicly available logic programming agent-oriented language  [10] [11] [12] [19] [20] [21] [22]. POMAS, as shown in Fig. 1  , is composed of several agents, where the two main ones are discussed in what follows. • The SmartHealthHelper agent. The functions of this agent are listed below. Specifically, the agent: • Analyzes the patient's vitals (heart rate, saturation, minimum and maximum pressure, temperature, weight) taken by wearable devices or by patients (or caregivers) themselves, returns immediate feedback on each parameter, and stores them in the corresponding list.
• For each therapy entered by human doctors, i. e. , a medicine to be taken regularly, sends a reminder to the user when necessary. • Periodically, sends a reminder for temperature, blood pressure and weight measurements. • Suggests immediate countermeasures for the patients to take if they feel unwell, pending medical intervention. • With each new entry of a weight value, compares it with the previous one and returns feedback indicating whether the user has lost or gained weight, and how many pounds, and informs the DoctorAgent. Vital parameters are, as mentioned, analyzed both over the short and over the long term by the Complex Event Processing Module. If a dangerous situation is suspected, the SmartHealthHelper will alert the DoctorAgent , which will be able to make a diagnosis and propose countermeasures, based on the complex data and events obtained by SmartHealthHelper , and through the advice provided by a human doctor.
• The DoctorAgent. The functions of this agent are listed below. Specifically, the agent: • Receives messages from the SmartHealthHelper agent, analyzes the communicated problem, e. g. , fever or tachycardia, or, in our case, symptoms and/or complex events potentially related to seizure, and responds by suggesting a medication to take or, in case of serious danger, it suggests going to the Emergency Room. It also provides feedback on when to take the next measurement. Such a feedback can partly be provided automatically, and partly by asking a human doctor. • Reports to the human medical doctor periodically and proactively in case of a suspected emergency. • Is able, at the request of the human medical doctor, to provide data and invoke a Deep Learning module, in our case PredictMed-Seizures, to estimate the actual probability of the presence of a dangerous event (in our case, seizure).
Complex event processing Within POMAS, we leverage Complex Event Processing techniques successfully exploited in other contexts, such as video surveillance  [ 23 ]  and social network analysis  [ 24 ]. More specifically, the related architecture is shown in Fig. 2  ; during step 1, the patient either uses a set of sensors (for instance, a thermometer, a pulse oximeter or an ECG device) or his/her observation skills to automatically store vitals (such as body temperature, oxygen saturation or ECG) and other useful information in a database (step 1 and step 3); in the meantime, such information can be also displayed on a smartphone (step 2). Then, a specific knowledge base containing the events models is triggered; specifically, it verifies which kinds of potentially dangerous short-term events have occurred in the current measurement window (step 3B), then stores them in the database, and detects and stores possible long-term events related to the patient's clinical history (step 4).
In fact, a possible instance of a very dangerous short-term event is a too-long epileptic seizure , which implies that an ambulance must be called immediately. On the other hand, a detailed long-term analysis of the patient's clinical history is carried out working on aggregated data at a different granularity, i. e. , by analyzing the outcomes of multiple measurements occurred in specific broader temporal windows. This is done by using the powerful functionalities of a specific language which further extends relational algebra and the wellknown Allen's interval relationships, named ISEQL (standing for Interval-based Event Query Language )  [ 26, 27 ]. Specifically, on the one hand, we exploit the language to formally define the longterm events we are interested in, focusing on specific time windows; this is done by using additional constraints of the language applied to the social network analysis  [ 28 ]  (i.
Additionally, the other intervals (i. e. , Seizure 1,. , Seizure 4) represent the temporal spans when specific instances of epileptic seizures have been detected by the short-term module. As a consequence, one scenario such as the one in Fig. 3  should be immediately identified as a longterm event, since there are multiple occurrences of anomalies in a relatively short temporal window. Thus, a medical doctor must be contacted immediately. As we may notice, from a technical point of view ISEQL allows a designer to easily model and detect these kinds of scenarios, by exploiting the cardinality constraints.
Epilepsy prediction and seizure detection For this study, we collected data from the Electronic Medical Record (EMR) of our 102 patients. The creation of computerprocessable algorithms to identify individuals with specific health conditions, diseases, or clinical events from EMR data is often referred to as "computational phenotyping"  [ 30, 31 ]. We have therefore defined the computational phenotype of children with DDs/CP at risk of epilepsy. Predict when a patient is likely to have a seizure before it happens is our next objective. It may allow sufficient time to put measures in place to avoid the face up the crises and the related consequences. We plan to create a Prediction Application to Detect and Predict Seizures (PredictMed-Seizures) that will provide ongoing monitoring with timely prediction of when a seizure is likely to happen.
Measurements and procedures Data from 2005 to 2021 on CP etiology, diagnosis, functional assessments, type of spasticity, epilepsy, and clinical history were collected from the medical records by members of the multidisciplinary team which included pediatricians, neurologists, orthopedic surgeons, physiotherapists, child psychologists and epidemiologists. Narrative notes were coded and entered into an electronic database  [6]. Assessments and data collection for the implementation of the "PredictMed" model were conducted in the last 6 months of 2018, and data analysis began in December 2019 and lasted 24 months. The functional assessments used are listed in the Appendix A (Tables  A   [6]. CP etiology was classified as prenatal (genetic, cerebral malformation, infection or vascular), peri-natal (anoxic, ischemic or infectious) or postnatal (cranial trauma, infectious, epilepsy or post-natal anoxic/ ischemic injury)  [6].
Statistical analysis and prediction model The guidelines of the "Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis" (TRIPOD) Statement were followed  [ 34 ]. All data were analyzed in anonymous form from a database including demographics, functional diagnosis, neurologic and cognitive assessments (Appendix A, Table  A. 3). Before implementing our algorithm we split our data into Training Data (70%) and Validation Data (30%). As first step to generate the predictive model, we divided our cohort in 2 groups (children with DDs/CP with and without epilepsy). The potential predictors' assessed were DDs/CP etiology (ET), intellectual disability level (ID), type of spasticity (SP), presence of scoliosis (NS), feeding ability (F), sex (SE), GMFCS, EDACS, CFCS (C), and autistic features (A).
The initial analysis included contingency tables  [6]  and Fisher's exact tests to identify associated factors, confidence intervals, and distribution frequencies. We calculated the odds ratios (logarithmic and linear), 95% confidence intervals and the z statistics  [6]  (Appendix A, Figure A. 7) using the OpenEpi software, a web-based epidemiologic calculator, and the MedCalc®statistical software  [6]. As second step, we generated all possible combinations (tuples) of the 10 predictors assessed with a total of 511 combinations/tuples (each tuple having a minimum of 1 and a maximum of 9 variables). As third step, we performed a logistic regression for each tuple and assessed its predictive performance. The significant variables ( p < 0. 2 ) were used as input variables in a bespoke multiple regression model using the open source software R  [6]  with the glm() function to predict each patient's probability of having epilepsy.
In accordance with the statistical learning theory described by Vapnik  [6]  , we split the patients' data in a "training set" and a "test set". We trained the logistic regression algorithm on a "training set" of 82 patients to predict the probability of having epilepsy for a new patient (belonging to a "test set" of 20 patients) by using the values of a tuple combining a sub-group of the 10 predictors: ID, A, ET, SP, SE, GMFCS, F, NS, EDACS, and C. For example, in the case of the 6-elements tuple ID + A + SP + ET + SE + GMFCS, we used the datasets of all 102 patients split into a training and a test set to build a logistic regression model to predict epilepsy based on the variables: ID + A + SP + ET + SE + GMFCS. To minimize the dependency from the compositions of training and test sets, we used cross-validation by randomly generating 20 different couples of training and test sets.
However, future studies are needed to validate diagnosis of CP disorders within administrative databases  [ 42 ]. The best machine learning model score performed an accuracy of 74%, sensitivity of 98%, specificity of 73%, and 82% average score. This particularly high sensitivity could allow clinicians to rule out an inadequate diagnosis of epilepsy in case of absence of obvious sign, or to predict it during medical follow-up with caregivers. The validity of the model is confirmed by the fact that the other parameters associated with epilepsy (spasticity, intellectual and motor disability) are in line and match with current literature  [ 7, [37] [38] [39] [40] [41]. The strengths of this study are the multicenter evaluation, no missing data and the cross-validation analysis. Our next step is to modify the model to perform also multi-class classification to work with dependent variables having more than two values as multiform types of epilepsies.
Conclusion We defined and prototypically implemented the PredictMed-MHS framework, which includes tools to efficiently and effectively detect and predict seizure to support the clinical decisions making, and to monitor and support patients. This framework is based on Multi-Agent Systems and Complex Event Processing, exploiting runtime data coming from wearable sensors or directly from the patient/caregiver, and encompassing "PredictMed-Epilepsy", a prediction model able to identify factors associated with epilepsy in children with DDs/CP. We illustrated the general architecture of the system and, more specifically, discussed in detail the function of the various components (agents) that we have already implemented, and of the Complex Event Processing module. We plan to thoroughly test all the functionalities of the integrated system on real patients and eventually integrate the system into the National Health System.
This paper adapts two existing strategies used in multivariate DTW, namely, Independent and Dependent DTW, to several commonly used elastic similarity measures. Using 23 datasets from the University of East Anglia (UEA) multivariate archive, for nearest neighbour classification, we demonstrate that each measure outperforms all others on at least one dataset and that there are datasets for which either the dependent versions of all measures are more accurate than their independent counterparts or vice versa. This latter finding suggests that these differences arise from a fundamental property of the data. We also show that an ensemble of such nearest neighbour classifiers is highly competitive with other state-of-the-art multivariate time series classifiers.
Many of these measures have only been defined for univariate time series. However, many significant tasks involve multivariate time series. This paper extends to the multivariate case seven univariate elastic similarity measures, and demonstrates that each supports strong nearest neighbour classification of different datasets. One elastic measure that has previously been extended to the multivariate case is DTW  (Shokoohi-Yekta et al, 2017). That work identified two key strategies for such extension. The independent strategy applies the univariate measure to each dimension and then sums the resulting distances. The dependent strategy treats each time step as a multi-dimensional point. DTW is then applied using Euclidean distances between these multidimensional points. It was shown that each of these strategies outperformed the other on some tasks.
Main Definitions Definition 1 Time Series A time series T of length L is an ordered sequence of L time-value pairs T = (t 1 , x 1 ), • • • , (t L , x L ) , where t i is the timestamp at sequence index i, i ∈ {1, • • • , L}, and x i is a D-dimensional point representing observations of D realvalued variables or features at timestamp t i. Each time point x i ∈ R D is defined by {x 1 i , • • • , x d i , • • • , x D i }. Usually, timestamps t i are assumed to be equidistant, and thus omitted, which results in a simpler representation where T = x 1 , • • • , x L. A univariate (or single-dimensional) time series is a special case where a single variable is observed (D = 1). Therefore, x i is a scalar, and consequently, T = x 1 , • • • , x L. Definition 2 Time Series Dataset A labelled time series dataset S consists of N labelled time series indexed by n, where n ∈ {1, • • • , N }.
Definition 3 Time Series Classification In a Time Series Classification (TSC) task, a time series classifier is trained on a labelled time series dataset, and then used to predict labels of unlabelled time series. The classifier is a predictive mapping function that maps from the space of input variables to discrete class labels. In this paper, to perform TSC tasks, we use 1-nearest neighbour (1-NN) classifiers, which use time series specific similarity measures to compute the nearest neighbours between each time series.
Definition 4 Similarity Measure A similarity measure computes a real value that quantifies the degree of similarity between two sets of values. For two time series Q and C, the similarity measure M is defined as M (Q, C) → R (1) All similarity measures used in our work compute a non-negative real value (i. e. R + 0 ). In many cases, a similarity measure is more useful with the following four properties, since they help with indexing and fast look up from large time series databases  (Chen and Ng, 2004). Similarity measure M is considered to be a metric if it has the following properties: 1. Non-negativity: M (Q, C) ≥ 0, 2. Identity: M (Q, C) = 0, if and only if Q = C, 3. Symmetry: D(Q, C) = D(C, Q), 4. Triangle Inequality: D(Q, C) ≤ D(Q, T ) + D(T, C) for any time series Q, C and T.
Deep learning methods include Time Series Attentional Prototype Network (TapNet)  (Zhang et al, 2020) , ResNet and InceptionTime. A recent paper for multivariare TSC algorithms  (Ruiz et al, 2020)  presents a comparison of most of these methods on the same set of datasets we use in this paper. The review compared 12 classifiers on 20 UEA multivariate datasets with equal length and which completed in a reasonable time. They found that the most accurate multivariate TSC algorithms are ROCKET (ranked 3. 8), InceptionTime (ranked 5. 15), MUSE (ranked 5. 25), CIF (ranked 5. 85) followed by HIVE-COTE (ranked 5. 9) in that order  (Ruiz et al, 2020, Figure   12 ).
Dependent Multivariate LCSS Definition 10 Dependent LCSS Dependent LCSS (LCSS D ) is similar to Equation  15 , except that to compute distance between two multivariate points we use Equation 8 and an adjustment is made to the parameter (see Section 3. 3. 3). In this case, parameter is selected using the standard deviation of the whole dataset. This is similar to the way it was selected in the univariate LCSS, in EE. ∆ LCSS D (i, j) =        L 2 (q i , c j ) 2 if i = 1, j = 1 1 + ∆ LCSS D (i -1, j -1) if L 2 (q i , c j ) 2 ≤ 2 • D • max ∆ LCSS D (i -1, j) ∆ LCSS D (i, j -1) otherwise, ( 18 ) LCSS U N D (Q, C) = ∆ LCSS D (L, L), (19) Similar to the univariate case, LCSS U N D is then normalised based on the series length L. LCSS D (Q, C) = 1 - LCSS U N D (Q, C) L. (20).
Dependent ERP Definition 12 Dependent ERP We define Dependent ERP (ERP D ) as, ∆ ERP D (i, j) = min    ∆ ERP D (i -1, j -1) + L 2 (q i , c j ) 2 ∆ ERP D (i -1, j) + L 2 (q i , g) ∆ ERP D (i, j -1) + L 2 (c j , g), (28) ERP D (Q, C) = ∆ ERP D (L, L). (29) In Equation  28 , note that the parameter g is a vector which represents the standard deviation of each dimension separately. This is in contrast to the univariate case in Equation 26 which uses the standard deviation of the whole training dataset (parameter g). In this case, all terms increases proportionally with respect to the increase in the number of dimensions. So we do not need to adjust for the parameter g as we adjusted for in LCSS in Section 3. 3. 3.
Experiments We conduct experiments to investigate three hypotheses. The first hypothesis is that there will be datasets for which each of the new multivariate distance measures is best suited. The next hypothesis arises from the observation that there are some datasets for which either the independent or dependent version of DTW are consistently more accurate than the alternative  (Shokoohi-Yekta et al, 2017). However, it is not clear whether this is a result of there being an advantage in treating all dimensions in lock step or the reverse, or due to some other property of the algorithms. It is credible that there should be some time series data for which it is beneficial to treat multiple variables in lock step. Suppose, for example, that the variables each represent the throughput of independent parts of a process and the quantity relevant to classification is aggregate throughput.
In this case, the sum of the values at each point is the relevant quantity. In contrast, if classification relates to a failure in any of those parts, it seems clear that independent consideration of each is the better approach. We seek to assess whether there are multivariate datasets for which each of lockstep and independent analysis are best suited, or whether there are other reasons, such as their mathematical properties, that underlie the systematic advantage on specific datasets of either DT W I or DT W D. The third hypothesis, inspired by EE  (Lines and Bagnall, 2015) , is that an ensemble of nearest neighbour classifiers, each using a different multivariate distance measure, will be more accurate than any single nearest neighbour classifier using a single distance measure. We start by describing our experimental setup and the datasets we used.
Experimental Setup We implemented a multi-threaded version of the multivariate similarity measures in Java. We also release the full source code in the github repository https:// github. com/dotnet54/multivariate-measures. In these experiments, for parametrization of the measures, we use leave-oneout cross-validation of 100 parameters for each similarity measure. We follow the same setting proposed in  (Lines and Bagnall, 2015). This parametrization is also used in HIVE-COTE, PF and TS-CHIEF. In this study, we use multivariate datasets obtained from https://www. timeseriesclassification. com. We also use the standard train/test splits provided in the repository. Out of the available 30 datasets, we use 23 datasets in this study. Since we focus only on fixed-length datasets, the four variable length datasets ("CharacterTrajectories", "InsectWingbeat", "JapaneseVowels", and "SpokenArabicDigits") are excluded from this study.
We also we omit "Eigen-Worms","MotorImagery" and,"FaceDetection", which was taking too long to run the leave-one-out cross-validation for 100 parameters in a practical time frame. Table  2  (on page 26) summarises the characteristics of the 26 fixed length datasets. Further descriptions of each dataset can be found in  Bagnall et al (2018). We ran experiments for all measures for z-normalised and unnormalised datasets. We z-normalised each dataset on a per series, per dimension basis. We found that the accuracy on these datasets are better when used as provided in the archive without normalisation (Note that 4 datasets on the archive are already normalised). This is also in agreement with a recent paper which conducted a similar experiment with normalisation using DT W I and DT W D  (Ruiz et al, 2020).
However, we note that this does not indicate that not normalising is always the optimal solution for all datasets. Sometimes normalisation can be useful when using similarity measures. For example, consider a scenario with two dimensions temperature (e. g a scale from 0 to 100 degree Celsius) and relative humidity as a percentage (between 0 and 1). In such a case, temperature will be dominating the result of the similarity calculation, and normalisation will help to compute the similarity with similar scales across the dimensions. We ran the experiments on a cluster of Intel(R) Xeon(R) CPU E5-2680 v3 @ 2. 50 GHz CPUs, with each experiment run on 32-threads. Total time to train all 23 datasets with leave-one-out-cross-validation was about 1648 hours. The two slowest datasets were "PEMS-SF" (291 hours) and "PhonemeSpectra" (1153 hours) The slowest measure to train is M SM D , which took a total of took 801 hours across all datasets.
Accuracy of Independent Vs Dependent Measures First we look at the accuracy of each measure used with a 1-NN classifier. Tables  3  and 4  (on page 27 and 28) present the accuracy for independent measures and dependent measures, respectively. For each dataset, the highest accuracy is typeset in bold. Of the values reported in Tables  3  and 4 , accuracy for measures other than than Euclidean distance (labelled "L 2 " in the table) and DTW are newly published results in this paper. Our first observation is that for every similarity measure there is at least one dataset for which that measure obtains the highest accuracy. This supports our first hypothesis, that each measure will have datasets for which it is well suited. To compare multiple algorithms over the multiple datasets, first a Friedman test is performed to reject the null hypothesis.
Hence, we refer to these visualisations as "average accuracy ranking diagrams". In Figure  2 , DT W I , which is to the further right, is the most accurate measure on the evaluated datasets. DT W I obtained a ranking of 7. 326 from the Wilcoxon test. By contrast, DDT W F D (ranked 14. 783) is the least accurate measure on these datasets. After Holm-Bonferroni's correction, computed p-values indicate that only the pairs DDT W F D and DT W I , and DDT W F D and W DT W D , are statistically different from each other. This could be because there is a large number of measures, which win or lose on similar datasets. 4. 3 Are Independent and Dependent Measures Significantly Different or Similar? In this section, we test if there are datasets for which independent or dependent version is always more accurate.