dataesr/scientific-paragraphs-categorization huggingface dataset:
Here we propose a new machine learning algorithm for classification of human activities by means of accelerometer and gyroscope signals. Based on a novel hierarchical system of logistic regression classifiers and a relatively small set of features extracted from the filtered signals, the proposed algorithm outperformed previous work on the DaLiAc (Daily Life Activity) and mHealth datasets. The algorithm also represents a significant improvement in terms of computational costs and requires no feature selection and hyper-parameter tuning. The algorithm still showed a robust performance with only two (ankle and wrist) out of the four devices (chest, wrist, hip and ankle) placed on the body (96. 8% vs. 97. 3% mean accuracy for the DaLiAc dataset). The present work shows that low-complexity models can compete with heavy, inefficient models in classification of advanced activities when designed with a careful upstream inspection of the data.
The proposed algorithm relies on a novel hierarchical system and a relatively small set of highly informative features extracted from the filtered signals, and was evaluated against the public Daily Living Activity (DaLiAc) dataset presented below  [20]. Because this algorithm was specifically designed for the classification task of the DaLiAc dataset, we further assessed its generalizability by testing it against another dataset, the mHealth dataset  [23]. Finally, given that many popular activity monitors (e. g. , ActiGraph or ActivPal in health studies) are not equipped with gyroscopes, we assessed the usefulness of adding gyroscopes to the accelerometers by comparing the performance of the algorithm when gyroscope data were included and when they were not. Sensors 2020, 20, 3090 3 of 12.
The DaiLAc Dataset The DaLiAc (Daily Living Activity) dataset consists of the signals of accelerometers and gyroscopes placed on the chests, wrists, hips and ankles of 19 adults performing thirteen daily activities in semi-controlled conditions. The activities include a wide range of simple and complex activities: lying, sitting, standing, dish washing, vacuum-cleaning, sweeping, walking, running, ascending stairs, descending stairs, bicycling with a resistance of 50 Watts, bicycling with a resistance of 100 Watts and rope jumping. Details about the subjects and the experimental designs can be found elsewhere  [20].
Classification Classification was done using a two-level hierarchical system, as illustrated in Figure  1. For all classification tasks in the system, the following classifiers were tested: LR (with a L2 regularization and a penalty coefficient equal to one); KNN with k = 5; gradient boosting (GB) (500 estimators, selecting 10 features at a time); and SVM. For additional comparability, a convolutional network was also tested (architecture in Figure  2 ) taking as input the four signals (AC, DC, accelerometer and gyroscope) and their Fourier transform. Classification was done using all 15 possible combinations of device locations on the subjects' body (e. g. , ankle, ankle + chest and ankle + chest + wrist). We used Python's Scikit-learn  [26]  and Tensorflow  [27]  libraries for the analysis, and unless otherwise specified, their default parameters.
Evaluation Method In order to evaluate the performances of the proposed models, a leave-one-subject-out procedure was followed: models were tested against data from one subject after being trained on all others, for each subject of the 19 subjects in the dataset. This procedure was adopted by the first study on the dataset and followed by several subsequent studies (Table  1 ). Reserving a fraction of each subject's data for testing instead a fraction of the subjects themselves can result in an upward bias of the estimate of the performance metric, since models learn the patterns that are specific to the subjects and can better classify them during testing. Moreover, averaging scores of all iterations in a leave-one-subject-out procedure is preferable to a single hold-out test on a several subjects, as it reduces bias in the accuracy estimator, especially in small datasets  [20].
Generalization on the mHealth Dataset The algorithm presented in this article was designed to address the specific classification task of the DaLiAc dataset. It was therefore deemed desirable to validate this algorithm on other data, collected in different conditions and presenting a different classification task. To do so, we used the algorithm on the mHealth dataset  [23]  that contains labelled body-worn accelerometer, gyroscope and magnetometer signals collected while subjects were performing different activities. The accelerometer, gyroscope and magnetometer sensors were placed on the lower arm and the ankle. In addition, a device placed on the chest recorded accelerometer data only. Data for the activities were collected in an out-of-the lab environment with no constraints on the way activities must be executed; subjects were asked to try their best when executing them.
The activities were the following: standing still, sitting and relaxing, lying down, walking, climbing stairs, bending the waist forward, frontal elevation of arms, bending the knees (crouching), cycling, jogging, running and jumping forwards and backwards. We trained and tested the data using the exact same algorithm, hyper-parameters and validation procedure as those presented here for the DaLiAc dataset. We used a flat classification, since classes seemed clearly distinct from each other.
Results for the DaLiAc Dataset For the five classification models (LR, GB, KNN, SVM and CNN), accuracy is reported for each combination of devices and for each task in the hierarchical system (Table  2 , and in Tables A and B in Supplementary Materials). Overall classification accuracy was highest for LR (based on data from all four devices) with 97. 30% accuracy, followed by GB (all devices) with 96. 94%, SVM (all devices) with 96. 84%, CNN (three devices, ankle, chest and wrist) with 95. 42% and KNN (three devices, ankle, chest and wrist) with 91. 82%. When looking at sub-tasks in the hierarchical classification system, GB is very slightly better than LR in the base-level classification (99. 23% vs. 99. 21%). GB outperformed LR also in distinguishing between standing and washing dishes (97.
40% vs. 97. 06%) and between walking, ascending and descending stairs (99. 08% vs. 98. 72%). When we combined the best classifiers for all sub-tasks, overall mean accuracy rose by 0. 04%. As this improvement remains very marginal, we refer to the system based exclusively on LR as the best algorithm. The confusion matrix for the final classification with LR is shown in Table  3. The training time varied significantly across the models studied. Using Google Colab (with GPU accelerator) and the parameters mentioned above, training and predicting data following the leave-one-out procedure (i. e. , 19 times) for the DaLiAc dataset lasted 4. 5 min with LR and KNN, 7. 2 min for SVM, 10. 7 min for GB and over half an hour for CNN (Table  2 ). The entire feature extraction phase for the 19 subjects (over six hours of observations in total) took about 30 s.
Results for the mHealth Dataset The ability of our algorithm to generalize was further validated on the mHealth dataset, using all accelerometer, gyroscope and magnetometer signals. We obtained very good average scores on the mHealth dataset with GB (98. 7% ± 2. 6%), LR (98. 2% ± 2. 7%) and SVM (97. 2% ± 3. 7%), but less good ones with KNN (92. 7% ± 4. 3%) and CNN (87. 7% ± 7. 5%).
Discussion Compared with previous works tested on the DaLiAc data set, the proposed algorithm, based on careful handcrafted features extracted from the signals, represents a threefold improvement. First, the proposed algorithm performs better than major works tested against the DaLiAc dataset (97. 30% accuracy with LR versus 96. 40% for the best model so far with CNN  [15] ) (see Table  1 ). Likewise, our algorithm with GB and LR yielded less than 2% classification error on the mHealth dataset. By comparison, Jordano et al. [31]  identified seven studies evaluated against the mHealth dataset, and when applying the same leave-one-subject-out procedure, the accuracy for the best algorithm was 94. 66%. Zdravevski et al. [18]  using a hold-out dataset for testing (subjects 7-10) reached 99. 8% accuracy.
By applying the same procedure and the same windowing strategy, we reached an accuracy of 99. 7% with our algorithm (LR). Second, compared to state-of-the-art CNN, the proposed algorithm performed best with fast-training models, such as logistic regression (32 min for the former versus 4. 5 min for the latter). Third, these superior results were obtained with simple and robust tools in machine learning that do not require preliminary hyper-parameter optimization and feature selection, such as LR. In fact, hyper-parameters optimization of classifiers (most notably neural networks) and feature selection can be a daunting, time-consuming task, and was shown to lead to over-fitting and poor generalization  [32]. This was corroborated by the validation of the algorithm against the mHealth dataset.
Simple classifiers based on handcrafted features, which required no or little hyper-parameter tuning, generalized very well on a new dataset, while CNN, which performed well on DaLiAc, for which it was tuned, yielded poor results on mHealth. It is difficult to fully explain how our algorithm outperformed previous algorithms using classical machine learning classifiers by around 4%, as authors do not always specify all the decisions that they make during data processing before reaching the results. Using the DaLiAc dataset, we undertook a few steps to identify the innovations that made our algorithm more accurate. First, running our algorithm with a flat classification system instead of the hierarchical system proposed here resulted in 1. 81% decrease in mean accuracy. Second, by extracting features performed on the acceleration signal only, without including the AC and DC components as we did, the decrease in accuracy amounted to 2.
This will help designers of future studies make an informed decision about the trade-off between cost and accuracy. Despite this promising improvement, two caveats need to be highlighted. The first caveat relates to the nature of our data. HAR algorithms are tested against clean data of activities performed in a characteristic manner as part of a relatively structured protocol. Realistic data, however, can contain fewer characteristic activities (e. g. , slouching) which represent a greater challenge to classify. To that extent, very recent attempts to create benchmark activity datasets simulating real conditions  [33]  are an important development in the field and new algorithms should preferably be assessed using these data. In addition, people in real conditions tend to switch rapidly between activities.
Motivation: During the last decade, trajectory inference (TI) methods have emerged as a novel framework to model cell developmental dynamics, most notably in the area of single-cell transcriptomics. At present, more than 70 TI methods have been published, and recent benchmarks showed that even state-of-the-art methods only perform well for certain trajectory types but not others. Results: In this work, we present TinGa, a new TI model that is fast and flexible, and that is based on Growing Neural Graphs. We performed an extensive comparison of TinGa to five state-of-the-art methods for TI on a set of 250 datasets, including both synthetic as well as real datasets. Overall, TinGa improves the state-of-the-art by producing accurate models (comparable to or an improvement on the state-of-the-art) on the whole spectrum of data complexity, from the simplest linear datasets to the most complex disconnected graphs.
, 2019) ,  StemID (Gru ¨n et al. , 2016)  and Monocle 3  (Cao et al. , 2019). In a recently published paper on TI,  Saelens et al. (2019)  compared 45 of the existing TI methods. Several interesting findings resulted from this study, including the strengths and weaknesses of existing tools as well as possible gaps in the field of TI. A first conclusion from this study was that no existing method was able to return accurate results for all the 350 datasets that were included in the study. Therefore, when facing a new unknown dataset, researchers need to apply several of the state-ofthe-art methods and then compare their results in order to be able to gain biological insight into the data. It could be argued that the methods that can model the most complex trajectories could be applied in general, since they should also be able to model the simplest trajectories.
Adaptive topology modelling using Growing Neural Gas In this article, we introduce TinGa, a fast and flexible TI method. It is the first method that applies the Growing Neural Gas algorithm  [GNG, Fritzke (1995) ] to infer trajectories. The basic idea behind this algorithm is to build a (possibly disconnected) graph that aims to fit a set of data points as well as possible using a graph structure that is iteratively adapted. The algorithm starts by building a graph that consists of two nodes, linked by an edge. An iterative procedure is then applied in which a random cell from the dataset is picked as input at every iteration and subsequently the graph is adapted to the data. An algorithmic description of TinGa is given in Algorithm 1. All nodes have an associated error that is representative of how well each node covers a certain region of the data space.
Datasets For this study, we used 350 datasets that were used in the benchmarking study described in  Saelens et al. (2019) , all of which have a known ground truth trajectory useful for evaluation. A large spectrum of topologies is represented in these datasets, from the simplest linear trajectories to the most complex disconnected trajectories. In Figure  2 , each of the nine possible topology types is represented as a graph. In bifurcations, a simple linear trajectory bifurcates into two branches. Converging trajectories are the exact opposite of bifurcations: two distinct branches merge into one. Trees consist of a succession of different bifurcations. Multi-furcations happen when a simple linear branch splits into more than two branches. Finally, some of the datasets are graphs; they can contain cycles or be acyclic, depending on the direction along the branches.
We have split the 350 datasets in two. Table  1  describes the 100 out of 350 datasets that were used for testing TinGa's robustness to its parameter setting, and fine-tuning of the max_nodes parameter. We then used the remaining 250 datasets to compare TinGa to 4 other TI methods. These 250 datasets contained 9 different types of trajectories, as can be seen in Table  2. Both the 100 datasets used for parameter tuning as well as the 250 datasets used for benchmarking to other methods contained comparable numbers of real and synthetic datasets. The synthetic datasets were generated using four simulators: dyngen  (Saelens et al. , 2019) , which simulates gene regulatory networks, dyntoy  (Saelens et al. , 2019) , which builds random gradients of expression in the reduced space, PROSSTT  (Papadopoulos et al.
, 2019) , which samples the expression from a linear model that depends on pseudotime and Splatter  (Zappia et al. , 2017)  insert a new node r between p and q. 29: errors of p and q are multiplied by a 30: r gets the mean error of p and q 31: p-q edge is removed, p-r and r-q edges are added 32: decrease error of all nodes by factor b 33: post-process the graph. : 34: build a graph from Nodes and Edges 35: apply a minimal spanning tree to the graph 36: identify nodes in the MST of degree 1 37: for each pair of nodes p 1 and p 2 of degree 1: 38: if graph edge p1 À p2 then 39: if edge doesn't result in a triangle then 40: if cell density along edge is sufficient then 41: add edge between p 1 and p 2. datasets were thus generated using these four simulators. The cells in each dataset were then post-processed to match a real dataset's characteristics such as the dropout rate.
Single-cell RNA-seq data preprocessing Real datasets were preprocessed following the standard bioconductor pipeline which uses both the scran and scater Bioconductor packages  (Amezquita et al. , 2020; Lun et al. , 2016). The same settings were used as in  Saelens et al. (2019) , with a filtering that removed genes that were expressed in less than 5% of the cells and had an average expression lower than 0. 02. Cell filtering was applied based on total counts, total amount of features, mitochondrial gene expression and if available, spike-ins, where cells with values higher than the median 6 3 MADs were removed. The most highly variable genes were selected by modelling the mean-variance relationship with a curve, and identifying genes that differed from this curve with a false discovery rate of 5% and a biological component (or effect size) higher than 0.
Benchmarking TinGa to state-of-the-art methods We compared TinGa to four top TI methods, as identified by the large-scale benchmarking study by  Saelens et al. (2019). These are Slingshot  (Street et al. , 2018) , PAGA  (Wolf et al. , 2019) , RaceID/ StemID  (Gru ¨n et al. , 2016)  and Monocle 3  (Cao et al. , 2019). Since the dynbenchmark package  (Saelens et al. , 2019)  contained wrappers for most of these methods, metrics for comparison, as well as 110 real and 240 synthetic datasets on which we could compare the methods, we re-used the same comparison settings. We created one new wrapper for Monocle 3, a method that was not yet included in the dynbenchmark package. Four metrics, earlier described in  Saelens et al. (2019) , were used to assess the performance of the method: • Hamming-Ipsen-Mikhailov (HIM): provides information on the difference in topology between a method's result and a gold standard, by taking into account both the edge lengths and the similarity in node degrees • CORRELATION: provides information on the correlation between the cell ordering in a method's results compared to a gold standard, taking the trajectory structure into account by using geodesic distances.
0001 to 1. • k: the iteration at which a new node can be added. Default: 200. Grid search was performed on values varying from 100 to 500. • age max : the maximum age of an edge before it is removed. Default: 200. Grid search was performed on values varying from 100 to 500. • a: the decay parameter for error when a new node is added. Default: 0. 5. Grid search was performed on values varying from 0. 1 to 0. 9. • b: the value by which all node errors decrease at every iteration. Default: 0. 99. Grid search was performed on values varying from 0. 2 to 0. 999. • max nodes : the maximum number of nodes allowed in the GNG graph. Default: 30. Grid search was performed on values varying from 4 to 30. We tested every resulting parameter setting on 100 randomly sampled datasets among the 350 datasets described in  Saelens et al.
(2019) , which we used as our training set. We then performed paired t-tests to assess whether the mean score of TinGa over the 100 training datasets would change significantly due to parameter tuning. Varying the parameters a, b, k and age max did not significantly change the results of TinGa over these datasets (with a P-value of 0. 05). We noticed that setting too high b and n values decreased the performance of TinGa, and we therefore advise to keep the values of these parameters equal to or lower than 0. 5 and 0. 01 for the b and n parameters, respectively. We believe that the fact that GNG nodes should not be allowed to move excessively under the influence of one cell input makes sense, since this allows the method to be more robust to outlier cells. The only parameter whose default value showed sub-optimal results was the max nodes parameter.
The GNG algorithm was originally designed to learn complex topologies, and the default number of nodes in the graph was set to a relatively high value (with a maximum of 30 nodes). In the context of TI, this number seems inappropriate, as allowing too many nodes in the graph can lead to the appearance of noisy structures, as can be seen in Figure  3. We tested various values for the max nodes parameter, ranging from 4 to the default of 30. The results of TinGa on the 100 datasets that we selected for training with different max nodes values can be seen in Figure  4. We observed that a maximum number of eight nodes was a good trade-off between performance, as assessed by the mean score in the figure, and running time. We set the max nodes parameter to eight and all other TinGa parameters to their default value for the rest of the study.
All TI methods use low-dimensional data as input. In the case of TinGa, we reduced the dimensionality of the count matrix of synthetic and real datasets to five dimensions using multi-dimensional scaling. In order to test the robustness of TinGa's results to the choice of the number of dimensions in the lower space, we tested different numbers ranging from 3 to 50, on the same 100 datasets we had already used for grid search on TinGa's parameters. The number of dimensions didn't have a high impact on TinGa's results, as can be seen in Figure  5. To confirm this, we performed paired t-tests to assess whether the number of dimensions that we selected would change the results of TinGa with regards to other methods. The Pvalues associated with these tests were all higher than 0. 05. TinGa is therefore robust to the number of dimensions in the data, and we fixed this number to 5 for the rest of the study.
Results We compared the performance of TinGa to a set of state-of-the-art methods for TI, namely PAGA, Slingshot, RaceID/StemID and Monocle 3. The performance of all five methods was assessed on 250 synthetic and real datasets offering a wide variety of complexities, from linear to disconnected trajectories. For each of these datasets, the ground truth trajectory is known, since it was either defined experimentally for the real datasets, or extracted from simulations for the synthetic datasets. Therefore, the results of any TI method can be compared to the ground truth trajectory and scored. We performed a comparison using four metrics that we described in Section 2. We report the results of the methods on the 178 synthetic and 72 real datasets separately.
Synthetic datasets TinGa and Slingshot are the methods that found the best cell ordering across all synthetic datasets, as shown by the correlation scores (Fig. 6a ). These two methods also found the best cell assignment across branches (Fig. 6b ). However, Monocle 3 performed better than Slingshot for recovering the topology of the datasets and the features expressed along the trajectory, as can be seen in the boxplots showing the Feature Importance score and the HIM score in Figure  6c  and d , respectively. TinGa, on the other hand, was consistently among the best methods for these four metrics when applied on the synthetic datasets. The scores of RaceID/StemID were greatly affected by the fact that it failed to return results on many datasets. In order to make the comparison of five methods on 250 datasets possible, we set a maximum memory use of 15 Gb for every method on every dataset.
RaceID/StemID systematically ran out of memory on datasets containing more than 5000 cells. Figure  6f  shows the time each method took to run on the datasets in function of the number of cells. All methods returned results in less than 10 s on datasets containing less than 1000 cells, except for RaceID/StemID, which already needed a few minutes on a dataset of 1000 cells. TinGa proved to be very scalable on larger datasets, while Slingshot and PAGA became significantly slower on datasets containing a few thousands of cells. Overall, the TinGa method obtained the best scores when compared to the four currently state-of-the-art TI methods on synthetic datasets, as can be seen in Figure  6e , where the Mean Score is the geometric mean of the four other metrics (Correlation, F1 Branches, HIM and Feature_Importance).