dataesr/scientific-paragraphs-categorization huggingface dataset:
1a, b, c ). This study represents the first stage in testing the device on patients in real-life situations to evaluate its technological capacities and usability by the patient, before applying and testing this technology in current care. Using the Smart Angel™ app (Evolucare, France), we hypothesized that this system would be able to record and transmit seven variables (pain, quality of recovery, nausea, vomiting, HR, MAP and SpO 2 ) for the first 5 days following ambulatory surgery. Before transmission to the website, these data were also reported by the patient on a paper form. The primary outcome was the proportion of clinical variables (self-monitored physiological data + questionnaire scores) transmitted correctly to the hospital via the IT system. This proportion was also compared with the number of time-matched measurements simultaneously recorded by the patients on the paper form as instructed.
Methods This was a single-cohort, open, prospective trial conducted at a French University Hospital (Hôpital Carémeau, CHU Nîmes, France). In accordance with the current French law and the Declaration of Helsinki, this study was approved by the institutional human investigation committee (Comité de Protection des Personnes, Sud Est V, Grenoble, France: 2017, A02790-53) and registered on ClinicalTrials. gov (NCT03464721; March 8, 2018) before starting  [15]. Written informed consent was obtained from all participants before inclusion.
1a, b, c ). 2 Remote physical parameters (HR, SpO 2 , MAP) with ranges (min. , max. ) for normal values were defined before starting the study and included in the algorithm (Fig. 1c ). In the event of a specific pathology or treatment (e. g. patient on beta-blockers), ranges can be adapted to specific patient characteristics and/or treatment. 3 The tablet and app technology uses a tablet (Samsung™, Korea) integrating the software (EvolucareLabs, France) which generates health questionnaires for the patient to answer with scores for pain relief, quality of recovery, nausea and vomiting (see above). This tablet communicates with a connected monitor at the patient's home to perform discontinuous measurement. The app assesses the self-report questionnaires and the wireless Bluetooth monitors record the physiological measurements (heart rate, mean arterial blood pressure and blood oxygen saturation).
The monitors used were: a wireless pulse oximeter clipped to the finger (iHeathlabs, USA) and a blood pressure monitor on the wrist (iHeathlabs, USA). When the patient is ready with the monitor, the software triggers the monitors and records the values. The patient can instantly see the tablet screen. Measurement data for each item is depicted with normal and abnormal ranges. All these data are then exported via the web (4G collection) to a secure server (Adista™, France). -Follow-up: From the day of surgery to postoperative Day 5, three times a day (morning, noon and evening), the application prompts the patient to complete the health questionnaire and follow-up with the monitors. Seven variables are recorded for each assessment: At the end of our 5-day study, patient monitoring was stopped and the equipment was returned to the hospital by express delivery.
Outcomes and data collection Surgical, anaesthetic and patient characteristics data were collected by the research nurse and investigators. Seven variables (pain, quality of recovery, nausea, vomiting, HR, MAP and SpO 2 ) were recorded by the app for all patients before their discharge from the centre and then three times a day from Day 1 to Day 5. In addition, the number of time-matched variables simultaneously recorded by the patients on a paper form was returned to the centre at end of the study. On Day 5, the patient had to answer a 10-item questionnaire with five response options ranging from "Strongly agree" to "Strongly disagree" (total: 1-100 points) and the result was converted into a System Usability Scale (SUS) based on a Lickert scale. The 10 items were: -I think I will use the SMART ANGEL device frequently.
Statistical analysis Statistical analysis was conducted using SAS (9. 4, SAS Inc. , Cary NC). Statistical results were expressed with mean (SD) or median with interquartiles [IQ] according to distribution. The numbers (with percentages, %) were given for categorical variables. The main judgment criterion was analysed in relation to a referential volume of theoretical information based on the following calculation: number of patients (n = 30) x number of data collection periods (i. e. one on Day 0 and 3 per day from Day 1 to Day 5, i. e. 16 in total) x number of parameters measured i. e. physiological parameters (heart rate, blood pressure, oxygen saturation and self-evaluation parameters (pain score, nausea, vomiting, comfort), i. e. 7 in total. Thus, the maximum reference volume of theoretical information is 30x16x7 = 3360.
Primary outcome For 29 patients, 3248 (29x16x7) data items were to be collected on paper or by remote monitoring technology. The remote monitoring technology recorded 2038 data (62%) (Table  1 ). The conventional paper form recorded 2656 (82%) data. The difference between remote monitoring and paper form was statistically significant (p = 0. 001) (Table  1 ). Figure  2a  and b  show the percentage of data recorded by patients per day at each time period on paper and via the web-solution, respectively.
2 Malfunctioning of the SmartAngel™ program installed on the tablet. Computer program patches were required to stabilize the program for 3 patients as it had failed to activate the remote MAP monitors (confidential data provided upon request). 3 Insufficient battery charge for discontinuous but frequent use of the monitors. This is crucial for patients and they must be informed of the need to charge up the batteries regularly. For the 62% of patients for whom all the data were correctly transmitted (Fig. 2b ), the Smart Angel™ tool represented a truly original monitoring dashboard which had never been proposed before, providing questionnaire data combined with physiological data. The number of patients included was insufficient to demonstrate the interest of the system as an aid to follow-up, but the data presented are a step forwards in ambulatory follow-up and would be useful for a multicentric study.
Our study shows a decrease in the data collected on paper on Days 4 and 5 (Fig. 2a ). This may suggest that optimal monitoring should be begun from the evening of Day 0 to the evening of Day 3 (with systematic measurements morning, noon and evening) and at least one measurement per day from Day 3 onwards to optimize patient adherence. Interestingly, the peak of postoperative complications classically occured between Day 1 and Day 3  [7]. In this study, the patient was monitored over the 5-day postoperative period without any manipulation from the expert centre. The high SUS score suggests good acceptance and usability by the patients  [21]. Indeed, they willingly accepted the small-sized connected objects and the fact that there was a bag to transport all the objects home probably facilitated acceptance of the device.
Acknowledgements The authors wish to thank the following people for their contribution: - Dr. Emilie Loup-Escande  for her help in developing the ergonomic part of the applications in relation with the  University of Picardie Jules Vernes, Amiens, France, UR 7273 CRP-CPO, Université Picardie Jules Verne, Chemin du Thil, 80000 Amiens, France  (emilie. loup-escande@u-picardie. fr). -Teresa Sawyers, British Medical Writer at our institution, for revising the manuscript.
Construction of graph-based approximations for multi-dimensional data point clouds is widely used in a variety of areas. Notable examples of applications of such approximators are cellular trajectory inference in single-cell data analysis, analysis of clinical trajectories from synchronic datasets, and skeletonization of images. Several methods have been proposed to construct such approximating graphs, with some based on computation of minimum spanning trees and some based on principal graphs generalizing principal curves. In this article we propose a methodology to compare and benchmark these two graph-based data approximation approaches, as well as to define their hyperparameters. The main idea is to avoid comparing graphs directly, but at first to induce clustering of the data point cloud from the graph approximation and, secondly, to use well-established methods to compare and score the data cloud partitioning induced by the graphs.
In particular, mutual information-based approaches prove to be useful in this context. The induced clustering is based on decomposing a graph into non-branching segments, and then clustering the data point cloud by the nearest segment. Such a method allows efficient comparison of graph-based data approximations of arbitrary topology and complexity. The method is implemented in Python using the standard scikit-learn library which provides high speed and efficiency. As a demonstration of the methodology we analyse and compare graph-based data approximation methods using synthetic as well as real-life single cell datasets.
Introduction Graph theory based methods play a significant role in modern data science and its applications to various fields of science, including bioinformatics. One of the applications of graphs in unsupervised machine learning is for dimensionality reduction, where the geometrical structure of a data point cloud is approximated by a system of nodes embedded in the data space and connected by edges in a more or less complex graph. For example, this graph can represent a regular grid as in Self-Organizing Maps (SOM)  [1] , Elastic Maps (ElMap)  [2, 3]  or principal curve approaches  [4, 5]. In this case, the graph can serve as a base for reconstructing a low-dimensional principal manifold  [3, 6]. However, in some applications of graph-based approximation, the geometry of a dataset can be characterized by more complex structures such as branching points, excluded regions and regions of varying local intrinsic dimensionality  [7] [8] [9].
For example, some large-scale clinical or single cell omics datasets can be modeled as a set of diverging and bifurcating trajectories connecting a root state (e. g. , a progenitor cell) with a number of final states (e. g. , corresponding to fully differentiated cells). In this case, a more complex graph than regular grid graphs should be used for efficient data approximation. Graph-based data approximation (GBDA) constructs a "data skeleton" which can be used to simplify, compress and denoise the complete data description. A challenging question in GBDA is to define the appropriate graph topology, which should be more complex than a regular grid and less complex than a general graph with arbitrary topology (in the latter case we might end up with a graph as complex as the data themselves).
One evident example of such a compromise consists in approximating the data point cloud by a graph with tree-like topology. A more advanced approach consists in introducing cubic complexes as more sophisticated graph types  [10]. The task of approximating a data point cloud by a graph can be split into two interconnected subtasks. The first subtask is to define optimal (in some mathematically defined sense) embedment of graph nodes in the multi-dimensional space, when the graph topology is fixed. The second, much more challenging subtask is to find the optimal (in some sense) graph topology matching the dataset structure. Several mainstream ideas have been suggested for solving these problems: one consists in exploiting the properties of Minimal Spanning Tree (MST)  [11]  and another consists in building principal graphs (PG)  [12].
The latter approach can be either equipped with a systematic search for the optimal graph topology as in the topological graph grammar approach  [10]  or using some heuristics to make an intelligent guess on the appropriate topology as in SimplePPT  [13]  or so called reversed graph embedding approaches  [14]. One particular important application of GBDA approach is the computational data analysis of single-cell omics datasets  [15]. A number of new methods have been developed in this field combined under the name cellular trajectory inference (TI)  [16]. TI methodology provides a useful tool for studying cell differentiation, cell cycle and other central biological processes with applications in cancer  [17]  and developmental biology  [18]. These methods essentially create a graph-based approximation of a data point cloud where each data point represents a molecular profile of a single cell allowing one to arrange the cells along the graph edges and branches, locally ranking the cells and thus obtaining insights into the underlying biological processes.
The availability and the complexity of single cell omics datasets serves a source of motivation for us here. The present study is devoted to improving and benchmarking existing GBDA approaches. The question of benchmarking the GBDA methods is significant since, for example, only in bioinformatics dozens of new GBDA methods have been introduced in the recent years. Most of them build linear, tree-like or simple circular graph topologies but among these methods there exist those building more general graph types (such as partition-based graph abstraction (PAGA)  [19] ). It is desirable to understand their advantages and disadvantages and to formulate rules for choosing the most appropriate ones for a given dataset or datatype (e. g. , gene expression vs. other types of experimental assays). This question has been mentioned in a recent review "Eleven grand challenges in single-cell data science"  [15] : "A pressing challenge is to assess how the different trajectory inference methods perform on different data types and importantly to define metrics that are suitable.
e. , assign each data point to the nearest graph segment (Figure  1 ). Each approximating graph topology and geometry "induces" certain clustering (data point partitioning) of a dataset. At the third step we suggest using the well-established methods to compare two clustering results (such as Rand or other scores). Since the induced clusterings are based on the graphs, one thus obtains a score how different/similar these graph approximations are. In addition, at the last step one can use the standard clustering quality scores (such as silhouette  [22] ) which measures how good is the clustering itself. In this way, one can adjust the (meta)-parameters of the graph approximation methods choosing those with the largest clustering quality score. The graph segments mentioned above may be thought as connected components of a graph without any branching point.
The exact definition is provided below in the main text. The proposed scores possess the main desired property for GBDA method benchmarking: They depend on "trajectories" rather than on "graphs", meaning that adding new nodes or rearranging them will not change the scores if "trajectories" remain unmodified. We argue that the proposed methodology provides a convenient scoring for comparing GBDA approaches as well as for parameter fine-tuning. It is implemented using Python sklearn library which provides highly standardised and efficient set of tools for calculation clustering quality and clustering comparison scores. In order to demonstrate the effectiveness of our approach we compare two mainstream GBDA methods, one is based on minimal spanning trees and another one based on application of elastic principal graphs.
Comparing and Benchmarking Graph-Based Data Approximation Methods Using Data Point Partitioning by Graph Segments In order to benchmark the graph-based data approximation (GBDA) methods, a standard strategy consists in creating synthetic datasets with known underlying ground truth graph topology as a generative model, applying the methods and comparing how far the reconstructions are from the ground truth graph model. The contribution of the current paper is an alternative approach for comparison of two graph-based approximations. Our proposal has certain advantages comparing with the approaches used before. First of all it compares "trajectories" rather than "graphs" meaning that (omitting the technical details) adding more nodes to the graph (or rearranging them), but preserving the same "trajectories" does not change scores.
Outlining the Method In order to compare (i. e. , calculate similarity score) two graphs approximating given dataset, we perform three steps: 1. Split each approximating graph into segments. By a segment we mean certain path from one branching point or leaf node to another. 2. Create clustering (partitioning) of the dataset by segments. Each data point is associated with the nearest segment of the graph. Thus, in general, if graph is partitioned into N segments at the previous step then the dataset will be partitioned into N clusters. 3. Compare the clusterings for the dataset using standard metrics (like adjusted Rand index or adjusted mutual information or any other suitable measure). Since the clusterings are produced from the structure of the approximating graphs, the score shows how two GBDA results are similar to each other.
For a graph which is an isolated cycle (not containing branching or leaf nodes), the whole cycle will be considered as a "segment". The same is true if a graph contains several connected components which are cycles: Then all of them are considered as separate "segments". The other exceptional case is case of nodes of degree zero (isolated nodes); they also will be considered as separate "segments". These exceptional cases cannot happen for connected principal trees which are the main object of the present study, they are just mentioned here for completeness. It is not difficult to prove that: Any graph can be uniquely partitioned into such type of segments. One can get the proof by a mild formalization of the argument with connected components mentioned above. The other way is to describe a constructive process to obtain such partitioning of the graph that is done in our Python implementation of the method.
We coded in Python a version of the depth-first search algorithm to produce a split into segments for an arbitrary graph. The main difference to the classical depth-first search is a storage of visited edges (not only nodes) of the graph to correctly process possible cycles in the graph. The algorithm starts from any branching or leaf node, and walks along edges in depth, joining them to the "current segment" until it meets a branching or a leaf node. Here, the "current segment" is terminated. In case of a leaf node one returns from the recursion, the same for already visited branching node. In case of a new branching node (not visited before) one goes into deeper level of depth-first recursive process.
Details on Step 2: Clustering (Partitioning) Data Points by Graph Segments Having obtained an approximating graph split into several segments (see the previous step), we can cluster the dataset by these segments. This means that for each datapoint we find the nearest segment. There is slight ambiguity how that can be done, however that ambiguity would not much effect the results (at least for the graphs quite many nodes) and so, for our purposes we choose the simplest way to define the proximity. We calculate the distances between all datapoints and all nodes of the graph. If the nearest node belongs to only one segment then the datapoint gets it as a label. If there are several segments to which the nearest node belongs, then we take the second nearest node (among nodes from the selected segments) and label the data point by the segment to which that second nearest node belongs to.
The algorithm is implemented in Python, Figure  2  illustrates its work. The alternative way to define the proximity would be for each datapoint calculate distance not only to nodes, but also to find nearest point on the graph (which might be somewhere at some edge, not node) and since all edges belong to segments-that segment give a label to that datapint. That way is theoretically better, however practically computationally more constly and the difference between the two methods would not be big for datasets of our primary concern-single cell data-which are quite noisy. Thus in what follows we consider the proximity defined as above. Let us also mention that with any way to define the proximity there can be some ambiguity-some points may be on equal distance from several segments; however, the set of such points would typically be of some measure zero, i.
Details on Step 3: Compare the Clustering Measures for the Datasets Having produced clustering of the dataset by several approximating graphs (see previous steps) one can use the well-established and efficiently implemented in Python sklearn library scores for clustering comparison. From our construction it follows that if the graphs are similar to each other then the clustering will be similar also, and the scores will reflect it, while dissimilar graphs will lead to dissimilar clusterings and the scores will show it. Thus these scores give efficient measure of similarity for graphs, for both geometry and topology at the same time. There are many scores used to compare clustering: Adjusted Rand index  [23] , adjusted mutual information  [24] , Fowlkes-Mallows score  [25] , V-measure  [26] , etc.
According to our experiments in most cases all the scores show similar results. So as a default choice we would suggest adjusted Rand index. Let us mention that "adjusted" (adjusted Rand index, adjusted mutual information) indicates a modification of the score such that for two random data clustering the score becomes near zero, that is a desirable property and such score modifications should be used by default. Figure  2  shows examples of graph comparison. One can see that all the scores lead to the same conclusions which perfectly fit the intuition. The suggested approach requires a dataset for which the compared graphs serve as an approximation, thus it does not allow to compare graphs on their own. It might seem as a disadvantage, but actually it is a way around for main applications, since the approach would give a more focus and weight for the regions of the data space where more data points are concentrated.
Unsupervised Scores for Comparing GBDA Methods and Parameter Fine-Tuning Another application of our approach is to provide a score for how good a given approximating graph is for a given dataset. In real situation there is no "ground truth" so having several GBDA methods (and typically many hyper-parameters for each of them) it is not always easy to choose which is the best one. Our suggestion provides certain insights how the question can be approached. The main idea is to transfer the methods used to score clustering to the GBDA area. We do not claim that the method always works very well, but we will demonstrate that it makes sense at least in some cases. That situation is similar to clustering setup, where choice of appropriate cluster number or a clustering method is frequently a subtle question for real-life datasets.
The approach is very similar to what is described in the previous section. To calculate a score of goodness of a given approximating graph for a given dataset we perform three steps: 1. Split each approximating graph into segments 2. Create clustering (partitioning) of the dataset by the segments 3. Use standard metrics and tools to calculate how good is the clustering:, e. g. , calculate silhouette, Calinski-Harabasz, Davies-Bouldin score etc. The first two steps are exactly the same as described in details in the previous section, so we do not comment on them here. The last step is standard for clustering techniques: There are several scores which provide certain estimates how good a clustering is. They are efficiently implemented in Python sklearn library. It should be noted that all three indices work better for convex clusters, rather than for other cluster shapes-that is a disadvantage in certain situations.