dataesr/scientific-paragraphs-categorization huggingface dataset:
Micro-learning Recently, micro-learning approaches have been successfully applied to graph structures to provide whatif scenarios (e. g. , in the context of power grid management  [10] ). Micro-learning, as explained by  [11] , refers to small fine-grained learning units as opposed to macro-learning which consists of a single coarse-grained learning unit. In these approaches, specific models of local data are built instead of one large model on the overall dataset. Micro-learning is typically useful for incremental predictive models, where one has to perform step-by-step decisions based on local properties, while yielding quality predictions. For instance for an IPTS, prediction of travel time or road reliability can be made using micro-learning over each edge and then results can be aggregated using a specific composition law (sum for travel time, product for reliability, etc.
). In DataTime, data analysis and predictions are made at the level of edges (provided a predictive model has been configured for this purpose). For graphs on which data can be aggregated from edge level to different implementations of parent paths, the prediction or analysis at edge level will be automatically aggregated to higher levels. Specific user defined composition laws are used for this purpose (e. g. , speed of a bus between two bus stops, aggregated to the whole bus line, water leaks at single pipes sections, aggregated to the whole pipe, etc. ). Figures  2  and 3  highlight the compositional prediction mechanism in IPTS context. They show how predictions are made on a complete line by aggregating the predictions on each inter-station. Furthermore, they enlighten the comparison with the classical macro approach, where the prediction is done directly on the whole line without taking into account the interstations.
More precisely, let p = (e 1 ,. , e ′ x ,. , e n ) be a path in the graph G, containing edges e for which training data exists in D and edges e ′ that are dedicated to p, i. e. , created ex-nihilo, with no training data in D. Our goal is to predict a given measure M(p) for any path p on this graph. We possess some local projections m(e) of M on a limited number of known edges e, and some local predictions m(e ′ ) of M on the other new edges e ′ along this path and, typically, a composition law C links these measures: M(p) = C(m(e 1 ),. , m(e ′ i ),. , m(e j ),. , m(e ′ k ),. , m(e n )). We then define our compositional prediction approach: Let G = (V, E) be a directed graph, M a target measure, a composition law C and a dataset D of observations m(e) on edges of G. Given a target path p = (e 1 ,.
Predictors The prediction package contains the abstract class Predictor that embodies the predictive system of DataTime. It encapsulates predictive models and associated tools for various purposes, from training to prediction, both in space and time. It proposes the following services: -Choice of the predictive model, in order to choose the best suited predictive model depending on the prediction issue (using the strategy design pattern). -Optimization of the model by searching for the best hyper-parameters. It embeds a grid search system that can either use a set of properties to seek from, or a random one (through the parameter optimize of Predictor. train() in Fig. 1 ). -Training of the model, by feeding it with a training dataset, hyper-parameters and choosing whether to optimize the predictive model using grid search (through the parameters of Predictor.
-Predictions, by returning a PredictedRun when called (Predictor. predict() in Fig. 1 ). To configure a predictor, a designer has to extend Predictor and implement the prepareDoctorData() method, which aims at preparing data for the health check-up of the predictive model and the training of a new one when needed. The doctor is called when one wants to predict something and that the predictive model is more than obsolescence days old compared to the current system date. The objective behind that is to keep predictors up to date and accurate, in respect to the evolution of data coming from the digital shadow. The second health check is based on real data and the model's accuracy against most recent data in the system. We achieve this by comparing the residuals obtained through predicting from a sample of the last available data from now to back_days back in time with their base error (the deviation of the predictive model obtained during the training phase).
If the predictive models yield an average residual that is over error_trigger times the base error, then a new model is trained with the data yielded by prepareDoctorData(). The doctor can be deactivated if needed (for example when the models are not subject to deviation because, e. g. , the data is quite consistent through time), by setting activate_healthcare to false. Predictors can use different predictive models thanks to the abstract class PredictiveModel. If one uses a single machine learning API in which predictive models are specializations of a single abstract class, then any model of such an API should be made available by encapsulating them in a class extending PredictiveModel. Then overriding the error() method for each model (because the error computation varies, and the result is not always kept into a trained model), which is mandatory for the doctor().
Digital Twin and Digital Shadow The Digital Shadow is either the representation of an existing data environment on which DataTime can be plugged in a read-only manner in order not to have any side effect on the existing information system. Also, it can be bound in an actual fully managed data environment dedicated to DataTime, making it responsible for the management of all the data flows between the digital twin and the real system. The digital shadow should also be responsible for the saving of the graphs and predictors instances. Moreover, it has the responsibility of creating time-series used for data analysis while keeping them optimized and consistent through time. In short, any data that is yielded either by the Digital Twin of DataTime or the real system sensors, etc. should transit and be governed by the Digital Shadow.
Features adaptation inter-stations contain Features such as length of the inter-station (in meters), type of road (one-way, two ways, reserved for buses, etc. ), number of traffic lights, number of pedestrian crossings, etc. All those features are physical features, extracted from Open-StreetMaps 1  Section and line features are obtained by aggregating their respective subpart features. In order to manage features in the bus network implementation, we created data instances that extend the abstract classes from the package data (following the typeobject design pattern). We then created several data features classes (e. g. , Length, TrafficSignalsCount, etc. ), one VolatileFeature class, called LineType and as many Metrics classes as needed amongst which Speed, TravelTime, etc.
The prediction on the new path p, M(p), is the composition of these micro-predictions. We can also apply other shortest path algorithm such as Dijkstra. Figure  6  gives an example of such a task, where we want to predict the duration for a given timestamp of trip S0 -S10 through S0 -S5 and S5 -S7 (detours), edges that do not exist in the example dataset. Figure  6 : What-if scenario: predict S0 -S10 while edges S0 -S5, S5 -S7 and S7 -S10 do not exist in the dataset.
Data mass and lazy loading Since an IPTS typically produces several GigaBytes of data per month, we soon end up with too much data for a local file system. We thus need a kind of lazy loading mechanism when querying trip data. Accordingly, runs are "built" on demand when a call is made through the method getMetric() from the !DigitalTwin. The resulting trips can be kept in memory up to the maximum available memory, but not saved. An advantage of lazy loading is that this allows the ingestion of data in pseudo-real time (e. g. , when there is a need to observe what is going on in the bus network within a short delay). etc. The data we gathered for 2 years uses more than 40GB.
DataTime in Practice at Keolis Our DataTime implementation was designed to be used by bus networks operators. Thanks to the framework we could develop the following features: -The creation of new elements over the bus network, i. e. , bus lines (or new bus lines sections), bus stops, inter-stations. -The analysis of any event on the network at any place and anytime in the bus network. For instance lines 1 to 20 in Algorithm 1 access past data using the method getMetric() with a time period located in the past -The prediction of any metric on the network at any place and anytime in the bus network, e. g. , Algorithm 1 lines 21 to 29 where the method getMetric() is call with a parameter meaning now -Providing the operators hints on which detour should be applied on a bus line depending on, e.
Research Questions We identify the following research questions: -RQ1: Does micro-learning ensure satisfying prediction accuracy ? -RQ2: Are a rich dataset and its associated feature set mandatory to fill missing edge observations in the dataset? -RQ3: Does the compositional prediction approach compete with the macro predication method (that apply in different contexts)? -RQ4: Can usable predictions (wrt. quality) be obtained with the compositional prediction strategy for the step-by-step, what-if scenarios? In the following experiments, we evaluate and discuss these questions in depth, on two sets of data. The first set comes from a synthetic bus network simulation, Algorithm 2: Detours finding (Scala inspired pseudo-code) where timing data is 100% correct by construction. The second set is real data coming from a real bus network: this data being quite imperfect as it is most often the case for real bus networks  [17, 24].
Experimental environment All experiments were run on a computer equipped with 16 cores 32 thread @ 3500MHz, 64GB DDR4 3200MHz 1TB SSD NVMe, running Ubuntu 20. 04 (Budgie flavour). Data were gathered into a normalized data lake built upon Apache Spark 2. 4. 5/Scala 2. 11. 12 jobs. All learning tasks were performed using GraalVM 20. 2 and Scala SMILE 2. 5. 3. It took around 70 minutes for overall model training and 2 minutes for each of the following predictions.
Experiment on Synthetic Data We created a data synthesizer that generates an artificial bus network based on real world features evenly distributed over it using uniform random generation. Using this code, we created an artificial bus network made of 2000 inter-stops (edges) representing 110 different bus lines, each made of 12 to 43 inter-stops. Hence, some inter-stations are shared between bus lines. Note that we do not need to generate nodes as long as nodes features are not used in our model which is edge based (nodes are virtually represented by the fact that edges are connected to each other within bus lines). We then generated 25 to 150 virtual bus trips for each of those lines, generating a target measure for each edge crossed in the trip. We used a negative exponential based function that takes edges features into account (a typical modeling in bus networks).
We then apply it as the mean of a normal law for which we generate a random standard deviation σ, corresponding to a random value situated between 2. 5 and 15% of the mean µ we generated earlier. We then use this normal law normal(µ, σ) to yield random values for our datasets. Doing so we generated two datasets: one for which measure is the travel time in seconds. Its compositional result is obtained by summing the travel time of each edge; another one for which measure is inter-stop reliability (i. e. , a real value between 0 and 1, 0 being unreliable and 1 totally reliable), with its compositional rule being made of product of edge reliability.
Experiment with Real Data We considered the bus transportation system of the French city of Rennes and its metropolitan area (about 500,000 inhabitants) managed by the Keolis Rennes company. Our graph contains 2110 bus stops and 116 bus lines, and the features of road portions are automatically extracted from OpenStreetMap (OSM in the sequel). As experimental candidates, we chose 4 groups of bus lines for which we could gather data all along their path. We tested 13 different bus lines in total. The four groups contain different class of bus lines: urbans, interdistrict, metropolitans and express. We considered only real data: no imputation was made in our dataset. We considered only full trips for which all data points were present. We also filtered historical data used to feed the model, dropping invalid speeds/nulls/values and outliers.
Macro-learning dataset The macro learning dataset is built using the raw dataset, from which we extract only the overall speed of every complete trip (i. e. , trips for which we have data for each inter-stop) of every bus line. We then add the bus line, its type and finally the OSM data aggregated all along the line (i. e. , total number of traffic signals, stops, etc. ). In order to keep as much data as possible, we did not use traffic and ridership data due to the fact that they cover smaller areas and periods in time.
Micro-learning Accuracy (RQ1) The experiment we led using artificial data yielded the following results: Tables  2  and 3  show that predictions for our synthetic datasets, the accuracy of our micro-model on edges is coherent for both speed and reliability prediction. But how would it behave with real, imperfect data that is typical of bus networks? Accuracy We tested (predicted) ∼12'500 trips for the urban bus lines group, ∼1'250 trips for the inter-district group, ∼5'800 for the metropolitan group and ∼800 trips for the express group. All those trips are distributed over the 6 holidays regime, 7 days and 17 periods in day over the 2019 year. Figure  7  shows the predicted speed of one of the urban bus lines using micro-learning. Each point represents the average predicted speed (purple) and average real speed (dashed) of each inter-station of the line on all the time periods for which data was available.
The leftest point of each curve represents the beginning of the bus line, the rightest one being the last stop. Observe that due to the bus line's data deletion from the training dataset, some inter-stops that are specific to this line are totally unknown to the model (orange dots). We also considered metropolitan, express and inter-district bus lines (not displayed)  3. Visually at first, we can see that microlearning captures well the daily behaviour of buses, even when the inter-stops are unknown to the model (orange dots). Table  4  sums up the model's prediction errors for each bus line, group of lines (urban, inter-district, metropolitan and express) and all the lines (global). For each model and bus line (resp. group of lines), the table presents Root Mean Squared Error (RMSE) in km/h (eq.
4), Mean Absolute Error (MAE) in km/h (eq. 5) and Mean Absolute Percentage Error (MAPE) (eq. 6) in percent. RMSE emphasizes large residuals (outliers) while MAE and MAPE are less sensitive to residuals and easier to interpret  [22]. RM SE = 1 N N 1 (Y t -Ŷt ) 2 Y t = actual, Ŷt = predicted (4) M AE = N 1 (Y t -Ŷt ) N Y t = actual, Ŷt = predicted (5) M AP E = 1 N N 1 Y t -Ŷt Y t * 100 Y t = actual, Ŷt = predicted (6) RMSE and MAE unit is km/h while MAPE's unit is percentage (difference against truth in percents). If we take a look at micro predictions in Globally, micro-learning yields variable quality results, with a higher error rate for unknown inter-stops. The features used for predictions probably need enhancement in terms of noise reduction and/or external data addition such as smart-card data, traffic status, etc.
Sensitivity with respect to features (RQ2) We checked the models sensitivity to OpenStreetMaps features and time features by training models with OSM-less datasets in order to assess how well those features describe the bus network. A global features analysis has been done in the PhD thesis work that yielded this paper  [14]. Table  6  shows the results of the predictions made without OSM features, hence using time discretization and built-in features only (period, day, holidays, line type, dwell-time). The values in table 6 (results without OSM features) are colored to compare them with table 4 (results with OSM features): Black for identical results, green for better results and red for worse results. We observed that, in most of the cases (i. e. , individual bus lines and groups), OSM features play an important role for speed prediction for micro, compositional and macro prediction.
This is the case for metropolitans bus lines and inter-district bus lines (Compred mostly). Impact of the size of the training dataset on the precision of the compositional predictive model: For all measures, the precision obtained for the prediction is considered to be satisfactory from the point of view of the bus operator (ie, Keolis Rennes). All tuples devoted to the training data set are used, which represents about 14. 5 million tuples. Although training the predictive model with this amount of data can be done in just over an hour on a standard workstation. It is interesting to explore whether such a large amount of data is necessary to achieve sufficient accuracy. We considered the impact of the reduction of the training dataset on the accuracy of the composition prediction model. Figure  8  shows that a few hundred or even tens of thousands of tuples allow training of a predictive model.
It yields compositional prediction results that are close to those that are obtained using the entire dataset. These results can be applied in the context of continuous learning of the model. Thus, a component could periodically compute the optimal training dataset's size and open the way to operate a continuous learning using a sample of the new data only. Micro-learning on graphs at edge level (e. g. , businter-stops level) yields consistent results on synthetic data. The accuracy turns out to be variable on a real dataset. The prediction error tends to be more important when there are no observations on this particular edge available in the learning dataset. These results point the importance of choosing the right features set for the model. However, Figure  7  shows that the micro learning model has a sensitivity to current features, this allows for what-if scenarios, when one wants to predict the behaviour of a new unseen path.
Applicability of the DataTime framework The DataTime framework as been applied to IPTS but the approach is not limited to this domain. As we describe in 1 the DataTime framework is domain agnostic and can be derived for any application. It is suited for domains having massive graph structureddata with a spatial dimension that evolves slowly over time. In addition, the predictive model relies on compositional prediction and require that the metric of interest can be obtained on each edge, and then aggregated following a compositional law. Such a law can be as simple as an addition, as in our example.
Technical considerations In practice, real world data is dirty  [21]. In our case, we gathered data on a 6 months period between early July 2018 and late January 2019. The data collected is based on the bus network AVL system which contains 116 bus lines. We made some data quality analysis of this data in order to build data-cleaning processes if needed. Table  8  shows the summary of the raw data. As one can see, the outliers in the dataset are quite preposterous: -996 km/h and 130 km/h are impossible speeds for buses. Moreover, readings for which bus speed was lower than 1 km/h or higher than the legal speed limit of 70 km/h (which respectively are low speed limit we defined and legal maximum speed for buses in France), can represent up to an important 5. 4% of the dataset. The main dataset came along with a twin that contains more reliable data (built over technologies that are more reliable than the main dataset), nonetheless at a coarser time grain (20 seconds instead of 1 second accuracy).
Hence, we curated the data by using both filtering (removing outliers) and a specific merge-join techniques. We managed to reduce the error rate to 0% while removing every outlier. We published a previous work that explains in depth the issues of these datasets and how we solved them  [15]. This kind of data quality issue is recurrent in data science, the data must be cleaned up to a certain extent, after which additional cleaning is no longer helpful. The difficulty is to identify when to stop it. This has been discussed in  [15].
The predictors, that are able of continuous learning (by automatically training new predictive models when needed), must be trained before being able to predict. The training time depends on many factors but there are 4 of them that will have a significant impact. Those are the type of prediction model, the size of the training dataset, the number of features and the hyper-parameters tuning. The latter can be the worst one if, e. g. , one uses grid search to find the best set of hyper-paremeters (using optimize=true in the method train(. )). We decided to train models with a training dataset that contains 1 or 2 month of data, with a set of 8 features and the default set of hyperparameters. These 8 features have been selected thanks to the state of the art in public transportation literature  [2, 4, 9, 17-19, 26, 27]  and the business knowledge of Keolis agents.
The training of such a model takes no longer than 10 minutes on a middle end computer (8 cores, 16 GB RAM). That is to say the continuous training if the predictive models eventually turn dull would be done in the same time range. This is thrilling knowing that a single model is able to yield predictions for the whole system. Finally, our implementation is scalable in these ways: 1. The digital shadow can rely on scalable databases, hence the storage and querying can be distributed amongst different machines if needed. 2. The implementation of the framework can be used on any machine and does not need a tremendous amount of computing power to analyze data or predict data: the data collection for analysis relies on the digital shadow's ability to scale, and the predictive models training time is short even on a single machine.
Impact at Keolis Industrial fields that are not computer-science focused often face an issue that we could call "the data overwhelming problem". This consists of having more and more complex information systems that generate more and more data, with a small team of IT engineers who are already too busy keeping the information system healthy. Although the company's data is valuable, its exploitation is hampered by operational constraints and the difficulty of exploiting it without appropriate tools. In companies that rely on stable and well known technologies like standard relational databases and spreadsheets to analyze them, such an amount of data yields the impossibility to study large samples of data. Specifically if those companies core workforce is made of domain experts for which computers are tools and services providers only.
As an example the bus network of Rennes generates nearly 2GB of data per month for the sole AVL system, and more than 10GB if we consider all the stakeholders of the bus network information system. Thus, for the domain experts of the bus network who are provided low to middle end computers, the analysis of large samples of data is quite a challenge when it is not merely out of reach. Thereby, a framework such as DataTime favors the data centralizing with a focus on scalability for data analysis. It makes it possible for domain experts to integrate past, present and future within a traditional information system containing a priori models. Moreover, DataTime could be even more user friendly by the providing of future DSLs, for, e. g. , facilitating the integration of new data sources, the edition of line topologies or make data analysis more fluid, etc.