dataesr/scientific-paragraphs-categorization huggingface dataset:
D-Reln D-Con D-Adv TEMPORAL:Asynchronous "and afterwards", "but afterwards", "after which", "then", "after that", "after this", "but, after that", "and after this", "after which", "eventually", "and eventually", "and in turn", "in turn", "which, in turn", "and then", "and so", "later", "and later", "but later", "next", "before", "followed by", "when", "thereafter", "and thereafter", "after which", "before that", "but before that", "although before that", "prior to this", "earlier", "and earlier", "formerly", "previously", "after", "and previously", "recently" "afterward(s)", "after that", "eventually", "in turn", "later", "next", "thereafter", "before that", "earlier", "previously" TEMPORAL:Synchrony "in the meantime", "but in the meantime", "whilst", "meanwhile", "while in the meantime", "while", "simultaneously", "and simultaneously" "in the meantime", "meanwhile", "simultaneously" CONTINGENCY:Cause "accordingly", "so", "as such", "and as such", "as a result", "and as a result", "however", "so that", "resulting in", "consequently", "and therefore", "and so", "with", "therefore", "which means", "which means that", "thus", "and thus", "thusly" "accordingly", "as a result", "consequently", "therefore", "thus" COMPARISON:Contrast "by comparison", "in comparison", "while", "compared to", "whilst", "by contrast", "in contrast", "and in contrast", "while", "although", "conversely", "and conversely", "nevertheless", "but", "none the less", "yet", "however", "on the other hand", "and on the other hand", "but on the other hand", "but", "whereas" "by/in comparison", "by/in contrast", "conversely", "nevertheless", "on the other hand" TEXPANSION:Conjunction "additionally", "and additionally", "and also", "and is also", "besides", "besides this", "aside from", "further", "furthermore", "and furthermore", "and further", "in addition to", "likewise", "and likewise", "moreover", "indeed", "similarly", "and similarly", "while" "additionally", "also", "besides", "furthermore", "in addition", "likewise", "moreover", "similarly" EXPANSION:Instantiation "for example", "for instance", "such as", "in particular" "for example", "for instance", "in particular" EXPANSION:Alternative "instead", "but instead", "though", "but rather", "rather" "instead", "rather"  triples for all other datasets, Conj:Conjunction, Inst:Instantiation, Alt:Alternative).
Experimental Setup All of our generative models use the BART architecture  (Lewis et al. , 2020)  and were trained on a computing grid using 4 Nvidia RTX 2080 Ti GPUs. Each experiment starts by fine-tuning the facebook/bart-base model hosted by Hugging-Face 4  , which has 6 layers in each of the encoder and decoder, a hidden size of 768, and was pretrained to perform reconstruction of corrupted documents on a combination of books and Wikipedia data. During training, we used a learning rate of 3e - 5  , a batch size of 16, and performed dropout with a rate of 0. 1 and early stopping as regularisation measures. For each experiment we set aside 5% of the training set for validation. During generation, we perform beam search with a beam size of 4. We compare the following models: Split Baseline (BL Split ) Pre-trained BART finetuned on a 1M example dataset of both syntax-and discourse-based splittings (WikiSplit).
This baseline allows us to compare training with very large heterogeneous training data (BL Split ) vs. learning from smaller, discourse-split data (BL DSplit ). Discourse-Split Baseline (BL DSplit ) Pretrained BART fine-tuned on a discourse-focused subset of WikiSplit (D-WikiSplit). This baseline is to be directly compared with BL Split. Parser Pipeline Baseline (P L P arse ) A pipeline of two models. The first uses the discourse parser process used to generate T s from Cs in Section 3 (C2T) and the second is a pre-trained BART finetuned on (T, S) data (T2S). We experimented training the T2S component on various datasets and found the best to be that trained purely on synthetic data. Thus, any pipeline mentioned in the remainder of this paper refers to a specific C2T component connected to this same T2S component.
This baseline allows us to compare pipeline models whose C2T component is learned on the split data vs. one where the C2T component uses an existing discourse parser. End-to-End Model (E2E) Pre-trained BART fine-tuned on discourse-split data. We report results for variants trained on D-CC-News-S (E2E Synth ), D-Wikisplit and D-MUSS (E2E Organic ), and all three combined (E2E Both ). Pipeline Model (PL) A pipeline of two models. The first model is pre-trained BART finetuned on (C, T ) data and the second is a pretrained BART fine-tuned on (T, S) data from D-CCNews-S. We report results for pipelines with a C2T component trained on all D-CCNews data (P L Synth ), D-MUSS data (P L Organic ), and D-CCNews combined with D-Wikisplit and D-MUSS data (P L Both ). Pre-training and Fine-tuning (PT+FT) Pretrained BART fine-tuned on one data set before being further fine-tuned on another.
Multi-Tasking (MTL) We prefix the training data with a control token indicating whether a training instance maps a complex input to a discourse tree (c2t), a discourse tree to a split text (t2s) or a complex input to a split output (c2s) and train pre-trained BART on this data. We use training examples from D-CCNews, D-WikiSplit and D-MUSS. At inference time, we prefix the input with the c2s control token for the end-to-end model; and with the c2t and t2s control tokens for the two components of the pipeline model.
This confirms our hypothesis that, because it includes multiple variants of the same discourse split using different connectives and orderings, the synthetic data helps to train discourse splitting models that are better able to generalise i. e. , are able to generate with different connectives for the same relation. For both E2E and P L models, combining organic and synthetic data (E2E Both and P L Both ) appears to reduce the performance trade-off of using one data type in isolation. Alternative ways of combining organic and synthethic data using either fine-tuning and pretraining or multi-tasking did not yield improvements. For both regimes, we experimented with multiple hyper-parameters and data combinations. The details of these experiments are given in Appendix A. 3. Generic-vs. Discourse-Split Data In terms of meaning preservation (BLEU, SAMSA), BL DSplit (trained on 371K instances) performs on par with BL Split (1M instances), showing that discoursefocused models can compete with standard splitting models when trained on much smaller, ded-icated datasets.
Conclusion In this paper we introduced the task of Discoursebased Sentence Splitting together with a largescale dataset of both organic and synthetic discourse splits. Experimental evaluation revealed that discourse-based, pipeline models have better discourse relation preservation capabilities than endto-end models, and that synthetic data is critical for learning models that can generalise i. e. that can generate mutliple variants of the same discourse relation. In future work, we would like to create more document-aware models incorporating both syntax-and discourse-based sentence splitting at the document level. In the case of multi-task learning, we trained a sequence-to-sequence model to simultaneously learn to perform the C2T, T2S and C2S tasks. The motivation behind this was that there could potentially be useful shared features between the tasks that would help overall learning performance.
We used C2T data from D-CCNews, D-MUSS, and D-WikiSplit; and both T2S and C2S data from D-CCNews-S, D-MUSS, and D-Wikisplit. For our pre-training and fine-tuning experiments we experimented with different dataset combinations and training strategies for both our end-toend model and our pipeline system. Initially, we tried pre-training first on synthetic data and then finetuning on organic data; either as D-MUSS and D-WikiSplit in combination, or one after the other. We also went on to experiment with pre-training on the standard WikiSplit dataset in an attempt to see whether useful features could be learned from training on a generic splitting task. For each of these, we also experimented with freezing/unfreezing different layers in the network (embedding, encoder, and decoder). As mentioned in the paper, we were unable to observe any improvements over our standard models for any of these experiments.
Acknowledgements Experiments presented in this paper were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including  CNRS ,  RENATER  and several Universities as well as other organizations (see https: //www. grid5000. fr). We thank the anonymous reviewers for their feedback. We thank the authors of (Martin et al. , 2020) who kindly provided us with their data for this study. We gratefully acknowledge the support of the  French National Research Agency  ( Gardent ; award  ANR-20-CHIA-0003 , XNLG " Multilingual, Multi-Source Text Generation ").
Related Works 2. 1. Vehicular Perception Weather is an external variable that cannot be controlled. A driver's visibility can be heavily hindered by the quality of the weather, and this phenomena has been intensely studied in the past, some studies dating back as far as the 1970s  [10]. In their 2019 work, Harith et al. [11]  identified 45 different works made in 21 countries and covering more than 500,000 accident cases caused by adverse weather, mostly rain and fog conditions. They concluded that a driver's vigilance is key to keeping the road safe. This concern is shared by Das et al. [12] , who agree that rainy weather is one of the most hazardous driving conditions, causing up to 25% of crashes in some areas. They assembled a dataset based on the crash records in the state of Florida, and managed to prove that there is a relationship between road accidents and poor visibility due to bad weather.
The same team used data mining on extensive data sets and aimed to isolate aggravating circumstances that can increase the accident rate when coupled with rainfalls  [13]. The patterns that emerge can provide valuable insight for safety professionals, but at the time of the study, a comparison with a clear-weather similar dataset was lacking. Andrey et al. [14]  point out that despite the accident rate increasing by up to 70% in case of rain, it returns to a normal value when the rain stops, despite the lingering effect of wet roads. This would mean that the main reason behind those accidents might actually be the poor visibility conditions rather than the slippery roads. Visibility is also affected by lighting conditions. Cameras are among the most present sensors in a vehicle, and they perform poorly if the lighting is not adequate.
Carlevaris-Bianco et al. [15]  built a dataset containing similar images with different illuminations, and underlined the difficulty an intelligent unit could have in processing the same situation if the lighting is different. Most of the causes of illumination variations are natural and independent of humans, for example night time or weather situations. The Gade et Moeslund  [16]  study says that lighting variation impacts many parameters of an image (intensity, color balance, etc. ), but also proposes the use of thermal camera instead, despite having their own sets of drawbacks, such as the inability to classify the detected objects. Visibility can also be improved in software via algorithms, as shown by Tarel et al. [17, 18]. They focused their works on foggy roads and showed great results for their filtering algorithm.
The necessary Line Of Sight (LOS) condition makes it difficult for a malicious agent to manipulate or intercept the data flow in a vehicular environment. However, VLC does have its own set of disadvantages and performs poorly in some environment, which is why it is more interesting to use it as a complementary protocol. VLC in hybrid works has already been considered in the past. In  [50] , it is stated that RF communication is sensitive to jamming attacks and interference, and even if the use of Cognitive Radio (intelligent detection of unused transmission channels) can minimize the risks, they still propose the addition of VLC communication to strengthen security. Another experiment was conducted by  [51] , where a joint 5G/VLC prototype was set up. The smart city sensors data were gathered and transmitted to the road infrastructures (traffic light) through 5G, and then to the cars via VLC.
Simulated Environment The model was tested and validated in a simulated environment. The interface was based on the Udacity  [72]  project, a car simulator built with the Unity engine  [73]. It allows the building of driving surroundings (roads, obstacles), driving conditions (rain, fog, physics constraints, etc. ), and the manual control of the vehicle. On a technical level, the driving data are logged in a JSON format and sent via an engine to the knowledge base. The reasoner will then be called to infer the environmental status. The communication between the simulator and the knowledge base is done through a socket connection. We considered two different software tools to do so: the Java OWL API  [60]  and the Owlready Python library. We compared both of those approaches in order to choose the optimal one for our study.
In Figures  4  and 5 , we compared both engines execution times. We executed both of them in a similar environment where the vehicle encounters some heavy-processing events. As we can see, and depending on the situation, the Python tool offers steady performances with a processing time of around 1 s. Java occasionally outperforms it, but it has more trouble in harder contexts: We can observe a high peak when the car encounters a fire hazard. The execution speed being an important factor, the choice was made to go with the Owlready tool for this study. An illustration of the process can be found in Figure  6 , in which the XML object represents the ontology. Each sensor has its own way of gathering and processing data. For example, cameras relying on deep learning algorithms in order to classify objects  [74] , LIDAR technology supporting this process with depth-computation  [75]  or radar sensors for the detection of close elements  [76].
An illustration of this process is shown in Figure  10. The raw data are sent from the simulator to the ontology through the Python pipeline. Details of the transmitted data can be found in Table  A3. Once logged in the knowledge base, they are processed by the set of logical rules and the tools described in Section 4. 1. If the system inference requires a perception enhancement, and if the conditions are verified (i. e. , available UAV in reach with all the correct sensors), the additional information will be displayed on screen, as shown in Figure  11 : a message informs the user that an obstacle is at a certain distance of the vehicle. This distance is computed thanks to the localization of the ego vehicle, and the localization of the obstacle detected by the UAV. The drivers are to encounter the following situations, as illustrated in Figure  12 : • The driver needs to take a turn in an intersection with limited visibility.
Without AR, work instructions are provided through standard work documents in a paper format containing the information needed for assembly. It consists of an overview of the equipment, standardized textual instructions, and detailed images for some sub-tasks. The AR tool was deployed with the industrial AR software solution Diota V2. 3. 0  (Diota, 2019). Diota is connected to the Catia Composer R2017X 3D creation software for the creation of content for the AR application. Diota Player software is used to play content created on selected hardware. The hardware is a shop floor workstation running the Windows 10 operating system equipped with a 27-inch touchscreen and an industrial HD camera. This configuration is installed on a standard mobile desk to facilitate integration into the workshop (Figure  2 ).
Hardware Selection Hardware takes many forms, from computers or tablets to the more unusual like glasses or projectors, each with their own advantages and disadvantages. Deciding between hardware forms is affected by the necessary mobility, the possibilities of manual handling, the software available, and the availability of the equipment for use in an industrial environment. The selected AR software can be used on hardware running the Windows 10 operating system such as a PC, tablet, HoloLens V1, or specific Diota projector. The projector was rejected because the way the information was displayed was not relevant to the use case and its volume. HoloLens have a small field of view, low autonomy, and require time for learning interactions. The tablet requires handling the device with one or both hands, which is problematic when it is necessary to interact with parts.
ACKNOWLEDGMENTS The content of this manuscript has been presented in part at the 16th EuroVR International Conference on Virtual Reality and Augmented Reality, EuroVR 2019, Tallinn, Estonia, October  23-25 ,  2019. Loizeau Q. , Danglade F. , Ababsa F. , Merienne F. (2019) Evaluating Added Value of Augmented Reality to Assist Aeronautical Maintenance Workers-Experimentation on Onfield Use Case. In: Bourdot P. , Interrante V. , Nedel L. , Magnenat-Thalmann N. , Zachmann G. (eds) Virtual Reality and Augmented Reality. EuroVR 2019. Lecture Notes in Computer Science, vol 11883. Springer, Cham. https://doi. org/10. 1007/978-3-030-31908-3_10.
Comparisons between three empirical (NRLMSISE-00, JB2008 and DTM2013) and two first-principles models (TIE-GCM and CTIPe) and neutral density data sets that include measurements by the CHAMP, GRACE, and GOCE satellites for 13 storms are presented. The models all show reduced performance during storms, notably much increased standard deviations, but DTM2013, JB2008 and CTIPe did not on average reveal a significant bias in the four phases of our metric. DTM2013 and TIE-GCM driven with the Weimer model achieved the best results taking the entire storm event into account, while NRLMSISE-00 systematically and significantly underestimates the storm densities. Numerical models are still catching up to empirical methods on a statistical basis, but as their drivers become more accurate and they become available at higher resolutions, they will surpass them in the foreseeable future.
The main sources of density data are satellite-drag inferred total densities by means of orbit perturbation analysis  (Jacchia & Slowey, 1963)  or accelerometers (e. g. Champion & Marcos, 1973) , and neutral mass spectrometers (e. g. Nier et al. , 1973) , with the latter providing composition measurements. Drag-inferred and spectrometer data have in common that they do not provide an absolute measurement of density. This is due to calibration issues and, e. g. , unknown oxygen recombination rates in case of mass spectrometers. The most recent and precise accelerometer-inferred density datasets are also not absolute. Their magnitude depends on the satellite model, and in particular the aerodynamic coefficient, which effectively is a scaling factor, that was assumed in the computation. As a consequence, the SE thermosphere models fit to the scales of the satellite models used in the respective databasesand these are rarely consistent between modelers.
Intercalibration is a necessary and complicated activity for all modelers, but it is not always entirely successful due to e. g. , no overlap in time or a drifting offset between datasets. JB2008 (up to 2008 at least) and DTM2013 predict densities that are close (often within 5%) to the US Air Force operational thermosphere model HASDM  (Storz et al. , 2005) , and therefore they are considered having the same scale.
Auroral precipitation is keyed to the hemispheric power index (PI), based on the TIROS/NOAA auroral particle measurements. The Weimer electric field model is keyed to the solar wind parameters impinging the Earth's magnetosphere, and its input drivers include the magnitude of the interplanetary magnetic field (IMF) in the y-z plane, together with the velocity and density of the solar wind. A combination of measurements from Advanced Composition Explorer and Wind spacecraft instruments, obtained at NOAA Space Weather Prediction Center, NASA's Space Physics Data Facility and Los Alamos National Laboratory, have been used to address data gaps and quality issues (e. g. ,  Skoug et al. , 2004). The (2,2), (2,3), (2,4), (2,5), and (1,1) propagating tidal modes are imposed at 80 km altitude  (Fuller-Rowell et al.
5 h (and then the satellite passes the same latitude and local time, but at a different longitude), even if the measurement cadence is 5 or 10 s. Precise density datasets inferred from accelerometer data of recent satellite missions are selected because only these are compatible with storm-time evaluation, notably precise measurements from pole to pole. However, the assessments are based on densities from one, or two satellites at best, and one cannot reconstruct the complete picture of the thermosphere at any given time; we only see the densities in a latitude-local time frame at a relatively constant altitude, with a temporal resolution of about 95 min. Table  2  lists the essential information of the three selected datasets, CHAMP  (Doornbos, 2011) , GRACE  (Bruinsma; unpublished)  and GOCE  (Bruinsma et al.
, 2014). The densities used in this study were smoothed to suppress variations with scales smaller than 600 km, and then down-sampled to 80 s cadence. The original GOCE densities can be obtained, after registration, on the ESA server (https://earth. esa. int). Table  3  lists the dates, minimum Dst and maximum ap/Kp of the storms, and the satellite data available for model assessment. The intervals are selected to cover a strong storm sequence that returns to low geomagnetic activity (low Kp), and secondly, observations from two satellites are preferred. This led to a slightly different selection of storms than proposed in  (Bruinsma et al. , 2018). Thirteen storms, eleven of which were CME-driven and two by High Speed Streams, were selected for this assessment using the available data provided by the three satellites.
The eight storms in 2005 are the so-called problem storms for the US Air Force  (Knipp et al. , 2013). Strong storms cause the largest satellite orbit perturbations, together with degraded tracking performance, and model assessment is most pertinent and needed for those events. The impact of weak storms is not dramatic, and only one is selected because  S. Bruinsma et al. : J. Space Weather Space Clim. 2021, 11, 12  it was a problem storm. While not strong but of moderate strength, the three storms in 2012 and 2013 were selected because they are the only ones for which high resolution density data is available in solar cycle 24, from GRACE and GOCE. The local time at the equator is also given in Table  3 , and it is clear that the distribution is sparse for any single storm.
This de-biasing procedure is used to minimize the effect of non-storm related model errors on the assessment. All data for each storm are selected 30-h before to 48-h after the minimum in Dst, which is defined as t 0. The Dst and densities centered on t 0 for all storms listed in Table  3  are displayed per satellite in Figure  2. The choice of fixed lengths of the intervals is supported by the density profiles shown in Figure  2. Note that GOCE is missing data in phase 1 for the storm in 2012; there is no data for GRACE. This example of a data gap in one of the phases, maintained in this analysis because of the few observed storms in solar cycle 24, is another reason for the sparseness of the storm density database. Data is regularly missing during storms. In this case for GOCE in 2012, the scaling factor for de-biasing was not determined according to the metrics, but with part of the data in phase 2.