dataesr/scientific-paragraphs-categorization huggingface dataset:
The HITRAN database targets a range of temperatures that does not substantially exceed those encountered on the Earth and with an exception of a few diatomic molecules (NO, OH, HF, HCl, HBr, HI and H 2 ) does not provide enough transitions. It is worth mentioning that if the temperature range covered in the experiments is not extensive it is better to fix the parameters associated with the second exponent term to be 0 and effectively use a single power law. We also recommend that in the case when there is a sufficient extent of data to fit to the DPL, one should report the new data in both new DPL and traditional single-power law formats if possible. This will allow researchers that work with the room temperature environment to continue using existing radiative transfer codes. [16, 33, 34]  determined from the ab initio νopt , γ 0 and δ 0 parameters, see Eqs.
Acknowledgments We thank  Jean-Michel Hartmann  and  Ha Tran  for the discussion and their remarks regarding the parameterization of the Hartmann-Tran profile and the strategy for improving accuracy when the parameters of the Hartmann-Tran profile are based on ab initio calculations. NS, HJ and PW contribution is supported by the  National Science Centre in Poland  through projects nos. 2015/19/D/ST2/02195  and  2018/31/B/ST2/00720. HC acknowledges the support by the  National Science Centre in Poland  through project no. 2014/15/B/ST4/04551. GK acknowledges the support by the  National Science Centre, Poland  through project nos. 2016/21/N/ST2/00334  and  2017/24/T/ST2/00242. RRG and BV are pleased to acknowledge support of this research by the  National Science Foundation  through grant no.
AGS-1622676. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the  National Science Foundation. The project is cofinanced by the  Polish National Agency for Academic Exchange  under the  PHC Polonium program  ( dec. PPN/X/PS/318/2018 ). The HI-TRAN database is supported by  NASA   AURA NNX17AI78G  and  NASA PDART NNX16AG51G  grants.
This raises the question whether purine modifications like m 1 G, m 2 G, m 2  2 G, m 1 A, m 6 A or m 6  2 A with altered Watson-Crick-Face show characteristic and distinguishable RTsignatures in RNA-Seq data and furthermore whether they can be clearly detected using computer-based analytical methods. Central and common problems in modification calling from RNA-Seq data are rooted in statistics and data treatment. As outlined above, interpretation of RNA-Seq datasets is frequently controversial, as different groups apply variegated statistical methods, but also because, in the absence of commonly accepted standards, many thresholds set for modification calling are set arbitrarily, or according to idiosyncratic reasoning. Consequently, there is a need to establish neutral and unbiased procedures for threshold determination.
Two examples of such supervised learning models for pattern recognition are support-vector machines (SVM)  (34)  and random forests (RF)  (35) , commonly used for binary classification. Random forest models were chosen as prediction tool in the study by  Hauenschild et al. (22)  operating by constructing decision trees, the building blocks of random forests, with training datasets and then being applied on test data for m 1 A recognition, according to the occurring RT-signatures. Random forests consist of a large number of individual decision trees, trained with different subsets of the training data, and each node of each decision tree being split using the feature with the best splitting properties (out of a random feature sampling) from the data. In operating as an ensemble, all prediction trees contribute to the overall classification, forming the model's prediction by majority voting (bagging)  (35).
Standard workflow The standard workflow performs automated trimming, mapping and feature extraction and was used to analyze all datasets. In general, the workflow is derived and adapted from Hauenschild et al. (44)  and implemented in Galaxy. Trimming. In the first trimming step, auxiliary sequences such as adapters, unique molecule identifiers (UMIs) and tailing-bases were removed from the reads using the Cutadapt (version 1. 16) software  (47). Due to the specifications of the library preparation, this step was performed separately for forward and reverse reads as they require different settings. The trimming step consisted of two phases: Removal of the Illumina adapters and the trimming of the UMI sequences, 10 bases (nine of them random) located at leftmost (forward read) or rightmost position (reverse read).
The two alignment files from forward and reverse reads were stored in BAM-format and then merged into a single file. The BAM file were then sorted and indexed using the SAMtools package  (49). Post-processing and feature extraction. The alignment data was converted into pileup format as this format allows to extract relevant information for each position in the reference. Remaining tailing bases were removed from the alignment using a custom python script (overhang trimming). The format was then converted to a tab-separated text-file termed 'Profile', containing information on multiple defined features such as the coverage, arrest rate, mismatch rate, the mismatch composition and the jump rate for each individual position within the reference. The features are defined as follows: Coverage: Describes the number of reads/nucleobases that were mapped to the position in the reference.
g. modified and unmodified instances) in a ratio of 1:1. Secondly, the model was enhanced by k-fold stratified cross-validation. This approach splits the available training dataset into k subsets. The random forest is then trained on k -1 subsets and performs a prediction on the remaining subset. The whole process is performed k times with a different subset being left out in each iteration. In our studies, we set k to 5. With this approach, the random forest model makes use of all available data as opposed to simply splitting the data into training and test samples as the test sample would not be used for training. Accordingly, cross-validation is especially useful for datasets of low sample sizes. To counteract any possible biases resulting from the 1:1-ratio of modified and unmodified instances and the thereof resulting reduction of the number of unmodified instances considered in the training process, the whole cross-validation process was repeated 10 times.
In each iteration, the unmodified instances were randomly selected. This approach not only generated a trained random forest as outcome, but also allowed the evaluation of the model's predictive accuracy. Assessment scores for the random forest performance such as the AUC, MCC, sensitivity and specificity were calculated and averaged over the 10 iterations of the training process. For the prediction process in this study, we trained a random forest model based on the instances of a specific modification from two independent tRNA datasets and subsequently performed classification on a third tRNA dataset.
Side-by-side comparison of 13 RT-signatures at known m 1 A sites To assess the influence of the enzyme on RT-signatures, we used the previously established workflow to detect variations in m 1 A-related signatures of a total of 13 enzymes (see Table  1 ). These 13 reverse transcriptases represent a selection of commercially available RTs at the beginning of the study (without claiming completeness). Briefly, yeast total tRNA was submitted to an RNA-Seq based on a library preparation protocol designed to capture misincorporation events as well as abortive cDNA  (43). The reads were mapped onto a reference set consisting of yeast tRNA sequences, compiled from the Modomics  (21)  and tRNA database  (51) , containing sequences of known modification status. From all m 1 A sites listed in Modomics and the tRNA database, those were excluded from further analysis, for which the coverage was insufficient to extract a statistically relevant RT-signature.
Of note, all used datasets contained at least 300 000 reads per RTenzyme (per replicate), and escalating comparative analyses with smaller datasets indicated no significant changes even at read numbers as low as 50 000 reads (see Supplement Figure  S10 ). From Figure  1A , it is apparent that the 13 reverse transcriptases display a wide distribution of features i, iii and iv, which is bracketed between RT #12, with highest mismatch, highest jump rate and lowest arrest rate on one hand, and RTs #4 and #10, with lowest mismatch rates, lowest jump rate and highest arrest rate, on the other hand. A distribution along the diagonal, which is immediately obvious upon visual inspection, strongly suggests an inverse correlation of arrest and mismatch rates, and indeed, the corresponding Pearson correlation coefficient (PCC) is -0.
In full analogy to the previous evaluation of m 1 A sites, the corresponding reads were mapped onto a reference set consisting of tRNA sequences from Saccharomyces cerevisiae, compiled from the Modomics (21) and tRNA database  (51). The data was inspected for signatures at known m 1 G and m 2  2 G sites. We found, that many m 1 G and m 2 2 G sites, which are typically present near the 5 -end at positions 9 and 26, displayed low coverage as a result of their location downstream of other RT-blocking modifications such as m 1 A or bulky modifications at position 34 and 37. We prepared three independent RNA-Seq datasets from three replicates of total tRNA yeast for each of the four reverse transcriptases, each total tRNA dataset (each replicate) containing 988 guanosines, including 16 m 1 G and 22 m 2 2 G sites.
In general, two replicates were used for random forest training and the remaining third replicate for testing (RF #1). In order to collect a sufficient number of sites for random forest training, we set a relatively low coverage threshold of 20 reads. After removal of the low-coverage instances from the two training datasets, between 22 (68. 8%) and 32 (100%) instances for m 1 G (dependent on the RT used; see Supplement Table  S5 ) and between 20 (45. 5%) and 38 (86. 4%) instances for m 2  2 G passed this coverage threshold (e. g. 32 m 1 G and 38 m 2  2 G for RT #12, see Supplement Figure  S4 (B) ). Along with a matching number of unmodified guanosines as negatives, these were used for training a random forest model in full analogy to those used for m 1 A. In a first random forest model we trained and tested for a joint m 1 G and m 2  2 G separation from other guanosines, so m 1 G and m 2  2 G sites were together used for random forest training.
Overall, the model prediction scored AUC values between 0. 9439 and 0. 9801 for the different RTs (e. g. 0. 9801 for RT #12), which favorably compares to our first runs on m 1 A  (22). After filtering to remove low coverage instances (coverage >20) we could correctly classify 80% of the m 1 G instances and 90% of the m 2  2 G instances (average percentages for all RTs used) in the respective test datasets (see Supplement Table  S5 ). Besides m 1 G and m 2 2 G other instances of Watson-Crick blocking modifications were classified as modified, filtering out non-modified guanosines in the process. An interesting instance of a 'false' positive concerned position 37 in yeast tRNA Phe , namely the wybutosine modification (see Supplement Figure  S5 ). This hypermodification is generated in a sophisticated multistep biosynthesis starting with m 1 G at G 37  (57).
DISCUSSION The central objective of this work concerns the performance of machine learning approaches for automated modification calling. We advocate the use of e. g. a random forest model as an unbiased entity, which determines threshold values without human input, yet leaves the possibility open to read out this parameter a posteriori. Here, we compared 13 commercially available reverse transcriptases of different origin, based on their individual behaviour at m 1 A sites in tRNA of S. cerevisiae. The presented data show large variance of RT-signatures, illustrated in the differently expressed features, arrest and mismatch. In addition, some of the reverse transcriptases show an increased susceptibility to deletions at modified sites, termed as jumps, which we used to extend the RT-signature feature set.
The only dataset featuring a potentially sufficient statistical basis would be that used by Grozhik et al. (29, 30) , who, significantly, conclude a very low number of m 1 A occurrences in human mRNAs. In further experiments, we then applied the three best performing RTs on other purine modifications with chemical alterations on their Watson-Crick face, very likely impeding cDNA synthesis, to evaluate the hypothesis, that, in general, every modification can be distinguished and detected. Indeed, our data on m 1 G and m 2  2 G instances suggests that the apparent limitations are most likely due to coverage at analysed modified sites and that with an optimized experimental setup, e. g. selection of the reverse transcriptase and sufficient sequencing output, the modifications can be predicted and differentiated.
ACKNOWLEDGEMENTS We are grateful to  Lilia Ayadi  and  Valérie Igel-Bourguignon  ( EpiRNA-Seq Core Facility ,  UMS2008  IBSLor,  CNRS-Université de Lorraine-INSERM, Nancy, France)  for excellent technical assistance in MiSeq sequencing. We want to further thank  Dr Lyudmil Tserovski  and  Dr Ralf Hauenschild  for their contributions during early stages of this project.
Study site and reference data The study area is the Alsace region located in the East of France (Figure  2a ). It covers an area of 8329 km² and includes four main land cover classes: urban footprint, agricultural areas, forest areas, and water surfaces. The urban footprint displays some level of class imbalance (Figure  2b ). Discontinuous UF is the most common class, followed by industrial or tertiary facilities and then the road and rail network. The key reason for choosing the Alsace region as a study site is its morphological characteristics typical of many European Western cities. The regional land use/cover vector database (BDOccsol©GeoGrandEst) produced by visual interpretation of aerial photography is used as reference data  [33]. The most up-to-date version (2012) available through the regional data infrastructure GeoGrandEst, maps the study area into 55 thematic classes.
These variations make these classes well suited for this study as we can already expect the densely built classes to have a more stable temporal response compared to the others (Figure  1 ). Reference data are rasterized at a 10 m spatial resolution. This process is applied at two levels: the first level with a binary map of the urban footprint, and the second level results in a multi-class map with the four UF classes (Figure  3   The regional land use/cover vector database (BDOccsol©GeoGrandEst) produced by visual interpretation of aerial photography is used as reference data  [32]. The most up-to-date version (2012) available through the regional data infrastructure GeoGrandEst, maps the study area into 55 thematic classes. The legend is organized into four levels of nomenclature. The first level categorizes land cover into four classes: (1) artificial surfaces, (2) agricultural areas, (3) forest areas, and (4) water surfaces.
Data Sampling It consists of subdividing the image into 256 × 256 pixel patches (Figure  3-Step1 ). The size should be sufficiently large as not to lose spatial context but should also take into account the available computational resources. Images with no spectral information are then removed. We perform a random shuffling of the dataset before splitting it into training and validation sets using a 10% ratio for validation and 90% for training. Data augmentation on the training set is also applied by random rotations, translations, and zooms to make our models more robust to variations and prevent overfitting. Table  4  shows the number of image patches obtained for each model. The number of patches is proportional to the distribution of defines classes on the study site. It is essential to discuss how to approach multi-label classification.
Two common strategies are often used: the first one consists in implementing a global model that outputs membership probabilities of all the classes of interest, whereas the second strategy, commonly referred to as the one-vs-all classification, consists in implementing separate binary classifiers for each class. Several factors influence this choice, like class distribution, number of target classes, and the available computing resources. As Figure  2b  illustrates, the dataset is very class imbalanced. Consequently, we use the one-vs-all approach  [34]. As a training strategy, we use transfer learning, which allows us to take advantage of existing pre-trained models. Transfer learning is also ideal when the number of training samples is limited  [35].
• Supervised classification of urban footprint using mono-temporal sentinel 2 data. Four distinct encoder-decoder architectures are implemented: FCN-32s [36], FCN-16s [36], SegNet  [26] , and UNet  [37]. Unlike the original paper, we use VGG-19  [38]  as the basis of the first two architectures. We initialize all encoders from the ImageNet weights and all decoders using Xavier Initialization  [39]. According to  [40] , when changing sensors, transfer learning can fail. We, therefore, employ Sentinel 2 RGB bands analogously to the ImageNet dataset. We nonetheless train from scratch a variant of the SegNet model on multispectral data to explore the impact on classification performance. As a loss function, we use the dice loss (DSC) defined below. We then conduct a performance comparison between the implemented models to select the best model for UF mapping.
• Supervised classification of UF using mono and multi-temporal Sentinel 2 data. The previous experiment showed that the UNet model offers the best performance. This will be further detailed in the results section. The model is then trained on mono-temporal data to map the four classes of UF. Afterward, we design a new architecture taking as a template the previous one to analyze multi-temporal data. The proposed architecture (Figure  4 ) takes as input a multi-temporal patch and outputs a classified map, and is capable of learning spatio-temporal features. The encoder consists of three VGG-16 branches carrying out the separate learning of spatial features for each timestamp. Then, a temporal-wise concatenation of the last convolutional layer is performed along the three branches for each VGG-16 block.
Another interesting remark is that the performance of the SegNet model trained on multi-spectral data is reduced compared with its RGB counterpart. Given the wide gap between the two variants of we will no longer consider the second one for the rest of the experiments. These findings support the importance of transfer learning for small datasets. To further analyze the model's performance, we examined in more depth the variations of their sensitivity and specificity using the ROC curve (Figure  8 ). UNet and RGB SegNet outperformed their counterparts in that their loss decreases more steeply and reaches lower values. Furthermore, it appears that the UNet model is holding a slight advantage. Another interesting remark is that the performance of the SegNet model trained on multi-spectral data is reduced compared with its RGB counterpart.
Given the wide gap between the two variants of SegNet, we will no longer consider the second one for the rest of the experiments. These findings support the importance of transfer learning for small datasets. To further analyze the model's performance, we examined in more depth the variations of their sensitivity and specificity using the ROC curve (Figure  8 ). UNet and RGB SegNet outperformed their counterparts in that their loss decreases more steeply and reaches lower values. Furthermore, it appears that the UNet model is holding a slight advantage. Another interesting remark is that the performance of the SegNet model trained on multi-spectral data is reduced compared with its RGB counterpart. Given the wide gap between the two variants of SegNet, we will no longer consider the second one for the rest of the experiments.
These findings support the importance of transfer learning for small datasets. To further analyze the model's performance, we examined in more depth the variations of their sensitivity and specificity using the ROC curve (Figure  8 ). UNet and SegNet outperformed the other models with a slight edge for the UNet. As an example, to reach 86% of true positives, UNet and SegNet only produce 1% of false positives compared with 7% and 10% for the FCN-16s and FCN-32s, respectively. The area under the ROC curve (AUROC) is also a meaningful indicator of a model's performance. Given that each ROC curve must depart as much as possible from the diagonal, AUROC must be as close as possible to 1 and as UNet and SegNet outperformed the other models with a slight edge for the UNet. As an example, to reach 86% of true positives, UNet and SegNet only produce 1% of false positives compared with 7% and 10% for the FCN-16s and FCN-32s, respectively.
Nevertheless, it is still unclear why, even for As we move away from the training dataset's sensing period, the model's performance gets worse. Moreover, the performance of discontinuous UF does not degrade as quickly as other classes. This can be explained by the number of available samples per class: a model trained over a large dataset has a better generalization capability. Nevertheless, it is still unclear why, even for discontinuous UF, performance degrades as we move further from training time. Afterward, we trained the multi-temporal UNet on two UF classes: Continuous UF and the class of industrial facilities. Results over the validation set are reported in Table  10. To answer the research question, we develop a testing methodology using multi-temporal Sentinel 2 imagery of the year 2018.
Another sound conclusion is that Sentinel 2 images, despite a spatial resolution of 10 m, can be used for urban land use mapping. Nevertheless, the contribution of the temporal dimension to UF mapping requires further testing with larger datasets and longer time series. With this mind, we think the positive results achieved in this research are worth building upon in future studies to further investigate the temporal domain as an input to urban land use classification models. Conducting experiments on a wide range of UF classes, using longer time series, and bigger datasets all represent good avenues for future research.
Conclusions and Perspectives In conclusion, this work explored the use of multi-temporal Sentinel-2 images for UF mapping. To this end, a novel multi-temporal fully convolutional network based on the famous UNet architecture was implemented. The proposed model learns a spatio-temporal representation of the data without the need for extensive feature engineering or heavy preprocessing and is trained in an end-to-end fashion. To investigate whether the temporal dimension can improve the classification performance of UF, we devised various testing methodologies using mono-temporal and multi-temporal images. We first assessed the robustness of four popular deep learning benchmarks to spatial and temporal variations and then went on to conduct a comparative analysis between the performance of a mono-temporal model and the proposed multi-temporal model.