dataesr/scientific-paragraphs-categorization huggingface dataset:
Of course, methods cannot be transferred directly, but it appeared that concepts and solutions might be easily adapted. A perfect example can be the work of our colleague Dr. Andre Kahles (ETH, Zurich, Switzerland). His concepts for graph-based sequencing data analysis took their origin in (cancer) transcriptomics. Over time, his team developed a much more powerful, generic framework for graphbased, large-scale metagenomics sequence analysis (https://metagraph. ethz. ch). Andre Kahles (ETH, Zurich, Switzerland). The MetaSUB project has been a huge catalyst for the development of the MetaGraph framework. Providing us with a very large, highly diverse dataset that was also ''Overall, what we are doing is a real example of a modern-day team science approach and bringing in colleagues with specialties in diverse fields.
We have people from different backgrounds such as microbiology, ecology, bioinformatics, computational genomics, design, engineering, diagnostics, and clinical care. '' annotated with detailed metadata, the project allowed us to develop and test the scalability of our approaches in a real-life context. As a result, we became pioneers in making all sequencing data of a project really full-text searchable. Another benefit is that since then many more use cases and research directions have emerged that will keep us busy with methods development for the coming years. Le Huu Song and Thirumalaisamy P. Velavan (Vietnamese German Centre for Medical Research, Hanoi, Vietnam -Universita¨tsklinikum Tu ¨bingen, Germany). Microbes are miraculous, they evolve much faster than humans, and this is also valid for MetaSUB.
It has truly been evolving over the years. Comparing microbial communities in different environments over time and space is very useful to understanding microbial dynamics, particularly in a hospital environment, which was our objective in Hanoi. We began collaborating with MetaSUB, understanding the consortium as a global multidisciplinary team ''united for purpose'' in identifying microbes in different environments with a harmonized research design. With an exceptional project management team, MetaSUB has consistently delivered successful project outcomes in a timely and sustainable manner, sharing results with community and policymakers alike. All data generated by MetaSUB are open for anyone to view and analyze. Frolova. At some point, I took an initiative to organize metadata collection, which means standardization of collected information (data types, which places to swab, input validation etc.
) and choosing appropriate software solutions to work without internet (if needed), on any device and with minimum risks of data loss. On the other hand, the collection process should be flexible enough, because there is no single identical transport system in the world and we had to take into account local characteristics. However, later I realized the most difficult thing was to keep everything concise. It was my first global City Sampling Day when we designed a new collection web form, which looked perfect to me and we heavily tested it before the 21st of June. However, when I personally had to input over 100 samples at the end of the day, I was completely exhausted. Field work made me rethink our design. In general, I love this aspect about MetaSUB, despite futuristic goals we stay very practical.
More details about these ongoing and future MetaSUB projects can be found at http://metasub. org/projects/, https://pngb. io/ metasub-maps, http://extrememicrobiome. org and https://dnaloc. ethz. ch database. Bezdan. One of my favorite MetaSUB spinoff projects is StuckOnU (http://metasub. org/stuckonu/), where we sequenced about 1,200 cellphones across six different conferences and quantified their microbiota. Given that individuals bring their smartphones to all occasions and touch them frequently in all situations, questions arise regarding the classification, persistence, migration, and dynamics of bacteria and viruses on such personalized devices. Since we collected an extensive amount of metadata, such as geolocation, eating habits, and antibiotic use, we were able to associate abundance and living habits in a novel and exciting way.
Malay Bhattacharyya (Indian Statistical Institute, Kolkata, India). The analyses of the MetaSUB data have also evolved through an interesting open science approach. MetaSUB had partnered with the CAMDA (Critical Assessment of Massive Data Analysis) conference to promote the analysis of MetaSUB data through an open data challenge. This started in 2017 with ''The MetaSUB Inter-City Challenge'' followed by  ''The MetaSUB Forensics Challenge'' in 2018 and 2019. This has generated a community of analysts who are continuously mining the MetaSUB data. George Grills (University of Miami, Miami, USA). What we are doing in many ways is we are creating dynamic maps of microbiome prevalence that are integrated with human data. The infrastructure that we are building and the approaches we are developing will allow us to create weather maps of microbiomes that could help predict where there might be surges of infectious diseases, e.
ACKNOWLEDGMENTS We thank all members of the  MetaSUB International Consortium  for their continued contributions and support for this study. We'd also like to acknowledge all organizations and grants for their support, including but not limited to the  WorldQuant Foundation , the  Bill and Melinda Gates Foundation , the  Alfred P. Sloan Foundation , and the  Rockefeller Foundation. Editorially,  Dr. Abul Arif  and  Dr. Michelle Muzzio  at iScience facilitated the transcription of the meeting of the co-authors as well as worked with the editing and flow of the Backstory.
They are much more difficult to predict than the canonical interactions. They are often organized in recurrent sets, which form substructures folding in an identical or almost identical way  [9, 10]. These small substructures appearing recurrently at various locations in different RNA molecules are called structural motifs or modules. These modules play a fundamental role in the basic architecture of RNA structures, and their detection can thus drastically increase the quality of predictions of 3D RNA structures. Structural motifs appearing within a secondary structure loop have been extensively studied, and listed in regularly updated databases  [11, 12]. Many methods to detect them in tertiary structures have been developed. Some of these methods search for modules by their geometry  [13] [14] [15] [16] [17] [18] , while others use graph theory algorithms by modelling the interactions within these modules, and sometimes using the associated sequences  [7, [19] [20] [21] [22].
Search for class predictibility The question is to determine how much knowledge of the context around an A-minor motif can help to infer its presence (and position). For answering this question, we define four kinds of contexts of A-minor motif. Then for each class and each kind of context, we search for occurrences of the context in the BGSU 2020 dataset (see section 3. 1), and we look at whether an A-minor motif is present or not within the context. We define below the four kinds of contexts and the main measures we use for quantifying the 'predictibility' for a given class and a given context. Examples of inputs and outputs for a given class for each context are presented in Fig. 4 , and a summary of the properties of each kind of contexts is given in Table  1.
Sequence context. The sequence context of a given class is defined by its sequence representative (see section 2. 3. 2). For every class, we search for its two sets of regular expressions in every sequence in the BGSU 2020 dataset. So we get two sets of occurrences: the occurrences of the strand of the A-minor motif involving the nucleotides 1 and 2 (helix strand), and (A) (B) Figure 4. Examples of how PPV is computed for a given class, for every context (sequence, all-interactions, sequence-canonical-short-range, sequence-allinteractions). The maximum common subgraph of the class is presented on the right. The input is indicated for every context (regular expressions and/or subgraphs). For the regular expressions, the letter N indicates that every letter is authorized. For the subgraphs, the non-canonical edges in blue (resp.
red) are short range (resp. long range). Every input is searched in the sequences and/or RNA graphs contained in one of the two datasets (validation (val. ) or test). The outputs indicate the number of occurrences of one strand of an A-minor motif (True positives (TP)), and the total number of occurrences (Positives (Pos. )) that were found in each dataset. With these two values, the PPV can be calculated. the occurrences of the strand of the A-minor motif involving the nucleotides 5 and 6 (loop strand). To quantify the predictibility of the class, we are then interested in the proportion of actual A-minor motif occurrences among the results. This measure is generally known as the Positive Predictive Value (PPV) or Precision  [36]. It calculates the number of true positives out of the total number of positives.
Note also that, for a pair of regular expressions that represents a given class, we potentially find several loop strands and several helix strands in an RNA sequence. We cannot decide which ones of them are corresponding, without having more information on the structure. We also use the True Positive Rate, that is the number of actual A-minor occurrences that are found (True Positives), divided by the total number of A-minor occurrences that are present in the dataset (True Positives plus False Negatives): TPR ¼ #actual A À minor occurrences found #actual A À minor occurrences (3) We consider here that an A-minor motif occurrence is found when both its strands are found, using the sequence representatives of the loop strand and the helix strand of the same class.
2018 A-minors dataset The 2018 A-minors dataset is composed of 374 intramolecular non-redundant occurrences of type I/II A-minor motif occurrences from PDB structures with a resolution of 3 Å or better. These occurrences are taken from the database CaRNAval  [23]  (November 2018) and the structural contexts of these The all-interactions context, plus the sequence context Sequence-canonical-short-range context The previous one, where only the short range canonical edges are kept occurrences are taken from the PDB and annotated with the FR3D programme  [18]. The vast majority of the A-minor motif occurrences are originated from ribosomes (91%), however the dataset also contains occurrences from ribozymes, riboswitches and introns (Table  2 ). These molecules are found in 16 different organisms, including 8 bacteria, 2 archaea, 5 eukaryotes and 1 virus, and there are 15 occurrences for which the organisms are unspecified.
BGSU 2020 dataset This dataset contains the representatives of equivalence classes from the BGSU RNA group (release 3. 136, retrieved in June 2020)  [37] , with a resolution of 3 Å or better. It is composed of 1582 non-redundant RNA structures of various sizes from the PDB. For every RNA molecule of this dataset, we extracted its RNA graph (see section 2. 1) by using the FR3D annotation program  [18]. This dataset can be divided in two parts. The first part, that we call the validation set is composed of the molecules from where the 374 A-minor motif occurrences of CaRNAval have been extracted (see Table  2 ). It is composed of 136 structures of molecules. The second part, called the test set, is composed of all the other PDB structures. It is composed of 1446 structures of molecules and contains 87 A-minor motif occurrences (see Table  3  and Table 4 ).
Detection of homologous A-minor motif occurrences using Gutell alignments To detect them, we first use the sequence-structure alignments of Gutell  [39]  (retrieved in April 2021). A-minor motif occurrences from homologous molecules that are aligned in these alignments will be considered as homologous. In these alignments, 75% of the A-minor motif occurrences of the 2018 A-minors dataset are found.
Detection of homologous A-minor motif occurrences using a dedicated method For the 25% of occurrences that are not present in Gutell's alignments, we developed a specific method. We search for pairwise sequence similarity and structure similarity among the A-minor motif occurrences from homologous molecules. Thus, we perform a global alignment of the sequences of 30 nucleotides on both directions around each strand of the motif, for each pair of occurrences of the 2018 A-minors dataset. To do so, we apply the Needleman-Wunsch algorithm implemented in the EMBOSS Needle tool from EMBL-EBI  [40] , with standard parameters (gap opening penalty of 10, and gap extending penalty of 0. 5). We thus obtain three alignments for each pair of A-minor motif occurrences, one for each strand of the motif.
Finally, we group homologous occurrences with the single link approach: two occurrences are in the same homology group if they belong to the same connected component in the graph. In this way, we obtain 49 groups of 3 or more homologous occurrences, 27 groups of 2 occurrences, and 11 occurrences not homologous to any other occurrence, in the 2018 A-minors dataset.
Results This section presents the results obtained using the datasets we previously presented. In a first part, several classifications of the occurrences of the 2018 A-minors dataset, based on 3D similarities (see section 2. 2) will be presented. For every classification, we fixed an order of extension k equal to 3. This value of k was chosen experimentally, among a range between 1 and 10. It allows us to optimize the consistency between 3D similarity (through RMSD, see section 2. 2) and graph similarity, as it is described in  [28]. By choosing a k equal to 3, we can also note that our 3D classification will easily distinguish between A-minor motif occurrences involving a tetraloop and A-minor motif occurrences involving an internal loop, because RMSD will be higher between a tetraloop and an internal loop than between two tetraloops or two internal loops.
The first classification is based on 3-1,2,5,6-extensions (see Definition 2. 2). It will be called 1,2,5,6-classification. The two other classifications are based on extensions around only 3 nucleotides of the motif (3-1,5,6-extension and 3-2,5,6-extension). These classifications will be, respectively, called 1,5,6-classification and 2,5,6-classification. We will describe these three classifications and their particularities. In a second part, we will present the results of the predictibility study, on the classes of the 1,2,5,6-classification, by using the four kinds of context defined in section 2. 4. The source code of this study implemented in Python3 is freely available at https://cbe. uqam. ca/aminor_analysis/, along with a jupyter notebook to test some features. Note that the search for topological representatives for all the classes takes around 5 hours on a PC (Intel Core i5-7440HQ 4 × 2.
Towards prediction of A-minor motifs What kind of information, contained in the structural context, can be useful to characterize and predict the presence and the position of A-minor motifs? This is the question we address in this section. In section 2. 4 we defined four kinds of sequence and topological contexts for a class of embedded A-minor motif occurrences, and we defined two measures for quantifying how a given context can infer the presence of an A-minor motif occurrence in a RNA graph (see section 2. 4. 1): the PPV by strand and by class, and the TPR which we calculate for all classes together. Here we focus on the 1,2,5,6-classification, and we consider only the classes which contain at least 3 occurrences in the 2018 A-minors dataset. We show the results of predicting A-minor motifs in the BGSU 2020 dataset, given their contexts.
31 on average for the loop strand and 0. 13 to 0. 23 on average for the helix strand, and for the test set, 0. 02 to 0. 19 on average for the loop strand, and 0. 01 to 0. 13 on average for the helix strand). On the other hand, taking both the sequence and the canonical short-range interactions gives no advantage, globally, to predict A-minor motif occurrences compared to the all-interactions context (without sequence), especially for the loop strand (average PPV of 0. 31 and 0. 32 for the validation set). This is even the contrary in the test set (mean PPV of 0. 12 for the sequence-canonical-short-range context and 0. 19 for the all-interactions context). Moreover, the average PPV values are better for the loop strand than for the helix strand, for every context and every dataset. This is especially the case for the all-interactions context in the validation set, where the average PPV for the loop strand is 0.
31 and 0. 23 for the helix strand. Table  9  shows the global True Positive Rate (TPR) on both datasets. For both datasets, the results are fairly good. Obviously, in the validation set, all the A-minor motifs belonging to a class with more than 3 A-minor motif occurrences are found, since all these classes and only them have been taken into account for this study. The missing A-minor motif occurrences in this dataset are in classes of size 2 or isolated in the classification. In the test set, more than 60% of the motif occurrences are found with the sequence and all interactions context, which is the most restrictive one.
Classes with different profiles according to the predictibility contexts Detailed results on classes are shown in Fig. 10  and Fig. 11. Fig. 10  shows the PPV for the different classes and the different contexts in the validation set. They are sorted by increasing PPV in the sequence-all-interactions context for the helix strand. As before, only the classes which contain at least 3 occurrences in the 2018 A-minors dataset (and thus in the validation set) have been considered. Fig. 11  shows the PPV for the subset of classes considered for the test set (see above), using the validation set and the test set, in the sequence-all-interactions context.
We compared our classifications with the classification of A-minor motifs presented in  [27]. The authors use secondary structure considerations: they classify A-minor motifs according to the type of secondary structure elements involved in the motifs and their relative position to each other (local or long range). Our datasets are slightly different because they use the DSSR program to annotate the A-minor motifs and a set of representatives structures from the BGSU group (release 3. 76) to have only non redundant structures. Moreover, we consider only intramolecular type I/II A-minor motifs, which is not their case. We have in common 311 A-minor motif occurrences. The similarities and differences between this classification and our classifications are presented in Table  10. We then see that a majority of pairs of occurrences belonging to the same class in our classifications are also grouped together in the classification in  [27].
Figure11. PPV for the sequence-all-interactions context, with the two datasets (validation set in yellow and test set in purple). Have been considered only the classes for which it exists at least one A-minor motif occurrence from the test set, that shares similar local 3D structures with at least one A-minor motif occurrence of the class (RMSD equal to 2. 5 Å or better). For some classes, no occurrence was found in the test set. This case is represented by a dashed bar of size 1, to differentiate it from a PPV equal to 1.
Reconstruction Algorithms There have been many algorithms proposed for the task of CS and SR. However, our goal in this paper is not to compare these algorithms, instead, we want to demonstrate the applicability of SR in general for urban GHG emission assessments. We, therefore, only solve the initial SR problem, given in Eq. (P1). To find the minimum, we use the cvx library, a matlab package for specifying and solving convex programs  (Grant and Boyd (2014, 2008) ), using the gurobi optimizer as a backend (Gurobi Optimization, LLC, 2021). For the noisy case, we solve a modified form of Eq. (P1), given by x = arg min x x 1 s. t. Ax -b 2 2 ≤ 2 2 , (P1 ) where is the noise vector. This is equivalent to the Lasso with the right choice of λ. Since we generate the noise by design, the optimization process simplifies by choosing without having to find the right λ value.
The Sparsity/Compressibility of Emissions in European Cities In the following, we determine the sparsity and compressability of real world emissions. To do so, we study the CO 2 emissions of European major cities, such as Berlin, Hamburg, Munich, London, Paris, and Vienna using data from the TNO_GHGco_v1. 1 emission inventory (van der  Gon et al. , 2019). For the areas and species we investigate, the database provides annual grided anthropogenic emissions with a resolution of about 1 km × 1 km. Furthermore, the emissions are classified by its proxy, such as public power, industry, stationary combustion, etc. Emissions of all proxies of the dataset are included for the emission data used, making the assumption that this provides a good representation of typical emission fields for the investigated cities.
Code and data availability. The code for this paper (Zanger et al. , 2022b)  is written in Matlab 2021a and available on GitHub: https:// github. com/tum-esm/recovery_of_sparse_urban_greeenhouse_gas_emissions (last access: 25 January 2022). The Gaussian plume footprints are available at https://doi. org/10. 5281/zenodo. 5901298 (Zanger et al. , 2022a). The TNO_GHGco_v1. 1 emission inventory (van der Gon et al. , 2019) is not publicly available. The latitude and longitude boundaries used to generate the city emission fields from the inventory is given in Appendix A.
Input Data Sources The following input data is utilized to calculate soil erosion based on the RWEQ equations and direction shown in Figure  2. The digital elevation model (DEM) data were downloaded from the Shuttle Radar Topography Mission (SRTM) website for the country of Mongolia  [31]. Wind speed was sourced from the ERA5-Land Dataset on the Copernicus Climate Data Store  [32]. Monthly average temperature and total precipitation came directly from Worldclim current conditions  [33]. Solar radiation was calculated according to Penman  [34]. Precipitation days information from a public website was used  [35]. To justify the use of this website, the data were compared to Climate Hazards Group InfraRed Precipitation with Station data (CHIRPS), and found to be only 1 or 2 days apart. As a varying number of rain days was shown to have almost no impact on wind erosion results based on the sensitivity analyses, the travel website data served as a reasonable source of the number of rain days.
Clay, silt, and sand fractions were from ISRIC SoilGrids 250 m  [36]. Soil organic matter and monthly snow cover are outputs of the Rangeland Production Model  [37] , driven by Worldclim historical vegetation. Monthly herbaceous vegetation cover was calculated from herbaceous biomass outputs from the Rangeland Production Model, and a simple linear regression relating standing biomass to vegetation cover estimated from vegetation collected in the study area as described by Ahlborn  [38]. The temporal inputs were collected on a monthly basis, excepting wind speed, which is a daily data value that is used to calculate a total monthly wind factor. Due to the differences in resolution of each of our input data sources, each GeoTiff file was scaled to the resolution of the elevation file with Python, averaging values for groups of pixels for higher resolution data.