dataesr/scientific-paragraphs-categorization huggingface dataset:
Module labels, descriptions, and scripts in scripted modules are compared using Levenshtein edit distance  [23]. pw3 compares single attributes in the same way as pw0 but uses higher, but not uniform weights for labels, script and service uri, followed by service name and service authority in the order listed, similar to  [34]. pll disregards all attributes but the labels, and compares them using the Levenshtein edit distance, resembling the approach taken in  [4]. plm disregards all attributes but the labels, and compares them using strict string matching as done in  [33, 18, 38].
Topological Workflow Comparison Regarding topological comparison of scientific workflows, existing approaches can be classified as either a) structure agnostic, i. e. , based only on the sets of modules present in two workflows,  [34, 33, 36, 4] ; b) based on substructures of workflows, such as maximum common subgraphs  [33, 18, 17]  or graph kernels derived from frequent subgraphs  [17] ; or c) using the full structure of the compared workflows  [38]. We include an approach to topological comparison for each of these classes. We denote the DAG of a workflow as G wf = (V wf , E wf ). Sets of Modules -Analogous to the similarity measure described in  [34, 33, 36, 4] , two workflows wf 1 and wf 2 are treated as sets of modules. The additive similarity score of the module pairs mapped by maximum weight matching (mw) is used as the non-normalized workflow similarity nnsimMS, with sim(m, m ) denoting a module pair's similarity value: nnsimMS = sim(m, m ) | (m, m ) ∈ mw(V wf 1 , V wf 2 ) Sets of Paths -As a slightly relaxed version of using the maximum isomorphic subgraph for workflow comparison  [33, 18, 17] , the sets of all paths two DAGs are comprised of can be used to compare them by their maximum similar subgraph  [22].
SUBDUE allows labels to identify nodes in a graph, which it uses during the graph matching process. To transform similarity of modules to identifiers, we set the labels of nodes in the two graphs to be compared to reflect the module mapping derived from maximum weight matching of the modules during conversion of the workflows to SUBDUE's input format. The workflows' non-normalized similarity is then computed as nnsimGED = -costGED For computing graph edit distance, we keep SUBDUE's default configuration which defines equal costs of 1 for any of the possible edit operations (as in  [32] ). Testing several different weighting schemes did not produce significantly different results.
1. 3). The step of normalization has been approached rather heterogeneously in previous work (see Table  1 ). Next to the consideration of workflow size taken therein, our general intention is to acquire similarity values in the range of [0,1]. For set based topological comparisons we resort to a variation of the Jaccard index for set similarity: The Jaccard index is used to express the similarity of two sets A and B by their relative overlap |A∩B|  |A∪B| which is equivalent to |A∩B| |A|+|B|-|A∩B|. Our modification reflects the fact that mapped elements of our sets of modules or paths are mapped by similarity, not identity. Thus, the size |V wf 1 ∩ V wf 2 | of the overlap between the two module sets, for instance, is replaced by the overlaps maximum similarity score captured by nnsimMS to derive the overall module set similarity of two workflows: simMS = nnsimMS |V wf 1 | + |V wf 2 | -nnsimMS The rational here is that where the classical Jaccard index compares the number of mutual elements in the sets with their overall number of (distinct) elements, our modification compares the amount of similarity of similar elements from the sets with the non-similar remainder.
For instance, one such class holds all web-service related module types. Modules within each such class may be compared and mapped onto each other. This introduction of type equivalences was motivated by the observation that in the used dataset of Taverna workflows, especially web-service modules are typed with a variety of identifiers, such as 'arbitrarywsdl', 'wsdl', or 'soaplabwsdl'. Importance Projection Preprocessing -Not all modules in a scientific workflow contribute equally to the workflows specific functionality: Modules used most frequently across different workflows often provide trivial, rather unspecific functionality, such as splitting a string into a list of strings  [35]. To account for this, we assign a score to each module indicating the importance of the module for a workflow's specific functionality.
EXPERIMENTAL SETUP The goal of this work is an unbiased and comprehensive benchmark of different methods for similarity search in a corpus of scientific workflows. To this end, we reimplemented all methods described in Section 3, structured according to the steps of our comparison framework. We collected a quite comprehensive corpus of similarity ratings for a set of Taverna workflows from the myExperiment scientific workflow repository  [32]. A second set of workflows was assembled from the Galaxy online repository  [20]. Gold standard ratings were obtained from 15 field experts and aggregated using median ranking. We compare all algorithms regarding ranking correctness and retrieval precision. These topics are discussed in more detail in the following. Note that all data will be made publicly available.
Workflow Dataset myExperiment provides a download of all workflows publicly available in its repository. For Taverna workflows, which make up approx. 85% of myExperiment  [35] , this dataset contains an rdf representation of the workflow structure, along with attributes of the modules used and annotations provided by authors. We transformed all workflows into a custom graph format for easier handling. During this transformation, subworkflows were inlined and input and output ports were removed. The complete dataset contains 1483 workflows. myExperiment -and thus our dataset -is generally domain agnostic. To match the expertise of our expert curators, we focussed on workflows from the life sciences domain in the evaluation. We will discuss possible implications of this decision in Section 6.
Expert Ratings We conducted a user study to collect manually assigned similarity ratings for selected pairs of scientific workflows. Overall, 15 domain experts from six different institutions participated. Ratings were obtained in two phases: In a first experiment, the goal was to generate a corpus of ratings independent of a concrete similarity measure to make it suitable for evaluation of large numbers of different measures, and measures to be developed in the future. 24 life science workflows, randomly selected from our dataset, (called query workflows in the following) were presented to the users, each accompanied by a list of 10 other workflows to compare it to. To obtain these 10 workflows, we ranked all workflows in the repository wrt a given query workflow using a naive annotation based similarity measure and drew workflows at random from the top-10, the middle, and the lower 30.
3. While we do see a few outliers, most experts are rather d'accord about how workflows are ranked. In a second experiment, the selected algorithms were run to each retrieve the top-10 similar workflows from our complete dataset of 1483 Taverna workflows for eight of the 24 query workflows from the first experiment. The results returned by each tested algorithm were merged into single lists between 21 and 68 elements long (depending on the overlap in the algorithm's top-10). Naturally, these lists did contain workflows already rated in the first experiment. Experts were now asked to complete the ratings using the same scale as before. The ratings provided in this second experiment qualify each workflow in the search results of each of the used algorithms wrt their user-perceived similarity.
Baseline Evaluation Figure  5  shows mean ranking correctness and completeness over all query workflows for each of the similarity algorithms under investigation. Note that for Graph Edit Distance, we allowed match cost computation of each of the 240 pairs of scientific workflows to take a maximum of 5 minutes. 23 pairs not computable in this timeframe were disregarded in the evaluation. All algorithms are used in their basic, normalized configurations with uniform weights on all module attributes. The simBW algorithm has the best results in terms of ranking correctness. simBT and the structural similarity measure simP S almost tie, followed by simMS. simGE delivers worst performance and is the only algorithm in this set with a statistically significant (p<0. 05, paired ttest) difference to simBW.
These findings largely confirm those of previous work (see Section 3) that annotational measures outperform structural workflow similarity -in certain settings. The good performance of simBW is not surprising: Well written titles and descriptions capture the main functional aspects of a workflow, and provide a strong means for judging workflow similarity. Interestingly, while most structural similarity measures are complete in their ranking, both annotational measures tie some of the compared workflows where experts see them differently ranked. This is especially true for simBT , which, additionally, is not able to provide rankings for four of the given query workflows due to lack of tags. These workflows were not considered for computation of simBT ranking performance. Note that around 15% of the workflows in our complete dataset lack tags.
Module Comparison (pX) Figure  6a  shows the impact of the different module similarity schemes (see Section 2. 1. 1 and Table  2  for an overview of the used notation) on ranking correctness by the example of the simMS algorithm. Trends are similar for the other measures (data not shown). Clearly, the uniform weighting scheme pw0 used in the baseline evaluation of Figure  5  performs worst (p<0. 05). Using only the edit distance of labels for module comparison (pll ) is on par with the more complex scheme pw3 (using refined weights on various attributes) in terms of mean ranking correctness. Note that there is a minimal reduction in ranking completeness for pll, resulting from the less fine grained similarity assessment. This reduction in completeness is much more pronounced when using label matching (plm) for assessing module similarity.
Further investigation showed that the striking increase in ranking correctness plm achieves is in fact due to this reduction in ranking completeness, as pairs of workflows tied by the algorithmic ranking are not accounted for when determining ranking correctness: while the most similar workflows are ranked high, less similar ones are not distinguished in terms of their similarity any more. The (to us) surprisingly good performance of the label only approaches, especially pll, shows that while the author base of the workflows in our dataset is heterogeneous, the labels given to modules mostly do reflect their function and thus convey measurable information about their similarity. Yet, the strict matching of labels, as used in many previous studies, offers to little fine grained similarity for differentiated assessment of workflow similarity.
We will see how this observation affects workflow retrieval in Section 5. 2. The change in the algorithms' ranking performance induced by both the pw3 and pll weighting schemes (see Fig. 6a  and b ) puts these algorithms ahead of simBW , if yet not significantly. The exception here is simGE, where the impact of different weighting schemes is less pronounced. The reason for this most probably lies in the specific type of structural comparison applied: The more overall workflow topology is taken into account, the less do differences in similarity assessment of single pairs of modules matter. From here on, we denote the module weighting scheme  used with an algorithm by specifying the corresponding pX in its name (e. g. , GE pw3 refers to the simGE using the pw3 module attribute weighting scheme for module comparison).
Including Repository Knowledge Module Pair Preselection (tX) -Figure  8a  (first bar) shows the impact of the type equivalence (te) module pair preselection strategy proposed in Section 2. 1. 5 for simMS. The trends of these changes are similar for all algorithms: While strict type matching significantly decreases the algorithms' ranking correctness (data not shown), the use of equivalence classes results in correctness values comparable to those without any restrictions on module pair selection. This is remarkable, as it shows that a) the technical classes of modules (web-service, script, local operation) do play an important role in determining a modules function; and b) this technical distinction of modules is also detected when comparing all pairs of modules -even when using only labels.
While te doesn't lead to an improvement of user perceivable ranking quality, the exclusion of certain module pairs  This positive impact of ip on the correlation of algorithm rankings with expert rankings confirms our intention of removing presumably unimportant modules, and confirms our selection of modules to keep. Yet, such manual selection is only possible with both in depth understanding of the types of modules used in a dataset of scientific workflows, and knowledge about their specific relevance to a workflow's functionality. An interesting line of research to be explored in future work are methods to perform such a preselection automatically, for instance, based on module usage frequencies. As a side effect, ip leads to a decrease in the average number of modules per workflow from 11.
3 to 4. 7 in our dataset, resulting in a significant increase in computational performance of all structural algorithms. This effect is especially relevant for Graph Edit Distance computation: where simGE was able to compute the similarities of only 217 of the 240 workflow pairs of our ranking experiment within the per-pair timeout of 5 minutes without using ip, it can now compute all pairs but one. From here on, we denote the use of importance projection with an algorithm by specifying 'ip' in its name, and 'np' for its omission.
Module Comparison Figure  10  graphs retrieval precision at k for simMS with various module similarity schemes pX, with and without ip and te. Three different relevance thresholds related, similar, and very similar are considered. Interestingly, the differences in retrieval quality decrease with the increase in relevance level up to the point where all configurations deliver similar performance for retrieval of very similar workflows. Apparently, finding the most similar results is independent of the module similarity scheme used. For the retrieval of related workflows, on the other hand, where more fine grained assessment of similarity is required, differences between the schemes become visible. The strict label matching approach of plm performs worst, confirming our observations from the previous experiment.
Workflow Similarity Figure  11  shows retrieval precision for the structural and annotational workflow comparison approaches under evaluation. The structural approaches are used with the pll module similarity scheme of edit distance comparison of labels, and have been applied with and without ip and te. Note that for simGE we only include results for preprocessed workflow graphs (ip). The strict topological comparison applied by simGE does find the most similar workflows equally well as its structural contenders, but provides much worse results when looking for similar or related workflows: As simGE puts a strong emphasis on workflow structure, it also retrieves workflows from other domains than the query workflow, which happen to be similar to the query workflow in terms of their graph structure.
Evaluation on Second Data Set Figure  12  shows ranking results on our second dataset of Galaxy workflows, for which we repeated our first experiment on workflow ranking using 8 query workflows. The module comparison schemes used are gw1, comparing a selection of attributes with uniform weights, and gll, comparing only module labels by their edit distance. The most striking observation is that simBW doesn't provide satisfying results on this data set, as the Galaxy workflows carry less annotations. While, overall, results for the structural algorithms are less convincing than on Taverna workflows, our observations are generally confirmed: structure agnostic comparison by simMS and comparison respecting substructures by simP S outperform the strict comparison of full workflow structure performed by simGE.
We do, however, believe that the life sciences are a particularly difficult domain for workflow comparison, due to the large number of different groups developing different tools (and workflows), even for similar tasks, leading to difficult task-matching issues. Our most important findings are: 1. For module comparison, the edit distance of module labels seems to be the best approach: It provides best results in retrieval and does not require refinement of multiple attribute weights; and it provides a more fine grained assessment of similarity than label matching, which, in turn, can only be recommended for retrieval of the few most similar results. Of course, these findings are only valid if labels are telling, i. e. , are indicative for the functionality of the labeled module. Such workflows include the studied Taverna workflows from the myExperiment repository, but also the majority of workflows found in the SHIWA repository  [2].
2. We have shown that structural approaches can outperform annotational ones when configured appropriately. Especially in repositories where workflows are not well annotated by descriptions or tags, such as the Galaxy repository  [20]  inspected here, or the CrowdLabs repository of VisTrails workflows  [28] , structural approaches are indispensible. While full structural comparison by Graph Edit Distance appears to be to strict -similar to label matching on the module level -, comparing workflows either by substructures such as paths or by the sets of modules they contain provide comparably convincing results. This is good news, as module set comparison is computationally far less complex than comparing substructures. Yet, the fact that Path Sets comparison is more stable in its results across different configurations indicates room for further research to include topological information with less computational complexity.
ACKNOWLEDGMENTS We sincerely thank  K. Belhajjame ,  J. Brandt ,  M. Bux ,  L. Childs ,  D. Garijo ,  Y. Gil ,  B. Haldemann ,  S. Kröger ,  P. Larmande ,  B. Ludäscher ,  P. Missier ,  C. Toffano , and  K. Zimmermann  for contributing workflow similarity ratings. This work was partly funded by  DFG  grant  GRK1651 ,  DAAD  grant  55988515  and  PHC   Procope grant. Work partially done in the context of the  Institut de Biologie Computationnelle, Montpellier, France.
Spatial and temporal genetic simulation using SPLATCHE2 Computer simulations mimic the processes that occur in the real world and allow us to study which patterns may affect systems. We have developed the program SPLATCHE2 (http://www. splatche. com) (Ray et al. 2010) , which performs spatially explicit simulations of genetic data under environmental variable scenarios and accounting for recombination, complex migration and long-distance dispersal. As input, the program requires a map (specified by a grid of demes) where the carrying capacity (K) and the migration rate must be user-specified for each deme. Optionally, both K and migration rate can change with time (moreover, anisotropic migration is also implemented). Other important inputs are related with demography (e. g. , initial population size and geographic origin, growth rate, generation time, total number of generations and a number of demographic models).
Then, SPLATCHE2 performs a demographic simulation over the map followed by a coalescent simulation based on user-defined samples (Figure  1 ). The coalescent simulation just traces the evolutionary history of the sampled genes going backwards in time until their most recent common ancestor. It is followed by a simulation of genetic data (DNA, STRs and SNPs) along the coalescent (gene) genealogy. Although the model makes several assumptions (like a molecular clock or non-overlapping generations) it is probably one of the most realistic software available and has been used in a variety of important publications. Below we describe some important applications of this program for scaling genetic diversity under complex evolutionary scenarios.
e. as measured by the genetic parameter F ST ). Similarly, sexbiased dispersal could be measured by comparing the level of genetic structure inferred from nuclear markers (inherited by both parents) to that indicated by mtDNA (which is transmitted only by females) or Y chromosome (transmitted only by males). If the level of genetic differentiation inferred from mtDNA is higher than that inferred from nuclear markers, male-biased dispersal may be assumed. Simulations, undertaken with a different program inspired by SPLATCHE2  (Rasteiro et al. 2012) , clearly show that different patterns of genetic differentiation can be detected under three scenarios, 1) bilocality (no sex-biased dispersal), 2) matrilocality (male-biased dispersal), and 3) patrilocality (female-biased dispersal, Figure  5 ).
Modern scientific experiments collect vast amounts of data that must be catalogued to meet multiple use cases and search criteria. In particular, high-energy physics experiments currently in operation produce several billion events per year. A database with the references to the files including each event in every stage of processing is necessary in order to retrieve the selected events from data storage systems. The ATLAS EventIndex project is developing a way to store the necessary information using modern data storage technologies that allow saving in memory key-value pairs and select the best tools to support this application from the point of view of performance, robustness and ease of use.
Introduction The ATLAS experiment has a catalogue of all real and simulated events in a database (TAGDB). The TAGDB is implemented in Oracle, as that was the only proven technology that could hold the impressive amount of expected data when this project started long before the start of LHC operations  [1]  [2][3]  [4]. The main purposes of this database were the selection of events on the basis of physical variables, the identification of files that contain a list of events already selected by other means, and consistency checks on the completeness of event processing. In TAGDB each event is recorded several times, once for each reconstruction cycle, and contains three main data blocks: information identifying the event (event and run number and trigger decisions), information allowing the localisation of the event data in files that contain it, and physical variables of the event, such as the number of reconstructed particles, their identification and properties, which might be useful to select events for specific analyses.
The TAGDB, with all related services and user interfaces, works for the retrieval of event information for limited numbers of events and for the technical checks described above; the event selection use case by physics variables turned out not to be generally useful because sometimes those quantities changed after the initial reconstruction and there was no mechanism in the system to update them. Nevertheless the TAGDB is slow, intrinsically complex and at times unreliable. In addition, the enormous amount of data (1 kB per record, which turns into several tens of TB in Oracle for all data collected by ATLAS so far) makes the implementation in Oracle particularly labour-intensive and expensive  [5]. Oracle storage capacity was deployed to store the ~8 billion real and simulated events recorded during LHC Run1; future data taking rates will be higher, exceeding the limits of Oracle scalability required by the existing workflow  [6]   [7].