dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgements We thank  Richard Wheeler  for providing uorescent micrographs of Trypanosoma brucei, as part of TrypTagDB. This work was supported by grants from the  Natural Sciences and Engineering Research Council of Canada (NSERC ,  RGPIN-2014-03994 ) and from the  Gordon and Betty Moore Foundation  (https://doi. org/10. 37807/GBMF9201 ) to P. J. K. N. A. T. I. was supported by an  NSERC   Canadian Graduate Scholarship and a Junior Research Fellowship  from  Merton College, Oxford. A. A. P. was supported by.
Data and availability All data, including proteomes, protein families, annotations, alignments, phylogenies, and Python scripts for phylogenetic analyses, contamination scoring, and functional enrichments are available from Dryad (https://datadryad. org/stash/share/jT_8Q2Yh3197gDLiAFh4JBiTs0-WbKYg_DYD-3Zqml4) (Reviewer accessible link).
Analyzing the dynamical properties of mobile objects requires to extract trajectories from recordings, which is often done by tracking movies. We compiled a database of two-dimensional movies for very different biological and physical systems spanning a wide range of length scales and developed a general-purpose, optimized, open-source, cross-platform, easy to install and use, self-updating software called FastTrack. It can handle a changing number of deformable objects in a region of interest, and is particularly suitable for animal and cell tracking in two-dimensions. Furthermore, we introduce the probability of incursions as a new measure of a movie's trackability that doesn't require the knowledge of ground truth trajectories, since it is resilient to small amounts of errors and can be computed on the basis of an ad hoc tracking.
Here we take a different approach and provide a software designed to be as general as possible. First, we created an open dataset comprising 41 movies of very different systems in order to test and benchmark general tracking softwares. Then, we created the FastTrack software that implements standard image processing techniques and a performant matching procedure. FastTrack can handle deformable objects as long as they keep a constant area and manages flawlessly a variable number of objects. For the end user, all these features allow to obtain excellent trackings in minutes for a very large panel of systems. To achieve perfect trackings, FastTrack also has a manual November 16, 2020 2/29 post-processing tool that displaces the workload from the fine-tuning of complex detection algorithms to the manual correction of a few remaining errors, if any.
This strategy does not require programming skills and with an ergonomic interface it is time-saving for small size datasets or when there is a very low tolerance for errors. Furthermore, we propose a new quantifier called the probability of incursions, that can be computed based on a statistical analysis of the dynamical and geometric properties of each movie. We show that this probability displays a remarkable scaling with the logarithm of the sampling timescale, and that one can easily derive a robust and practical ad hoc characterization of virtually any movie. Finally, we implemented an algorithm to determine nearly-optimized tracking parameters automatically. FastTrack has already been used in a few publications  [15, 16]  and is currently used for several research projects in Physics and Biology.
Dataset We compiled various video recordings to form a large dataset called the Two-Dimentional Tracking Dataset (TD 2 ). It is open to new contributions and available for download at http://data. ljp. upmc. fr/datasets/TD2/. All videos have been either previously published  [13, [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26]  or have been kindly provided by their authors and are licenced under the Non-Commercial, Share-Alike Creative Commons licence (CC BY-NC-SA). Each movie has an unique identifier composed of three letters and three digits (e. g. ACT 001), that we use in the sequel anytime we need to refer to the data. The dataset comprises 41 sequences involving different types of objects at various scales: bacteria, paramecia, cells in a dense tissue, 7 animal species (including some whose behavior is commonly studied: fruit flies, medaka, zebrafish and mice), self-propelled particles, passive hard particles, droplets in microfluidic channels, centimetric robots, macroscopic objects on a conveyor belt, humans playing sports and traffic.
A summary of the key features of each movie in the dataset is presented in S1 The number of objects is constant in about half of the movies  (22) , while in the other half some objects appear or disappear at various locations in the field of view. November 16, 2020 3/29 Independantly, in about half the movies  (20)  the objects are moving in a strict two-dimensional space, while for the other half the objects evolve in a quasi-2D space and can at least partly overlap on some frames.
Software design and implementation FastTrack is written in C++ and respects the object-oriented programming paradigm. We use the OpenCV library for image processing and the Qt framework for the graphical user interface. The software wokflow is depicted in Fig 1 , and can be broken down in three main phases: detection, matching and post-process. As an entry point, the user defines data sources as video files (all formats supported must have a naming convention with left-padded zeros to ensure a correct ordering of the frames (e. g. frame 000001. pgm). The whole process described below can be applied on a movie-per-movie basis or for a batch of movies. In the latter case, the user can define different sets of parameters and background images, either manually or via a configuration file, and select subsets of movies for which the software will use those sets.
Processing of the dataset Being able to track objects in virtually any movie is an overwhelming challenge, and achieving this with one single software is, to date, utopic. This is primarily due to the versatility of the imaging conditions which multiply the number of approaches that are needed to correctly perform the detection phase. Two main pitfals can be discerned: the variations due to illumination (e. g. reflections as in GRA 001, shadows as in SOC 001 and TRA 001) and the partial overlaps between objects (e. g. HXB 001, ZFA 001). November 16, 2020 10/29 Still, many experimental setups in academia and production lines in industry are designed to mitigate these issues and it is common to have a uniform, diffuse and constant illumination, and either a compartimentation, strict two-dimensional confinment or low density of objects to avoid overlaps.
In the TD 2 dataset, approximately half of the movies (23, see S1 Table ) had sufficiently good illumination conditions to be tracked directly with FastTrack. The other movies required an additional pre-processing step, that was performed upstream with custom Matlab scripts. Severe and frequent overlaps impeded the processing of two movies (HXB 001 and ZFL 001), that were discarded for the remaining of the analysis. Then, the workflow performed robustly and we could track the objects in the remaining 39 movies. The Kuhn-Munkres algorithm performs in O(n 3 ) polynomial time  [42]  so the matching phase is generally fast; on a modern workstation we observed processing peaks at 500 frames/s and it took at most a few minutes with up to more than 2,000 objects in the field of view over 1,000 frames (GRA 001).
Classification of the dataset based on incursions To characterize and classify the different movies of the dataset, we introduce the notion of "incursions". Incursions happen every time an object travels sufficiently between to time points to exit its own Voronoï cell, defined at the initial time, thus entering one of it's neighbor's cell. With this definition the Voronoï cells are static boundaries defined by the initial frame, and the likelyhood of an incursion highly depends on the timescale τ over which the displacement if observed (Fig  3-A ), with τ = 1 corresponding to the time between two successive frames. On a global scale, the amount of incursions depends on the distribution of displacements but also on the density d (defined as the number of objects per unit area), the complex statistical properties of the geometry of Voronoï cells and the degree of motion alignment.
So, unless the ground truth trajectories are accessible, in most cases the single value of P inc at τ = 1 cannot be used as an ad hoc measure for the trackability of a movie. A timescale-varying analysis allowed us to extract more robust quantifiers. As p inc (ρ) does not depend on τ and R(ρ) is shifted to the high values of ρ when τ increases, one can easily expect that P inc (τ ) has a sigmoid-like shape. We thus computed P inc for various values of τ : for τ > 1 we tooks integer values (i. e. keep one frame every τ ) while for τ < 1 we linearly interpolated the displacements (i. e. multiplied ρ by τ ). We represented the results in Fig  3-C  for the 39 movies that could be tracked in the dataset. Strikingly, all P inc followed a logistic curve when τ is November 16, 2020 12/29 log-scaled so we used fits of the form: P inc = L 1 + e -k(log(τ )-x0) (3) and, noting τ 0 = e x0 , the fitting function can be rewritten as: P inc = L 1 + τ0 τ k (4) The fits are shown in ) follows the standard logistic sigmoid function f (x) = 1 1+e -x.
S5 Fig-B displays how P swap evolves as a function of h r for three recordings in the dataset. For low values of h r , P swap is essentially imposed by the distribution of the objects' displacements, since a high number of errors are generated when the objects are not allowed to move sufficiently. For higher values of h r , the distribution of the distances to the neighborsas defined by the Voronoï tesselation -starts to influence P swap as the algorithm becomes sensitive to incursions. It can also be more easily fooled by entries and exits at the boundaries of the region of interest when the number of objects in the scene varies. In between, for most recordings there is a gap yielding the minimal probability of error; this is particularly true when the objects are densely packed, since the distribution of distances to neighbors is sharper, like for DRP 001 where P swap drops to zero on a range of h r.
The acquisition framerate also has an important role here: with highly-resolved movies the distribution of displacements is shifted to the left, leading to a clear separation and low values of P swap , while for poorly-resolved movies like ZFJ 001 the two distributions overlap and P swap is always bound to high values. Similar analysis can be performed on the other tracking parameters. S5 Comparison with other softwares To assess the performance of FastTrack, we ran a benchmark with two state-of-the art tracking softwares that have been applied to various types of two-dimensional data: idtracker. ai [45]  and ToxTrac  [46]. First, it is worth mentionning that these softwares are much more difficult to install than FastTrack, and that they have strong intrinsic limitations as compared to FastTrack: both require a good framerate and image quality, with sufficient contrast and number of pixels per object, and a constant number of objects in the scene.
The benchmark was thus performed on a dataset constituted of a selection of videos that were provided with each software and some movies of the TD 2 dataset that meet these requirements. interface. The average tracking duration and the standard deviation were averaged over 5 runs using the built-in timer feature implemented in each software. The accuracy was evaluated manually using the built-in review feature implemented in each software. The number of swaps and the number of non-detected objects were counted in each movie. Occlusion events were ignored in this counting. The accuracy was computed as: A = n obj * n img -(2 * N swap + N undetected ) n obj * n img (10) with N swap the number of swaps, N undetected the number of non-detected objects, n obj the number of objects and n img the number of images.
For 100Zebra, the accuracy was computed only over the 200 first images. All the results are presented in Fig 5. In terms of speed FastTrack is orders of magnitude faster than idtracker. ai and significanly faster than ToxTrac on all tested videos. In terms of accuracy, all softwares performed extremely well, except idtracker. ai on ZFJ 001. FastTrack had a perfect accuracy on 8 movies out of 11 and always had an November 16, 2020 20/29 accuracy above 99%. In order to correct for the few errors, the ergonomic post-processing interface of FastTrack can be used to reach a perfect tracking within a few more minutes. Altogether, FastTrack offers many assets as compared to idtracker. ai and ToxTrac: it is more versatile and the total time spent to track a movie is globally lower -in some cases by orders of magnitude -without sacrificing the tracking accuracy.
Availability and Future Directions The source code of FastTrack is licensed under the GNU GPLv3 and can be downloaded at the following Github repository: https://github. com/FastTrackOrg/FastTrack. Developers implementing algorithmic improvements or new features are encouraged to submit their code via Github's pull request mechanism for inclusion in a forthcoming release. Pre-built binaries (AppImages) for Windows, Mac and all versions linux can be downloaded on the FastTrack website (http://www. fasttrack. sh). We also provide a PPA for Ubuntu 18. 04 that ensure proper system integration. The Windows and PPA versions incorporate an automatic update manager such that users always stay up-to-date with the latest stable release. For the Mac and Linux AppImages, updates have to be performed manually by downloading the latest version.
User's and developer's documentation are available on the FastTrack website. A video tutorial for beginners illustrating the complete workflow is available in Movie S2. We also provide an additional video presenting the post-processing panel in detail (Movie S3). The FastTrack algorithm only uses past and current information and is relatively fast, so it is in principle amenable to live tracking. Currenty the software doesn't natively integrate a live tracking feature, but it can be integrated inside any C++ project with minimal programming skills as explained in the developer's documentation (http://www. fasttrack. sh/API/index. html). In the future, we will continue the effort to make the TD 2 dataset grow, and greatly encourage new submissions from all fields of science. It is a useful basis for the development of general-purpose tracking softwares and a convenient material for benchmarking.
Data Collection and Extraction A data abstraction form was designed a priori with Excel (Microsoft Corp, Redmond Washington, DC, USA) to standardize data collection. Two independent reviewers (E. Buscail. and C. Ca. ) scanned all abstracts identified by search cross-referencing. The full text was then identified for each study that potentially met the inclusion criteria. Two reviewers (E. Buscail and C. Canivet) independently reviewed the full-text eligibility. If no consensus could be reached by the two reviewers and after discussion, a third specialist author was consulted and made the decision (B. Chaput). Articles that did not meet the inclusion criteria were excluded (Figure  1 ). Data extraction included general study information (age, sex ratio, BMI), neoadjuvant treatment (long/short-course radiotherapy and chemotherapy), surgical data (surgical resection technique, perineal reconstruction type), postoperative perineal wound healing details (total, major, and minor complication, and flap failure), and the postoperative length of stay and the duration of follow-up.
Literature Search and Results The initial search resulted in 2588 citations, and one citation was added with a manual cross-reference. Then, 2088 titles remained after removing duplicates. After screening, 71 publications were retrieved for full-text review. When data was not available, the corresponding authors were contacted: four of them answered  [18] [19] [20] [21] , and two of them could provide data details  [18, 21]. From the 71 articles, 53 articles were excluded, and the reasons are detailed in Figure  1. Finally, eighteen articles were included in quantitative synthesis (Figure  1 )  [6, 7, 9, 10, 18, [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33].
25,911 New Zealand dollars, p = 0. 012). Billig et al. compared a group of 2363 patients with primary closure and 194 patients with flap closure and found no significant difference between the two groups in terms of total and direct costs  [40]. Regarding cost-effectiveness, it was higher for the primary closure group ($259/healthy day) than for the flap group ($186/healthy day)  [40] ; nevertheless, this result was not significant (p = 0. 17). Currently, three ongoing multicenter trials could provide some answers as to which strategy to choose. The functional aspect was the question raised by the NEAPE trial (NCT 01347697)  [41]  comparing the gluteus flap to mesh closure. The issue of the gluteus flap is also studied by the BIOPEX-2 trial (NCT04004650)  [42]  with a primary closure control group.
Acknowledgments: PubMed citable collaborators: Denmark:  S. K. Perdawood ;  Department of Surgery, Slagelse Hospital , Ingemannsvej 18, 4200 Slagelse; Denmark:  Emilie Colov ,  Department of surgery C, Rigshospitalet, University of Copenhagen, Copenhagen ; Brazil:  Rodrigo Oliva Perez, MD ;  Ludwig Institute for Cancer Research Sao Paulo Branch ,  Rua Manoel da Nó  brega  1564 ,  Saõ Paulo ,  SP 04001-005 ; France:  Gilles Houvenaeghel, MD ;  Institut Paoli-Calmettes , 232  Bd de Sainte Marguerite ,  13009 ,  Marseille.
Classical data management solutions, i. e. , databases or data warehouses, only manage previously modeled structured data (schemaon-write approach). However, archaeologists need to store data of all formats and they may discover the use of data over time. Hence, we propose the use of a data lake  [2] , i. e. , a scalable, fully integrated platform that can collect, store, clean, transform and analyze data of all types, while retaining their original formats, with no predefined structure (schema-on-read approach). Our data lake, named ArchaeoDAL, provides centralized storage for heterogeneous data and data quality checking, cleaning, transformation and analysis tools. Moreover, by including machine learning frameworks into ArchaeoDAL, we can achieve descriptive and predictive analyses. Many existing data lake solutions provide architecture and/or implementation, but few include a metadata management system, which is nevertheless essential to avoid building a so-called data swamp, i.
To illustrate the flexibility and completeness of ArchaeoDAL's architecture, we provide a concrete implementation dedicated to the HyperThesau project. With a fully integrated metadata management and security system, we can not only ensure data security, but also track all data transformations. The remainder of this paper is organized as follows. In Section 2, we review and discuss existing data lake architectures. In Section 3, we present ArchaeoDAL's abstract architecture, implementation and deployment. In Section 4, we present two archaeological application examples. In Section 5, we finally conclude this paper and present future works.
ARCHAEODAL'S ARCHITECTURE, IMPLEMENTATION AND DEPLOYMENT In this section, we propose a generic, flexible abstract data lake architecture that covers the full data lifecycle (Figure  2 ) and contains eleven layers. The orange layers (from layer 1 to layer 6) cover the full data lifecycle. After data processing in these layers, data become clearer and easier to use for end-users. The yellow layers cover non-functional requirements. After the definition of each layer, we present how each layer is implemented in our current ArchaeoDAL instance. As this instance is dedicated to the project HyperThesau, it does not cover all the features of the abstract architecture. For example, real-time data ingestion and processing are not implemented. However,real-time or near real-time data ingestion and processing feature can be achieved by adding tools such as Apache Storm 2  or Apache Kafka 3  in the data ingestion and data insights layers.
Data Source Layer In the data source layer, we gather the basic properties of data sources, e. g. , volume, format, velocity, connectivity, etc. Based on these properties, data engineers can determine how to import data into the lake. If metadata are required, data engineers must also find the best fitting metadata model to govern input data. In our instance of ArchaeoDAL, we have data sources such as relational databases and various files stored in the archaeologists' personal computers.
Data Ingestion Layer The data ingestion layer provides a set of tools that allow users to perform batch or real-time data ingestion. Based on the data source properties that are gathered in the data source layer, data engineers can choose the right tools and plans to ingest data into the lake. They must also consider the capacity of the data lake to avoid data loss, especially for real-time data ingestion. During ingestion, metadata provided by the data sources, e. g. , the name of excavation sites or instruments, must be gathered as much as possible. After data are loaded into the lake, we may lose track of data sources. It is indeed more difficult to gather this kind of metadata without knowledge about data sources. ArchaeoDAL's implementation exploits Apache Sqoop 4  to ingest structured data and Apache Flume 5  to ingest semi-structured and unstructured data.
Data Storage Layer The data storage layer is the core layer of a data lake. It must have the capacity to store all data, e. g. , structured, semi-structured and unstructured data, in any format. ArchaeoDAL's implementation uses the Hadoop Distributed File System 6  (HDFS) to store ingested data, because HDFS stores data on commodity machines and provides horizontal scalability and fault tolerance. As a result, we do not need to build large clusters. We just add nodes when data volume grows. To better support the storage of structured and semi-structured data, we add two tools: Apache Hive 7  to store data with explicit data structures and Apache HBase 8  that is a distributed, versioned, column-oriented database that provides better semi-structured data retrieval speed.
Data Distillation Layer The data distillation layer provides a set of tools for data cleaning and encoding formalization. Data cleaning refers to eliminating errors such as duplicates and type violations, e. g. , a numeric column contains non-numeric values. Data encoding formalization refers to converting various data and character encoding, e. g. , ASCII, ISO-8859-15, or Latin-1, into a unified encoding, e. g. , UTF-8, which covers all language symbols and graphic characters. ArchaeoDAL's implementation uses Apache Spark 9  to clean and transform data. We developed a set of Spark programs that can detect duplicates, NULL values and type violations. Based on the percentage of detected errors, we can hint at data quality.
Data Insights Layer The data insights layer provides a set of tools for data transformation and exploratory analysis. Data transformation refers to transforming data from one or diverse sources into specific models that are ready to use for data application, e. g. , reporting and visualization. Exploratory data analysis refers to the process of performing initial investigations on data to discover patterns, test hypotheses, eliminate meaningless columns, etc. Transformed data may also be persisted in the data storage layer for later reuse. ArchaeoDAL's implementation resorts to Apache Spark to perform data transformation and exploratory data analysis. Spark also provides machine learning libraries that allow developers to perform more sophisticated exploratory data analyses.