dataesr/scientific-paragraphs-categorization huggingface dataset:
Dimensionality reduction: Correlation with QC measures To evaluate the dependence of the inferred low-dimensional signal on unwanted variation, we computed the absolute correlation between each dimension (e. g. , principal component) and a set of quality control (QC) measures. For the V1 dataset, FastQC (http://www. bioinformatics. babraham. ac. uk/projects/fastqc/) and Picard tools (https://broadinstitute. github. io/picard/) were used to compute a set of 16 QC measures. These measures are available as part of the Bioconductor R package scRNAseq (https://bioconductor. org/ packages/scRNAseq). For the Glioblastoma and mESC datasets, we used the scater Bioconductor R package  [60]  (Version 1. 2. 0) to compute a set of 7 QC measures. For the S1/CA1 dataset, we used a set of 6 QC measures provided by the authors (http://linnarssonlab.
Dimensionality reduction: Correlation of pairwise distances between observations For simulated data, we assessed different dimensionality reduction methods (ZINB-WaVE, PCA, and ZIFA) in terms of the correlation between pairwise distances between observations in the true and in the estimated reduced-dimensional space. In particular, we monitored the influence of the number of unknown cell-level covariates K when the other parameters are set correctly (V is a column of ones 1 J and genewise dispersion). For each simulation scenario, the correlation between true and estimated pairwise distances was computed and averaged over B datasets.
Bias and MSE of the ZINB-WaVE estimators For data simulated from the ZINB-WaVE model, let θ and θb , b = 1,. , B, respectively denote the true parameter and an estimator of this parameter for the b th simulated dataset. Then, for each scenario, the performance of the estimator θ can be assessed in terms of bias and mean squared error (MSE) as follows: Bias = 1 B B b=1 ( θb -θ), MSE = 1 B B b=1 ( θb -θ) 2. Bias and MSE were computed for β µ , β π , γ µ , γ π , ζ, W α µ , W α π , ln(µ), and logit(π). When the parameter to be estimated was an n × J matrix, the matrix was converted to a 1 × nJ row vector and bias and MSE were averaged over the elements of the vector.
(d-e) Discovery of novel cell types for the 10x Genomics 68k PBMCs dataset. (d) Scatterplot of first two t-SNE dimensions obtained from 10 components of W from ZINB-WaVE; cells are color-coded by cluster. Clustering was performed on the 10 components of W (see Methods). (e) Heatmap of expression measures for marker genes for the 18 clusters found by our procedure: columns correspond to clusters and rows to genes; the value in each cell is the average log expression measure per cluster, centered and scaled so that each row has mean zero and standard deviation one.
See Supplementary Figure S27 for the same scenario but with n = 10, 000 cells and Supplementary Figure S28 for additional scenarios. (e-g) Average silhouette widths (over n samples and B = 10 simulations) for true clusters vs. zero fraction, for n ∈ {100, 1, 000, 10, 000} cells, for datasets simulated from the Lun & Marioni [42] model, with C = 3 clusters and equal number of cells per cluster. While ZINB-WaVE was relatively robust to the sample size n and zero fraction, the performance of PCA and ZIFA decreased with larger zero fraction. Between-sample distance matrices and silhouette widths were based on W for ZINB-WaVE, the first two principal components for PCA, and the first two latent variables for ZIFA. ZINB-WaVE was applied with X = 1 n , V = 1 J , genewise dispersion, and K ∈ {1, 2, 3, 4} (only K = 2 is shown in (e-g)).
Acknowledgments We would like to thank  Olivier Mirabeau  from the  Curie Institute  for the pre-processing of the Glioblastoma dataset. We also thank  Aaron Lun  and the  Marioni Lab  for making the code needed for the simulations available online. Finally, we are grateful to  Russell Fletcher  for his help with the re-analysis of the OE data. DR and SD are supported by the  National Institutes of Health BRAIN Initiative  (grant  U01 MH105979 , PI:  John Ngai ). SG and JPV are supported the  French National Research Agency  (grant  ABS4NGS ANR-11-BINF-0001 ). JPV is supported by the  European Research Council  (grant  ERC-SMAC-280032 ), the  Miller Institute for Basic Research in Science , and the  Fulbright Foundation.
2 m 3 , MASCOT carries four scientific instruments, for a total mass of 3 kg: a camera CAM  (Jaumann et al. 2016, this issue) , a magnetometer MAG  (Hercik et al. 2016, this issue) , a radiometer MARA  (Grott et al. 2016, this issue) , and MicrOmega, an hyperspectral microscope designed, developed and tested at IAS/Orsay, under full CNES (Centre National d'Etudes Spatiales) support. MASCOT, as a system, has been developed by DLR in Bremen, in cooperation with CNES. MASCOT measurements will both provide unique data sets acquired in situ, and complement orbital measurements performed from the mother spacecraft, as well as serving as references for future analyses of the returned samples. MicrOmega, onboard the MASCOT lander, will have a central role in achieving these objectives. In particular, Mi-crOmega will acquire reflectance spectra in the near-IR of ∼3.
Electronics Hardware Electronic functions are implemented on two boards with separated functions; one board is implemented inside the MASCOT common E-box (MicrOmega Electronic Unit /MEU) while the second (MicrOmega Proximity Electronic) is implemented inside the MSU. The MicrOmega Electronic Unit provides simple power interface between MASCOT PCDU and the instrument, the SpaceWire/data interface (nominal and redundant link) and the control of all electrical subsystems of MSU. It also manages some simple data processing and a stand-alone test mode. The MicrOmega Proximity Electronic is mounted inside the MSU, close to the analog parts of the instrument (detector, temperature sensors), to minimize the potential perturbations. There are 3 interconnect harnesses (MIH), under DLR responsibility, between MASCOT E-box and MSU.
Software To insure optimization at system level, the MicrOmega instrument has a very limited data handling and operational autonomy and relies on the MASCOT On Board Computer (MOBC) to sequence its operations and to process the data before transmission to Earth. The 3D hyperspectral data (x, y, λ) are obtained by sequentially acquiring 2D images (x, y) at different wavelengths (λ). The basic interface level is the request by the MOBC of one image from MicrOmega and the transmission of the acquired data. After reception of the image, the next one is requested. The number of images and the specification of all the parameters associated with each image (e. g. integration time, summing, RF frequency) are set by the MOBC, based on a ground command defining the sequence of observations. The 3D data cubes (x, y, λ) are compressed onboard via the Langevin and Forni (2000) algorithm before transmission to the ground.
1) Performance Margin Model Development: The performance margin model is developed in two steps. First, a model for the p HSA is developed based on hydraulic principles, with the help of commercial software AMESim. The AMESim model is given in Figure  7. Coherently with (2), the model in Figure  7  is written as p HSA = g HSA (x HSA ). ( 14 ) Second, as p HSA is a STB performance parameter, the performance margin of the HSA can be determined according to  (1) : m HSA = 1 p th (p th -g HSA (x HSA )). (15) 2) Aleatory Uncertainty Evaluation: The x HSA comprises six parameters, namely, the clearances on diameters (CoDs) of the six components of the HSA. The CoDs are subject to aleatory uncertainties from production and manufacturing processes, which are quantified by the tolerances in Table  III.
B. Single Board Computer A SBC, as shown in Figure  8  [41], is chosen to demonstrate the time-dependent belief reliability analysis for electrical systems. A probabilistic reliability analysis was conducted in [41] based on the parts-counting reliability prediction method in  [42]. The times to failure of both the components are assumed to be exponentially distributed and their failure rates are predicted based on the database in [42], as shown in Table  V. The failure rate of the SBC can, then, be calculated by summing over all the components' failure rates. Hence, the predicted probabilistic reliability is R p (t) = exp{-1. 186 × 10 -6 t}, (17) where the unit of t is hour. The probabilistic reliability in (  17 ) is a time-dependent function. To further evaluate the belief reliability, first note by substituting (5) into (8), we have R B = 1 1 Z Rp 2 + σ e m d 2.
ACKNOWLEDGMENT This work has been performed within the initiative of the  Center for Resilience and Safety of Critical Infrastructures (CRESCI , http://cresci. cn). The research of  Dr. Zhiguo Zeng  is partially supported by  Natural Science Foundation of China (NSFC)  under grant No. 71601010. The research of  Dr. Meilin Wen  is supported by  Natural Science Foundation of China (NSFC)  under grant No. 71671009. The research of  Prof. Rui Kang  is supported by  Natural Science Foundation of China (NSFC)  under grant No. 61573043.
The underlying structure of and relationships between each stage in a PMx analysis, the assumptions associated with these, and the transitions between stages (supported by decisions) can be highly complex. Figure  1  provides a detailed illustration of a typical PMx workflow. 2 The PMx analysis consists of a sequence of tasks requiring the integration of different tools and analysis methods-for example, preparing/cleaning/merging datasets from various sources (which might have different formats, and originate from different databases), performing exploratory analyses, making assumptions based on those, preparing for estimation (further data manipulation, setting initial estimates, defining task execution information), estimating parameters in the model, examining model diagnostics, performing model validation, testing assumptions made, and collating all information to make inferences and inform decisions.
It is essential to ensure data integrity, maximize quality assurance and reproducibility in order to enhance the contribution of model-based methods within the regulatory submission and review cycle. 3 n addition, capturing and reporting provenance information is important in supporting compliance with regulatory expectations of a computer system producing material that may form part of a regulatory interaction (FDA regulation 21 CFR Part 11). 4  A system that can perform this function will support these regulatory requirements, and will encourage transparency of the rationales and details of the work for communication both internally and externally with regulators and other collaborators. The Drug Disease Model Resources (DDMoRe) consortium was created to improve the quality, efficiency, and cost-effectiveness of model-informed drug discovery and development, a set of concepts, principles, and recommendations that have evolved into what is now termed MID3.
Building on an established standard also brings with it a large body of experience in other applications besides PMx. Briefly, the PROV-O ontology defines a series of terms relating to entities, activities, and agents (Figure  2 ). An entity is a physical, digital, conceptual, or other kind of thing with some fixed aspects, which may be real or imaginary (examples include a model, a dataset, an output file, a script, a decision, or an assumption). An activity is something that occurs over a period of time and acts upon or with entities, and may encompass consuming, processing, transforming, modifying, relocating, using, or generating entities (such as estimating the parameters of a model, or performing a visual predictive check). Finally, an agent is something that bears some form of responsibility for an activity taking place, for the existence of an entity, or for another agent's activity.
An agent can be an organization, a person, or a software application. PROV-O defines a set of relationships that describe the interactions between entities, activities, and agents. Entities may be derived from other entities (a model based on another model), may be generated by activities (a model output may be generated by an estimation step), and are attributed to agents (such as NONMEM). Activities may use entities (a model fit may use a data file), may be informed by other activities (a previous model development step), and are associated with agents (NONMEM, or the analyst, or both). Agents may act on behalf of other agents (NONMEM may be used by the analyst, and the analyst may be working on behalf of an organization). PROV-O also supplies the necessary framework to invalidate entities and/or activities, allowing the effects of making a change in an analysis to be assessed based on the relationships of the affected entity or activity with downstream activities and entities: everything dependent on the change could, for example, be flagged as invalid, or as requiring reassessment.
This has obvious benefits during scientific and quality review. In line with the principles of good practice (GxP), descriptions, rationales, as well as comments for each activity should be provided in order to identify the key analysis steps (base and final models, for example), to highlight model assumptions, and to document rationales and justifications at decision points. Recording, storing, and querying this information has distinct benefits for accurately capturing and describing how a model was developed, evaluated, and ultimately applied to address a drug development question. The application of the PROV-O standard to support the PMx use case necessitated some focus in how entities, activities, and agents from the PROV-O model were described and annotated. There are a number of metadata items, such as flags or states, that are important to be able to be attached to entities, such as the importance of an output ("pivotal" or not), its quality control (QC) review status ("pass" or "fail," or null if not assessed), and its significance in the analysis (such as flagging base and final models as such).
Other pieces of metadata related to entities might include file names and sizes, dates and times of file creation or most recent modification, details related to the analyst who created or most recently modified an entity, as well as version and location control. Metadata for an activity might include the name of the analysis tool(s) and their versions, compiler information (if relevant), the command line used for execution, the hardware and platform on which it was executed, etc. Metadata for an agent might include their role in the analysis and within the PMx team. Transparency in the setting and evaluation of assumptions that may impact model application is of great importance in the planning and documentation of any model-informed activity. 3 It was necessary to create a structured definition for assumptions to enable them to be stored, associated with other entities or activities, to be peer-reviewed, and to be reported.
IMPACT The benefits of the proposed Thoughtflow system are manifold. Filtered, interactive views and queries of the data according to user role (analyst, manager, reviewer) and appropriate export functionality, in combination with tagging of entities and activities by analysts to identify key model development steps, would enable the large amounts of information typically associated with a PMx analysis to be reduced to a manageable level, and support the streamlining of tracking and reporting activities. Analysts would be able to apply provenance information to rapidly generate documentation for analyses, from run records to complete reports, in a reproducible and structured manner. A well-structured "Thoughtflow" capture and reporting tool would enhance knowledge sharing and communication among team members: an individual analyst would be able to rapidly familiarize themselves with an analysis started by a colleague, or on projects where several analysts are collaborating, for example.
This avoids the repetition or duplication of prior work and helps assure quality and reproducibility by ensuring the use of the same versions of analysis datasets and software. Using a Thoughtflow-based tool, it would be more straightforward for analysts to repeat an analysis either completely or in part, if required (if source data changed mid-analysis, for example). Using tagging and appropriate queries, a Thoughtflow tool could also facilitate the rapid generation of complete reports (through LaTeX (https://www. latex-project. org) and support tools such as knitr  11  ) and summary outputs, such as run logs, model development summaries that include the source (provenance), version and location of key input and output files for QC and review purposes, high-level project summaries for management, and complete audit trails, as well as complete reports.
This has the added benefit of avoiding error-prone hand-generated tables. Objective and detailed summaries of the model development process are possible using this approach, including descriptions of the assumptions being made as well as the objectives and rationales for individual tasks. This allows reviewers and regulators to assess precisely what activities have been completed and evaluate the appropriateness of the results and conclusions, potentially reducing delays associated with missing or incomplete information. Additionally, the analysis could be reproduced either completely or in part if necessary, using a minimal number of steps (keystrokes or mouse clicks). Standardization of the way the provenance data are stored, imported, and exported between applications would facilitate straightforward sharing of databases between analysts and organizations.
Finally, managers would be able follow the process of a data analysis project at whatever level of detail wished. Since progress would be updated in real time with respect to the analysis, this could potentially reduce the time spent supervising an analyst. It would also be easier to track the Figure  3  Examples of a provenance approach to pharmacometrics workflow. In (a), entity "model2" is derived from entity "model1" via activity "clone. " Agent "user1" was associated with the activity. In (b), the parameters of entity "model1" are estimated using entity "data. csv" during activity "run," generating a range of new entities including "model1. lst. " Two agents were associated with this activity: the analyst "user1" and the software "NONMEM. " In (c), entity "model1" is assessed by quality control reviewer "user2" during activity "QC" to generate a new version of entity "model1," with the flag qcStatus set to "pass.
SPECIFICATIONS AND PROTOTYPE The proposed DDMoRe Thoughtflow specifications describe cross-platform, robust support for tracking, reporting, and replicating elements of a modeling and simulation pharmacokinetics, pharmacodynamics (PKPD) project (across all phases of development), either in part or as an entire analysis. A technical specification for provenance capture and suggestions for implementation has been developed (included as electronic Supplementary Material) which describes the input-to-output task workflow, including decisions made, as well as the PMx knowledge management suitable for describing a complete, end-to-end modeling and simulation exercise. Any Thoughtflow solution, in order to be generalizable, must facilitate the flexible integration of varying operational processes and standards in industry, small and medium enterprise, and academia, as well as diverse computational tools for data handling, visualization, modeling, simulation, and reporting.
Commonly-used tools for data handling and visualization include R (R Foundation, Vienna, Austria), Mat-Lab (MathWorks, Natick, MA), SAS (SAS Institute, Cary, NC), and a myriad of others. Tools used for modeling and simulation include nonlinear mixed-effects tools such as NONMEM (ICON, Dublin, Ireland), R, SAS, Phoenix NLME (Certara, Princeton, NJ), and Monolix (Lixoft, Antony, France), as well as Bayesian tools such as WinBUGS (MRC Biostatistics Unit, Cambridge Institute of Public Health, Cambridge, UK), OpenBUGS,  12  and Simcyp (Certara). Reports are typically authored using literate programming tools, such as Rmarkdown (http://rmarkdown. rstudio. com) and LaTeX/ knitr, or Microsoft Word (Microsoft, Redmond, WA). New tools such as Stan  13  and Julia 14 may come into more common use in the future, hence it will be essential to allow their integration as well.
Owing to the sheer number and diversity of options available, it is not possible to explicitly support every tool that can be used for PMx data analyses. The DDMoRe strategy focuses instead on an application programming interface (API) for capturing provenance information, with prebuilt hooks for a limited number of widely used tools, such as NONMEM, R, and Monolix. It is nonetheless possible, in principle, to support any tool via the API. Ultimately, an implementation of the Thoughtflow standards should facilitate the comprehensive reexecution of analyses, either completely or in part, based on changes to key elements in the analysis (such as source data). A tool should be able to identify dependencies of the changed object(s), and will allow these to be selectively updated without the need to completely rerun the entire analysis.
Provenance database In contrast to the traditional relational database approach, a Resource Description Framework (RDF) triplestore, which is a "graph" database (a database that uses graph structures with nodes, edges, and properties to represent and store data) designed for the storage and retrieval of "triples" using semantic queries via the RDF format lies at the heart of the system. A "triple" is a data construct composed of a subject, a predicate, and an object, and can be visualized as two nodes linked by a relationship: for example, "dataset1. csv was created by createData. R," or "run5. mod was associated with assumption1. " Thoughtflow information is captured as a series of interconnected nodes, joined together by these relationships, which capture all the decisions, activities, inputs, and outputs in a single graph.
The graph can be traversed using semantic queries, matching on nodes across the graph, using the RDF format. Queries can return multiple downstream nodes (for example, finding outputs that are affected by a change to an upstream input), and support the use of ontologies-a relationship or node can have subtypes (for instance, both a graph and a table can be an output of an activity, and can be handled separately, enabling one to "find graphs" or "find outputs"). Provenance information will be encoded in the triplestore using the PROV-O standard. The triplestore is queried via standard SPARQL queries via a SPARQL server (Apache Fuseki in our implementation). The prototype uses Apache Jena (https://jena. apache. org), a free and open source Java framework for building semantic web and linked data applications to support the interface with the triplestore.
Provenance services A suite of services will allow a client to query the content of the provenance database using a simple representational state transfer (REST) web interface, insulating them from the underlying implementation and negating the need to craft their own queries. A request from the client is then implemented using SPARQL (a semantic query language for databases: SPARQL Protocol and RDF Query Language, https://www. w3. org/TR/rdf-sparql-query). The services will support queries to add, update, and retrieve information-for instance, to determine the provenance of an entity, to retrieve the information to create a project run record, or to retrieve a list of invalidated entities and activities following an update to an upstream entity or activity.