dataesr/scientific-paragraphs-categorization huggingface dataset:
All structural equation models were estimated with the software Mplus 5. 2  (Muthén and Muthén, 1998-2007). Model fit was evaluated by various indices: the chi-2 goodness-of fit statistic, the root-mean-square error of approximation (RMSEA), the comparative fit index (CFI), the Tucker-Lewis index (TLI), and the standardized root-mean-square residual (SRMR). A nonsignificant chi-2 goodness-of fit statistic indicates a good fit. The corresponding probability value indicates the probability of finding a multivariate difference of a certain size between the specified model and the sample data given that the specified model is the "true" model in the population. RMSEA values below 0. 05 indicate a good model fit  (Browne and Cudeck, 1993). CFI and TLI values larger than 0. 95 and SRMR values close to 0.
Introduction We consider the problem of multivariate nonparametric regression with errors in variables. We observe the i. i. d dataset (W 1 , Y 1 ),. , (W n , Y n ) where Y l = m(X l ) + ε l and W l = X l + δ l , with Y l ∈ R. The covariates errors δ l are i. i. d unobservable random variables having error density g. We assume that g is known. The δ l 's are independent of the X l 's and Y l 's. The ε l 's are i. i. d standard normal random variables with variance s 2. We wish to estimate the regression function m(x), x ∈ [0, 1] d , but direct observations of the covariates X l are not available. Instead due to the measuring mechanism or the nature of the environment, the covariates X l are measured with errors. Let us denote f X the density of the X l 's assumed to be positive and f W the density of the W l 's.
Acknowledgements: The research of  Thanh Mai Pham Ngoc  and  Vincent Rivoirard  is partly supported by the  french Agence Nationale de la Recherche  ( ANR 2011 BS01 010 01  projet Calibration). Michaël Chichignoud now works at  Winton Capital Management , supported in part as member of the German-Swiss Research Group  FOR916  ( Statistical Regularization and Qualitative Constraints ) with grant number  20PA20E-134495/1.
Introduction: The vaccine vial monitor (VVM) registers cumulative heat exposure on vaccines over time. As low-and lower-middle-income countries transition beyond support from the Global Alliance for Vaccines and Immunization (Gavi), they will assume full responsibility for vaccine financing and procurement. It is unclear to what extent countries transitioning out of Gavi support will continue to include VVMs on their vaccines. This paper aims to systematically review evidence on VVM availability and use in low-and middle-income countries to document factors behind global access to and country demand for VVMs. Such results could help identify actions needed to ensure continued use of VVMs in countries that transition out of Gavi support. Methods: We performed a systematic review of electronic databases, reference lists, and grey literature in English and French languages with publication dates from 2005 onwards.
Search strategy An electronic literature search was conducted in February-March 2016 that applied the PICOS framework to inform the search strategy  [16]. The search aimed to identify peer-reviewed articles, unpublished documents or conference proceedings that reported the following program experiences in low-or middle-income countries: VVM availability and deployment; VVM perceptions and practices; vaccine introduction and decision-making processes (including procurement); and Gavi graduation and vaccine program sustainability. These themes were selected to capture information about the context within which decisions on inclusion of VVM would be made. Since there are no studies specifically looking at critical factors for VVM inclusion on vaccines, data on factors influencing vaccine introduction decision-makers were used as a proxy for conceptualizing decision-making in relation to inclusion of VVM.
The primary outcome was global access to and country demand for VVMs. The search was restricted to English or French language documents published in 2005 or later, which is the period when VVM was included as a minimum requirement for vaccines procured by United Nations agencies. Case studies, evaluation reports, cross-sectional surveys, qualitative studies, review articles, conference abstracts or proceedings, and cost-effectiveness analyses were considered for inclusion in this review. Policy briefs, technical briefs and organizational strategies were considered if retrieved from websites of organizations actively working in this area. The search strategy included three different methods. First, a systematic database search was conducted in PubMed and Google Scholar using the following terms: ''vaccine vial monitor" ''vaccine AND temperature monitor" ''immunization AND temperature monitor" ''vaccine AND temperature monitoring" ''immunization AND temperature monitoring".
Data extraction A single reviewer performed the literature search and screened titles and abstracts to identify eligible studies for full-text review. The second screening consisted of a full-text review of identified articles and documents after abstract screening. This full-text review used a standard form to abstract data and a protocol to assess study quality using an assigned grading system (Supplementary Materials, S1-S2). The standard form extracted basic information including study design and methodology as well as primary and secondary findings. Data quality was assessed according to six criteria including the research question, methodology, design, findings, interpretation, and goal-achievement, leading to a maximum grade of 14 points.
Data review and analysis Studies included were analyzed for the following outcomes: (1) availability and deployment of VVM-labeled vaccines; (2) VVM practices and perceptions in the immunization system (e. g. , perceived or known benefits of VVMs in relation to key immunization indicators including coverage, wastage, quality control, stock management, and perceived risks (if any) of non-inclusion of VVMs at global, national, and sub-national levels); (3) vaccine introduction and decision-making processes; (4) Gavi graduation and vaccine program sustainability. Results were synthesized and described according to these outcomes.
Search results The initial database search identified 719 citations, and 695 of these citations were excluded based on review of their titles (n = 689) or duplication with other titles (n = 6). Twenty-four citations were retained after this initial review that were included in a further abstract screening. This abstract screening eliminated nine additional citations because publication was outside of the stipulated time criteria. Fifteen citations were retained after abstract screening. After further review, five articles were eliminated because of limited or no relation to access and demand for VVM and 10 selected for study inclusion. An additional four publications were found through manual reference reviews of included studies, leading to a total of 14 publications (Fig. 1 ). An additional four publications were included based on the search of organizational websites and unpublished documents, leading to a total of 18 publications included in analysis.
Study characteristics Tables 1 and 2 describe the publications identified according to search terms and outcomes. Among these 18 publications, three publications focused on VVM availability and deployment; nine publications examined VVM practices or policies; four publications reported VVM knowledge and perceptions; five publications described vaccine introduction and decision-making processes; and three publications related to Gavi graduation. Some publications addressed more than one outcome and were therefore included across multiple results categories. The included studies used a combination of methodologies including analysis of program records, key informant interviews and focus group discussions, and a literature review (Table  3 ). Data were collected at different levels including global, national, and sub-national.
Benchmarking surveillance systems requires realistic simulations of disease outbreaks. However, obtaining these data in sufficient quantity, with a realistic shape and covering a sufficient range of agents, size and duration, is known to be very difficult. The dataset of outbreak signals generated should reflect the likely distribution of authentic situations faced by the surveillance system, including very unlikely outbreak signals. We propose and evaluate a new approach based on the use of historical outbreak data to simulate tailored outbreak signals. The method relies on a homothetic transformation of the historical distribution followed by resampling processes (Binomial, Inverse Transform Sampling Method-ITSM, Metropolis-Hasting Random Walk, Metropolis-Hasting Independent, Gibbs Sampler, Hybrid Gibbs Sampler).
Introduction Many outbreak detection algorithms (ODA) for disease surveillance have been proposed in the literature (see for example Chen's review  [1] ), and designers of epidemiological and laboratory surveillance systems need to opt for the most suitable. The choice of an algorithm should be based on reproducible evaluations carried out under conditions as close to real life as possible  [2]. Several authors including Buckeridge  [3]  and Jackson  [4]  have shown that algorithm evaluation depends on the characteristics of the outbreak curve (shape, duration and size) and the baseline (mean and variance). The evaluation process thus depends heavily on the quality of test data and the control of characteristics known to influence these results  [5]. To evaluate an ODA, specific datasets composed of outbreak curves over a baseline are usually needed.
These two components can be obtained from real data, synthetic data or from a combination of the two (semi-synthetic dataset), as described in Mandl  [6]  and Buckeridge  [5]. The baseline is often obtained from the real, daily, routine data of disease surveillance systems, which are specific to the population under surveillance  [3, 7]. Useful simulation tools also exist for simulating a baseline (such as Project Mimic, http://www. projectmimic. com/). However, obtaining a sufficient quantity of outbreak curves with a realistic shape and covering a sufficient range of agents, size and duration, is known to be very difficult  [3] [4] [5] 8]. With regard to disease outbreak curves, several authors have pointed out  [3, 5, 7, 9]  that the use of real data for outbreak signals is hindered by the shortage of real datasets that include enough clearly-defined outbreak signals.
Natural outbreak signals cannot enable the control of features required for evaluating ODA (such as outbreak size, duration and shape). Buckeridge et al. [5]  show that among 22 studies using natural outbreak signals, only five had enough outbreaks to evaluate and report quantitative measures such as sensitivity and timeliness. They conclude that evaluations based on natural signals should be reserved for qualitative evaluation. Similarly, Wagner  [9]  believes that a minimum of roughly 10 outbreaks is required to build a quantitative evaluation. In contrast, synthetic outbreak curves have the potential to express the diversity of biological threats that a surveillance system could encounter and must detect. Synthetic datasets allow precise specification of the outbreak curve and make it possible to build large test sets.
They are more convenient for sensitivity analyses and for exploring the effects of curve changes. Most evaluation studies in the last decade have used semi-synthetic datasets that combine real baseline data with synthetic outbreaks  [5]. Several methods can be used to build synthetic outbreak curves. Mechanistic disease models, such as the Susceptible-Infected-Recovered (SIR) model, are commonly used  [10, 11]. Agent-based models (ABM) such as EpiSims  [12]  build a network-based model of disease spread instead of a compartment-based one  [9]. However, the purpose of most studies that simulate curves based on these approaches is not to generate outbreak curves to evaluate ODA, but is usually to predict an overall outbreak trend (until real-time quality forecasting is achieved and to identify realistic averaged parameters for describing outbreaks.
The principal limitation of evaluations using semi-synthetic datasets is that they only find the smallest peak that could be detected, leaving the critical question of detectability unanswered: "What is the smallest outbreak that can be detected?". We have known since Wagner  [9] , Buckeridge  [3]  and Jackson  [20]  that the answer is influenced by the outbreak shape for the selected size and duration, leading us to believe that ODA evaluations based solely on the injection of geometric shape can only be considered as partial evaluations. Bollen  [21]  considers that a curve reflects latent determinants, which are hidden. Baker  [22]  and Tucker  [23]  also consider that observations of repeated measures over time are the result of latent factors and characteristics that can only be indirectly observed using real curves.
Material and methods If we want to test algorithm ability and limitations under a large range of real surveillance situations, an adequate simulation tool must also be able to create the entire spectrum of outbreak curves, even those considered to be "atypical" but already-observed curves (qualified as "outliers") must be considered, provided their atypical nature is recognized. The risk of a statistical modeling approach is that it might restrict the scope of the simulation used for the algorithm evaluation to an average behavior of the epidemics, which could be non-representative of the large range of shape varieties observed for the same pathogen. Using outlier curves precisely allows algorithm limitations to be evaluated against unusual outbreaks. This is why we chose real datasets from published disease outbreaks, representing different modes of transmission (Fig  1 ) and a range of shape varieties.
In Fig  3 , for a given GOF, the lower the surface (greens and blues) in the cube, the better the quality of the simulation. We can observe that algorithms based on MCMC are less precise, regardless of the GOF chosen, than those based on independent sampling (Table  1 , Fig  3 ). The binomial algorithm had the best performance and was fast, since we implemented it using Rbinom, an optimized function based on C language directly implemented in the core package of R. While generally accurate and fast, the ITSM algorithm was slower (Table  1 ) than most other algorithms with increasing size or duration of the simulated curve. In theory, the "burnin" process penalizes all MCMC algorithms, but when the number of cases and days increases, the study shows that ITSM takes twice as much time as RWMH.
Another advantage of resampling methods resides in the fact that our goal can be achieved without generating and validating a plausible model of transmission and surveillance. The strength of Monte Carlo resampling methods is their ability to produce realistic and customized curves by rebuilding the original underlying distribution directly from a single observed curve (the epicurve probability distribution), even if the outbreak curve is unusual, provided that the outbreak PMF (probability mass function) can be extracted from real data. Creating a database of outbreak curves (Disease or shape curve/Number of cases/Number of days) supplemented with contextual information (such as population under surveillance/time/ location) can provide the material for various simulations and may help to define what constitutes a realistic and plausible epidemiological curve faced by the surveillance system.
In this study, we present only four shapes for a proof of concept, but our database is currently made up of 893 real outbreaks resulting from 58 diseases under surveillance, allowing the generation of many other curves. However, the quality (accuracy) of the choice of curve to support the resampling process during the simulation is clearly the major limitation of the method. As for all simulation approaches, an evaluation of the epidemiological situation (population, agent, system, location, period, etc. ) of the original observed curve/model is required when choosing the one that will support the simulation. Until our proposal, simulation of the complex shape of the measles, dengue or anthrax outbreaks (presented in Fig 1 ) for evaluating a disease surveillance system and ODA was based on the curve/shape set presented in S1 Fig, which is considered the reference  [6, 7, [16] [17] [18] [19]  for simulating outbreaks with controlled size and duration.
Similarly, we intend for future experiments to corroborate the estimates presented here by measuring the precursor and post-shock plasma conditions in xenon with, e. g. X-ray Thomson scattering, which so far has only been done for radiative shocks in argon  [39]. This work was supported by STFC and AWE through their academic access programme, in part by The Royal Society, EPSRC and Labex PLAS@PAR. The authors would like to acknowledge Robert Charles, Jim Firth, Paul Treadwell, Rob Johnson, David Hillier, Nick Hopps and the entire Orion team for their help and support during the experiments.
For computations in practice, availability of R codes for generating DTP tables and figures are available from the authors upon request. Also, the DTP can easily be implemented using existing software (e. g. , the "dfcrm" package in R  (31) ) to inform all possible outcomes in the following cohort(s). Calibrating a model is key to the success of using a model-based dose-finding design. Simulation is a common tool to evaluate the statistical properties of a selected design over the long runthis can be viewed as the Big Picture. However, it is vital too to focus on the finer, Small Details, to ensure that the model operates in ways that adhere to the investigators' judgement, such as coherence in escalation/de-escalation  (11, 32) , stopping early for excessive toxicity and any trial's explicit requirements.
Acknowledgements CY and LJB were funded by grant  C22436/A15958  from  Cancer Research UK. We acknowledge the support of the  Trials Acceleration Programme (funded by Bloodwise)  for funding the Viola trial which was used as a case study ( 13019 ). Celgene has provided free drug and an unrestricted educational grant for the trial. We would like to thank the editor and referees for their comments which have helped to improve the manuscript.
2] [3]  NLMEM for continuous PK/PD data use nonlinear dynamic models that draw on physiological or pharmacological principles to provide a reasonable approximation of the dynamics of the drug in the body and of their effects. They describe both population and subject-specific characteristics, represented as fixed parameters for population characteristics and random parameters for subjects. NLMEMs are widely applied because of their ability to quantify several levels of variability, to handle unbalanced data, and to identify individual specific covariates. In any regression modeling, after fitting a model to a dataset, it is essential to assess the goodness-of-fit between the model and the dataset and to determine whether the underlying model assumptions seem appropriate. In this tutorial, we refer to this procedure as "model evaluation," although, in literature, it has been described under several more or less equivalent terms, such as "model diagnostics," "model adequacy," "model assessment," "model checking," "model appropriateness," and "model validation.
" Model evaluation has to be clearly distinguished from "model building" and "model qualification" processes, which are two steps of model development that require model evaluation but imply different concepts. "Model building" is the process of developing a model on a given dataset to achieve clearly defined analysis objectives. "Model qualification" is the assessment of the performance of a model in fulfilling the analysis objectives. "Model evaluation" is required for both processes to diagnose one or several intermediary or key models in a model-building step or evaluate a selected model with respect to the modeling objectives. In this tutorial, we will only focus on the model evaluation step and describe various tools for evaluation of a NLMEM, regardless of whether it is an intermediary model, a key model in the modelbuilding step, or a best model that can be used for further inferences.
For each example, we simulated data from a "true" model and fitted different models, including the "true" and various "misspecified" models to highlight some types of model deficiency. To show the availability of several software tools developed for NLMEM estimation and simulation, the steps of simulating data, estimating parameters, and computing evaluation metrics were performed using a variety of software, including R (https://cran. rproject. org/), NONMEM (http://www. iconplc. com/innovation/ solutions/nonmem/), MONOLIX (http://lixoft. com/products/ monolix/), and PHOENIX (https://www. certara. com/software/ pkpd-modeling-and-simulation/phoenix-nlme/). The graphical displays presented in the tutorial are a suggestion using R scripts but it is not a recommendation from the ISoP Model Evaluation group.
Study design and data simulation We considered a balanced design, with five sampling times per patient (t 5 0. 5, 1, 4, 12, 24 hours after treatment), which we call the standard design. Two other designs, referred to as the sparse and very sparse designs, with two or one sampling times per patient, respectively, were also used to illustrate the influence of shrinkage on some of the evaluation graphs. In these sparse sampling designs, samples were randomly selected from the five sampling times above. A dataset of 180 patients with three treatment groups each receiving one of the three different doses of 10, 100, and 1,000 mg, was simulated using the model with the sampling designs described above and parameters provided in Supplementary Table  S1.
Parameter estimation, computation of evaluation metrics, and software Population parameters were estimated using the default option of the Stochastic Approximation Expectation Maximization algorithm implemented in MONOLIX version 4. 3. 3 (http://lixoft. com/products/monolix/). The simulated data and medians of predictions for different models are shown in Supplementary Figure  S1. Individual parameters were then estimated as EBEs which are the mode of the conditional posterior distribution, given the observed data and the population model. Let p g i jy i ; W ð Þthe conditional distribution of g I. The EBE estimate of g i is given by: ĝi 5argmax g i p g i jy i ; W ð Þ ð Þ 5 argmax g i p y i jg i ; W ð Þ 3p g i jW ð Þ p y i ð Þ (7) At this stage, l has been estimated. Therefore, once ĝi is estimated, ĥi 5h l; ĝi ; z i ð Þcan be easily calculated.
Once the model parameters were estimated, evaluation metrics were computed using additional software. In this paper, all the goodness-of-fit graphs were generated using R and several R packages for which scripts are available in BASIC EVALUATION TOOLS Evaluation based on population predictions Population predictions. By definition, population predictions (xPRED, with x 5 1, C, or P) are the expectation of the model, E(y i ), given the individual designs and covariates. There are several methods for computing xPRED. The simplest way is model linearization using the First-Order (FO) linearization (Eq. 8) (i. e. , the prediction assuming all random effects equal 0; denoted PRED). The corresponding predicted profile, given design and covariates, is often called the prediction for typical individual or typical profile.