dataesr/scientific-paragraphs-categorization huggingface dataset:
9. A case study for France. We applied the SEIR-based ESMDA data assimilation method with data from the pandemics in France to assess the lockdown effect and run possible scenarios following the intervention. The first occurrences of confirmed cases of COVID-19 in France were reported at the end of January 2020. However, biological a posteriori sample testing showed that there were isolated cases as early as November 2019. The first clusters of infectious patients were identified at the end of February in the Oise district (north of Paris area) and in the city of Mulhouse (east of France) at the beginning of March. A critical stage of public intervention, which recognizes the pandemics, started March 14th with the closure of schools and universities, and the enforcement of the first systematic restrictions on public meetings.
Hence the number of confirmed cases is by far not representative of the actual number of infected people. Inaccuracies and delays in reporting deaths in care homes to Santé Publique France profoundly impact the accuracy of the total deaths observations. Moreover, we believe that, at least for France, we cannot model the deaths in care homes with the same SEIR-like dynamics as we do for the rest of the country. Care homes comprise confined spaces with an aged, weakened population with a very different R-matrix than the rest of the country. Thus we do not use these data in the conditioning. By contrast, the numbers of deaths at hospitals are likely more reliable. That is why, as opposed to the Québec case, the death number in care homes is not added to the death toll here. 9. 2. Exp. 1: Calibrated reference.
The first experiment (Exp. 1) serves as a reference for the application of ESMDA on the data for France. We have chosen the model parameters based on several sensitivity simulations, which yield a calibrated reference analysis. We focused on (i) the hospitalized patients and (ii) the accumulated deaths at hospitals. We have also experimented with the total deaths, which include the deaths in care homes. This approach yielded a significant misfit with the data, which is consistent with the above explanation that, for the French data, one cannot easily model the sum of hospitals and care homes deaths by using a global   SEIR model. By contrast, we expect the reported deaths at hospitals and the hospitalized to be reliable and representative so that we choose a standard observation deviation of 1%, with a maximum error of 50 individuals.
Furthermore, they were somewhat limited at first but increased later, which may change the fraction of undocumented cases and affect the statistics. In late April, several authorized laboratories spread around the country started to analyze the tests. The Health Ministry produces a daily official report with the number of positive cases and the number of accumulated deaths. These reports are available on the Health Ministry web page. We used these data to conduct the data assimilation experiments of the virus transmission in Argentina. There is still some capacity available in the health system up to the end of May (Health Ministry web page), and we consider the number of deaths due to COVID-19 to be rather precise. The standard deviation of the observed deaths was four at the start of the pandemic and increased up to six the last examined day, which contains an accumulated number of deaths of 539.
Regrettably, the information on hospitalized people is not well updated on the available official databases. Therefore we have been unable to use this information in the data assimilation experiments. Since the first detected cases were around early March and because of the early lockdown, we conducted the data assimilation experiments for Argentina with a smaller number of initial exposed and infected individuals than the other countries, 40 and 10 respectively, and standard deviation of 8 and 2. We kept the rest of the parameters to be the same as for the other experiments (Table  1 ). We set the prior values for the effective reproductive number, R 1 = 3. 7, between March 1st and March 20th, R 2 = 1. 0 during the lockdown between March 21st and April 27th, and R 3 = 1. 5 after the relaxation of the lockdown.
On the other hand, New York is currently on track to gain control of the outbreak. 13. Discussion. We have demonstrated an approach for parameter estimation and model calibration in an age-stratified SEIR model to model and predict the SARS-CoV-2 epidemics. By assigning uncertainties to all poorly known parameters and characterizing them by specifying appropriate prior distributions, we can sample large ensembles of parameter realizations. The model predictions originating from the prior ensemble of parameters constitute our previous knowledge of the epidemic evolution, before data are taken into account, and have typically significant errors. An ensemble data-assimilation method allows for estimating both static model parameters and the time-varying effective reproductive number. Our ensemble data-assimilation approach, conditioning the model on observations, implies constraining the prior ensemble and compute an updated ensemble of parameter estimates that constitute the posterior solution.
A sensitivity study indicated that using more than 5,000 ensemble members will not lead to visible changes in the results. Thus, we have used 5,000 realizations for all the final cases presented in the paper. We present a more detailed discussion regarding the convergence properties of ESMDA in Sections A. 1 and A. 2 of the Appendix. A document with supplementary plots, illustrating the convergence of ESMDA, is available from the Github repository: https://github. com/geirev/EnKF_seir/blob/master/doc/sensitivity. pdf. 13. 3. Model initialization. Given a prior set of model parameters, e. g. , the defaults in Table  1 , we can initialize the model by setting the start date, the initial number of infectious I 0 , and exposed E 0. These numbers interact with the reproduction number R(t) for the initial period between the start date and the interventions' introduction.
When we assign uncertainties to the fraction of patients going to the hospital, we can obtain an excellent match to the observations of the number of deaths. If the case fatality rate p f and the hospitalization rate p s are unknown, it is impossible to constrain the SEIR model with the observed numbers of deaths and hospitalized. However, we can condition the SEIR model on the observations by using prior estimates of these parameters. The observed number of cases would be useful if they were unbiased observations since they relate directly to the SEIR model variables. In theory, we could obtain reliable case data by testing the whole or an unbiased sample of the population. With such data, we would not only much better constrain the model, but we could even accurately estimate the case fatality rate p f and the hospitalization rate p s.
We introduced a variant of the SEIR type model and obtained realistic predictions of the epidemics, with uncertainty estimates, by conditioning the model ensemble on observed deaths and hospitalization numbers. The system clearly illustrates the inherent instability in infectious illnesses where the disease's development primarily depends on the effective reproductive number, R(t). The data assimilation system allows for online and automated estimation and monitoring of R(t). Unfortunately, there is a lag of two-three weeks between infections and registered hospitalization and deaths. However, we handled this challenge by using a smoother approach when assimilating data to constrain the model simultaneously on all data present in a time window. This lag causes that, when one observes a too high value of R(t), it may already be too late to avoid another round of significant restrictions on society.
From visual inspection of the results, it is hard to justify more than 16 steps. We decided to use 32 steps in all the simulations presented in this manuscript to ensure convergence with a margin. A document with supplementary material, including the results of the sensitivity tests, is available from the Github repository: https://github. com/geirev/EnKF_seir/blob/master/doc/sensitivity. pdf. A. 2. Sensitivity of ESMDA to the size of the ensemble. We have examined the convergence of ESMDA concerning the ensemble size. ESMDA, being based on a Monte Carlo algorithm, means that we can always improve the solution by increasing the ensemble size. However, we need to decide on a tradeoff between ensemble size, N , and the number of ESMDA steps, N mda , due to limitations on computing power. While the number of ESMDA steps impacts the algorithm's actual convergence towards the correct solution, the ensemble size impacts the precision of the statistical estimate (i.
Thus, we use 5000 realizations in all the simulations in this paper. We have included the results in the supplementary material found at: https://github. com/geirev/EnKF_seir/blob/master/doc/sensitivity. pdf A. 3. Uniqueness of the solution. Finally, we will comment on the degrees of freedom in the model parametrization versus the amount of information in the data. There are dependencies between several of the uncertain parameters. E. g. , suppose we specify too few initial infected. In that case, this may be compensated for by a higher value of R(t) in the first period. If the model predicts too many infectious, it is still possible to match the accumulated deaths and hospitalizations by reducing the fractions p s and p f. Thus the system appears to be under-determined. However, by specifying prior distributions for the uncertain parameters, we ensure one unique posterior solution, and we obtain it by running the model with the ensemble of estimated parameters as input.
Acknowledgments. G. Evensen was supported by internal funding from  NORCE  (and his family was generous enough to allow and encourage him to work long days and nights given the urgency of this work). J. Amezcua ,  A. Carrassi , and  A. Fowler  were supported by the  NERC  award  NCEO02004. F. C. Vossepoel was supported by a  Delft Technology Fellowship. M. Pulido was supported by  ANPCYT  under grant  CORR01   COVID-19 Federal. Both C. Jones  and  C. Sampson  were supported by the  US Office of Naval Research  under grant  N00014-18-1-2204. CEREA is a.
This NONMEM tutorial shows how to evaluate and optimize clinical trial designs, using algorithms developed in design software, such as PopED and PFIM 4. 0. Parameter precision and model parameter estimability is obtained by assessing the Fisher Information Matrix (FIM), providing expected model parameter uncertainty. Model parameter identifiability may be uncovered by very large standard errors or inability to invert an FIM. Because evaluation of FIM is more efficient than clinical trial simulation, more designs can be investigated, and the design of a clinical trial can be optimized. This tutorial provides simple and complex pharmacokinetic/pharmacodynamic examples on obtaining optimal sample times, doses, or best division of subjects among design groups. Robust design techniques accounting for likely variability among subjects are also shown.
A design evaluator and optimizer within NONMEM allows any control stream first developed for trial design exploration to be subsequently used for estimation of parameters of simulated or clinical data, without transferring the model to another software. Conversely, a model developed in NONMEM could be used for design optimization. In addition, the $DESIGN feature can be used on any model file and dataset combination to retrospectively evaluate the model parameter uncertainty one would expect given that the model generated the data, particularly if outliers of the actual data prevent a reasonable assessment of the variance-covariance. The NONMEM trial design feature is suitable for standard continuous data, whereas more elaborate trial designs or with noncontinuous data-types can still be accomplished in optimal design dedicated software like PopED and PFIM.
Very complex designs considering the best time samples and dose may be considered in optimal designs which could be missed in trial-and-error clinical trial simulations. 5] [6] [7] [8] [9]  As one would hope, the different software tools will generally give the same answer to the same calculations. 7 However, the tools do differ, with some methods and approaches only implemented in certain software. In addition, for NONMEM  10  users, the implementation of models in these software programs will require learning a new modelling language (often implemented in R  11  or MATLAB  12  ). In this work, we present the $DESIGN tool in NONMEM, which implements basic evaluation and optimization of the FIM. 5] [6] [7] [8] [9]  The design algorithms in NONMEM have been modeled after PopED by Hooker et al.
(https://andre whook er. github. io/PopED), and PFIM by Mentre et al. (http://www. pfim. 5] [6] [7] [8] [9]  These articles outline the theory of clinical trial evaluation, of which a brief description is given in the section Design Theory in Supplementary Materials. A description of the options to the $DESIGN are listed in the Introduction to NONMEM 7 manual (intro7. pdf) to the NONMEM software,  10  version 7. 5 and higher. In general, to evaluate or optimize a trial design using the FIM you will need a model, a set of model parameter values, a design, and the tasks that are to be performed (and a design space, if one wants to optimize the design). At its most basic $DESIGN can be used with any NONMEM model file and dataset combination to retrospectively evaluate the model parameter uncertainty one would expect given that the model generated the data (simply by updating the model parameter estimates and replacing the $ESTIMATION line with a $DESIGN line in a NONMEM control file).
GENERAL TRIAL DESIGN LAYOUT IN NONMEM Population designs are usually composed of one or several elementary designs and the number of subjects associated with each elementary design. Each elementary design is defined by design elements, such as the number of samples and their allocation in time, they may also contain a dosing protocol, response measurements, and covariates. In NONMEM, the data file pointed to by the $DATA record in an NMTRAN control stream contains the layout of the elementary designs, where the ID data item refers to the elementary design number. Thus, an elementary design is actually a subject template of records in a data file, indicating its dosing protocol, sample times, covariates, etc. , just as if this data set were being used for simulation. A very basic elementary design structure will be introduced in the first example.
By default, each elementary design represents one subject, and elementary designs are equally weighted. We shall see how this can be altered by using GROUPSIZE to multiply the weighting of the entire template data set, and in a later example by using the STRAT and STRATF data items to alter the weighting among elementary designs. Any design elements that are data record specific (such as dose, time, covariate, etc. ) are placed in the data file, much as what one would do for clinical trial simulation. It is most economical to place these design elements in the data file, rather than specifying these data-record specific details in the control stream. NONMEM users who are used to using the data file as a template for simulation will be familiar with using a data file template for specifying data-record specific design elements for optimal design/evaluation.
EXAMPLE 1: WARFARIN PHARMACOKINETIC DESIGN EVALUATION We first consider a simple example of design evaluation. Using the example of warfarin. ctl (control stream shown in Table  1 ). We see that most of the records will be familiar to NONMEM users. Note, however, the new record specific for trial design: The csv file specified in the $DATA record for this problem is very simple, and contains a single elementary design with a bolus dose record and three observation records. Although the csv file contains only one subject (that is one subject type or elementary design), the subject number size may be specified with GROUPSIZE in the $DESIGN record. Here, we request the variance-covariance expected from a design with 32 subjects, all with the same elementary design (equivalent time values, doses, and covariates) defined in the csv file.
We define for each patient the three following sampling times: one, four, and eight. There can be more than one elementary design in the csv, each with their own design (different doses, sample times, etc. ), as shown in a later example. Each elementary design would be multiplied by the GROUPSIZE scale factor. The FIMDIAG = 1 option indicates evaluation of the FIM with block-diagonal modality, as described in the introduction (the default in NONMEM is FIMDIAG = 0, but FIMDIAG = 1 is considered usually the most appropriate one to use). 7, 13 he problem may be executed using the standard NONMEM script as follows: where warfarin. ctl is the control stream and warfarin. res will be our NONMEM report file. This model has MU referencing of its thetas. MU referencing is not required, but can be helpful in evaluating FIM analytically, providing greater significant digit precision and speed.
A correspondingly large expected shrinkage for the individual ETA (empirical Bayes estimate) associated with this parameter (94. 8%) can also be seen. The reported values for this example are very similar to results computed by both PFIM and PopED (results not shown). Note that design evaluation of an existing dataset is possible using, simply, the command "$DESIGN" and a realized dataset substituted for the elementary design dataset. For standard NONMEM analyses, $DESIGN can be used before parameter estimation, using some reasonable initial estimate of parameters to explore if parameters using this model and particular data set are expected to be estimable. In addition, after estimation using the estimated parameters obtained from, say, a first order conditional estimation analysis, $DESIGN can be used to see what the RSE of parameters one would expect given that the model generated the data.
EXAMPLE 2: WARFARIN PHARMACOKINETIC DESIGN OPTIMIZATION In this example, we attempt to optimize the trial design proposed in example 1. We add the following specifications. First, additional data items describing the designs space TSTRAT, TMIN, and TMAX are needed (we shall see later how they are used), and added to the elementary design: These are then referred to in the $INPUT record of the control stream (complete controls stream warfarin2. ctl and data file are in Supplementary Materials); Next, we have added several options to the $DESIGN record: We specify MAXEVAL>0, to indicate that we wish to have a design optimization performed. If we do not specify the method (NELDER/FEDOROV/STGR/RS/DISCRETE), the default is NELDER (Simplex method in PFIM software). We also ask that intermittent results every 20 iterations be printed out (PRINT = 20).
56 (second column of values in Table  5 ), similar to the ideal -118. 27 in the continuous sample time assessment, so there was very little compromise here in rounding the sampling times. Another design consideration would be to require that four PK/PD samples be taken in one elementary design (PK and PD times are restricted to be taken at the same time), and another four PK/PD samples be obtained in another elementary design (so, two elementary designs or two subject types). The data template to describe this is shown in Table  4  (warfarin_pkpd2. csv), where two elementary designs are templated. Notice that each PK/PD sample pair (one from CMT = 2 and the next from CMT = 3) share the same initial TIME value, and they share a TSTRAT index. This ensures that the TIMEs of the PK/PD sample pair moves together during the optimization.
EXAMPLE 5: DS -OPTIMALITY EXAMPLE FOR FINDING THE BEST SAMPLE TIMES (USING UNINT) FOR A TWO -COMPARTMENT PROBLEM For this example, we now use the Ds-optimality criteria where some parameters (structural and variance fixed effects) are declared "uninteresting" for design optimization. For this problem OFVTYPE = 6 is selected, which takes into account interesting versus uninteresting parameters. In the previous examples, we have T A B L E 4 Elementary designs for problems warfarin_pkpd and warafarin_pkpd2 been implicitly using the default the d-optimalitylog(det(FIM), OFVTYPE = 1, for optimization of the design. A list of various OFVTYPE option values are listed in Table  S1 , and in the description of the $DESIGN record in intro7. pdf to the NONMEM software. 10 For the present example, we use a two compartment PK model and opt-design2.
ctl (Supplementary Materials), and we specify that the Omega values are UNINTed (that is, are declared uninteresting), and that only the thetas and the residual proportional error are of interest:  Note: The -log(det(FIM)) and SE's are obtained from the raw output file(. ext) or the report file, shrinkage is obtained from the. shk or the report file, and sample times are obtained from the $TABLE file (. tab, optimization) or data file (. csv, evaluation). The OFVTYPE = 6 objective function is structured as -log(det(FIM)) + log(det(FIMuninteresting)) Final optimal times are listed in Table  6 , as are the RSEs and shrinkage. Comparing with the results when the UNINT are removed, and OFVTYPE = 1 (second column in Table  6 ), and when UNINT is replaced with FIXED, and OFVTYPE = 1 (third column in Table  6 ), we can notice that the RSEs of V2 omega and Sigmaconstant for the UNINT setting are quite large.
This is comparable to determining best group size, as described in an example in ref. 6. The STRAT and STRATF are data items that should be defined in the data file as follows: STRATF = data item containing fraction representation for the associated elementary design STRAT = data item containing stratification index pertaining to the STRATF design element. Because STRAT and STRATF describe a property of the entire elementary design (all data records belonging to the same ID), the only the STRAT and STRATF of the first record of the ID is considered. As an example, suppose we have a STRAT index of four, and its STRATF value is 0. 4. All elementary designs that have the same STRAT of four on their first record will share a STRATF value of 0. 4, and this represents their weight of influence on the FIM.
If STRAT and STRATF are specified, and there is at least one STRAT value greater than 0, then the STRATF values are optimized, and represent the weight to the contribution of that elementary design to the information matrix. For STRAT values less than or equal to 0, then their STRATF values are not optimized, and remain fixed at their initial values, but are still used as weights to the information matrix. It is up to the user to ensure that the initial sum of STRATF values among unique STRAT indices sum to one. If value of STRATF is less than 0. 0, then that elementary design is not included in the assessment. The effective group size for elementary designs with STRATF greater than zero is then GROUPSIZE*STRATF value for that elementary design. The data file laying out the design is shown in Supplementary Materials, tmdd2.
CONCLUSIONS To our knowledge, $DESIGN in NONMEM is the first optimal design tool implemented in a pharmacometrics software that was primarily developed for model fitting. This allows the user to easily evaluate designs using approaches based on the FIM, using already implemented models, parameters values, and sometimes designs. Using an optimal design approach, instead of CTS, for design evaluation is timely, as, indeed, many more designs can be evaluated. Furthermore "true" design optimization can presently be done extensively only by using these optimal design approaches. Note, however, that SEs and RSEs predicted by the FIM are often close, but theoretically only lower bounds of the true uncertainty for any given dataset, especially for designs of limited size. Because the FIM is presently computed using a linear approximation,  we suggest the users to use this tool to define one or a few designs that should then be evaluated by clinical trial simulation.