dataesr/scientific-paragraphs-categorization huggingface dataset:
This degree of importance depends on the representativeness of each experiment of the database (and a clear analysis is needed in Element 2) as well as on the type of SRQs considered. This should be integrated in the development of the quantification method to avoid under or overweighting the influence of a given pair. The simplest and the most widespread method of invers quantification of model uncertainties is the quantification on the basis of SETs, where singular measurement representing the phenomenon can be compared with associated calculated parameter  (Skorek, 2004 ), e. g. measured void fraction in the channel is associated with phase relative velocity, only. The population of point values (multipliers) obtained from the comparison can be approximated as Probability Density Function (PDF) representing model uncertainty, in this care relative velocity model.
Adapting the works of  (Hemez et al. , 2010)  or  (Oberkampf et al. , 2007)  to the IUQ framework, this analysis in the validation domain should be combined with the adequacy of the experimental database. It leads to the evaluation of the maturity of the IUQ for the intended use. This evaluation should be carried out in a loop approach. If the maturity is proved as not sufficient, an iteration step is performed. It means the maturity is improved according to findings during quantification and validation and the whole procedure is repeated. A lack of maturity can be due to different choices adopted in each SAPIUM element. One reason could be the lack of adequacy of the experimental database. In this case, it is preferable to consider integral experiments in the quantification (and not only in the validation) and to iteratively revise the quantified model input uncertainties from these new data.
The NPP accident scenario identification and IUQ definition process can rely heavily on expert opinion and can be subjective. Therefore, iteration of the process, based on experimentation and analysis, is important. It is recommended to use the phenomenon identification and raking table (PIRT) technique  (Diamond, 2006; Wilson and Boyack, 1998)  to first identify and rank the physical phenomena, and use the sampling-based global sensitivity analysis (GSA) technique to confirm the PIRT (also called Q-PIRT)  (Luo et al. , 2010; Martin, 2011; Yurko and Jacopo, 2012). It is important to keep in mind the phenomena ranking could be subject to the limitations of the knowledge of the expert, the experimental databases or simulation models. An iteration with other elements may be necessary if such limitations are identified during the SAPIUM process.
Element 2 -development and assessment of the experimental database It is recommended to follow a structured and transparent approach to perform the construction of a scenario-dependent experimental database. It first requires describing in a standard format (including information on covered phenomena, geometry, scaling effect, experimental uncertainties) all available experiments coming from SETs, IETs and CETs. Various scales IETs are mandatory for the application to the reactor case. Extra experiments might be also required if the adequacy of the database is not sufficient. For a transparent and reproducible adequacy assessment, it is recommended to use mathematical tools to quantitatively perform the analysis. Multi-Criteria Decision Making approach can be exploited in order to objectively and automatically evaluate the representativeness leading to a ranking of experiments.
The evaluation of the completeness of the experimental database is based on a completeness index. This index should be restricted to the database and not be applied to the whole quantification/validation process as it is classically done in VVUQ. Finally, a special attention should be devoted to the splitting of the experimental database for input uncertainty quantification and validation. If the number of available experiments is too limited to perform this splitting, all available experiments should be considered for the quantification step and the validation step should be adapted. The SAPIUM approach is proposed to improve the IUQ by expert judgement. However, in some situations the expert judgement could still be used. A typical situation is when available measured parameters are not sufficient to determine separate uncertainties.
Element 3 -selection and assessment of the simulation model The assessment of the applicability of the SM is an iterative process that applies to all experimental tests of the validation database. The possible failure of the process for one test implies the improvement of the SM (either code or nodalization or both) and the repetition of the assessment of the applicability of an SM for all tests of the validation database. The validity of a simulation model is defined over the domain of model form, inputs, parameters, and responses. This fact effectively limits use of the model to the particular application for which it was validated; use for any other purpose would require the assessment of the applicability of the SM to be performed again. In other words, the assessment of the applicability process cannot prove that an SM is correct and accurate for all possible conditions and applications, but, rather, it can provide evidence that a SM is sufficiently accurate.
Therefore, the assessment of the applicability process is completed when sufficiency is reached. The SM shall not be tuned to a particular data set and the data used to assess the SM have not been deliberately selected to make the SM appear to be more accurate than it truly is. An important aspect of the simulation model assessment is the nodalization strategy and model option selection. They should be consistent between the experiment and the nuclear power plant. In the assessment process, a special attention should be devoted to the construction of consistency indicators to evaluate the accuracy between simulation and experiment. The selection of important uncertain input parameters (including nature of uncertainties e. g. aleatory, epistemic) should be confirmed by sensitivity analysis methods, in order to reduce the subjectivity by expert judgement.
Element 4 -model input uncertainty quantification Inverse methods can also be used to calibrate a model (simultaneously or not with quantification). The experience in PREMIUM indicates that the IUQ should be performed without recalibration. This is especially advisable in case of lack of adequacy of the available experimental database. Inverse methods should allow assigning different weights to different pairs of simulated/experimental value in order to avoid under or overweighting the influence of a given one. It is the case for example for probabilistic ones that use the so-called likelihood function in their inferences, and a possibility is to construct a weighted likelihood. The least-square technique, used in the solution of inverse problems, can also assign different weights in the construction of the sum of squares to be minimized.
Element 5 -model input uncertainty validation The main recommendations can be split with respect to 4 different topics. The first one concerns the validation experiments. If the experimental database is large enough, they should correspond to experiments which are not used for the uncertainty quantification. When the number of experiments is not sufficient to split the database, a leave-one-out cross-validation which is a classical technique in statistics, e. g. (Wackernagel, 1998) , can be exploited. Another option could be to enforce the connections between experimentalists and developers to design new validation experiments following specific requirements  (Oberkampf and Trucano, 2007). It is also important to control (or evaluate) the impact of the methodological assumptions on the validation result.
Besides a consistency checking, it is advisable to take into account extra features such as the concentration of the information provided by the uncertainty analysis by combining informativeness and calibration type indicators. Finally, the last topic is focused on the analysis of the validation results. The computation of the validation indicators is not sufficient if the objective is application to NPPs. It is recommended to construct a maturity model combining the acceptability of the validation results in the validation domain with the adequacy of the experimental database. The evaluation of the predictive maturity should be carried out in a loop approach. If the lack of maturity can be explained by a lack of adequacy of the experimental database, it is preferable to extend the database by considering integral experiments for the quantification and to iteratively revise the quantified model input uncertainties from these new data.
Adequacy of the experimental database IUQ process is based on the comparison between simulation and experimental results. Therefore, the quantified model input uncertainties strongly depend on the adequacy of the experimental database. Two main aspects of this topic should be further investigated. The first one is related to the quantitative analysis of a database and to the construction of representativeness and completeness indices. Several generic tools are already developed but their extension to the framework of IUQ still remains an open issue. The second one concerns the lack of experiments. In the case of a poor validation, the SAPIUM approach provides some tools and recommendations to evaluate if the experimental database should be enlarged (e. g. computation of adequacy indicator, loop approach, cross-validation).
Conclusions This paper was devoted to the description of the different elements and steps of a good practice guidance for a transparent and rigorous model IUQ. The main developments have been performed by the SAPIUM group that included contributors from 10 organizations including Technical Support Organization (TSO), industry and university. The starting point was the available state of knowledge coming from previous related OECD/NEA projects as well as current practices in regulation, industries and research. The main outcome is an original structured approach that offers a general framework to develop or analyze IUQ. This work clearly emphasized that IUQ shouldn"t be reduced to the application of inverse methods as it was the case during the PREMIUM activity. It is a more general process that involves a clear specification of the problem and efficient strategies to construct adequate experimental database and to combine the information coming from different experiments.
It also requires assessing the simulation model before quantifying input uncertainties. Finally, the validation of the quantified model input uncertainties has to be taken into account in the whole process in order to check the acceptability of the results for the intended use. The description of the different elements revealed the need to use mathematical tools integrating the physical knowledge in order to be fully rigorous, transparent and reproducible. Even if part of the material to perform model IUQ is available, several remaining open issues should be tackled. This paper includes some development axes for future works. They concern  the adequacy analysis of the experimental database: construction of quantitative indices to evaluate the adequacy, development of a strategy for a limited number of experiments,  the construction of inverse propagation methods: integration of a reliability degree of the information to back-propagate, treatment of aleatory and epistemic uncertainties by uncertainty theories,  the evaluation of the acceptability of the results: construction of predictive maturity indices devoted to IUQ, evaluation of the impact of other uncertainty sources on the validation, construction of a matured extrapolation process to exploit the quantified input uncertainties in the application domain.
For industrial application, it is important to focus on the practicability of the proposed tools. This can be achieved by exercises of comparison and benchmark of IUQ constructed in the SAPIUM framework. As a SAPIUM follow-on activity, demonstration cases could be conducted, with the objectives to verify (i) the applicability of the best-practices, (ii) that whatever the problematic, the best-practices allow to deal with, and (iii) the issues identified during PREMIUM can be avoided by following the SAPIUM best-practices. For that, these demonstration cases could be split into the following three steps:  1st step: The experimental database contains enough data and there are few influential phenomena: the problem is well posed and seems easy to treat.  2nd step: In this case, several influential phenomena are involved, but the experimental database provides enough data of different types.
An example of this case is the critical flow at the break, which is of significant relevance during a LOCA.  3rd step: In the last case, several influential phenomena are present, but the experimental database does not contain enough varied data (such as in PREMIUM benchmark). This progressive approach could help to treat the key issues identified and to use gradually the recommendations developed in SAPIUM, and give a feedback on the proposed methodology. Acknowledgments: The authors are grateful to OECD/NEA for organizing the SAPIUM project, and to their respective organizations for supporting their participation in this project.
Available data As explained before, the meta-model is constructed from in-situ measurements, although for practical applications of structural design such data are not available. It is of common practice for structural design to rely on hindcast databases (see e. g. [7] ). Hindcast databases are based on numerical models and hence are available on a regular time step, over a fine spatial grid, in many regions of the world. The Ifremer IOWAGA wave database (see  [8] ), which is a recently developed model that provides accurate estimation of waves based on WAVEWATCH III R has been used to provide long time series of H s. The wind from the NCEP CFSR  [9]  reanalysis dataset is used as forcing of the numerical wave model, and is also used here for wind time series. The global HYCOM reanalysis  [10]  is used for estimation of currents.
No circulation model forced by NCEP CFSR in North sea was available for this work. HYCOM was chosen for the availability of 3-hourly surface current speed over a long period (from 1993). The concomitant use of three databases used for metocean environment provide a consistent hourly dataset, see e. g. [11]. Thus, the simultaneous use of these hindcasts is relevant for combining their independent physical effects on offshore structures. For each dataset, we used the nearest neighbour to the semi-submersible location. The resulting database spans from 1st January 1993 to 31st December 2015, with a hourly time-step. The sea-state is described by 8 parameters, detailed in Table  1. Since the platform is almost symmetric, hence not very sensitive to the directionality, only the mean direction is considered for each variable (e.
Data analysis Before going into the core of the methodology, a more precise insight into the data is provided in Figure  6. This figure presents an extract of the full dataset during January 2003, where two important events of tension in the mooring lines can be observed. First in the upper-left corner, one can see the time series of the tension in a mooring line, and the corresponding environmental forcings in the other plots : H s in the upper left, W s in the bottom left and C s in the bottom right plot. Two vertical lines are superimposed on the plot, corresponding to extreme tensions. The left one, in yellow, corresponds to an extreme in the tension caused by conjunction of high wave height and strong wind, none of them being at its maximum as it can be seen from the plot : the H s is still increasing while the wind is decreasing.
g. [6] , one can assume, that for a given sufficiently high threshold u GPD , the distribution of X i conditionally on being above the threshold u GPD follows a Generalized Pareto Distribution (GPD) with parameters σ > 0 and ξ ∈ R, best described by its cumulative distribution func-tion (cdf) : P(X < x|X > u GPD ) = 1 -1 + ξ x -u s σ -1/ξ + , with (z) + = max(z, 0). We have chosen to estimate the parameters of the GPD through a maximum likelihood method. Results are shown in Table  2. We have two parameters to fix here, u s and u GPD. They are respectively fixed as the 97. 5% quantile and the 99% quantile of the original dataset. Only the second one is reported since it impacts the estimated parameters. The duration d s separating two clusters has been fixed to 48 hours. We also checked (not shown here) that these values correspond to sensible choices by using the classical tools, such as residual life plot, the stability of estimates above the threshold, and the validity of the Poisson process hypothesis for threshold excesses, using the Dispersion Index plot (  [13] ).
Acknowledgements This research work has been carried out in the frame of the  CITEPH  project  MulAnR , funded by both partnership and sponsors. The authors thanks ENGIE E&P International, SAIPEM and Doris Engineering for their financial support and their fruitful discussions. ENGIE  E&P Norge  is acknowledged for the provision of storm time-series measured at Gjøa platform. The authors are also grateful to two anonymous referees for their valuable comments that leaded to great improvement of the paper.
Introduction Like many scientific disciplines, economics has experienced a data and computational revolution. Today, most economics papers are empirical and rely on complex scripts analysing rich and often non-shareable datasets  (Christensen and Miguel (2018) ;  Vilhuber (2020) ). As computational results now account for a large part of the output and contribution of an academic paper, it is more important than ever that empirical results are computationally reproducible. This means that another researcher can regenerate the published quantitative results from a set of files and written instructions provided by the original authors  (Kitzes et al. (2017)). In short, one can check whether "same data + same code = same results"  (Barba, 2018). 1 This step is fundamental to guarantee that empirical results can be trusted by both academics and policymakers.
However, we find no evidence that authors are more reluctant to share their code when they are more senior, more cited, or affiliated with more prestigious universities. The evidence thus does not support the received idea that increasing reproducibility standards would discourage the best researchers from submitting their work  (Harvey, 2014). Second, verifying reproducibility imposes high costs on journals, a point which has been neglected in the literature. We use the example of the cascad certification agency to provide concrete estimates of the costs of verifying the reproducibility of empirical research in economics. 2 The main components of verification costs are labor costs and the cost of accessing new commercial or administrative datasets. We find that verifying a paper takes on average 10 hours, and estimate that accessing a new dataset costs on average around 5,000 EUR (5,600 USD).
As several papers can use the same dataset, economies of scale are large. We estimate that verifying the results of all papers published by ten leading economics journals would lead to an average cost around USD 365 per paper. We then analyze the competition effect. Following  Jeon and Rochet (2010) , we model journals as platforms intermediating between authors and readers. Two academic journals choose their submission fees, subscription fees, and a reproducibility level. We show that reproducibility is suboptimally low in equilibrium. The reason is that journals are competing fiercely to attract authors: as each paper can only be submitted to one journal, authors derive market power from the uniqueness of their papers and form a "competitive 2 cascad stands for Certification Agency for Scientific Code And Data and its website is www.
Finally, published papers can later be replicated on different datasets or with different methodologies, which happens with probability ρ(q). Replication attempts always succeed for correct paper and always fail for wrong papers. We assume that a reader gets a utility v(q) from reading a correct paper, and -w from reading a wrong paper. The reader rationally discards papers that have been shown to be wrong, either because they failed the reproducibility test or because they failed the replication test, but reads the others. 4 In this simple model, the reader will read all the correct papers, but will also read those wrong papers that still passed the reproducibility test and were not considered for replication. Mathematically, the expected utility of the reader for a given paper is: π v(q) Further studies -(1 -π) (1 -p(q)) Control (1 -ρ(q)) Stimulate w.
Stimulate: As reproducible research calls for the disclosure of code, it stimulates the criticism of existing results by simplifying the conduct of replication studies running the original methodology on another dataset. Replication decreases the probability of reading a wrong paper by 1 -ρ(q), where ρ should be seen as increasing in q. Considering a sample of articles published in top-50 economics journals,  Mueller-Langer et al. (2019)  show that a mandatory data-disclosure policy has a positive effect on the replication probability by six percentage points. They conclude that replication efforts could be incentivized by promoting data disclosure and hence reducing the cost of replication. Further studies: reproducibility can encourage further studies by communicating more information to the academic community on how exactly to conduct a given analysis.
Indeed, the pressure to publish is indeed ranked first among all the reasons put forward by scientists when surveyed by Nature about the impediments to reproducible research  (Baker (2016) ). A second important cost is that sharing one's code and data will make the authors face more competition when writing follow-up papers. These costs depend again on an article's characteristics X i. Some types of research may be particularly discouraged by a strict reproducibility policy, in particular research using proprietary datasets  (Harvey, 2014). The costs could also be larger for more productive authors and/or more innovative papers. A third cost is reputational risk. Making one's code and data available makes it easier for others to spot errors (coding or other), which is socially beneficial but privately costly  (Azoulay et al.
(6) As is well known, if Oi and Ci follow i. i. d. extreme value distributions, then the probability p open i that U open i ≥ U closed i , and that the authors hence choose to make their article open, is given by: ln We collected information about available code and data from the JFE data and program webpage. p open i 1 -p open i = [α O (Y j ) -α C (Y j )] + [β O (Y j ) -β C (Y j )] X i (7) and the coefficients α = α O (Y j ) -α C (Y j ) and β = β O (Y j ) -β C (Y j ) can ln p open i 1 -p open i = α+β 1 •T op10 i +β 2 •International i +β 3 •Seniority i +β 4 •Citations i +ε i (8) We define the different variables in Table  1. Panel A gives summary statistics for the four explanatory variables, and Panel B reports the results of different regression specifications. We draw several insights from our analysis.
Despite the existing competition among top-5 journals, the market power of the AER seems to have been high enough to keep its ambitious 2004 data policy, and even taking the next step of verified reproducibility in 2018 for conditionally accepted papers (as shown in Figure  2 ). 23 Moreover, the 2004 data policy became a standard that has been adopted by other journals. Indeed, all top-5 journals now have a similar data policy, and two of them (JPE and QJE ) explicitly mention that they adopted the AER's 2004 policy.