dataesr/scientific-paragraphs-categorization huggingface dataset:
In this example, the function cdfcomp can be used to report CDF values in a logscale so as to emphasize discrepancies on the tail of interest while defining an HC5 value (Figure  4 ). R> data("endosulfan", package = "fitdistrplus") R> ATV <-endosulfan$ATV R> fendo. ln <-fitdist(ATV, "lnorm") R> library("actuar") R> fendo. ll <-fitdist(ATV, "llogis", start = list(shape = 1, scale = 500)) R> fendo. P <-fitdist(ATV, "pareto", start = list(shape = 1, scale = 500)) R> fendo. B <-fitdist(ATV, "burr", start = list(shape1 = 0. 3, shape2 = 1, + rate = 1)) R> cdfcomp(list(fendo. ln, fendo. ll, fendo. P, fendo. B), xlogscale = TRUE, + ylogscale = TRUE, legendtext = c("lognormal", "loglogistic", "Pareto", + "Burr")) None of the fitted distributions correctly describes the right tail observed in the data set, but 1 The plotdist function can plot any parametric distribution with specified parameter values in argument para.
Empirical and theoretical CDFs R> quantile(fendo. B, probs = 0. 05) Estimated quantiles for each specified probability (non-censored data) p=0. 05 estimate 0. 2939 R> quantile(ATV, probs = 0. 05) 5% 0. 2 In addition to the ecotoxicology context, the quantile generic function is also attractive in the actuarial-financial context. In fact, the value-at-risk VAR α is defined as the (1 -α)-quantile of the loss distribution and can be computed with quantile on a 'fitdist' object. The computation of different goodness-of-fit statistics is proposed in the fitdistrplus package in order to further compare fitted distributions. The purpose of goodness-of-fit statistics aims at measuring the distance between the fitted parametric distribution and the empirical distribution: e. g. , the distance between the fitted cumulative distribution function F and the empirical distribution function F n.
For this purpose, a generic quantile function is provided for class 'bootdist'. By default, 95% percentile bootstrap confidence intervals of quantiles are provided. Going back to the previous example from ecotoxicology, this function can be used to estimate the uncertainty associated to the HC5 estimation, for example from the previously fitted Burr distribution to the endosulfan data set. R> quantile(bendo. B, probs = 0. 05) (original) estimated quantiles for each specified probability (non-censored data) p=0. 05 estimate 0. 2939 Median of bootstrap estimates p=0. 05 estimate 0. 3004 two-sided 95 % CI of each quantile p=0. 05 2. 5 % 0. 1785 97. 5 % 0. 5000 The estimation method converged only for 1000 among 1001 bootstrap iterations.
Alternative methods for parameter estimation This subsection focuses on alternative estimation methods. One of the alternatives for continuous distributions is the maximum goodness-of-fit estimation method also called minimum distance estimation method (D'  Agostino and Stephens 1986; Dutang et al. 2008). In this package this method is proposed with eight different distances: the three classical distances defined in Table  1 , or one of the variants of the Anderson-Darling distance proposed by  Luceno (2006)  and defined in Table  2. The right-tail Anderson-Darling (AD) gives more weight to the right-tail, the left-tail AD gives more weight to the left tail. Either of the tails, or both of them, can receive even larger weights by using second order AD statistics. To fit a distribution by maximum goodness-of-fit estimation, one needs to fix the argument method to "mge" in the call to fitdist and to specify the argument gof coding for the chosen goodness-of-fit distance.
This is carried out by providing, for the optimization process, a lower and an upper bound for each parameter. The code below calls the L-BFGS-B optimization method in optim, since this quasi-Newton method allows for box constraints 2. We choose to match moments defined by Equation  4 , and so a function for computing the empirical raw moment (called memp in our example) is passed to fitdist. For two-parameter distributions (i. e. , d = 2), Equations 4 and 5 are equivalent. R> library("actuar") R> fdanish. P. MLE <-fitdist(danishuni$Loss, "pareto", + start = c(shape = 10, scale = 10), lower = 2+1e-6, upper = Inf) R> memp <-function(x, order) sum(x^order)/length(x) R> fdanish. P. MME <-fitdist(danishuni$Loss, "pareto", method = "mme", + order = 1:2, memp = "memp", start = c(shape = 10, scale = 10), + lower = c(2+1e-6, 2+1e-6), upper = c(Inf, Inf)) R> cdfcomp(list(fdanish.
The quantile matching is carried out numerically, by minimizing the sum of squared differences between observed and theoretical quantiles. R> fdanish. ln. QME1 <-fitdist(danishuni$Loss, "lnorm", method = "qme", + probs = c(1/3, 2/3)) R> fdanish. ln. QME2 <-fitdist(danishuni$Loss, "lnorm", method = "qme", + probs = c(8/10, 9/10)) R> cdfcomp(list(fdanish. ln. MLE, fdanish. ln. QME1, fdanish. ln. QME2), + legend = c("MLE", "QME(1/3, 2/3)", "QME(8/10, 9/10)"), + main = "Fitting a lognormal distribution", xlogscale = TRUE, + datapch = 20) Above is an example of fitting a lognormal distribution to the danishuni data set by matching probabilities (p 1 = 1/3, p 2 = 2/3) and (p 1 = 8/10, p 2 = 9/10). As expected, the second QME fit gives more weight to the right-tail of the distribution. Compared to the maximum likelihood estimation, the second QME fit best suits the right-tail of the distribution, whereas the first QME fit best models the body of the distribution.
Customization of the optimization algorithm Each time a numerical minimization is carried out in the fitdistrplus package, the optim function of the stats package is used by default with the "Nelder-Mead" method for distributions characterized by more than one parameter and the "BFGS" method for distributions characterized by only one parameter. Sometimes the default algorithm fails to converge. It is then interesting to change some options of the optim function or to use another optimization func-tion than optim to minimize the objective function. The argument optim. method can be used in the call to fitdist or fitdistcens. It will internally be passed to mledist, mmedist, mgedist or qmedist, and to optim (see ?optim for details about the different algorithms available). Even if no error is raised when computing the optimization, changing the algorithm is of particular interest to enforce bounds on some parameters.
method = "CG", + control = list(maxit = 10000))) R> if (is(fCG, "try-error")) + fCG <-list(estimate = NA) It is also possible to use another function than optim to minimize the objective function by specifying the argument custom. optim in the call to fitdist. It may be necessary to customize this optimization function to meet the following requirements. (1) the custom. optim function must have the following arguments: fn for the function to be optimized and par for the initialized parameters. (2) custom. optim should carry out a minimization and must return the following components: par for the estimate, convergence for the convergence code, value = fn(par) and hessian. Below is an example of code written to wrap the genoud function from the rgenoud package  (Mebane, Jr. and Sekhon 2011)  in order to respect our optimization "template".
The rgenoud package implements the genetic (stochastic) algorithm. R> mygenoud <-function(fn, par,. ) { + require("rgenoud") + res <-genoud(fn, starting. values = par,. ) + standardres <-c(res, convergence = 0) + return(standardres) + } The customized optimization function can then be passed as the argument custom. optim in the call to fitdist or fitdistcens. The following code can for example be used to fit a gamma distribution to the groundbeef data set. Note that in this example various arguments are also passed from fitdist to genoud: nvars, Domains, boundary. enforcement, print. level and hessian. The code below compares all the parameter estimates (α, λ) by the different algorithms: shape α and rate λ parameters are relatively similar on this example, roughly 4. 00 and 0. 05, respectively.
To illustrate the use of package fitdistrplus to fit distributions to censored continuous data, we will use another data set from ecotoxicology, included in our package and named salinity. This data set contains acute salinity tolerance (LC50 values in electrical conductivity, mS. cm -1 ) of riverine macro-invertebrates taxa from the southern Murray-Darling Basin in Central Victoria, Australia  (Kefford, Fields, Clay, and Nugegoda 2007). R> data("salinity", package = "fitdistrplus") R> str(salinity) 'data. frame': the expectation-maximization approach of  Turnbull (1974)  to compute the overall empirical CDF curve with optional confidence intervals, by calls to survfit and the plot method for 'survfit' objects from the survival package (Therneau 2014; Therneau and Grambsch 2000, Figure  10  shows the Turnbull plot of data together with two fitted distributions).
R> plotdistcens(salinity, Turnbull = FALSE) As for non censored data, one or more parametric distributions can be fitted to the censored data set, one at a time, but using in this case the fitdistcens function. This function estimates the vector of distribution parameters θ by maximizing the likelihood for censored data defined as: As fitdist, fitdistcens returns the results of the fit of any parametric distribution to a data set as an S3 class object that can be easily printed, summarized or plotted. For the salinity data set, a lognormal distribution or a loglogistic can be fitted as commonly done in ecotoxicology for such data. As with fitdist, for some distributions (see  Delignette-Muller et al. 2015, for details) , it is necessary to specify initial values for the distribution parameters in the argument start.
The discrete nature of the variable is automatically recognized when a classical distribution is fitted to data (binomial, negative binomial, geometric, hypergeometric and Poisson distributions) but must be indicated by fixing argument discrete to TRUE in the call to functions in other cases. The toxocara data set included in the package corresponds to the observation of such a discrete variable. Numbers of Toxocara cati parasites present in the digestive tract are reported from a random sampling of feral cats living on Kerguelen island  (Fromont, Morvilliers, Artois, and Pontier 2001). We will use it to illustrate the case of discrete data. R> data("toxocara", package = "fitdistrplus") R> str(toxocara) 'data. frame': 53 obs. of 1 variable: $ number: int 0 0 0 0 0 0 0 0 0 0. Emp. and theo.
CDFs Data CDF q q q q q q q q q q q q q q q q q q q q q empirical theoretical The fit of a discrete distribution to discrete data by maximum likelihood estimation requires the same procedure as for continuous non-censored data. As an example, using the toxocara data set, Poisson and negative binomial distributions can be easily fitted. R> (ftoxo. P <-fitdist(toxocara$number, "pois")) For discrete distributions, the plot of an object of class 'fitdist' simply provides two goodnessof-fit plots comparing empirical and theoretical distributions in density and in CDF  (Figure 11). Function cdfcomp can also be used to compare several plots to the same data set, as follows for the previous fits (Figure  12 ). R> plot(ftoxo. P) R> cdfcomp(list(ftoxo. P, ftoxo. nb), legendtext = c("Poisson", + "negative binomial")) When fitting discrete distributions, the Chi-squared statistic is computed by the gofstat function using cells defined by the argument chisqbreaks or cells automatically defined from q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0 20 40 60 the data in order to reach roughly the same number of observations per cell.
Conclusion The R package fitdistrplus allows to easily fit distributions. Our main objective while developing this package was to provide tools for helping R users to fit distributions to data. We have been encouraged to pursue our work by feedbacks from users of our package in various areas as food or environmental risk assessment, epidemiology, ecology, molecular biology, genomics, bioinformatics, hydraulics, mechanics, financial and actuarial mathematics or operations research. Indeed, this package is already used by a lot of practitioners and academics for.
Acknowledgments The package would not have been at this stage without the stimulating contribution of Régis Pouillot and  Jean-Baptiste Denis , especially for its conceptualization. We also want to thank  Régis Pouillot  for his very valuable comments on the first version of this paper. The authors gratefully acknowledges the two anonymous referees and the Editor for useful and constructive comments. The remaining errors, of course, should be attributed to the authors alone.
RegulonDB (http://regulondb. ccg. unam. mx) is one of the most useful and important resources on bacterial gene regulation,as it integrates the scattered scientific knowledge of the best-characterized organism, Escherichia coli K-12, in a database that organizes large amounts of data. Its electronic format enables researchers to compare their results with the legacy of previous knowledge and supports bioinformatics tools and model building. Here, we summarize our progress with RegulonDB since our last Nucleic Acids Research publication describing Reg-ulonDB, in 2013. In addition to maintaining curation up-to-date, we report a collection of 232 interactions with small RNAs affecting 192 genes, and the complete repertoire of 189 Elementary Genetic Sensory-Response units (GENSOR units), integrating the signal, regulatory interactions, and metabolic pathways they govern.
These additions represent major progress to a higher level of understanding of regulated processes. We have updated the computationally predicted transcription factors, which total 304 (184 with experimental evidence and 120 from computational predictions); we updated our position-weight matrices and have included tools for clustering them in evolutionary families. We describe our semiautomatic strategy to accelerate curation, including datasets from high-throughput experiments, a novel coexpression distance to search for 'neighborhood' genes to known operons and regulons, and computational developments.
INTRODUCTION RegulonDB is a relational database that offers, in an organized and computable form, updated knowledge on transcriptional regulation in Escherichia coli K-12  (1). Regu-lonDB, first published in 1998, captures the results of a continuous effort to this day  (2). Our curation work also feeds the EcoCyc database  (3) , which together with Reg-ulonDB are the major sources of organized information for the best-known bacterial genome model organism. For years we have expanded the number of objects and their properties in our database, always enriching the modeling of the molecular components governing transcription initiation, as we strive to keep up-to-date with new methodologies. We have also enriched the modeling of gene regulation, proposing new concepts, such as regulatory phrases (4) and, more recently, GENSOR units (genetic sensoryresponse units) that link signals, the associated regulatory interactions and the regulated response as metabolic and cellular capabilities  (5).
Briefly, RegulonDB facilitates access to organized information on the mechanisms of tran-D134 Nucleic Acids  Research, 2016, Vol. 44 , Database issue scription initiation; more precisely, RegulonDB organizes the available information on the shadows and fingerprints of these mechanisms in the genome. Here we present progress since the last Nucleic Acids Research (NAR) paper, published in 2013  (1). We have kept our curation up-to-date, including regulation by small RNAs (sRNAs); we report a rather complete repertoire of elementary GENSOR units, each one integrating the network of regulatory interactions and metabolic pathways affected by one transcription factor (TF). This additional information represents major progress for our focus on integrative approaches to facilitate not only information but also summarized knowledge given the high granularity of most biological processes.
Curation of high-throughput (HT) datasets (i. e. , chromatin immunoprecipitation [ChIP] variants, microarrays, gSELEX and transcriptional start site [TSS] mapping) is a delicate issue, since we do not want to dilute the high-quality classical experiments with the massive but more fragmented knowledge that these methodologies produce. Our current solution, as discussed in detail below, is at the crossroads of two paradigms: the classic one of a relatively well-organized genome with promoters and binding sites involved in regulation of transcription initiation, and one inundated by scattered promoters and binding sites, many of which we do not yet know if they are involved in transcriptional regulation. Another avenue linking RegulonDB data at this time with HT-generated profiles of expression is the capability we have implemented for evaluating the similarity of coexpression of any two genes, based on the COLOMBOS microarray library of experiments.
If any enzyme is part of a multimeric complex, all the monomers of the complex are added (even if they are not directly regulated by the TF). These 144 new GENSOR units have been curated by a semiautomatic method that starts with a pipeline of programs that extract all information pertinent to a GENSOR unit from the RegulonDB and EcoCyc databases; such data are subsequently manually revised and curated and used to generate the visual map available in RegulonDB. The full methodology, motivation and relevance of this integrative new concept will be published elsewhere (Ledezma et al. , manuscript in preparation). GENSOR unit components and their interactions place a TF and its regulatory mechanisms in a larger context, providing evidence in many cases for the TF's role in decisionmaking processes and information flux from the signal to the elicited genetically encoded response.
RegulonDB 9. 0 hosts a GENSOR unit for each local TF  (6)  for which there is experimental evidence in the database. A total of 103 TFs have a known effector in RegulonDB, including 25 two-component systems. When available, the four components of the GENSOR unit are highlighted: the signal, the signal processing, the genetic switch and the response. By default, effectors are considered signals, unless a reaction that produces the effector is present in the GEN-SOR unit, in which case the substrate of that reaction is deemed the signal and the reaction itself becomes part of the signal processing. Directionality of reactions is considered in the identification of the four components, as well as for the addition of reaction gaps. A total of 78 GENSOR units have their four components highlighted; 119 include the genetic switch and the response, and 2 contain only the genetic switch.
Updated set of predicted TFs We updated the set of computationally predicted TFs, based on recent work by Perez-Rueda et al. (9). A total of 184 TFs experimentally characterized and for which information was deposited in RegulonDB (1) were used as seeds in BLASTP searches against the complete proteome of E. coli. E-values of ≤1e-6 and a coverage of 70% were required for a TF to be considered a putative TF. In addition, TFs specifically associated with E. coli K-12 and deposited in the DBD, HAMAP  (10) , Superfamily DB  (11)  and PFAM  (12)  databases were retrieved. Superfamily and family assignations were based on Superfamily annotations  (11) , PFAM  (12)  and the Conserved Domain Database (CDD)  (11) (12) (13) (14). Forty-two groups of paralogs defined by BLASTP (  14 ) comparisons for which the E-values were ≤1e-6 and for which coverage was at least 50% of any of the proteins in the alignment were identified in the total set of TFs.
This set of predicted binding sites corresponds to 12 574 TF → gene regulatory interactions; this represents a recovery of 52% of the 1592 annotated regulatory interactions in the database for the 93 TFs for which we have a PWM, which represents a 9% improvement from the previous RegulonDB version. If only TFs with a good-quality PWM are taken into account, the total number of predicted TF → gene interactions is 8714, recovering 672 (57%) of annotated interactions for this TF subset. The TFBS predictions can be obtained from the 'Dataset' menu in the 'Computational Prediction' section.
org/abs/ 1105. 0121). (iii) The hierarchical tree was cut using as the threshold a combination of different metrics values; the tree is cut in a collection of trees (a forest). (iv) Each tree is used as a guide to create a progressive alignment of the PWMs. (v) The clusters are represented both as trees and as heat maps (see http://www. rsat. eu/). The browser that enables the user to see the collection of PWMs in a hierarchical tree is available via the 'Integrated Views & Tools' menu, in the 'Browse RegulonDB' section in the 'Clustering of RegulonDB PWMs' option. Also in the same section, there is a link to display a circular browser (Figure  2 ), developed with the D3. js JavaScript library (http://d3js. org/), that integrates the information from families, the TFs and their PWMs.
Identification and annotation of methods All data added to the database contain the evidence that demonstrates the existence of each object or interaction of regulation. We use a set of evidence codes (see http:// regulondb. ccg. unam. mx/evidenceclasification) for this purpose. These evidence codes are derived from more than one method that is reported in the literature, as in the case of the evidence to identify TSSs of promoters, 'Transcription initiation mapping,' that could be related to the methods of primer extension, S1 mapping or 5'-RACE, among others. This level of description is common to major resources, such as the GO (Gene Ontology) and EcoCyc databases. A virtue within RegulonDB that we implemented since 2008 is, on the one hand, a simplified classification of 'weak' and 'strong' levels of confidence for all evidence sources.
Northern blotting is commonly used to identify TUs, whereas primer extension is used to identify TSSs of promoters. The objects that were identified with these methods are listed in RegulonDB with strong evidence codes. We took all the papers that are linked to only one promoter or one TU in RegulonDB, where the promoter or TU had the evidence code for 'Transcription initiation mapping' and 'Length of transcript experimentally determined,' respectively. Subsequently, using ODIN filters, we did a search for the words 'primer extension' and 'northern blot' in each set of papers. To increase the confidence in these text-mining strategies, we included the requirement that in the phrase(s) that identifies the method, the name of the TU/promoter has to be present too. Of course, we read all these phrases and checked if they were correct.
CURATION OF HIGH-THROUGHPUT DATASETS HT experiments generate a large number of scattered fragments of knowledge. By default, we add such information, with peaks already processed by the authors of the curated papers, as datasets separated from the database, but available, for instance, for display as tracks as part of the different resources in RegulonDB. Our manual curation efforts are focused on extracting the subset of objects (binding sites and promoters) that have additional evidence supporting them, as well as to combine different experiments that congruently support an object. For instance, ChIP-based experiments identify sites that occur within coding regions  (23) , which may have nothing to do with transcriptional regulation or at least there is not yet evidence of such involvement.