dataesr/scientific-paragraphs-categorization huggingface dataset:
Asymptotics of LOO-CV and the widely applicable information criterion (wAIC) Bayesian LOO-CV is asymptotically equivalent to the wAIC, which is an adaptation of the AIC and, more fundamentally, the TIC, to the Bayesian case  (Watanabe, 2007). The wAIC (per site) takes the following form : wAIC = 1 n i ln E post [p(X i | θ)] - 1 n i V post [ln p(X i | θ)] (8) where E post is the expectation, and V post the variance, over the posterior distribution under the complete dataset X. In practice, these theoretical expectation and variance terms are replaced by their Monte Carlo counterparts (empirical mean and variance over the MCMC sample). In terms of interpretation, the first term of wAIC can be seen as the self-fit, that is, the fit of the training set under the parameter value estimated on that training set.
Because it uses the data twice, this measure of the fit is optimistic. The second term represents an estimate of this optimism bias. As such, it plays the same role as the dimensional penalty in AIC. Of note, in spite of their similar form, the two terms in equation 8 are not of the same order of magnitude. Owing to the asymptotic concentration of the posterior, the variance of the log likelihood at a typical site (the second term) decreases as a function of data size, whereas the mean log likelihood (the first term) remains asymptotically macroscopic. The same situation holds for the AIC and other classical information criteria, for which the dimensional penalty becomes negligibly small compared to the log likelihood term for sufficiently large datasets. The numerical stability and the asymptotic equivalence between the wAIC and LOO-CV (Watanabe, 2010) were empirically assessed by conducting a scaling experiment, consisting of randomly subsampling a large phylogenomic dataset and plotting the fit (LOO-CV and wAIC) of the GTR, CAT-Poisson and CAT-GTR models (relative to LG), as a function of data size (Figure  4 ).
Overall, LOO-CV and the wAIC give very similar results. The discrepancy between them decreases as the data size becomes larger, giving nearly indistinguishable numerical estimates for the largest data sizes considered here. Even for smaller data size, the difference between LOO-CV and the wAIC is visible but small compared to the difference in fit between the models. In terms of numerical stability, both estimators, of LOO-CV and of wAIC, tend to be numerically more stable for larger data size. In addition, the wAIC tends to be more accurate and less sensitive to variation in Monte Carlo sample size than LOO-CV (Supplementary Information, section 2). In the end, for large datasets (more than 4000 positions), for which the asymptotic approximation of the wAIC is very accurate, it is possible to get numerically satisfactory estimates of the wAIC based on a thinned sample of 100 points regularly spaced over a well-converged MCMC run.
Discussion In The main conclusions are as follows. As suggested previously  (Gelfand et al. , 1992; Bernardo & Smith, 1994; Konishi & Kitagawa, 2007) , Bayes factors are inadequate for selecting the bestapproximating model, and cross-validation appears to be more adequate for this purpose. Among CV methods, LOO-CV stands out as the best choice, both statistically and computationally. It also has a clear asymptotic connection with information criteria, and more specifically with the widely applicable (or Wanatabe-Akaike) information criterion (wAIC  Watanabe, 2009). For large datasets, wAIC is easily implemented and offers a good complement to LOO-CV.
Leave-one-out cross-validation stands out as the computationally most efficient and most easily implemented cross-validation approach. LOO-CV also has a clear asymptotic connection with information criteria, and more specifically with wAIC in the Bayesian case. For large datasets, wAIC is easily implemented. It has a broad range of applicability, at least among i. i. d. models, and the control of its numerical error is easier than for LOO-CV. In the end, based on the results presented here, the practical recommendations for finding the best approximating model are relatively simple. LOO-CV and the wAIC represent the most practical and reliable approaches. Both can be obtained from a single pass over the MCMC sample, but LOO-CV may require larger MCMC sample sizes (or less thinning) than wAIC to pass the quality checks.
Practical consequences and perspectives The arguments exposed here in favor of LOO-CV and wAIC over Bayes factors for model approximation purposes are at odds with the general perception in the applied Bayesian community that Bayes factors represent a general gold standard for model selection  (Kass & Raftery, 1995; Lartillot & Philippe, 2006; Xie et al. , 2011; Oaks et al. , 2019). This raises the question of the practical consequences of the use of Bayes factors thus far, in situations where cross-validation might have represented a logically more adequate criterion. As illustrated by the analysis of the normal case (Figure  1 ), for large datasets and for models that don't differ too much in their dimensionality, all model selection approaches agree in their selection. Thus, in practice, previous results based on the application of Bayes factors on large datasets, such as multi-gene phylogenetic analyses, are unlikely to be qualitatively incorrect, although the case is less clear for smaller-scale analyses.
Definitions and relations between alternative Bayesian measures of model fit In this subsection, the alternative Bayesian measures of model fit are formally defined. A homogeneous mathematical notation is introduced, so as to emphasize the connections and the differences between them, leaving aside in a first step the numerical and algorithmic problems. Suppose we have a dataset made of n observations, X = (X i ) i=1. n. In the context of phylogenetic inference, these observations would typically be the columns of a multiple sequence alignment. In the following, we will adopt a frequentist perspective and assume that these observations are iid from an infinite population of unknown distribution. We then consider a model M parameterized by θ. In a Bayesian framework, this model is endowed with a prior p(θ) and then conditioned on data X, giving the posterior distribution p(θ | X): p(θ | X) = p(X | θ)p(θ) p(X) (9) where p(X) = p(X | θ)p(θ)dθ (10) is the marginal likelihood.
As mentioned in the introduction, the general idea is to split the dataset into two subsets, using one subset (noted X t ) for training the model and then evaluating the fit of the model over the remaining subset (noted X v , for validation). In the context of Bayesian inference, a natural procedure to implement cross-validation is to average the likelihood under the validation set over the posterior distribution under the training set, i. e. computing: p(X v |X t ) = p(X v |θ) p(θ|X t ) dθ. (14) The resulting cross-validation score is then log-transformed and averaged over multiple random splits of the original dataset into a training and a validation sets. Of note, alternative approaches have been proposed, such as computing the cross-validated likelihood on a plug-in estimate, typically, the posterior mean  (Konishi & Kitagawa, 1996).
However, this approach is not invariant by reparameterization. It is also not applicable for singular and redundant models, such as mixture models  (Plummer, 2008). Based on this general idea, multiple settings can be contemplated for implementing crossvalidation. These alternative settings differ in how the dataset is split, how the replication procedure is defined, or whether the likelihood is averaged over the posterior distribution jointly for all observations of the validation set, or independently for each of them. In k-fold cross-validation, the dataset is split into k subsets of equal size. Then, each subset is considered in turn as the validation set, while the other k -1 subsets are pooled together to make the training set. There are thus k replicates in total. In a variant of this approach, used in phylogenetics  (Lartillot & Philippe, 2008) , each replicate is obtained independently of other replicates, by randomly splitting the dataset into a fraction f of the observations, which is set aside for validation, while the remaining fraction 1 -f used for training.
It is thus close to the original version of k-fold cross-validation, with f = 1/k, except that the replicates are not obtained by systematic rotation of the subsets. As a result, the number of replicates can be arbitrary. In practice, for computational reasons, a small number of replicates is used, typically m = 10. The fraction f is typically set to 0. 1 or 0. 2, or equivalently, k = 10 or 5. In the following, this approach will also be called k-fold cross-validation, even if it does not exactly correspond to the original version. To more formally describe cross-validation, assume that l = 1. L replicates are considered, each based on a random split of the dataset into X = (X t l , X v l ), and that the training and validation sets are of size q and r, respectively. Thus, in k-fold CV, q = (1 -f )n and r = f n, with f = 1/k, but the definitions introduced below are valid for more general settings.
Noting I v l the subset of 1. n corresponding to the indices of the observations assigned to the validation set in replicate l, the sitewise cross-validation score is thus defined as: cv s = 1 r 1 L l i∈I v l ln p(X i |X t l ) (16) Again, the score is per future observation. The sitewise approach has apparently not been used previously in the context of phylogenetics. However, it can be useful to consider it in the general comparative evaluation conducted below. Finally, in leave-one-out cross-validation, each observation is taken in turn and set aside for validation, using the n -1 remaining observations to train the model. Noting X (i) the training set (of size n -1) obtained by removing observation i, the score is defined as: cv l = 1 n n ln p(X i |X (i) ) (17) Whichever setting is used, from a frequentist perspective, the primary quantity of interest, implicitly targeted by cross-validation, is the expected log-likelihood cross-validation score, the expectation being taken over multiple independent draws of the entire dataset X from the population.
General settings across all experiments For the experiments shown in Table  1 , under the normal model, 100 datasets of size n = 1000 observations were simulated under the following parameter values: dimension p = 100, 300, 1000, true mean θ * = 0. 04, and variance parameter σ 2 = 1. For each replicate, the analytical values of the log Bayes factor and the relative cross-validation score between the two models were computed (see Appendix). In the case of cross-validation, since the results are ultimately averaged over the 100 independent simulation replicates, only one choice for the splitting of the dataset into a validation and a training set is considered for each replicate, taking the first 800 data points for training and the last 200 for validation. For sIS, the first approach (fixed K for all data points) was used, with K such as indicated in Table  1.
For the phylogenetic analyses, two previously published empirical datasets were considered: • EF2: a multiple sequence alignment of elongation factor 2 in 30 eukaryotic species (627 aligned positions), taken from  Lartillot & Philippe (2006) ; • Metazoa: and a concatenation of genes (35371 aligned positions) across 132 metazoans, along with 2 choanoflagellates and 12 fungi for the outgroup  (Philippe et al. , 2005). In the case of EF2 (Table  2 ), for k-fold CV (both joint and site-wise), 10 random reshufflings of the sites of the original dataset were created. Each of these reshuffled version of the dataset was further split into a training and a validation set, on which nIS was applied, running the MCMC for 1100 points (saving every cycle for JTT, LG, GTR and every 3 cycles for CAT) on the traning set and then computing the CV score of the validation set based on the 1000 last points of the MCMC.
For sIS, a preliminary run on the original dataset and under each model was conducted for 1100 points (saving every 3 cycles for CAT). Empirical posterior means and variances for defining the reference priors for sIS were obtained based on the last 1000 points of this preliminary run. The self-tuned version of the sIS method was then used, with the following parameter values: B = 10, K 0 = 30, K max = 1000 and v 0 = 0. 1, with L = 2 independent runs for each data replicate (using the original dataset for computing the marginal likelihood and the 10 random reshufflings of the sites for k-fold CV). For LOO-CV, L = 2 independent runs were conducted on the complete dataset and under each model, again for 1100 points and saving every 3 cycles for CAT, discarding the first 100 points of burn-in and applying the CPO method on the 1000 remaining points.
For the experiments shown in Figure  2 , the Metazoan dataset was first filtered to remove sites with more than 20% of missing data, leaving a total of 9804 sites. Then, 4 jackknife replicates of sizes n = 200 to 800 were randomly sampled. For figure  3 , a standard MCMC was run under the complete dataset (9804 sites) and under the LG model, for a total of 1100 points. Then, 10 posterior predictive replicates were simulated, based on 10 samples regularly spaced across this MCMC chain, discarding the first 100 points and taking one every 100 points. Each of these simulated datasets was then jackknifed, yielding replicates of sizes n = 200 to n = 800. Both the empirical and the simulated jackknife replicates were then used for assessing the fit of the model by marginal likelihood or LOO-CV, using the same settings as for EF2.
For Figure  4 , the complete dataset was used, and 4 jackknife replicates of size ranging from n = 200 to n = 16000 were randomly sampled. The LOO-CV scores and the two versions of the wAIC were computed by running L = 2 independent chains of 1100 points (saving every 3 cycles for CAT) on each jackknife replicate, using the last 1000 points, with or without a 10-fold thinning (in which case the Monte Carlo estimates of the posterior averages are based on 100 points).
Data and software availability All methods for phylogenetic models were implemented in PhyloBayes  (Lartillot et al. , 2013) , version 1. 9, available at https://github. com/bayesiancook/pbmpi. The empirical data used here are also available trough this repo, along with example scripts. The methods for the analytical and Monte Carlo results under the normal model are available at https://github. com/bayesiancook/ normcv.
Source classifications, stellar masses, and star-formation rates are presented for ≈80 000 radio sources from the first data release of the Low Frequency Array Two-metre Sky Survey (LoTSS) Deep Fields, which represents the widest deep radio surv e y ev er undertaken. Using deep multi-wavelength data spanning from the ultraviolet to the far-infrared, spectral energy distribution (SED) fitting is carried out for all of the LoTSS Deep host galaxies using four different SED codes, two of which include modelling of the contributions from an active galactic nucleus (AGN). Comparing the results of the four codes, galaxies that host a radiative AGN are identified, and an optimized consensus estimate of the stellar mass and star-formation rate for each galaxy is derived. Those galaxies with an excess of radio emission o v er that e xpected from star formation are then identified, and the LoTSS Deep sources are divided into four classes: star-forming galaxies, radio-quiet AGN, and radio-loud high-excitation and low-excitation AGN.
Ho we v er, to e xtract the maximum science from deep radio surv e ys, it is essential that they are carried out in regions of the sky that are extremely wellstudied at other wavelengths across the electromagnetic spectrum. The ancillary data are required to identify the radio source host galaxies, to estimate their redshifts, to classify the nature of the radio emission (star formation versus radiatively efficient AGN versus jetmode AGN), and to determine the physical properties of the host galaxies (stellar mass,  SFR, environment, etc). Until recently, the state-of-the-art in wide-area deep radio surv e ys was the VLA-COSMOS 3 GHz surv e y (Smol či ć et al. 2017a ), which used the Very Large Array (VLA) to co v er 2 deg 2 of the Cos-mic Evolution Surv e y (COSMOS) field, arguably the best-studied de gree-scale e xtragalactic field in the sk y.
2021 , hereafter Papers I and II, respectively). An e xtensiv e optical and near-infrared cross-matching process has identified and provided detailed photometry for over 97 per cent of the ≈80 000 radio sources detected o v er the central regions of the target fields where the best ancillary data are available (a combined area of 25 deg 2 ;  Kondapally et al. 2021 , Paper III). These data have been used to provide high-quality photometric redshifts  (Duncan et al. 2021 , Paper IV). In this paper, the fifth of the series, these data are combined with far-IR data to carry out detailed spectral energy distribution (SED) fits to the multiwavelength photometry from ultraviolet (UV) to f ar-IR w av elengths, using sev eral different SED-fitting codes. Using the results of this analysis, the radio sources are classified into their different types, and key physical parameters of the host galaxies, such as their stellar masses and SFR, are determined.
The layout of the paper is as follows. In Section 2 , the LoTSS Deep Fields surv e y is described: this section outlines the choice of target fields and places the first data release in the context of the eventual full scope of the surv e y. Section 3 then describes the data that will be used in the paper and outlines the application of the SED-fitting algorithms. Section 4 describes how the results are used to identify the (radiative-mode) AGN within the sample. The results of the different SED-fitting algorithms are compared in Section 5 and used to define consensus measurements for the stellar mass and SFR of each host galaxy. Combining this information with the radio data, Section 6 then describes the identification of radio-excess AGN. Section 7 summarizes the final classifications of the objects in the sample and investigates the dependence of these on radio flux density , luminosity , stellar mass, and redshift.
The four target fields are the European Large Area ISO Surv e y Northern Field 1 (ELAIS-N1;  Oliver et al. 2000 ) , the Bo ötes field  (Jannuzi & Dey 1999 ) , the Lockman Hole  (Lockman, Jahoda & McCammon 1986 ) , and the North Ecliptic Pole (NEP); these are described in more detail in Section 2. 3. Table  1  outlines the anticipated final depths of each field based on awarded observing time. Scaling by depth and area from radio source counts in shallower LoTSS Deep observations, the final LoTSS Deep Fields are expected to detect more than 250 000 radio sources within the central 35 de g 2 , o v erlapping the best multiwavelength data. Fig. 1  compares the sensiti vity, field-of-vie w, and angular resolution of the LoTSS Deep Fields to other completed and on-going radio surv e ys. The final LoTSS Deep Fields dataset will be unri v alled in its combination of depth and area.
LoTSS Deep DR1 This paper considers the radio source catalogues from the first LoTSS Deep Fields data release. LoTSS Deep DR1 released the reduced LOFAR images and catalogues constructed from data taken before October 2018 (Paper II, Paper I), along with the optical/IR catalogues and host galaxy identifications (Paper III) and photometric redshifts (Paper IV). These LoTSS Deep DR1 LOFAR observations focused on the ELAIS-N1, Bo ötes and Lockman Hole fields, due to the earlier availability of the multiwavelength data in those fields. The LoTSS Deep DR1 LOFAR images included only the data from the Dutch LOFAR stations, not the international stations, due to the additional complications associated with calibrating the long baselines and the associated computing requirements (see e. g. Morabito et al.
Multiwavelength data in the LoTSS Deep Fields ELAIS-N1, Bo ötes, Lockman Hole, and NEP are the premier largearea northern extragalactic fields, with vast amounts of telescope time across the electromagnetic spectrum invested in observing these fields o v er the last two decades. Imaging at optical and near-IR wavelengths reaches 3-4 magnitudes deeper than typical all-sky surv e ys, allowing host galaxy identifications for o v er 97 per cent of the hosts of the radio sources in LoTSS Deep DR1 (Paper III) compared to just 73 per cent using all-sky surveys in the LoTSS DR1 release  (Williams et al. 2019 ). Other datasets, such as deep Herschel and Spitzer data in these fields, are irreplaceable, and add greatly to the scientific potential: Herschel data are a key tool to constrain obscured SFRs, while the mid-IR wavelengths covered by Spitzer contain the diagnostic emission from the AGN torus.
This range of complementary data makes these excellent fields to study not only the high-redshift AGN and luminous SFGs detected by LOFAR, but also to understand how this activity sits within the wider cosmological context of the underlying galaxy population. As well as their combined benefit of sky area and sample size, each of the four LoTSS Deep Fields possesses unique characteristics or datasets that further enhance its specific scientific potential, whilst complementing each other. The specific data available in each field are summarized here; a more complete description of the available data in the ELAIS-N1, Lockman Hole, and Bo ötes fields (but not NEP, as it was not included in the LoTSS Deep DR1) can be found in Paper III, which also provides the coverage maps of each survey and the resulting catalogues.
Bo ötes The Bo ötes field is the target of some of the deepest wide-field optical imaging, in the B W , R , and I filters from the NOAO Deep Wide Field Surv e y  (Jannuzi & Dey 1999 ) , in the z band from the zBo ötes surv e y  (Cool 2007 ) , and in the U and Y bands from the Large Binocular Telescope  (Bian et al. 2013  ), all co v ering around 10 de g 2. The same sk y re gion has been observ ed in the near-IR J , H , and K bands  (Gonzalez et al. 2010  ) and using Spitzer from 3. 6 to 8. 0 μm as part of the Spitzer Deep Wide Field Surv e y (SDWFS;  Ashby et al. 2009  ). Catalogues of galaxies in the Bo ötes field were generated by  Brown et al. ( 2007 Brown et al. ( , 2008 ) ). Bo ötes has also been observ ed by Hersc hel as part of HerMES, and by Spitzer -MIPS, adding far-infrared measurements to the dataset.
g. 'inside-out' or 'outside-in' growth) o v er large samples of massiv e galaxies with z < 1. Given these forthcoming datasets, a number of photometric surv e ys hav e been recently undertaken to provide matching observations at other wavelengths, including the Hawaii Two-0 surv e y  (McP artland et al. in preparation). Additionally, the Euclid / WFIRST Spitzer Le gac y Surv e y has obtained mid-infrared imaging o v er the central 10 deg 2 of the field using Spitzer that is ∼0. 8 mag deeper than the SERVS data available in ELAIS-N1 and Lockman Hole. As shown in Table  1  , the NEP is not included in LoTSS Deep DR1, and hence not included in the analysis of this paper, as the radio data were not available at the time of the optical cross-identification. An image from 72-hrs of data is now available and will be published by  Bondi et al.