dataesr/scientific-paragraphs-categorization huggingface dataset:
8-A , just as we discussed in the context of the artificial 1D dataset. Then, in a second learning phase, the SMC relaxation time falls back to measurable values using the transverse directions learned by the RBMs. Even though SMC thermalization times remain quite high, SMC sampling becomes competitive with TMC sampling in terms of physical computation times because TMC relaxation times grow rapidly when m 0 is no longer able to break metastabilities, and because TMC sampling is much slower than SMC because the update of visible variables cannot be parallelized. Nevertheless, quite unexpectedly, we find that TMC-RBMs have much faster SMC relaxational dynamics than SMC-RBMs and that the factor between the two increases with the number of parameter updates, see Fig. 8-B. When we compare the samples generated with the two learned RBMs at t age = 1001 parameter updates, we find that those obtained with the TMC-RBM are qualitatively better than those from the SMC-RBM.
A B Figure  8 : A: Histogram of the dataset along the first direction ω 0 , together with the tethered potential Ω( m0 ) and the corresponding reconstructed TMC probability p( m0 ) of both RBMs in the early stages of learning (here t age = 61). We clearly observe the formation of two metastable states in both machines. In practice, we observe that Markov chains using SMC dynamics are trapped in one of these metastable states since initialization and cannot visit the rest of the phase space. Exactly as described in the synthetic 1D dataset. B: We estimate the SMC integrated autocorrelation time τ SMC int of m 0 (t) as a function of the number of training updates used to train the two machines: TMC-RBM and SMC-RBM. The shaded area marks the region where τ SMC int exceeds 10 7 MC sweeps, while τ TMC int (for the slower m constraints ) remains below 15 MC sweeps.
Human genome dataset We considered the population genetics' dataset of Ref. [35, 40]. This dataset corresponds to a sub-part of the genome of a population of 2504 individuals (each providing two strands of DNA, yielding a dataset of 5008 samples) sampled from 26 populations in Africa, East Asia, South Asia, Europe, and the Americas. Each variable is binary {0, 1} and indicates whether or not a particular gene is altered compared to the human reference genome. A total of 805 alterations are reported, reflecting a high proportion of the population structure present in the whole genome dataset. The dataset is therefore made of 805 binary variables and 5008 independent samples. For our study, the dataset was made of 4500 samples for the training set. This dataset has a very clustered nature, as can be appreciated in the dataset's projection along the second and third principal directions shown in Fig.
10  where each dataset sample is represented as a black dot. One can also see that SMC dynamics get completely trapped in these different clusters in a well-trained machine, as we illustrate with different color trajectories each corresponding to an independent sampling of the RBM. We therefore constrain these directions for the training of the RBM using the TMC method. For this case, we will exhibit two clear advantages of this method. First, we show the feasibility of training the RBM on a real dataset using a set of two constrains. In this setting, the TMC method is not only useful to estimate better the negative term of the gradient, but it also gives the possibility to monitor how-well the equilibrium measure of the RBM is matching the data through the TMC effective potential. It is worth of mentioning that evaluating the quality of an RBM (that is, ensuring a proper adjustment of the density peaks with those of the dataset) with traditional SMC sampling methods is extremely hard, because jumps between the different metastable clusters are very rare and there are four of such clusters involved.
In fact, on Fig. 11  we show the TMC effective potential and the reconstructed distribution at various stages of the learning. It is interesting to see that it seems quite difficult to adjust correctly the 4 clusters to their correct weight. Thanks to the TMC method, we can easily track the position of the RBM's clusters and diminish the learning rate to fine-tune the learning. On Fig. 11 , we can see that after t age ≈ 1200, the TMC-RBM seems to be well-learned. The second crucial improvement is that, only with the TMC method we can guarantee that all the clusters are visited regularly. Indeed, we already showed in Fig. 10  that we suffered strong break of ergocity issues even after 10 5 SMC sweeps. This fact tells us that, on this dataset, each persistent chain of the PCD will remain trapped in certain regions of the phase space and will not be representative of the equilibrium measure of the RBM.
As a consequence, the persistent chains will not estimate correctly the negative term of the gradient. Finally, it is possible to try to train this dataset using the classical PCD -SMC algorithm. However, what we see is that the learning dynamics has a lot of trouble adjusting correctly the different minima, and that in addition, even if one can "cherry pick" the exact number of updates where the trained SMC-RBM has all the good dataset minima, such a process requires using the TMC method to track the presence of the good minima in the effective potential because SMC samplings are completely stuck. Even in that case, we observe that again TMC-RBMs relax faster than the equivalent SMC-RBM (for the same t age ). Yet, SMC dynamics are that slow for both kinds of machines, that computing τ SMC int , as done in Fig.
Conclusion We have demonstrated the feasibility and benefits of using a biased Monte Carlo sampling strategy for both training -i. e. , to better estimate the negative terms of the gradient in order to update the parameters of the model -and for generating new samples in trained RBMs. The largest improvements are observed for highly clustered datasets, where classical local MCMC moves get stuck in one or several of these clusters and fail to sample ergodically the entire phase space (in reasonable time). We show that the Tethered Monte Carlo method is extremely efficient for sampling the equilibrium measure of these types of models, as long as one can find good observables to uniquely identify each cluster. We show that the projection along the first PCA directions in the early stages of training are good order parameters as motivated from the mean-field results  [16, 32].
We show that this TMC strategy is crucial to correctly learn artificial low-dimensional clustered datasets. On these datasets, the standard learning approach that combines parallel local MCMC updates with PCD completely fails to obtain reliable models for the data. On real datasets, we find that as learning progresses, more ordering parameters are needed to avoid the presence of metastabilities during sampling, which means that the TMC strategy with only one or two constraints also starts to suffer from ergodicity problems. We note, however, that even if TMC does not thermalize easily, the models trained with TMC exhibit significantly faster relaxation dynamics than the models obtained with the standard approach. Moreover, we find that even for RBMs trained with standard sampling techniques, the TMC sampling approach is extremely useful to assess the quality of a given trained machine, since it allows a direct calculation of the probability distribution of the model, projected along a set of selected directions, thus providing the possibility to estimate how well the distribution of the model matches the empirical one.
The developed approach is thus a practical alternative to the most commonly used Gibbs sampling algorithm when the intrinsic properties of the dataset drive learning toward a model with well-separated modes. Despite the impressive performance of the TMC model on clustered datasets, it has two important drawbacks: First, it is much slower than the traditional alternative sampling method in terms of computation time because the global constraint introduces correlations between the visible (or hidden) variables, thus removing the conditional independence of the two layers that favors parallelization. For this reason, alternative approaches involving only local constraints should be considered in the future to restore parallel performance. A possible improvement could be to constrain the value of some hidden nodes with a condition that does not remove the conditional independence.
Acknowledgments This work was supported by the  Comunidad de Madrid  and the  Complutense University of Madrid (Spain)  through the  Atracción de Talento programs (Refs. 2019-T1/TIC-13298  for A. D. and  2019-T1/TIC-12776  for B. S. ), the  Banco Santander  and the  UCM  (grant  PR44/21-29937 ); and  Ministerio de Economía y Competitividad ,  Agencia Estatal de Investigación  and  Fondo Europeo de Desarrollo Regional (FEDER) (Spain and European Union)  through the grants. PID2021-125506NA-I00  and  PGC2018-094684-B-C21. N. B. would like to thank the  Department of Theoretical Physics of the Complutense University of Madrid  for the kind reception and support during his stay.
25°. The chemical scheme used in PYVAR-CHIMERE is MELCHIOR-2, with more than 100 reactions (CHIMERE, 2017), including the secondary production of CO through the oxidation and photolysis of hydrocarbons and its sink with OH. Initial and boundary conditions for several key gaseous species responsible for the oxidation capacity of the lower atmosphere (e. g. , CO, NO, NO 2 , O 3 ,H 2 O 2 , HCHO, etc) were specified using monthly climatological data from LMDz-INCA global model  (Szopa, 2008). CO emissions from fires, that count for about 2% of the European total COemissions (San-Miguel-Ayanz and Steinbrecher, 2019), are not taken into account in this study. CO biogenic emissions are assumed to be negligible and are not taken into account. The prior estimate of CO anthropogenic emissions is derived from the recent TNO-GHGco-v3 gridded inventory for the period 2005-2018.
, 2022). The temporal disaggregation is based on temporal profiles provided per GNFR sector code with typical month to month, weekday to week-end and diurnal (at 1-hour scale) variations. The TNO-GHGco-v3 inventory is aggregated at the 0. 5°x0. 5°horizontal resolution of the CHIMERE grid. The resulting prior anthropogenic CO emissions are illustrated in Figure  2a : CO emissions are high over large cities and over industrial areas (e. g. over the Benelux, the Po Valley in Italy, northwestern Germany, southern Poland). In addition to CO, the chemical scheme MELCHIOR-2 needs emissions from other species, such as NMVOCs or nitrogen oxides (NO x =NO+NO 2 ). Anthropogenic NO x emissions are from the TNO-GHGco-v3 inventory while NMVOC anthropogenic emissions are from the EMEP inventory  (Vestreng et al.
Finally, there is no significant inter-annual variability from 2011 to 2019, neither in the prior nor in the posterior CO emissions. A particular attention has consequently been put on the possible detection of an inter-annual anomaly linked to the policies implemented in response to the COVID-19 in 2020. impact is expected for CO than for NO 2 , particularly because of smaller contributions from lockdown-affected sources  (Clark et al. , 2021). At the European scale, the CO posterior emission estimates derived from the MOPITT data decrease by about -1. 3% in April 2020 compared to April 2019. This decrease is lower than the estimates of about -4. 7%, -6. 4%, -7. 6% and -8. 2% of, respectively,  Guevara et al. (2023 ), Doumbia et al. (2021) ,  Forster et al. (2021)  and from the officially reported emissions from EMEP/CEIP (CEIP, 2022).
5km x7 km resolution since August 2019 should be well suited for such a perspective. The large increase in the number of observations with this mission is expected to increase the capabilities to monitor CO emissions and to address air-quality-related emissions at the national to subnational scales. MOPITT version 8 products are freely available through NASA's EarthData portal at https://earthdata. nasa. gov/,  (Deeter et al. , 2019). The TNO-GHGco-v3 inventory  (Super et al. , 2020)  is available upon request from TNO (contact: Hugo Denier van der Gon, hugo. deniervandergon@tno. nl). The CHIMERE code is available here: www. lmd. polytechnique. fr/chimere/,  (Menut et al. , 2013; Mailler et al. , 2017). The CIF inversion system  (Berchet et al. , 2021).
Acknowledgements. We acknowledge the  NCAR MOPITT group  for the production of the CO retrievals. A large part of the development and analysis were conducted in the frame of the H2020  VERIFY  project, funded by the  European Commission   Horizon 2020 research and innovation programme , under agreement number  776810. We wish to thank all the persons involved in the preparation, coordination and management of this project. This project has also received funding from the  French ANR  project  ARGONAUT  under grant agreement No  ANR-19-CE01-0007  and from the  French PRIMEQUAL project LOCKAIR  under grant agreement No  2162D0010. This work was supported by the  CNES (Centre National d'Etudes Spatiales) , in the frame of the  TOSCA ARGOS  project. This work was granted access to the HPC resources of TGCC under the allocations  A0100102201  and  A0110102201  made by  GENCI.
This budget closure is seen as an important component of the success of the dynamical regime identification. Note that sea ice is taken into account in the present analysis as is included in ECCOv4, and is reflected in the momentum budget of the underlying ocean. The global ECCOv4 model is preferred to a regional model, as this represents the global co-variance space given by the ocean dynamical drivers in a more consistent manner. This is because of the unbalanced nature of the dataset, where those regions in covariance space that do not account for a large percentage of the overall data, have more opportunity to be recognized as a coherent pattern, if this is what they are, rather than being classified as noise. Note that a regional model could result in the opposite effect, where noise is mistakenly identified as a region of interest.
A cluster number of 50 was chosen, where there were 5 dominant regimes and the remaining 45 were grouped into the 'nonlinear' regimes as they were predominantly nonlinear and as a whole consistently classified (see ref. 6  for details). The method and workflow, in combination with working on a closed BV budget, were designed for interpretability, and to be appropriate for the geoscientific application. The interpretability comes from the method being decomposable into separate steps, where the steps are humanly tractable, and using an algorithm that offers transparency through its simplicity and clarity of assumptions. The dynamical regimes were originally presented in ref. 6  , where more details on the method can be found as well as extensive code and explanation at https://github. com/maikejulie/DNN4Cli.
Focusing on the upwelling of carbon  87  shows a similar relationship with topography, but only considers the location where a particle crosses 1000 m and discards the areas associated with sea ice. Both Tamsitt et al. and Brady et al. 74, 87  do not consider the upwelled volume and may not have time integrations long enough to capture the impact of the circulation in areas with longer residence times like the Weddell and Ross Seas. Future work assessing the upwelled volume of CDW waters using a lagrangian framework could offer fruitful insight. Our study uses the state estimate (ECCO), which offers the closest product that can act as a comprehensive observational dataset capable of closing budgets of vorticity. We note that the low resolution of the state estimate means that direct comparisons to observational data are difficult.
Acknowledgements The authors are grateful for valuable discussions with  S. Griffies ,  V. Balaji ,  A. Adcroft ,  L. Talley ,  Y. Nakayama ,  D. Jones ,  J. Deshayes ,  I. Rosso. ,  E. Wilson  and  A. Thompson. We are grateful to  Y. Cho  who offered assistance with Figs. 2 and8. MS funding:  Cooperative Institute for Modelling the Earth System, Princeton University , under Award  NA18OAR4320123  from the  National Oceanic and Atmospheric Administration, U. S. Department of Commerce. The statements, findings, conclusions, and recommendations are those of the authors and do not necessarily reflect the views of  Princeton University , the  National Oceanic and Atmospheric Administration , or the  U. S. Department of Commerce. RL funding: The  Make Our Planet Great Again (MOPGA ) funding from the  Agence National de Recherche  under the " Investissements d'avenir" programme  with the reference  ANR-17-MPGA-0010.
Data availability Data used here for the dynamical regimes is the same as for ref. 6  but collected for the reader's convenience here: https://zenodo. org/record/7783326#. ZCSzC47MJqI with https://doi. org/10. 5281/zenodo. 7783326. Additional data is taken from the ECCOv4 dataset available here: ECCO Consortium, Fukumori, I. , Wang, O. , Fenty, I. , Forget, G. , Heimbach, P. , & Ponte, R. M. (date accessed: 29 March 2023). ECCO Central Estimate (Version 4 Release 4). Retrieved from URL: https://www. ecco-group. org/products-ECCO-V4r4. htm.
1a ). We determined the HIO forming efficiency of each concentration by counting the number of spheroids plated in a single well at day 0, and again counting the number of HIOs present after 10 days in culture (formula: # of day 10 HIOs/# of day 0 spheroids=forming efficiency). Experiments were repeated for 3 separate experiments (batches), in 3 different iPSC lines with 3 technical replicates. We saw minimal difference in HIO forming efficiency when comparing ligand or concentration, with growth in 10 ng/mL EREG having the only statistically significant increase in forming efficiency across conditions compared to the EGF control cultures (Extended Data Fig. 2a ). We compared morphologic features between the culture conditions, including area, circularity, roundness, solidity, and aspect ratio.
4a ). Small portions of undefined or non-intestinal cells have previously been reported in HIOs  19  , and these cells were excluded from further analysis. To confirm cluster annotations based on marker gene expression (Fig. 2d ), we carried out a label transfer of EREG-tHIO data onto a previously published fetal intestine reference dataset  26  (Fig. 2e  and Extended Data Fig. 1a ), and confirmed the accuracy of annotations based on marker genes. tHIO datasets robustly label transferred onto their counterparts in the human fetal datasets using Seurat's integrated label transfer function  27  (Fig. 2e , Extended Data Fig. 4b ). To further benchmark tHIO cell type identity against our reference dataset, we used a cohort of highly enriched genes for each major cell type in the reference (endothelial, neural, smooth muscle, immune, epithelial, and mesenchymal), that allowed us to generate a "score" for each cell in the tHIO sample based on the enrichment of these genes.
3e  -left, grey). With these data suggesting functional muscle and the presence of neurons in these organoids, we hypothesized the presence of a functional neuromuscular unit in EREG-tHIOs. To examine if the neuronal populations functionally regulate smooth muscle contractions, we excited neurons by using the selective 3-nicotinic receptor agonist dimethylphenylpiperazinium (DMPP) to stimulate neurotransmitter release and activate neurons in EREG-grown tHIOs (Fig. 3f ). Following DMPP treatment, explants were treated with tetrodotoxin (TTX) to block action potentials which successfully blocked the neurons' ability to be depolarized again following another dose of DMPP, thus supporting ENS-dependent contractile activity within the tissue (Fig. 3g ). Finally, we assessed the function of nNOS-expressing neurons by inhibiting them with NG-nitro-L-arginine methyl ester (L-NAME), and cholinergic neurons by blocking them with a dose of atropine.
Stem cell lines and generation of human intestinal organoids (HIOs) This study includes data from HIOs generated across 3 hPSC lines: Human ES line H9 (NIH registry #0062, RRID: CVCL_9773, female) with an mCherry reporter, human iPSC lines WTC11 (RRID: CVCL_Y803, male) and 72. 3  31. All experiments using hPSCs were approved by the University of Michigan Human Pluripotent Stem Cell Research Oversight Committee. All stem cell and organoid lines were routinely monitored for mycoplasma using the MycoAlert Mycoplasma Detection Kit (Lonza Cat#LT07-318).
HIO shape and area quantification To compare shape and area of different HIO conditions, 5 organoids per condition were grown for 30 days in vitro and a 10x bright-field image of each organoid was outlined manually using the freehand selection tool in ImageJ. Outlines were measured in ImageJ with measurements set to capture area and shape descriptors including area, solidity, aspect ratio, circularity, and roundness. This was completed on three stem cell lines and measurements were graphed in Extended Data Fig. 2.
Quantification and statistical analysis (for RT-qPCR etc) All quantitative experiments were completed in 3 different organoid lines for 3 different batches with 3 technical replicates per batch. All statistical analysis was performed in GraphPad Prism Software. See figure legends for number of replicates used, statistical test performed, and the pvalues used to determine the significance for each separate analysis. All t tests were ra"n twotailed, unpaired with welch's correction.
Sequencing data analysis To generate cell-by-gene matrices, raw data was processed using the 10X Cell Ranger package, and sequenced reads were aligned to the human genome hg38. All downstream analysis was carried out using Scanpy 37 or Seurat  27  (depending on package usage needs). For primary human tissue sample analysis in Extended Data Fig. 1 , we reanalyzed the human whole cell fetal dataset published in our lab's previous work  10, 19, 26. Samples included a 47-day proximal intestine, a 59-day proximal intestine, two 72-day duodenum, 80-day duodenum and ileum, an 85-day duodenum, 101-day duodenum and ileum, two 127-day duodenums, 132-day duodenum. All samples were filtered to remove cells with less than 500 or greater than 10,000 genes, or greater than 60,000 unique molecular identifier (UMI) counts per cell.
For Extended Data Fig. 3 , all organoid whole cell samples (1 ng/ml EREG, 10 ng/ml EREG, 100 ng/ml EREG, 100 ng/ml EGF) were filtered to remove cells with less than 700 or greater than 6,800 genes, or greater than 33,000 UMI counts per cell, and 0. 1 mitochondrial cell counts. Data matrix read counts per gene were log normalized prior to analysis. After log normalization, highly variable genes were identified and extracted, no batch correction was needed as these samples were processed at the same time. Data was then scaled by z-transformation. Dimension reduction was performed using PCA and then UMAP on the top 10 PCs and 15 nearest neighbors for visualization. Clusters of cells within the data were calculated using the Louvain algorithm within Scanpy with a resolution of 0. 4. The 10 ng/mL EREG sample alone was filtered to remove cells with less than 700 or greater than 8,000 genes, or greater than 50,000 UMI counts per cell, and 0.
1 mitochondrial cell counts. Data matrix read counts per gene were log normalized prior to analysis. After log normalization, highly variable genes were identified and extracted. Data was then scaled by z-transformation. Dimension reduction was performed using PCA and then UMAP on the top 10 PCs and 15 nearest neighbors for visualization. Clusters of cells within the data were calculated using the Louvain algorithm within Scanpy with a resolution of 0. 4. The 1 ng/mL EREG sample alone was filtered to remove cells with less than 500 or greater than 8,000 genes, or greater than 45,000 UMI counts per cell, and 0. 1 mitochondrial cell counts. Data matrix read counts per gene were log normalized prior to analysis. After log normalization, highly variable genes were identified and extracted. Data was then scaled by z-transformation.