dataesr/scientific-paragraphs-categorization huggingface dataset:
In each training iteration, a mini-batch of training instances was randomly selected from the training sample, consisting of imagelabel pairs. The images were randomly flipped and rotated with 90 deg steps under the assumption of spatial invariance. For the SDSS dataset, the baseline method and step 1 were conducted using the whole training sample of 393 219 instances. In steps 2 and 3, the constructed nearly balanced training subset was down-sampled to 60 000 instances to fit the soft Gaussian labels (Eqs. (  4 ) and (  8 )). Since we only need to train the output unit of a network with fewer training instances after step 1, the number of iterations and the mini-batch size were reduced. For the CFHTLS datasets, the CFHTLS-DEEP training sample of 22 256 instances and the CFHTLS-WIDE training sample of 115 937 instances were mixed for the baseline method and step 1 with both high-quality and low-resolution spectroscopic redshifts.
For steps 2 and 3, the CFHTLS-DEEP sample and the CFHTLS-WIDE sample had to be considered separately, however, due to potential systematics, producing two separate models for the two datasets. In addition, only the instances with highquality spectroscopic redshift in the training samples were used for bias correction, similar to those in the test samples. There are 11 220 high-quality instances in the nearly balanced training subset for the CFHTLS-DEEP sample and down-sampled 60 000 instances for the CFHTLS-WIDE sample. Finally, we leveraged the strategy of averaging an ensemble of models to improve the performance. For the baseline method, we trained the network five times. The training was conducted in the same way each time, but with a different initialization of weights and different selections of mini-batches.
Correction for z spec -dependent residuals and mode collapse The results of correcting z spec -dependent residuals and mode collapse are illustrated in Figs. 6 and 7 with the SDSS and the CFHTLS-WIDE test data, respectively. To exclusively demonstrate the outcome of bias correction, we do not show the result for the CFHTLS-DEEP dataset due to limited statistics, but we evaluate the quality of z photo estimation for this dataset in Appendix E. The redshift residual of each data instance indicates the difference between the estimated photometric and A36, page 9 of 28 A&A 662, A36 (2022) Fig. 6. Main results of correcting z spec -dependent residuals and mode collapse with the SDSS data and the network Net_P. Upper panel: mean residuals as a function of spectroscopic redshift for the baseline method and steps 1-3 of our methods with the SDSS data and the network Net_P.
We note that the multichannel output unit, the restricted subset of training data, and the soft labels would all contribute additional errors compared to the baseline method. While not strong for the SDSS data, this compromise is more significant for the CFHTLS data because the redshift range is wider and there are fewer training data. σ MAD = 1. 4826 × Median|∆z -Median(∆z)| where ∆z = z photo -z spec 1 + z spec. ( 11 ).
Upper panels: residuals of mean redshifts (different from mean residuals) in each z photo tomographic bin for the baseline method and step 4 of our methods using the network Net_P with the SDSS data, the CFHTLS-DEEP data, and the CFHTLS-WIDE data. A magnitude cut r < 24 is applied to the CFHTLS-DEEP data to create a restricted sample. The shaded areas indicate the requirement for the accuracy of photometric redshift estimation (i. e. , |∆z| < 0. 002). Steps 1-3 are applied in advance of the calibration by step 4. Middle panels: same as the upper panels, but residuals of mean redshifts are estimated in r-band magnitude bins. Lower panels: 1-Wasserstein distance between the z photo and z spec sample distributions (Eq. (  13 )) as a function of r-band magnitude. the Wasserstein distances. Noticeably, the bias correction contributes to a boost of the Wasserstein distances as a result of the compromise between biases and errors.
The methods we propose in this work for data-driven models may complement SED fitting approaches to deal with such effects. For the z photo -dependent residuals, our results from step 4 and the reference results from the data-driven approaches are similar. The drop at z photo ∼ 0. 25 for the SDSS data from  Pasquet et al. (2019)  is eliminated by our methods, and the plateau at z ∼ 0. 3 observed by  Pasquet et al. (2019)  is removed as a result of correcting for mode collapse. Although our methods do not result in biases as strong as produced by the SED fitting method for the CFHTLS-WIDE data, there are non-negligible errors at high redshifts that confine the robustness of our methods, suggesting that a good control of estimation errors is a precondition for bias correction.
-z photo -dependent residuals were resolved by reconstructing the distribution of the test sample with the training data and subtracting the mean z photo -z spec deviation in each cell of the z photo -magnitude grid, which is a direct calibration independent of CNN models. We applied our methods to the SDSS data that cover 0 < z spec < 0. 4, as well as to the CFHTLS data in a wider range of 0 < z spec < 4. 0 from the Deep and Wide fields. In terms of bias control, our methods achieve better performance than the baseline method, the KNN method from  Beck et al. (2016) , the SED fitting method using the LE PHARE code  (Arnouts et al. 1999; Ilbert et al. 2006) , and the CNN models from  Pasquet et al. (2019) and Treyer et al. (in prep. ). We also examined a few implementing details and assumptions behind our methods, and analyzed their effects ), and SED fitting using the LE PHARE code  (Arnouts et al.
In addition, biases scale with estimation errors in such a way that correcting large biases caused by large errors would yield even larger errors. It is thus important to have a good constraint of estimation errors before the bias correction. In spite of increasing errors, we prioritized bias correction in this work because it is critical to cosmological analysis. The calibration of residuals with the CFHTLS data in highredshift intervals is unable to meet the accuracy requirement of |∆z| < 0. 002 for a Euclid-like survey. This partial failure for the CFHTLS data points to the difficulty of applying datadriven methods to the high-redshift regime provided with a shortage of spectroscopic training data with good S/Ns. Given limited spectroscopic follow-ups for the upcoming imaging surveys, there is an increasing need for optimizing the exploitation of training data, either at the data level (data augmentation, resampling, etc.
As discussed in Section 3. 2. 3, the influence from underpopulated bins can be modeled by assuming a Gaussian scattering profile linearly modified by the local number density. This assumption might be violated if the peak number density is significantly high, as suggested by the growth of σ MAD and the overly corrected slopes with the SDSS data for step 3 when the threshold is above 200. Therefore, it is necessary to trim the peak densities before modeling the scattering profiles. Based on our observations from the figures, we take 200 and 100 as the thresholds in the main experiments for the SDSS data and the CFHTLS-WIDE data, respectively. Each of them is a tradeoff between suppressing the overpopulated bins and avoiding overfitting.
11) , the slopes of the < ∆z > -z spec piecewise linear fit (Eq. 9) and the total variation distance between the z photo and z spec sample distributions (Eq. 10) as a function of the balancing threshold that defines the nearly balanced training subset for steps 2 and 3 of our methods. As the threshold approaches the maximum number density (2,626 in this case), the results from step 2 are expected to converge to those from step 1, and those from step 3 are expected to converge to those from step 3\2 in which step 2 is skipped. The dotted curves show the total variation distances for the simulated z photo sample distributions that are expected to have no mode collapse. We compare the methods using the SDSS data and the network Net_P.
D. 3. Impact of the number of iterations The performance of a model is influenced by the number of iterations conducted in training, as it (together with the mini-batch size) determines the average number of times each instance is fed into the model and how well the model fits the training sample. In general, a model trained from scratch would gradually depart from a state of underfitting as the number of training iterations increases, and there might be a trend of overfitting if the volume of information in the training sample does not match the complexity of the model. Using the SDSS data and the network Net_P, we explore this evolution trend by continuously training the network and saving the trained parameters at a set of different iterations for the baseline method and step 1. We do not change the numbers of iterations for steps 2 and 3, however.
The dotted curves show the total variation distances for the simulated z photo sample distributions that are expected to have no mode collapse. The baseline method and steps 1 -3 of our methods are compared using the SDSS data and the network Net_P. the validity of our bias correction methods is verified in all these cases. We use a number of 60,000 iterations for representation learning in the main experiments, but mild underfitting or overfitting would not produce any major impact. We continue discussing the effect of overfitting in Appendix D. 4.
D. 4. Impact of the sample size The size of the training sample is one of the most important factors influencing the performance of a model. Predominantly, a complex model would easily overfit on a small training sample. As a complement to the experiment of varying the number of training iterations (discussed in Appendix D. 3), we investigate the effect of overfitting by controlling the size of the training sample for the baseline method and the representation learning in step 1 using the SDSS data and the network Net_P. However, we use the original training sample to construct a nearly balanced subset for steps 2 and 3. As illustrated in Fig. D. 7, the estimation accuracy (indicated by σ MAD ) gradually drops as a result of increased overfitting when the sample size decreases, in general leading to higher residuals suggested by the steeper < ∆z > -z spec curves, but weaker mode collapse suggested by the lower total variation dis-   Eq.
11) , the slopes of the < ∆z > -z spec piecewise linear fit (Eq. 9) and the total variation distance between the z photo and z spec sample distributions (Eq. 10) as a function of the number of training iterations conducted for the baseline method and step 1 of our methods. The number of iterations for steps 2 and 3 remains the same as in the main experiments. The dotted curves show the total variation distances for the simulated z photo sample distributions that are expected to have no mode collapse. We compare the methods using the SDSS data and the network Net_P. tances between the z photo and z spec sample distributions, which is similar to the consequence of conducting more training iterations. On the other hand, the performance of bias correction appears robust for different degrees of overfitting.
D. 5. Impact of labeling errors In our main experiments, the labels used in training are assumed to contain no errors for the SDSS data and the high-quality CFHTLS data. Using the SDSS data and the network Net_P, we examine whether Gaussian labeling errors would have an effect on the behavior of the biases and the performance of our methods. We randomly draw Gaussian errors with a series of given dispersions (σ z s) and add them to the initial spectroscopic redshifts for the training data (but not the test data). The erroneous spectroscopic redshifts are fixed and those within the redshift range [0, 0. 4] are converted into labels used for the baseline method and the representation learning in step 1. As shown in Fig. D. 8, introducing labeling errors leads to an increase in estimation errors indicated by σ MAD and may have an impact on the slopes of the < ∆z > -z spec curves.
D. 6. Impact of the model complexity The complexity of a model determines the volume of information that can be learned from data and thus affects the perfor-A36, page 25 of 28 mance of the model. A more complex model would generally have lower errors and biases in its prediction if not overfitted. We investigate whether our methods are robust with models that have different complexities using the SDSS data. In addition to the networks from  Pasquet et al. (2019) and Treyer et al. (in prep. ) (denoted with "Net_P" and "Net_T", respectively), we experiment on other three simple networks that differ in the number of layers (summarized below). Likewise, the penultimate fully connected layer of each network is used to obtain a representation.
The dotted curves show the total variation distances for the simulated z photo sample distributions that are expected to have no mode collapse. We compare the methods using the SDSS data and the network Net_P. Eq. 11) , the slopes of the < ∆z > -z spec piecewise linear fit (Eq. 9) and the total variation distance between the z photo and z spec sample distributions (Eq. 10) as a function of the dispersion of Gaussian errors σ z introduced to the redshift labels for the training data used for the baseline method and step 1 of our methods. The Gaussian errors are not applied to the training data in steps 2 and 3, or the test data. The dotted curves show the total variation distances for the simulated z photo sample distributions that are expected to have no mode collapse. We compare the methods using the SDSS data and the network Net_P.
In addition, all these cases show similar < ∆z > -z spec curves in spite of different slopes, suggesting that the uneven data distribution inherently determines the existence of class-dependent residuals. (Eq. 11) , the slopes of the < ∆z >spec piecewise linear fit (Eq. 9) and the total variation distance between the z photo and z spec sample distributions (Eq. 10) as a function of different network architectures with increasing complexity (defined in the text). The dotted curves show the total variation distances for the simulated z photo sample distributions that are expected to have no mode collapse. The baseline method and steps 1 -3 of our methods are compared using the SDSS data. Finally, we caution that it might not be statistically meaningful to evaluate global properties for heterogeneous samples with these metrics.
Acknowledgements. This project has received funding from the  European Union 's  Horizon 2020 research and innovation programme  under the  Marie Skłodowska-Curie  grant agreement  No713750. Also, it has been carried out with the financial support of the  Regional Council of Provence-Alpes-Côte d'Azur  and with the financial support of the A*MIDEX (n •  ANR-11-IDEX-0001-02 ), funded by the  Investissements d' Avenir  project funded by the  French Government , managed by the  French National Research Agency (ANR). This work has been carried out thanks to the support of the DEEPDIP  ANR  project ( ANR-19-CE31-0023 ) and the  Programme National Cosmologie et Galaxies (PNCG)  of  CNRS/INSU  with  INP  and  IN2P3 , co-funded by  CEA  and  CNES. This work makes use of  Sloan Digital Sky Survey (SDSS) data.
- 3 × 3 Fig. D. 8. σ MAD(Eq. 11), the slopes of the < ∆z > -z spec piecewise linear fit (Eq. 9) and the total variation distance between the z photo and z spec sample distributions (Eq. 10) as a function of the dispersion of Gaussian errors σ z introduced to the redshift labels for the training data used for the baseline method and step 1 of our methods. The Gaussian errors are not applied to the training data in steps 2 and 3, or the test data. The dotted curves show the total variation distances for the simulated z photo sample distributions that are expected to have no mode collapse. We compare the methods using the SDSS data and the network Net_P.
Acknowledgements This study was initiated while T. B. was visiting the  Department of Mathematical, Physical and Computer Sciences, University of Parma , with the financial support of the  Ecole Normale Superieure Paris-Saclay. The authors thank the support by the  University of Parma , by the  Italian National Group of Mathematical Physics  ( GNFM-INdAM ), and by the  Italian National Research Project  " Multiscale phenomena in Continuum Mechanics : singular limits, off-equilibrium and transitions" (Prin  2017YBKNCE ).
r. t. ) µ. Our main result states that Λ µ t disintegrates (or factorizes) into (1. 2) Λ µ t (x, y) = Λ φ t (x) • Λ νx,t t (y) , ∀(x, y) ∈ R n+1 , ∀t ∈ N , where Λ φ t (resp. Λ νx,t t ) is the Christoffel function of the marginal φ of µ on X (resp. of some probability measure ν x,t on R, given x ∈ X). Moreover, for every fixed x ∈ X, one can compute explicitly the Hankel moment matrix of the measure ν x,t by solving a single convex optimization problem on positive definite matrices with log det(•) as objective function. Research supported by the AI Interdisciplinary Institute ANITI funding through the french program Investing for the Future PI3A under the grant agreement number ANR-19-PI3A-0004. The author is also affiliated with IPAL-CNRS laboratory, Singapore. Notice how (1. 2) mimics the disintegration  (1.
Acknowledgement I would like to thank warmly my three advisors:  Francesco Costantino  for his guidance throughout the discovery of this subject and the editing of this article;  Joan Bellier-Millès  for all the time he spent in explanations and  David Jordan  for very helpful conversations and comments. I am grateful to  Patrick Kinnear  for his remarks and advises, and to  Romane Clavié  for her constant support. This research took place in the  Institut Mathématique de Toulouse  and was supported by the  École Normale Supérieure de Lyon.
Ant Specimens We analyzed a total of 80 specimens of three ant species of the Odontomachus genus. All included specimens were taken from the Natural History Museum of the National Science Museum (Thailand) and were collected throughout Thailand between 2002 and 2018 (Table  S1 ). Only worker ants were used for this study. The workers of Odontomachus are monomorphic [2,19], i. e. , their size variation is generally limited, with only a single distribution mode. Using the contour of the head, we compared 25, 33, and 22 specimens of O. monticola, O. rixosus, and O. simillimus, respectively. For the pronotum, two specimens of O. simillimus were removed from the data because of an accidental deformation of their contour. The head and pronotum were photographed using a DS-Ri1 SIGHT digital camera attached to a Nikon AZ 100M stereomicroscope (Nikon Corp.
Characteristics of the Study Population Overall, we analyzed samples from 71 patients (males 62%) (Table  1 ). Regarding their clinical features, data were provided for 66 of them (93%), while for 5 of them (7%) there were no data provided. Out of those, 17 were hospitalized (26%) in severe or critical condition during their infection, while 49 were not hospitalized (74%), showing mild COVID-19 clinical symptoms, including cough, sore throat, mild fever below 38 • C, and loss of smell. Sample collection was conducted at various time-points up to fifteen months after patients' first positive PCR test (T0) and the dates of the sample collection were grouped into four categories: 1-3 months, 4-5 months, 6-8 months, and 12-15 months. Sampling occurred two (24%) or three times (76%) for each patient.
Acknowledgments. We are grateful to  Francesco Bonsante  and  Bruno Martelli  for interesting discussions, useful advices, and encouragement. We also thank  Jeffrey Danciger  and  Gye-Seon Lee  for interest in this work and related discussions, and  Clément Guérin  and  Andrea Maffei  for ad-hoc explanations about character varieties. We are grateful to an anonymous referee whose suggestions highly improved the exposition and results of this paper. We thank the  mathematics departments of Pavia, Luxembourg and Neuchâtel , for the warm hospitality during the respective visits while part of this work was done. The stage of this collaboration was set during the workshop "Moduli spaces", held in Ventotene in September 2017: we are grateful to the organisers for this opportunity.
Widely applicable, accurate and fast inference methods in phylodynamics are needed to fully profit from the richness of genetic data in uncovering the dynamics of epidemics. Standard methods, including maximum-likelihood and Bayesian approaches, generally rely on complex mathematical formulae and approximations, and do not scale with dataset size. We develop a likelihood-free, simulation-based approach, which combines deep learning with (1) a large set of summary statistics measured on phylogenies or (2) a complete and compact representation of trees, which avoids potential limitations of summary statistics and applies to any phylodynamics model. Our method enables both model selection and estimation of epidemiological parameters from very large phylogenies. We demonstrate its speed and accuracy on simulated data, where it performs better than the state-of-the-art methods.
In the following, we introduce our vectorial tree representation and the new summary statistics designed for BDSS. We then present the deep learning architectures trained on these representations and evaluate their accuracy on simulated datasets in terms of both parameter estimation and model selection. We show that our approach applies not only to trees of the same size as the training instances, but also to very large trees with thousands of tips through the analysis of their subtrees. The results are compared to those of the gold standard method, BEAST2  [12, 13]. Lastly, we showcase our methods on an HIV dataset  [24, 25]  from the men-having-sex-with-men (MSM) community from Zurich. All technical details are provided in Methods and Supplementary Information. Our methods and tools are implemented in the PhyloDeep software, which is available on GitHub (github.
Inspired by these approaches, we designed our concise, easily computable, compact, and bijective (i. e. 1-to-1) tree representation that applies to trees of variable size and is appropriate as machine learning input. To obtain this representation, we first ladderize the tree, that is, for each internal node, the descending subtree containing the most recently sampled tip is rotated to the left, Fig. 2  a (ii). This ladderization step does not change the tree but facilitates learning by standardizing the input data. Moreover, it is consistent with trees observed in real epidemiological datasets, for example Influenza, where ladder-like trees reflect selection and are observed for several pathogens  [1]. Then, we perform an inorder traversal  [30]  of the ladderized tree, during which we collect in a vector for each visited internal node its distance to the root and for each tip its distance to the previously visited internal node.
Simulated datasets For each phylodynamic model (BD, BDEI, BDSS), we simulated 4 million trees, covering a large range of values for each parameter of epidemiological interest (R0, infectious period: 1/γ, incubation period: 1/ε, the fraction at equilibrium of superspreading individuals: fSS, and the superspreading transmission ratio: XSS). Of the 4 million trees, 3. 99 million were used as a training set, and 10,000 as a validation set for early stopping in the training phase  [31]. Additionally, we simulated another 10,000 trees, which we used as a testing set, out of which 100 were also evaluated with the gold standard methods, BEAST2 and TreePar, which are more time consuming. Another 1 million trees were used to define confidence intervals for estimated parameters. For BD and BDEI we considered two settings: one with small trees (50 to 199 tips, in Supplementary Fig.