dataesr/scientific-paragraphs-categorization huggingface dataset:
These approximations can lead BEAST2 to a lack of convergence (2% cases for BDEI and 15% cases for BDSS) or a convergence to local optima. We suspect BEAST2 of converging to local optima when it converged to values with high relative error (>1. 0; 8% cases for BDEI and 11% cases for BDSS, Fig. 3 b-c ). Furthermore, our deep learning approaches showed a lower bias in parameter estimation than BEAST2 (Supplementary Tab. 2). As expected, both approaches, FFNN-SS and CNN-CBLV, get more accurate with larger trees (Supplementary Fig. 3 ). We tried to perform maximum likelihood estimation (MLE) implemented in the TreePar package  [5]  on the same trees as well. While MLE under BD model on simulations yielded as accurate results as BEAST2, for more complex models it showed overflow and underflow issues (i.
Showcase study of HIV in MSM subpopulation in Zurich The Swiss HIV Cohort is densely sampled, including more than 16,000 infected individuals  [24]. Datasets extracted from this cohort have often been studied in phylodynamics  [8, 25]. We analysed a dataset of an MSM subpopulation from Zurich, which corresponds to a cluster of 200 sequences studied previously by Rasmussen et al. [25]  , who focused on the degree of connectivity and its impact on transmission between infected individuals. Using coalescent approaches, they detected the presence of highly connected individuals at the beginning of the epidemic and estimated R0 to be between 1. 0 and 2. 5. We used their tree as input for neural networks and BEAST2. To perform analyses, one needs an estimate of the sampling probability. We considered that: (1) the cohort is expected to include around 45% of Swiss individuals infected with HIV  [24]  ; and (2) the sequences were collected from around 56% of individuals enrolled in this cohort  [34].
5 a-b ). The values obtained with FFNN-SS and CNN-CBLV are close to each other, and the 95% CI are nearly identical. We inferred an R0 of 1. 6 and 1. 7, and an infectious period of 10. 2 and 9. 8 years, with FFNN-SS and CNN-CBLV, respectively. Transmission by superspreading individuals was estimated to be around 9 times higher than by normal spreaders and superspreading individuals were estimated to account for around 7-8% of the population. Our R0 estimates are consistent with the results of a previous study  [8]  performed on data from the Swiss cohort, and the results of Rasmussen et al. [25]  with this dataset. The infectious period we inferred is a bit longer than that reported by Stadler et al, who estimated it to be 7. 74 [95% CI 4. 39-10. 99] years  [8]. The infectious period is a multifactorial parameter depending on treatment efficacy and adherence, the times from infection to detection and to the start of treatment, etc.
In contrast to the study by Stadler et al, whose data were sampled in the period between 1998 and 2008, our dataset covers also the period between 2008 and 2014, during which life expectancy of patients with HIV was further extended  [37]. This may explain why we found a longer infectious period (with compatible CIs). Lastly, our findings regarding superspreading are in accordance with those of Rassmussen et al. [25]  , and with a similar study in Latvia  [5]  based on 40 MSM sequences analysed using a likelihood approach. Although the results of the latter study may not be very accurate due to the small dataset size, they still agree with ours, giving an estimate of a superspreading transmission ratio of 9, and 5. 6% of superspreading individuals. Our estimates were quite robust to the choice of sampling probability (e.
DISCUSSION In this manuscript, we presented new methods for parameter inference and model selection in phylodynamics based on deep learning from phylogenies. Through extensive simulations, we established that these methods are at least as accurate as state-of-the-art methods and capable of predicting very large trees in minutes, which cannot be achieved today by any other existing method. We also applied our deep learning methods to the Swiss HIV dataset from MSM and obtained results consistent with current knowledge of HIV epidemiology. Using BEAST2, we obtained inaccurate results for some of the BDEI and BDSS simulations. While BEAST2 has been successfully deployed on many models and tasks, it clearly suffers from approximations in likelihood computation with these two models. However, these will likely improve in near future.
Combining both representations (e. g. , during the FFNN phase) is certainly an interesting direction for further research. Note, however, that the predictions of both approaches for the three models we studied are highly correlated (Pearson coefficient nearly equal to 1 for most parameters), which means that there is likely little room for improvement (at least with these models). A key advantage of the deep learning approaches is that they yield close to immediate estimates and apply to trees of varying size. Collection of pathogen genetic data became standard in many countries, resulting in densely sampled infected populations. Examples of such datasets include HIV in Switzerland and UK  [24, 42]  , 2013 Ebola epidemics  [6]  , several Influenza epidemics and the 2019 SARS-Cov-2 pandemic (www.
Reduction and centering of summary statistics Before training our NN and after having rescaled the trees to unit average branch length (see above), we reduce and center every summary statistic by subtracting the mean and scaling to unit variance. To achieve this, we use the standard scaler from the scikit-learn package  [50]  , which is fitted to the training set. This does not apply to CBLV representation, to avoid losing the ability to reconstruct the tree.
Parameter and model inference using neural networks We implemented deep learning methods in Python 3. 6 using Tensorflow 1. 5. 0  [51]  , Keras 2. 2. 4  [52]  and scikit-learn 0. 19. 1  [50]  libraries. For each network, several variants in terms of number of layers and neurons, activation functions, regularization, loss functions and optimizer, were tested. In the end, we decided for two specific architectures that best fit our purpose: one deep FFNN trained on SS and one CNN trained on CBLV tree representation. The FFNN for SS consists of one input layer with 99 input nodes (98 SS + the sampling probability), 4 sequential hidden layers organized in a funnel shape with 64-32-16-8 neurons and 1 output layer of size 2-4 depending on the number of parameters to be estimated. The neurons of the last hidden layer have linear activation, while others have exponential linear activation  [53].
The batch size is set to 8,000. To train the network, we split the simulated dataset into 2 groups: (1) proper training set (3,990,000 examples); (2) validation set (10,000). To prevent overfitting during training, we use: (1) the early stopping algorithm evaluating MAPE on the validation set; and (2) dropout that we set to 0. 5 in the feedforward part of both NNs  [55]  (0. 4, 0. 45, 0. 55 and 0. 6 values were tried for basic BD model without improving the accuracy). For model selection, we use the same architecture for FFNN-SS and CNN-CBLV as those for parameter inference described above. The only differences are: (1) the cost function: categorical cross entropy, and (2) the activation function used for the output layer, that is, softmax function (of size 2 for small trees, selecting between BD and BDEI model, and of size 3 for large trees, selecting between BD, BDEI and BDSS).
The picked subtree corresponds to the top part of the initial tree's clade with the same root, while the tips sampled after Dlast are pruned. The picked subtrees do not intersect with each other and cover most of the initial tree's branches: 98. 5% (BD), 97. 3% (BDEI) and 82. 4% (BDSS) of the initial tree branches on the 'huge' tree datasets (5,000 to 10,000 tips). For the BDSS model, this percentage is lower than for BD and BDEI, because of the narrower subtree size interval (200-to-500 tips versus 50-to-500 tips) corresponding to current PhyloDeep training set settings. Subtree Picker performs a postorder tree traversal (tips-to-root) and requires O(n 2 ) computing time in the worst case, where n is the number of tips in T. In practice, Subtree Picker takes on average 0. 6 (BD), 0. 8 (BDEI) and 0.
Confidence intervals For all NN-based parameter estimates, we compute 95% CI using a form of parametric bootstrap. To facilitate the deployment and speed-up the computation, we perform an approximation using a separate set of 1,000,000 simulations. For each simulation in the CI set, we store the true parameter values and the parameter values predicted with both of our methods. This large dataset of true/predicted values is used to avoid new simulations, as required with the standard parametric bootstrap. For a given simulated or empirical tree T, we obtain a set of predicted parameter values, {p}. The CI computation procedure searches among stored data those that are closest to T in terms of tree size, sampling probability and predicted values. We first subset: • 10% of simulations within the CI set, which are closest to T in terms of size (number of tips).
Parameter inference with BEAST2 To assess the accuracy of our methods, we compared it with a well-established Bayesian method, as implemented in BEAST2 (version 2. 6. 2). We used the BDSKY package  [4]  (version 1. 4. 5) to estimate the parameter values of BD simulations and the package bdmm  [12, 13]  (version 1. 0) to infer the parameter values of BDEI and BDSS. Furthermore, for the inference on BDSS simulations, instead of BEAST 2. 6. 2 we used the BEAST2 code up to the commit nr2311ba7, which includes important fixes to operators critical for our analyses. We set the Markov Chain Monte Carlo (MCMC) length to 5 million steps for the BD model, and to 10 million steps for the BDEI and BDSS models. The sampling probability was fixed during the estimation. Since the BD, BDEI and BDSS models implemented in BEAST2 do not use the same parametrizations as our methods, we needed to apply parameter conversions for setting the priors for BEAST2 inference (Supplementary Tab.
Let n be the size of T (number of tips). This tree decomposition requires one preorder tree traversal, with computing time in O(n). However, the picking strategy (2) requires another O(s) time to extract ∆ (with appropriate data structure and pre-treatments). Thus, the whole computing time (computing (2) for each node in the tree traversal) is in O(n 2 ), in the worst case. In practice, the subtrees extracted by Subtree Picker cover on average 98. 5% (BD), 97. 3% (BDEI) and 82. 4% (BDSS) of the initial tree branches on the 'huge' tree datasets (5,000 to 10,000 tips). For the BDSS model this percentage is lower than for BD and BDEI, because of the narrower subtree size interval (m=200, M=500 versus m=50, M=500) corresponding to current PhyloDeep training set settings. In terms of computing time, Subtree Picker takes on average 0.
Computation of 95% CI We compute 95% CI using parametric bootstrap. To facilitate the deployment and speed-up the computation, we perform an approximation using a separate set of 1,000,000 simulations for calculation of CI. For each simulation in the CI set, we store the true parameter values (i. e. , values with which we simulated the tree) and the parameter values predicted with both of our methods. This large dataset of true/predicted values is used to avoid new simulations, as required with the standard parametric bootstrap. For a given simulated or empirical tree T, we obtain a set of predicted parameter values, {p}. The CI computation procedure searches among stored data those that are closest to T in terms of tree size, sampling probability and predicted values. We first subset: • 10% of simulations within the CI set, which are closest to T in terms of size (number of tips), thus obtaining 100,000 CI sets of true/predicted parameter values.
Comparison of time efficiency For FFNN-SS and CNN-CBLV, we reported the average CPU time of encoding a tree (average over 10,000 trees), as reported by NextFlow workflow manager  [56]  , a pipeline software that we used. The inference time itself was negligible. For BEAST2, we reported the CPU time averaged over 100 analyses with BEAST2 as reported by NextFlow. For the analyses with BDEI and BDSS models, we reported the CPU time to process 10 million MCMC steps, and for the analyses with BD, we reported the CPU time to process 5 million MCMC steps. To account for convergence, we recalculated the average CPU time considering only those analyses for which the chain converged and an ESS of 200 was reached across all inferred parameters. The calculations were performed on a computational cluster with CentOS machines and Slurm workload manager.
ACKNOWLEDGEMENTS We would like to thank  Dr Kary Ocaña  and  Tristan Dot  for initiating experiments on machine learning and phylogenetic trees in our laboratory. We would like to thank  Quang Tru Huynh  for administrating the  GPU farm at Institut Pasteur and the INCEPTION program  ( Investissement d'Avenir  grant  ANR-16-CONV-0005 ) that financed the GPU farm. We would like to thank  Dr Christophe Zimmer  from  Institut Pasteur, Sophia Lambert  and  Dr Hélène Morlon  from  Institut de Biologie de l'Ecole Normale Supérieure IBENS  and  Dr Guy Baele  from  Katholieke Universiteit KU Leuven  for useful discussions, and  Dr Isaac Overcast  from  IBENS  and  Luc Blassel  from  Institut Pasteur  for critical reading of the manuscript. We would like to thank  Dr Tanja Stadler  and  Jérémie Sciré  for their help with BEAST2 and MLE approaches.
JV is supported by  Ecole Normale Supérieure Paris-Saclay  and by  ED Frontières de l'Innovation en Recherche et Education ,  Programme Bettencourt. VB would like to thank  Swiss National Science Foundation  for funding ( Early PostDoc  mobility grant  P2EZP3_184543 ). OG is supported by  PRAIRIE  ( ANR-19-P3IA-0001 ).
DATA AVAILABILITY The data generated in this study (simulated trees, BEAST2 logs, and the results of BEAST2 and PhyloDeep runs), as well as the HIV phylogenetic tree for Zurich epidemic (a showcase application) are provided on GitHub (github. com/evolbioinfo/phylodeep, version 0. 3) and have been deposited in the Zenodo database under accession code https://doi. org/10. 5281/zenodo. 6646668. The simulated trees were obtained with our simulator (see Code availability), the HIV tree was previously published by Rasmussen et al  [25]  and is available on their GitHub: github. com/davidrasm/PairTree (all confidential information has been removed).
CODE AVAILABILITY The PhyloDeep package (version 0. 3) is under the GPL v3. 0 license and uses Python (3. 6) and Python libraries : ete3 (version 3. 1. 2 under GNU general licence); pandas (version 1. 1. 5); numpy (version 1. 19. 5); scipy (version 1. 1. 0); scikit-learn (version 0. 19. 1); tensorflow (version 1. 13. 1); joblib (version 0. 13. 2); h5py (version 2. 10. 0); Keras (version 2. 4. 3 under Apache 2. 0 license); matplotlib (version 3. 1. 3 under PSF license). We provide (i) the source code of PhyloDeep, (ii) the code of the tree simulators used to train the deep learners and (iii) the log files obtained with BEAST2 on GitHub (github. com/evolbioinfo/phylodeep). The code has been deposited in Zenodo  [57]. We used the version 2. 6. 2 of BEAST2 for BD and BDEI inferences and BEAST2 compiled up to the commit nr2311ba7 for BDSS inferences, and version 3.
Examples To illustrate how efficient can be the introduction of Stokes constraints for volume computation, we consider the simple setting where K is a Euclidean ball included in B the unit Euclidean ball. Indeed drastic improvements on the convergence are observed. All numerical examples were processed on a standard laptop computer under the Matlab environment with the SOS parser of YALMIP  [15] , the moment parser GloptiPoly  [8]  and the semidefinite programming solver of MOSEK  [4].
Data Collection In this study, the daily number of reported cases and deaths of COVID-19 was obtained from the provincial and municipal health commissions in China, and the meteorological data were obtained from the website of the China Meteorological Administration (http://www. cma. gov. cn/). Since only one cumulative confirmed case was found in Tibet, and meteorological data were not available for Hong Kong Special Administrative Region, Macau Special Administrative Region, and Taiwan Province, data from 682 conventional meteorological stations (Supplementary Figure  1 ) in 30 provinces, autonomous regions, and municipalities directly under the central government were finally used in this study. The meteorological data included temperature ( • C), relative humidity (%), precipitation (mm), sunshine hours (h), air pressure (hPa), and wind speed (m/s) at each station.
On December 7, 2019, the first confirmed case of the COVID-19 was reported in China, while on January 10, 2020, the case was first reported on the official website of the Wuhan Municipal Health Planning Commission. Considering the completeness and continuity of the data, the starting date of data collection was January 10, 2020. The first imported case in China was reported on February 26, 2020, and to control for the impact of the imported cases on our study, we collected national outbreak data before the end of February 26, 2020. We also included holiday as a categorical variable in the generalized estimation model to control for potential confounding effects. In China, there are traditional holidays, including the Spring Festival, in addition to Saturdays and Sundays, which are included in this study period.
Parameter Estimation Model parameter values and their sources were listed in Table  1. There were 10 parameters in this model, namely br, dr, β, κ, p, 1/ω, 1/ω', f, 1/γ and 1/γ '. (1) The birth rate br and the mortality dr were derived from the statistical year books of corresponding regions. (2) The actual report data of COVID-19 were fitted by SEIAR model to obtain the transmission relative rate β. (3) In the early study, the transmissibility of asymptomatic infections was unable to determine, so this study was based on one article previously published by our team and set κ to 0. 526  (19). (4) In the early study, the proportion of asymptomatic infections was unable to determine, so this study was based on the articles previously published by the team and set P to 0. 526  (19). (5) Our study set the incubation period to 5 days  (19, 22) , and set the latent period to 5 days, that was ω = ω' = 0.
Indicator for Assessing Transmissibility The R 0 was usually used for quantitatively assessing the transmissibility  (19, (23) (24) (25) (26) (27) , but it described the natural transmissibility of the disease in an ideal state. In actual situations, the transmissibility of infectious diseases was generally measured by R eff. In this study, the formula for calculating R eff is as follows:   (22, (28) (29) (30) (31) (32). Then, we set the tolerance to. 001 to solve the differential equation. We used coefficient of determination (R 2 ) calculated by IBM SPSS 21. 0 to evaluate the curve fitting. We organized the data by Microsoft Office Excel 2010 and draw the spatial distribution map of 682 meteorological stations. We calculated the daily incidence rates, temperature ( • C), relative humidity (%), precipitation (mm), sunshine hours (h), air pressure (hPa), and wind speed (m/s) by IBM SPSS 21.
0, and the daily meteorological data adopted the arithmetic average of all stations in the province and city; GraphPad Prism 7. 0 was used for charting the above indicators. R The daily meteorological data of each city were set as independent variables. The daily R eff value, incidence rate and the number of reported cases in each province were set as dependent variables, respectively. We used the generalized estimation equation to evaluate the short-time effect of independent variables on the dependent variables by SAS 9. 4 software. The working correlation matrix was used to evaluate the correlation of each repeated measurements and provided effective variance estimation for parameter estimates  (33, 34). That is the correlation between the daily measurements of the dependent variables. Taking the R eff value as an example: we can calculate the correlation coefficient between the R eff value of the i th day and the other R eff value of the (i+1) th day by working related matrix, to evaluate the correlation mentioned above.
This working correlation matrix include four different methods: (1) Autoregressive (1): AR (1): the correlation was related to the number of measurement intervals. The correlation was weak when the measurement intervals were far apart; (2) Exchangeable (EXCH) referred to the same correlation between any two measurements; (3) Unstructured (UN) referred to the off-diagonal data in a matrix formed by repeated measurements differing with each other; and (4) Independent (IND): it meant that there was no correlation between dependent variables of repeated measurement. In matrix selection, data types and generalized estimation equations fitting criterion quasilikelihood under the independence model criterion (QIC) results were combined, in which the lower the QIC value, the better the model fit. For example, take six repeated measurements as follows:   1 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 AR(1) :         1 ρ ρ 2 ρ 1 ρ ρ 2 ρ 1 ρ 3 ρ 4 ρ 5 ρ 2 ρ 3 ρ 4 ρ ρ 2 ρ 3 ρ 3 ρ 2 ρ ρ 4 ρ 3 ρ 2 ρ 4 ρ 4 ρ 3 1 ρ ρ 2 ρ 1 ρ ρ 2 ρ 1         EXCH :           1 ρ ρ ρ 1 ρ ρ ρ 1 ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ 1 ρ ρ ρ 1 ρ ρ ρ 1           UN :   1 ρ 1 0 0 0 1 0 0 0 1   During the data analysis, it was necessary to create different databases to explore both immediate and lag effects.
First, we needed to create a database to explore immediate effects; second, the dependent variable data for 1 day should be dropped and the dependent variable foe the next day corresponds to the meteorological factor values for the first day. By analogy, a database exploring lagged effects was established. In this study, model 1 is a model of the fitness effect, model 2 is a model of the 1-day lag effect, model 3 is a model of the 2-day lag effect, model 4 is a model of the 3-day lag effect model, model 5 is a model of the 4-day lag effect model, and model 6 is a model of the 5-day lag effect model.
At the national level, the firsthand data available were various types of meteorological data from meteorological stations across the country. In contrast, daily meteorological data for each province were not directly available. In this study, data at the provincial level were obtained by averaging the daily meteorological data from the stations included in each province. The processing method was the same as other existing studies  (1, 4, 7). Also, according to the national distribution map of weather stations, the distribution of weather stations within each province was relatively uniform. Also, the average values of the data from the stations included in each province were representative. We included the categorical variable of holiday in the generalized estimation model to control the potential confounding effects of other independent variables on the three outcome variables.
FUNDING This project was partly supported by the  Bill & Melinda Gates Foundation  (No. INV-005834 ), it was also supported by  National Key Research and Development Program of China  (No. 2021YFC2301604 ), and the  Scientific Research Grant of Fujian Province of China  (No. Z0230104 ). The funders had no role in the study design, data collection and analysis, decision to publish, and preparation of the manuscript.
Acknowledgements I am indebted to  M. A. Klatt , with whom I had many discussions about hyperuniformity and Gaussian excursions. I also wish to thank  S. Torquato  for comments on the final manuscript. I am particularly grateful to  Yann Bugeaud  for insights about diophantine approximation. This study was supported by the  IdEx Université de Paris ,  ANR-18-IDEX-0001.