dataesr/scientific-paragraphs-categorization huggingface dataset:
It is this combination that we have chosen to generate unconstrained diversity (cf. section "Case 1: unconstrained molecular generation"). On the other hand, when optimizing the QED property, we studied CheckMol, IFGs and shingles separately to observe the impact of the choice of descriptors (cf. section "Case 2: goal-directed molecular generation"). One would expect this choice to have an impact on the balance between diversity and drug-likeness. This study is also not exhaustive and focuses on three structural descriptors that we think to be relevant for the diversity of chemical functional groups. It illustrates how the choice of descriptors is problem dependent. The method we propose is not dependent of a particular generator. It allows to evaluate the diversity contribution of a compound in a dataset at a given time of any generation process.
As stated before, the size of the dataset is constant and operations are done in batches. Then, the flexibility (10) � ′ s (m r , m a , X) = ω ifg � ′ s ifg (m r , m a , X) + ω scaf. � ′ s scaf. (m r , m a , X) + • • • of this generator allows to quickly adapt it while its interpretability allows to visualize the impact of the diversity on the chemical space exploration (see section "Case 2 : goal-directed molecular generation"). Finally, due to its very unconstrained nature, it is very efficient in optimization and generates molecules that are sometimes unrealistic and not very stable. In our case, it is a useful property to increase the chemical diversity of reference datasets as we will demonstrate in section Case 1: unconstrained molecular generation.
Case 1: unconstrained molecular generation In this large-scale experiment, we aim to maximize chemical diversity using our methodology presented above. The newly generated compounds respect all the constraints of QM9 and PC9, i. e. maximum nine heavy atoms among C, N, O and F. Since our objective is an optimization of diversity, we have named the resulting dataset OD9. OD9_0 will refer to QM9 ∪ PC9 , and OD9_1 the newly generated compounds. More than one million new compounds have been generated and all OD9 molecules have been calculated in DFT with the same calculation parameters and a strict quality control. Due to high failure rate of the DFT and the quality control with the new compounds, the whole chain was executed 6 times with slightly different parameters. To guarantee the uniqueness of the generated compounds, already known molecules were removed after each of the 6 iterations.
For the first two runs, the descriptors used were IFGs and scaffolds, while in the following runs we used IFGs and generic scaffolds. For the first four runs, the starting dataset was QM9 ∪ PC9 , while for the two last runs we only started from methane. Since the differences in parameters did not result in significant changes, we made the union of the results of the 6 executions in OD9_1 to analyze them together. The entire workflow is presented below. Step 0 (generation) The first step consists in generating with EvoMol about 210k molecules, i. e. approximately the size of the union of PC9 and QM9 without duplicates. EvoMol was used with primary actions only (append, substitute and remove atom, change bond) with the sole objective of maximizing diversity of equal weight between IFGs and Scaffolds ( ω ifg = ω scaf.
In the rest of the article, when we talk about stability, it will refer to this stability to the DFT calculation. A final uniqueness check is performed without taking into account the stereochemical information. Berkeley Open Infrastructure for Network Computing (BOINC) We have set up an instance of BOINC, named QuChemPedIA@home, to allow us to distribute our numerous calculations on our machines and on the computers of the volunteers. Inputs generation for NWchem was automated and a native wrapper for Linux operating systems was developed. The contribution of other operating systems is allowed by the use of a virtual machine and the official wrapper with BOINC and Virtualbox. So much heterogeneity implies many calculation errors, but the system is designed for this and failed tasks will be submitted a number of times on different hosts before being declared in error.
Case 1: unconstrained molecular generation Diversity analysis on scaffolds and IFGs As a starting dataset we have gathered all the SMILES of the union of QM9 and PC9. All these molecules were then recalculated twice with the BOINC project. Following the workflow presented in section Case 1: unconstrained molecular generation, some compounds are discarded if they are radicals or duplicated, if they are too far from a global minimum, if their 3D structure does not allow to find a canonical SMILES or if their final SMILES is different from the original SMILES. At the end of this process, 122,227 QM9 compounds and 77,790 PC9 compounds were retained. The cleaning of radicals and duplicates has had a significant impact on PC9. The union of these two sets is noted OD9_0 to indicate that this is our starting point.
After step 3, this union contains only 184,158 molecules after elimination of the duplicates between QM9 and PC9. The detail of the evolution of the number of compounds at each step for all datasets is given in Table  1  of Additional file 1. The chemical diversity in terms of generic scaffolds, scaffolds and IFG is reported in Table  1. Comparing QM9 and PC9, we can confirm our previous analysis  [12]. There is less topological diversity (generic scaffolds), alkyl diversity (acyclic scaffolds) and heteroatoms diversity (distinct IFG) in QM9 than in PC9. There is however more diversity in QM9 when we consider the unsaturations (see the distinct scaffolds column) since the double bonds were placed with a combinatorial approach when in PC9, which is a subset of PubChemQC  [16] , there is predominantly aromatic compounds.
Analysis on non optimized scores or descriptors We can also compare the chemical diversity between OD9_0 and OD9_1 through properties that we did not directly optimize. We have selected two indices of synthesizability and some electronic properties to evaluate as distributions the diversity of those two datasets. The CLscore and SAscore indices have been designed to estimate the synthesizability or complexity of a molecule mainly by comparing it with the most common fragments of respectively the ChemBL and the PubChem  [54, 57]. A high CLscore is expected for a molecule mainly composed of common fragments in ChemBL. A high SAscore is in principle associated with a molecule that should be difficult to synthesize. Figure  3  represents the distribution of these two scores for OD9_0 and OD9_1 (steps 1 and 3).
Concerning the energies of the frontier molecular orbitals, we can observe that the HOMO level is finally always centered around a value of about -6 eV. In contrast to the very low energy of the LUMO (and therefore the small gap) for newly generated molecules. The new chemical diversity seems to correspond to very acceptor and unstable molecules. In the chemistry of molecular materials, many problems are related to electronic properties. A training dataset with much wider distributions of electronic properties should lead to more robust and relevant models for molecular materials. Since the number of IFGs has skyrocketed during the generation, we decided to look at the shingles as descriptors. The diversity in terms of molecular shingles is reported in Table  2. The number of first neighbors configurations (radius of 1, denoted r1) is much lower in QM9 than in PC9.
In fact, almost all QM9 shingles r1 exist in PC9 as revealed by the union of QM9 and PC9. However, when the radius of shingles increases, taking into account neighbors of neighbors (radius of 2) and so on (radius of 3), more chemical diversity in QM9 appears. QM9 is composed of less basic bricks than in PC9, but they are used in a combinatorial way, leading to what could be called a combinatorial diversity. EvoMol was able to propose 1007 shingles of radius 1 absent from QM9 and PC9 and an absurd amount of combinatorial diversity with 642,265 new shingles r2 and 4,568,964 new shingles r3. 80% of those are present only once (unique) in all the dataset. Just like for the IFGs, the drastic evolution of those numbers after DFT and after filtering, shows that exotic combinations of chemical environments are subject to common geometrical and electronic re-optimizations.
They often cannot be written as Lewis structures, i. e. SMILES with a clear alternations between single and double bonds. The widespread SMILES representation can be an hindrance for de novo generation of complex electronic structures. Nevertheless, we managed to generate a lot of new molecular sub-graphs and combinations, stable in DFT. So much that it appears that r2 and r3 shingles are too specific to be used as descriptors to assess the molecular diversity. To observe redundancy in the data would then require huge data sets. In Fig. 5 , the top three r1 and r2 shingles are represented along with the percentage of molecules presenting this descriptor. The top three are different in both datasets. Again we can observe that the percentage in OD9_1 (step 3) are lower, indicating a more evenly distributed distribution.
Hoping to be able to better understand the chemical differences between OD9_0 and OD9_1, we decided to analyze their bond types and chemical functions according to CheckMol. Looking at a heatmap of the bonds in Fig. 6 , we can see that CH and CC bonds represent a large Fig. 2  Cumulative plots of the generic scaffolds, the scaffolds and the IFGs for the OD9_0 (QM9UPC9 in black straight lines) and for the newly generated OD9_1 step 3 compounds (in red dashed lines) Fig. 3  Distributions of the CLscore and SAscore. The black straight lines represent the OD9_0 dataset (QM9UPC9). The new generated compounds OD9_1 are represented by the blue dotted lines for the step 1 or the red dashed lines for the step 3 of the workflow majority of all the chemical bonds in OD9_0 (42. 46% + 24. 58%). Also, 15 of those 25 bonds are represented less than 1% including 6 bonds that are represented less than 0.
05%. In the 1M SMILES generated with EvoMol, OD9_1 at step 1, there is in proportion far less CH and CC and much more CN and NH bonds. Many rare bonds concern two heteroatoms. We can see that using the IFG in our objective function was a great incentive for the generation of new examples of such bonds. There is still 12 bonds under 1% but no bond is under 0. 05%. The DFT  calculations and the step 3 of curation has a major impact on some bonds. Almost all N=O bonds and most N=N or aromatic CN bonds are discarded with our workflow. When we dig deeper into this study using functional groups (FGs) as detected by the Checkmol program, we found almost 100 distinct FGs. The table of all occurrences in all the datasets are given in the Additional file 2. We have selected FGs whose proportions show a strong evolution between the different datasets and represented their proportions in a heatmap (see Fig.
7 ). In all datasets, heterocyclic compounds are heavily represented and the use of the IFGs in our diversity objective has further amplified their proportion. We can then see that we have generated many hydrazine derivatives, hydroxylamines, aminals and imines. This is consistent with the significant increase in NN, CN, NH, C=N and NO bonds (see Fig. 6 ). Some rare functions have been generated quite often such as hydroperoxide, guanidine, peroxide, hydrazone, azo, oxime, diaryl ether, imidoyl halide, diarylamine. This partly justifies the increase in the proportion of oo (aromatic peroxyde), OO, C=N, CN, NN, N=N, NO, CF bonds. It can also be noted that three new FGs have appeared in OD9_1, nitroso, nitrite and ketene impacting the N=O, C=N and C=C bond ratios. On the contrary, EvoMol, cannot propose nitro compounds because in its actual stage, it does not handle formal charges and zwitterions.
The scale being logarithmic, the improvement is of an order  of magnitude, with no notable difference between the descriptors. The generator finds a compromise between intensification and exploration, so that it does not focus too quickly exclusively on the part of the chemical space that seems to be the most promising. On top of that, as dataset diversity increases over time, the solutions proposed are more diverse. In the end, when all the proposed solutions have good scores, the generator can continue to improve the dataset by offering more diversified solutions. On the other hand, when too much weight is put on diversity (red curves), the generator diverges to the point where even the best dataset solution declines over time. It is quite simple to understand that too much diversity will be counterproductive.
In our experiments, the dataset is small, 1000 molecules, and the number of descriptors can be quite large as we have just seen. After some time, it is not enough to propose molecules with new descriptors, they must also be concentrated. If the weight is too important, it is the density of the descriptors that prevails over the initial objective. The exploration trees of Fig. 9  represent the relation of inheritance between successive individuals (edges) and the score of each individual (color). We present trees for different type of descriptors and different weights. The reference experiment, i. e. without entropy, will be used as a baseline for comparison. What is remarkable for this exploration tree is the low number of nodes and the very large number of direct descendants per node, materialized by the black triangles, which are juxtapositions of edges.
Conclusions In this paper we present a fast and generic method to evaluate diversity according to descriptors that can be chosen or defined according to an application. Our innovative approach allows to individually estimate the impact of a solution to the diversity of a set, which allows for effective incremental evaluation. This metric can be used alone or in combination with other objectives. We have realized and made available an implementation in EvoMol, our molecular generator. This method can of course be easily incorporated into other generators based on population-based algorithms. It can be easily used to select a diverse subset of solutions from a large dataset, for active learning for instance. It is even possible to adapt it to force the diversity of deep learning molecular generators, as did Nigam et al.
[36]  to help his GAN model not fall into a mode collapse issue. In terms of applications, we have shown two interesting results of adding a criterion of chemical diversity in de novo generation. Firstly, a diversity criterion based on scaffolds and chemical functions (IFGs for example) with a genetic algorithm has allowed to efficiently enhance the chemical diversity of reference datasets. Underrepresented chemical functions have been proposed. We have been able to study in detail the chemical diversity generated thanks to chemical functions, chemical bonds, shingles but also thanks to distributions of structural or electronic properties. For example, we observed an unprecedented exploration of the chemical space of acceptor molecules (low LUMO energy). The OD9 dataset with 435k molecules, thus represents an important improvement if the goal is to train a ML method with good performances in generalization.
Acknowledgements The authors would like to thank  Béatrice Duval  for the enriching discussions and  Jean-Mathieu Chantrein  for his technical help since the calculation resources were provided by the  LERIA  and  MOLTECH-Anjou laboratories. TC and BDM would especially thanks the  BOINC community , since the more than 1. 2M compounds corresponding to the OD9 dataset have been recalculated at the  B3LYP/3-21G  level of theory using our BOINC collaborative computing project, called QuChemPedIA@home (https://quchempedia. univ-angers. fr/athome/) and  Théo Dézé  who set up the associated BOINC platform.
Availability of data and materials The EvoMol code source is available in the following github repository:https:// github. com/ jules-leguy/ EvoMol. The OD9 dataset is available as a figshare collection. It encompasses the 435 032 NWChem logfiles calculated thanks to the QuChemPedIA@home BOINC project. https:// doi. org/ 10. 6084/ m9. figsh are. c. 51805 13. v1.
Dataset Size Distinct shingles r1 Distinct shingles r2 Unique shingles r2 Distinct shingles r3 Unique shingles r3 QM9 step 3 122,227 229 28,053 7162 376,852 273,423 PC9 step 3 77,790 1295 39,725 18,718 223,127 158,226 OD9_0 step 3 184,158 1297 57,741 22,130 544,460 392,637 OD9_1 step 1 1,023,624 1007 642,265 282,311 4,568,964 3,675,203 OD9_1 step 2 854,059 3585 979,596 548,870 4,255,262 3,513,467 OD9_1 step 3 250,874 762 213,034 103,858 1,156,813 929,228 OD9 step 1 1,276,171 2447 691,715 301,669 5,156,545 4,064,788 OD9 step 2 1,088,773 3714 1,013,639 557,832 4,798,140 3,870,539 OD9 step 3 435,032 1563 250,163 116,483 1,665,725 1,293,995.
UCYN-C is the third marine unicellular group identified thus far by nifH sequence, and is most closely related to the free-living unicellular diazotroph Cyanothece sp. ATCC 51142  15  , and less studied. Finally, noncyanobacterial diazotrophs (NCDs), including Archaeal and Bacterial lineages, co-occur with the cyanobacterial diazotrophs in the surface ocean and additionally below the photic layer. The distribution and in situ activity of NCDs are poorly constrained and difficult to estimate  [16] [17] [18]. The first global ocean database of diazotrophs was compiled for the MARine Ecosystem DATa (MAREDAT) project  19. It includes cell counts for diazotrophs that can be identified by microscopy (Trichodesmium, Richelia, and Calothrix), as well as nifH qPCR datasets which additionally cover UCYN-A, UCYN-B, and UCYN-C (Supplementary Fig.
S1a ). A recent update of the MAREDAT dataset resulted in more than doubling of the nifH observations  20. However, both MAREDAT and the updated version still have low coverage in vast regions of the global ocean (Supplementary Fig. S1a ). Several of these poorly sampled areas were sampled during the Tara Oceans circumnavigation (2009-2013)  21  (Supplementary Fig. S1b ). Tara Oceans collected discrete size fractions of plankton using a serial filtration system  21  ; some samples were used to generate parallel molecular and imaging datasets. The Tara Oceans gene catalog from samples enriched in free-living prokaryotes is based on the assembly of metagenomes and is highly comprehensive  22, 23. However, the larger plankton size fractions (>0. 8 μm) enriched in eukaryotes are genomically much more complex, and thus current Tara Oceans gene catalogs from these fractions are based only on poly-A-tailed eukaryotic RNA  24, 25.
Hence, the prokaryotes from these larger size fractions have been unstudied and are limited to specific taxa based on these poly-A assembled sequences  [26] [27] [28]  , and thus the signal is difficult to interpret quantitatively  29. The Tara Oceans imaging dataset  30  is also underutilized, especially due to the lack of well-established workflows. Overall, the cyanobacterial diazotrophs, especially those with diverse lifestyles (colonial, symbiotic, chain formers), have been poorly characterized (with the exception of UCYN-A  11, 16, 22, 26, 28, 31  ). Here, we report the diversity, abundance, and distribution of symbiotic, colony-forming, and particle-associated diazotrophs in the global ocean by mining >1300 metagenomes  22, 24, 25  and >2,000,000 high-throughput images from Tara Oceans.
Results and discussion Diazotroph abundance and biovolume based on imaging methods. We first used machine learning tools (see "Methods"  30  ) to search for diazotrophs in the Tara Oceans high-throughput confocal imaging dataset derived from 61 samples of the 20-180 μm plankton size fraction collected at 48 different sampling locations (Supplementary Fig. S2 ). More than 400 images of DDAs and almost 600 images of Trichodesmium-free filaments were predicted (Figs. 1 and 2; https://www. ebi. ac. uk/biostudies/ studies/S-BSST529); all images were from the tropical and subtropical regions and consistent with the molecular analyses (see later). The same image recognition searches were performed on the confocal microscopy images from the 5-20 μm size fraction  30  , which covers 75 samples from 51 stations.
These searches detected only 8 images of short Trichodesmium filaments and 4 single Hemiaulus-Richelia symbiotic cells. Thus our observations are in agreement with earlier studies showing that most filaments are collected by their length rather than their diameter during sampling  34. Several new locations to our knowledge not previously reported in diazotroph databases were identified (see below; Fig. 3 , Supplementary Figs. S3  and S4 ). In addition, we detected colonies of Crocosphaera-like cells and others living in the more rarely reported diatom Climacodium  [12] [13] [14]. Notably, a few Crocosphaera cells (1-2 cells) were detected by their strong phycoerythrin signal emitted in the alexa 546 channel while both the Crocosphaera cells and chloroplast of each Climacodium host emitted autofluorescence signals in the red channel (638 nm), demonstrating the combination of signals is a key attribute in detection of inconspicuous plankton by the image recognition model (Fig.
In the case of Trichodesmium colonies, it was determined using in situ images from the UVP5. Boxplots depict the 25-75% quantile range of the dataset without zeros (the corresponding biologically independent seawater samples are indicated in the plot), with the center line depicting the median (50% quantile); whiskers encompass data points within 1. 5× the interquartile range. Source data are provided as a Source data file. Fig. S6 ), and confirms the tendencies observed in culture experiments  42. Single-cell free-living NCDs were estimated by combining flow cytometry estimates of free-living bacterial densities with diazotroph relative abundances derived from metagenomic sequencing of the 0. 22-1. 6/3 μm plankton size fractions (see "Methods"). We detected concentrations up to ~2. 8 × 10 6 cells L -1 , with the highest values in the Pacific Ocean (Fig.
3a ). Our estimates agree with recent reports based on the reconstruction of metagenomeassembled genomes  16. Combining the imaging datasets from Tara Oceans (flow cytometry, confocal microscopy, and UVP5) enabled the conversion of abundances to estimates of biovolumes for the NCDs, Trichodesmium, and DDAs (Fig. 3c ). Single-cell free-living NCDs are by far the most abundant diazotrophs in the surface ocean, however, Trichodesmium dominates in terms of biovolume by two orders of magnitude over NCDs (Fig. 3c ). Cell density and biovolume of NCDs have not been previously reported at a global scale, thus the work presented here expands our understanding on the relative contributions for these recently recognized important diazotrophs. Diazotroph diversity and abundance using metagenomes from size-fractionated plankton samples.
To gain further insights into the abundance and distribution of diazotrophs across the whole plankton size spectrum, we compared the imaging data with metagenomic reads from the 5 main size fractions mapped against a comprehensive catalog of 29,609 unique nifH sequences (see "Methods"). The nifH catalog represents most of the genetic diversity reported for diazotroph isolates and environmental clone libraries (although it has some redundancy; see "Methods"), with 30% of the sequences derived from marine environments and the rest from terrestrial and freshwater habitats. Around 2. 5% of these nifH sequences (762 out of 29,609) mapped with at least 80% similarity to the 1164 metagenomes, retrieving a total of 96,215 mapped reads. Of the 762 sequences, 167 retrieved only one read. Mapped nifH reads were detected in slightly more than half of the samples (66% or 771 of 1164 metagenomes), which highlights the broad distribution of diazotrophs in the Tara Oceans datasets (blue circles in Fig.