dataesr/scientific-paragraphs-categorization huggingface dataset:
3 Å using molecular replacement with 1RMG  17  (Fig. 1A , Extended data Fig. 2A ). PGLR crystallised as a single molecule in a P1 asymmetric unit. The crystal structure of ADPG2 (420 aa, 1-41 and 406-420 aa not modelled, PDB: 7B8B) was resolved at a resolution of 2. 0 Å using PGLR as the search model (Fig. 1A , Extended data Fig. 2B ). ADPG2 crystals belonged to the orthorhombic space group P212121 with chains A and B having a Cα root mean square deviation (rmsd) of 0. 924 Å. PGLR and ADPG2 fold in right-handed parallel β-helical structure, which is common to pectinases (Fig. 1A )  11. This β-helix is formed by three repeating parallel β-sheets -PB1, PB2 and PB3 which contain 11, 12, and 11 parallel β-strands respectively, as well as a small β-sheet, PB1a, having only 3 β-strands (Extended data Fig.
Sequences retrieval and analysis Arabidopsis thaliana PGLR (At5g14650) and ADPG2 (At2g48150) coding sequences were retrieved from publicly available genome database TAIR (https://www. arabidopsis. org/). The presence of putative signal peptide was predicted using SignalP-5. 0 Server (http://www. cbs. dtu. dk/services/SignalP/). Glycosylation sites were predicted using NetNGlyc 1. 0 Server (http://www. cbs. dtu. dk/services/NetNGlyc/). Sequence alignments were performed using MEGA and Clustal Omega multiple sequence alignment programs  36.
5, with a flow rate of 100 µl. min -1 at 25°C with different concentrations of various analytes: PGA (81325, Sigma), citrus pectin with DM 24-30% (P9311, Sigma) and pool of OGs centred on DP12DM5, DP12DM30 and DP12DM60 at 25, 50 and 100 μM. The fluorescence traces were analysed with the switchANALYSIS® software (V1. 9. 0. 33, Dynamic Biosensors). The association and dissociation rates (kon and koff), dissociation constant (KD = koff/kon) and the error values were derived from a global single exponential fit model, upon double referencing correction (blank and real-time)  38. The experiments were performed in three replicates.
Crystallization of proteins PGLR and ADPG2 were concentrated at 10 mg. ml -1. Crystallization was performed using the sitting-drop vapour-diffusion method. Crystallisation conditions were screened using a mosquito robot (TTP Labtech) and the PACT premier plate (Molecular dimensions, Sheffield, UK). PGLR and ADPG2 (100 nL) were mixed with an equal volume of precipitant (1:1). The crystals that resulted in best diffraction data were obtained with 0. 2 M sodium fluoride, 0.
Structure solution and refinement and analysis For PGLR and ADPG2 structure and function prediction I-TASSER prediction software was used  41. ColabFold was used for AtPGIP1 and AtPGIP2 models  42. The structure of PGLR was solved by molecular replacement using Phaser  43. The data were phased using rhamnogalacturonase A (PDB: 1RMG, Uniprot: Q00001) as a search model  17. Model was built using Autobuild and refined using Refine from PHENIX suite  44. The model was iteratively improved with Coot  45  and Refine. ADPG2 structure was solved by molecular replacement using PGLR as a starting model and the above-mentioned iterative procedure. The final structure for PGLR and ADPG2 have been deposited in the Protein Data Bank as entries 7B7A and 7B8B, respectively. UCSF Chimera was used for creation of graphics  46.
Poisson-Boltzmann calculations of electrostatic potentials The protonation states of each amino acid were assigned according to the pKa curves calculated at pH = 4 for PGLR and pH = 5 for ADPG2, using the PROPKA software 58. Atomic charges and radii for the protein atoms were assigned using the PDB2PQR software 59 according to the parameters of the AMBER14SB_parmbsc1 forcefield  48  , while atomic charges and radii for the sugar atoms were obtained from our previous work 60. The surface electrostatic potentials for WT PGLR and ADPG2 were then calculated solving the non-linearized form of the Poisson-Boltzmann equation through the APBS (Adaptive Poisson-Boltzmann Solver) software on a cubic grid composed of 193 grid points across the x-, y-and z-directions 61. These calculations followed a stepwise approach where the Poisson-Boltzmann equation is first solved on a coarse mesh grid with a length of 155 Å and a spacing of 0.
For each of these conditions, measurements of primary root lengths were done using ImageJ software with NeuronJ plugin. For each condition, 30-40 plants were measured. For cell lengths determination, approximately 1 mm from the tip of the root of 3 to 7 plants were photographed under a steromicroscope (ZEISS SteREO Discovery. V20). Images were assembled using MosaicJ plugin from Image J. The length of the firsts 50 epidermal cells, starting from the first cell of the columella, were measured using image J software with NeuronJ plugin. Phenotypical      Model of PGLR and ADPG2 processivity A) PGLR shows low processive dynamics where enzyme-substrate association is followed by hydrolysis and dissociation of the substrate from the enzyme. This low processivity produce OGs of variable DPs. B) ADPG2 sliding motion after forming enzyme-substrate complex allows multiple substrate hydrolysis while staying attached to the substrate showing highly processive dynamics.
Extended Data Fig 1: Purification and biochemical characterization of ADPG2 A) Western blot analysis of ADPG2 with anti-His antibodies on de-glycosylated form of ADPG2 obtained after digestion by PNGase F (1) and non-digested native sample (2). B) pH-dependence of ADPG2 activity. The activities were measured after 1 hour of incubation with PGA at 25°C at various pHs. C) Temperaturedependence of ADPG2 activity. The activities were measured after 1 hour of incubation with PGA at pH 5. 2 at various temperature. D) Determination of Km, Vmax and Kcat for ADPG2 and PGLR. Activity was assessed using various concentrations of polygalacturonic acid (PGA) at 25°C and pH 5. 2 using the 3,5-dinitrosalicylic acid method. E) Substrate specificity of PGLR and ADPG2. Activity was measured at 50°C and pH 5.
PGLR and ADPG2 sequence and structure identity with selected fungal enzymes A) Sequence alignment of PGLR and ADPG2 with characterized fungal PGs. Selected PGs; Pectobacterium carotovorum PG1 (PcPG1, PDB: 1BHE), Aspergillus niger PGI (AnPGI, PDB: 1NHC) and PGII (AnPGII PDB: 1CZF), Fusarium phyllophilum PG1 (FpPG1, 1HG8), Aspergillus aculeatus (AaPG1, PDB: 1IB4) and Chondrostereum purpureum (CpPG1, PDB: 1KCD). The aa of the active are red-boxed while and the conserved aa are blue-boxed; The alignment was performed using ClustalO. B) Superimposition of PGLR, ADPG2 and AaPG1 structures 11. A B Extended Data Fig 5. PGLR and ADPG2 N-terminal loops In PvPGIP2-FpPG1 interaction N-terminal loop with N121 play a key role in PG-PGIP interaction (FpPG1 aa in grey). In PGLR (blue) this loop is rich in methionine (pink) while ADPG2 loop (in brown) is rich in lysine (green) residues.
Acknowledgements This work was supported by a grant from the  Agence Nationale de la Recherche  ( ANR-17-CE20-0023 ) and by the  Conseil Regional Hauts-de-France  and the  FEDER (Fonds Européen de Développement Régional)  through a PhD studentship awarded for to J. S. We wish to thank  Pierre Legrand  and all the staff at Proxima1 beamline ( Synchrotron SOLEIL ,  Gif sur Yvette, France ) for X-ray diffraction and data collection. The technical assistance of  Maša Boras , a former master student is gratefully acknowledged.
1997 Extended Data Fig 4. PGLR and ADPG2 sequence and structure identity with selected fungal enzymes A) Sequence alignment of PGLR and ADPG2 with characterized fungal PGs. Selected PGs; Pectobacterium carotovorum PG1 (PcPG1, PDB: 1BHE), Aspergillus niger PGI (AnPGI, PDB: 1NHC) and PGII (AnPGII PDB: 1CZF), Fusarium phyllophilum PG1 (FpPG1, 1HG8), Aspergillus aculeatus (AaPG1, PDB: 1IB4) and Chondrostereum purpureum (CpPG1, PDB: 1KCD). The aa of the active are red-boxed while and the conserved aa are blue-boxed; The alignment was performed using ClustalO. B) Superimposition of PGLR, ADPG2 and AaPG1 structures 11. A B Extended Data Fig 5. PGLR and ADPG2 N-terminal loops In PvPGIP2-FpPG1 interaction N-terminal loop with N121 play a key role in PG-PGIP interaction (FpPG1 aa in grey). In PGLR (blue) this loop is rich in methionine (pink) while ADPG2 loop (in brown) is rich in lysine (green) residues.
Synthetic biology often involves engineering microbial strains to express highvalue proteins. Thanks to progress in rapid DNA synthesis and sequencing, deep learning has emerged as a promising approach to build sequence-toexpression models for strain optimization. But such models need large and costly training data that create steep entry barriers for many laboratories. Here we study the relation between accuracy and data efficiency in an atlas of machine learning models trained on datasets of varied size and sequence diversity. We show that deep learning can achieve good prediction accuracy with much smaller datasets than previously thought. We demonstrate that controlled sequence diversity leads to substantial gains in data efficiency and employed Explainable AI to show that convolutional neural networks can finely discriminate between input DNA sequences.
Deep learning algorithms, in particular, can uncover relations in the data on a scale that would be impossible by inspection alone, owing to their ability to capture complex dependencies with minimal prior assumptions  20. Although deep learning models can produce highly accurate phenotypic predictions  12, 21, 22  , they come at the cost of enormous data requirements for training, typically ranging from tens to hundreds of thousands of sequences; see recent examples in Supplementary Table  S1. Little attention has been paid to deep learning models in synthetic biology applications where data sizes are far below the requirements of state-of-the-art algorithms and, moreover, there is a poor grasp of what makes a good training dataset for model training. This is particularly relevant in applications where the cost of strain phenotyping is a limiting factor, as this places an upper ceiling on the number of variants that can be screened.
The challenge is then to design a limited set of variants so that the resulting data can be employed to train useful predictors of protein expression. For example, if the sequence space has a broad and shallow coverage, i. e. composed of distant and isolated variants, the resulting data may be difficult to regress because each sample contains little information that correlates with expression. Conversely, if the coverage of the screen is narrow and deep, i. e. composed of closely related sequence variants, models may be accurate but generalize poorly to other regions of the sequence space. Here, we trained a large number of sequence-to-expression models on datasets of variable size and sequence diversity. We employed a large screen of superfolder GFP-producing (sfGFP) strains in Escherichia coli  23  that was designed to ensure a balanced coverage of the sequence space.
We sampled these data so as to construct training datasets of varying size and controlled sequence diversity. We first establish the baseline performance of a range of classic, non-deep, machine learning models trained on small datasets with various phenotype distributions and a range of strategies for encoding DNA sequences. This analysis revealed that for this particular dataset, accurate models can be trained on as few as a couple of thousand variants. We moreover show that convolutional neural networks (CNN), a common deep learning architecture, additional improve predictions without the need to acquire additional data. Using tools from Explainable AI  24   23  of an sfGFP coding gene preceded by a variable 96nt sequence. The variable region was designed on the basis of eight sequence properties previously described as impacting translational efficiency: nucleotide content (%AT), patterns of codon usage (codon adaptation index, CAI, codon ramp bottleneck position, Btl P , and strength, Btl S ), hydrophobicity of the polypeptide (mean hydrophobicity index, MHI) and stability of three secondary structures tiled along the transcript (MFE-1, MFE-2, and MFE-3).
A total of 56 seed sequences were designed to provide a broad coverage of the sequence space, and then subjected to controlled randomization to create 56 mutational series of ~4000 sequences each. After removal of variants with missing measurements, the dataset contains 228,000 sequences. Violin plots show the distribution of the average value of the eight properties across the 56 mutational series; the biophysical properties were normalized to the range [0, 1] and then averaged across series. For all violins, the white circle indicates the median, box edges are at the 25th and 75th percentiles, and whiskers show the 95% confidence interval. B Two dimensional UMAP  27  visualization of overlapping 4-mers computed for all 228,000 sequences; this representation reveals 56 clusters, with each cluster corresponding to a mutational series that locally explores the sequence space around its seed; we have highlighted five series with markedly distinct phenotype distributions (labels denote the series number).
We finally demonstrate that in limited data scenarios, controlled sequence diversity can improve data efficiency and predictive performance across larger regions of the sequence space. We validated this conclusion in a recent dataset of ~3000 promoter sequences in Saccharomyces cerevisiae  25. Our results provide a systematic characterization    1 ), plus an additional mixed encoding consisting of binary one-hot augmented with the biophysical properties of Fig. 1A ; in all cases, one-hot encoded matrices were flattened as vectors of dimension 384. We considered four non-deep models trained on an increasing number of sequences from five mutational series with different phenotype distributions (Fig. 1B ). B Impact of DNA encoding and data size on model accuracy. Overall we found that random forest regressors and binary one-hot encodings provide the best accuracy; we validated this optimal choice across the whole sequence space by training more than 5000 models in all mutational series (Supplementary Fig.
Size and diversity of training data We sought to compare various machine learning models using datasets of different size and diversity. To this end, we employed the genotype-phenotype association data from Cambray et al. 23. This dataset comprises fluorescence measurements for an sfGFP-coding sequence in Escherichia coli, fused with more than 240,000 upstream 96nt regions that were designed to perturb translational efficiency and the resulting expression level. The library of upstream sequences was randomized with a rigorous design-of-experiments approach so as to achieve a balanced coverage of the sequence space and a controlled diversity of variants. Specifically, 96nt sequences were designed from 56 seeds with maximal pairwise Hamming distances. Each seed was subject to controlled randomization using the D-Tailor framework  26  , so as to produce mutational series with controlled coverage of eight biophysical properties at various levels of granularity: nucleotide sequence, codon sequence, amino acid sequence, and secondary mRNA structure (Fig.
1A ). The complete dataset contains 56 mutational series that provide wide coverage of the sequence space, while each series contains ~4000 sequences for local exploration in the vicinity of the seed. The dataset is particularly well suited for our study because it provides access to controllable sequence diversity, as opposed to screens that consider fully random sequences with limited coverage or single mutational series that lack diversity. To further characterize sequence diversity across the library of 56 mutational series, we visualized the distribution of overlapping 4-mers using the Uniform Manifold Approximation and Projection (UMAP) algorithm for dimensionality reduction  27. The resulting twodimensional distribution of sequences (Fig. 1B ) shows a clear structure of 56 clusters, each corresponding to a mutational series.
Moreover, the sfGFP fluorescence data (Fig. 1C , Supplementary Fig. S2 ) display marked qualitative differences across mutational series, including near-Gaussian distributions, left-and right-skewed distributions, as well as bimodal and uniform distributions. This indicates that the dataset is diverse in both genotype and phenotype space, and thus well suited for benchmarking machine learning models because it allows probing the impact of both genetic and phenotypic variation on model accuracy.
Impact of DNA encoding and size of training data To understand the baseline performance of classic (non-deep) machine learning models, we trained various regressors on datasets of varying sizes and with different DNA encoding strategies (Fig. 2A ). Sequence encoding is needed to featurize nucleotide strings into numerical vectors that can be processed by downstream machine learning models. We considered DNA encodings on three resolutions (Table  1 , Fig. 2A ): global biophysical properties (Fig. 1A ), DNA subsequences (overlapping k-mers), and single nucleotide resolution (one-hot encoding). We trained models on five mutational series chosen because their phenotype distributions are representative of those in the whole dataset (Fig. 1B , Supplementary Fig. S2 ), and with an increasing number of sequences for training (from ~200 to ~3000 sequences per series).
Given the variation in phenotype distributions, we stratified training samples to ensure that their distribution is representative of the full series. We considered four nondeep models: ridge regressor  28  (a type of penalized linear model), multilayer perceptrons  29  (MLP, a shallow neural network with three hidden layers), support vector regressor  30  (SVR, based on linear separation of the feature space with a radial basis function kernel), and random forest regressor  31  (RF, based on axis-aligned splits of the feature space). We chose this array of models because they markedly differ in their principle of operation and underlying assumptions on the shape of the feature space. We tuned model hyperparameters using grid search and 10-fold cross-validation on datasets assembled from aggregated fractions of all mutational series; this allowed us to determine a fixed set of hyperparameters for each of the four models with good performance across the whole dataset (see Methods and Supplementary Fig.
S3 ). In all cases, we assessed predictive accuracy using the coefficient of determination, R 2 defined in Eq. (  1 ), between measured and predicted sfGFP fluorescence computed on a set of ~400 test sequences (Supplementary Fig. S3 ) that were held-out from model training and validation. In line with expectation, the results in Fig. 2B  show that models trained on small datasets are generally poor irrespective of the encoding or regression method. Linear models (ridge) display exceptionally poor accuracy and are insensitive to the size of training set. In contrast, a shallow neural network (multilayer perceptron) achieved substantial gains in accuracy with larger training sets, possibly owing to its ability to capture nonlinear relationships. Our results show that mildly accurate models (R 2 ≥ 50%) can be obtained from training sets with ~1000 sequences using random forests and support vector regressors (Fig.
2C ). We found random forest regressors to be the most accurate among the considered models, consistently achieving R 2 ≥ 50% for datasets with more than 1000 samples and showing a stable performance when trained on other mutational series (Supplementary Fig. S5 ). To produce robust performance metrics, the R 2 scores in Fig. 2B  are averages across five training repeats with resampled training and test sets (Monte Carlo cross-validation). We also observed a sizeable impact of DNA encodings on prediction accuracy. Subsequence-resolution encodings achieve varying accuracy that is highly dependent on the specific mutational series and chosen model (Fig. 2B , Supplementary Fig. S5 ). Overall we found a strong preference for base-resolution encodings, with binary one-hot representations achieving the best accuracy.
Deep learning improves accuracy with the same amount of data Prior work has shown that deep learning can produce much more accurate predictions than non-deep models  16, 19. Deep learning models, however, typically require extremely large datasets for training; some of the most powerful deep learning phenotypic predictors, such as DeepBIND  12  , Optimus 5-prime  9  , ExpressionGAN  34  , and Enformer  14  were trained with tens to hundreds of thousands of variants. In the case of sequence-to-expression models, recent literature shows a trend towards more complex and data-intensive models (see Supplementary Table  S1 ); the most recent sequence-to-expression model employed ~20,000,000 promoter sequences to predict protein expression in Saccharomyces cerevisiae  25. But it is unclear if the accuracy of such predictors results from the model architecture or simply from the sheer size of the training data.
In addition to the components shown in Fig. 3A , we also included a dropout layer to prevent overfitting and max pooling to reduce the number of trainable parameters. Similar as with the non-deep models in Fig. 2B , hyperparameter optimization was performed by splitting the data into separate sets for training and cross-validation (details in Methods and Supplementary Fig. S3 ). This allowed us to find a single CNN architecture with good performance across the individual 56 mutational series and the whole dataset. When trained on up to 75% of the full dataset (~160,000 sequences), our CNN model produced excellent predictions in test sets covering broad regions of the sequence space (average R 2 = 0. 82 across five cross-validation runs, Fig. 3B  and Supplementary Fig. S6 ). This suggests data size alone may be sufficient for training accurate regressors.
However, since data of such scale are rarely available in synthetic biology applications, we sought to determine the capacity of CNNs to produce accurate predictions from much smaller datasets than previously considered. To this end, we trained CNNs with the same architecture in Fig. 3A  on each mutational series, using ~1000-3000 sequences in each case; details on CNN training can be found in the Methods, Supplementary Fig. S7 , and Supplementary Tables S4-S5. We benchmarked the accuracy of the CNNs against nondeep models trained on the same 56 mutational series. As benchmarks we chose two non-deep models: a shallow perceptron (MLP) because it is also a type of neural network, and a random forest regressor because it showed the best performance so far (Fig. 2B ). We found that CNNs are consistently more accurate than non-deep models, regardless of the size of the training data (Fig.
The idea is to design a sequence space for training that can enlarge the highconfidence regions of the predictors with a modest number of variants. To this end, we performed a computational experiment designed to test the impact of sequence diversity on the ability of CNNs to produce accurate predictions across different mutational series. We trained CNNs on datasets of constant size but increasing sequence diversity (Fig. 5A , Supplementary Fig. S12 ). We considered an initial model trained on 5800 sequences sampled from the aggregate of two series chosen at random, e. g. 2900 sequences from series 13 and 23, respectively (Fig. 5A   S12 ). The decreasing number of training sequences per series reflects better data efficiency, thanks to an increasingly diverse set of training sequences. To quantify sequence diversity, we counted the occurrence of unique overlapping 5-mers across all sequences of each training set, and defined diversity as 1= P 100 i = 1 c i , where c i is the count of the i-th most frequent 5-mers.