dataesr/scientific-paragraphs-categorization huggingface dataset:
Validation of the discriminatory power of host-segregating markers To validate the capability of the selected SNPs to discriminate isolates from different populations, STRUCTURE tests were run again using the marker-determination dataset and different sets of markers: SNPs contained in the same CDS, all SNPs determined from OR12, HC2-48 and ZV1224 alignments and all SNPs from all alignments. One hundred tests were then performed using each set of SNPs and 60 random isolates per test for self-attribution (POPFLAG = 0) ("no admixture model", K = 3, 10,000 iterations and a burn-in period of 10,000 iterations). Additionally, source attribution of 299 validation isolates of known source reservoirs, which were not used for SNP determination, was performed. Specifically, each SNP was obtained using samtools mpileup option.
STRUCTURE was run 10 times using marker-determination isolates as training dataset (n=450) and validation dataset as unknown source isolates (POPFLAG = 0). STRUCTURE model parameters remained unchanged. Each validation isolate was attributed to its source based on the average of attribution rate of all 10 tests. An isolate was considered correctly source re-attributed with a STRUCTURE score greater than 70%. In each case the same method was performed simultaneously with MLST alleles to compare the discriminating strength of both type of markers (SNP or allele). The minimum spanning tree was generated using GrapeTree from the sequence types of all 896 C. coli isolates, based on 7 MLST genes (aspA, glnA, gltA, glyA, pgm, tkt and uncA) extracted using PubMLST platform. Orange color represents isolates isolated from chickens, green color from cattle, magenta color from pigs and red color are for clinical isolates.
DATA AVAILABILITY All 900 C. coli genomes are available using IDs listed in Dataset S1, S2 and S3: BioSample and PubMLST IDs for NCBI and PubMLST databases respectively. Personnal VCF filter Python script available on GitHub: QuentinJehanne. (2020, April 8). QuentinJehanne/ccoli_2020: v1 of a personal VCF filter (Version v1. 0. 0). Zenodo. http://doi. org/10. 5281/zenodo. 3744758.
XL-MS uses chemical cross-linking agents (crosslinkers) together with mass spectrometry and database searching in order to identify pairs of residues covalently bonded by the cross-linker. If we know that two residues of a given complex are cross-linked, one can deduce that the distance between them cannot be longer than the length of the crosslinker. This information can be used in two ways: as a criterion if a particular conformation is compatible with the chemical cross-link, or as a restraint to guide molecular mod-eling. In this work, we mostly focus on the first aspect, but we will discuss how the method can be efficiently used directly in molecular modeling. Several strategies have been proposed for the structural interpretation of cross-links. The simplest is to use the distance between the C α carbons of the cross-linked residues as a Euclidean metric  (Robinson et al.
The conformation of minimum energy is selected as the representative cross-linker structure between the given residues, and its energy is used as a classifier. The method is described in more detail in the METHOD DETAILS section. To evaluate NRGXL and compare it to the other available methods, we used the entries from the cross-link database (XLdb)  (Kahraman et al. , 2013)  (see XLdb), a database of pairs of chemical cross-links and the independently solved corresponding X-ray crystal structure. For each entry in the XLdb, we determined the set of compatible lysine pairs as all pairs with a Euclidean distance shorter than the length of the cross-linker plus the two side chains (28. 42 Å when using Bissulfosuccinimidyl suberate (BS3) as the cross-linker and two lysines as the cross-linked residues), see Binary Classification study.
In the static model, 24 systems had at least one compatible lysine pair satisfying the above criterion, with a total of 236 compatible lysine pairs with an experimentally detected cross-link associated (EyXL) and 7613 compatible lysine pairs without (EnXL). In contrast, in the flexible model, we could include 27 systems in our analysis, accounting for 273 compatible lysine pairs with an associated experimentally detected cross-link, and 13089 compatible lysine pairs without. The nine extra conformations can easily explain the differ- 2/18 ence in number of systems and number of compatible lysine pairs between the static and flexible models per database entry in the latter. The sampling induces differences also in the distances in lysine pairs. If a particular distance is shorter in at least one of the nine additional conformations than in the static conformation, the corresponding residue pair is included in the analysis.
For a Binary Classification, one has to set a threshold for a quantitative feature (here, a distance or an energy value), which is then used to classify the elements. In the present case, we classified compatible lysine pairs into two sets according to an upper threshold: the set predicted to be in cross-linkable conformations (CyXL) when the considered value is lower than the threshold and, the set predicted not to be in cross-linkable conformations (CnXL) for values higher than the threshold. As one would expect, the distributions of distances for Eu-cDist for the two cases EyXL and EnXL (compatible distance with and without cross-link in the database, respectively) are different, with the mode of the distribution shifted towards larger values for EnXL (Figure  3 ). We observed the same behaviour for SASD and energies (data not shown).
Interestingly, Degiacomi et al. (Degiacomi et al. , 2017 ) also found that the inclusion of different conformations like in the flexible model can be important when explaining the experimental data. Moreover, we also computed the SASD using the XWalk software  (Kahraman et al. , 2011) , one of the most established programs to compute cross-link distances based on an SASD evaluation. We found an AUC of 0. 75 ± 0. 02 for the static model, which is sensibly better than our SASD implementation but comparable with the AUC obtained using DynamXL (see Figure  5 ). To assess the effect of the size of the protein on the predictive ability of the classifiers, we split the database into three groups based on the size of the protein (0-2000, 2000-4000 and 4000-6000 residues). We saw that the gain of the energy classifier relative to the EucDist classifier and the SASD is even higher for larger systems (see Figure  S1 ).
Discussion NRGXL, the approach to assess the propensity of a residue pair to be cross-linked departs from the traditional, geometrical cross-link interpretation to focus on the physico-chemical interactions between the cross-linker and the protein. In terms of XL-MS data interpretation, the two main improvements of our sampling method are: (1) a more realistic structural model of the cross-linker (2) and an efficient and accurate model to treat the flexibility of the XL, the side chains, and the whole complex. The Binary Classification study, where we analyzed the performance of different XL statistics as predictors of crosslinkability, clearly showed that spatial statistics used by previous studies such as Euclidean distance  (Robinson et al. , 2015; Chen et al. , 2010; Ferber et al. , 2016)  or SASD  (Degiacomi et al.
Further studies using these thresholds also as values for restraints during modeling should be performed. We note that the optimal EucDist value found in this study is not only close to the value used in our modeling of the Pol III RNA polymerase  (Ferber et al. , 2016)  but also in the range of a more recently proposed threshold by  Ferrari et al. (Ferrari et al. , 2019a)  which was measured by an independent approach in a different database. Our method outperforms the geometrical approaches especially by classifying better the FP to TN (Figure  7 ).
For the number of TP, our predictor does not show significant improvement compared to the geometrical approach. This tendency is in part due to the incompleteness of XL-MS data since only a subset of cross-linkable lysines is detected as being linked, thus inducing cross-links labeled as FP which should be labeled as TP. Due to its efficiency and accuracy, NRGXL can be used as a more realistic description of cross-linking restraints during modeling. However, the computational cost of NRGXL is evidently higher than calculating an Euclidean distances. Therefore, NRGXL could be used as a post-modeling filter, as it has already been suggested for SASD  (Merkley et al. , 2014) , to assess the quality of the models resulting from a classical modeling approach using Euclidean distance restraints, or be used as part of a modeling strategy that uses a full calculation of realistic distances relatively rarely (e.
g. ,  (Ferber et al. , 2016) ). The efficiency is achieved without neglecting the importance of flexibility. Residue side chains experience rapid motions in solution  (Henzler-Wildman and Kern, 2007) , and XL-MS data represents an ensemble of different conformations. Hence, sampling several conformations for a given cross-link allows our method to deal with multiple side chain orientations and therefore incorporate side chain flexibility. The importance of treating also the overall flexibility is evidenced by the comparison between the static and flexible models. While in the static model we only considered side chain flexibility, in the flexible model we explicitly generated multiple conformations of the complete system with an anisotropic network model. This improves the performance not only of the approaches based on EucDist and SASD but also of NRGXL.
Anisotropic Network Model To assess the influence of the global flexibility of the entire system, we used normal mode analysis with an anisotropic network model  (Atilgan et al. , 2001 ) (ANM) with ProDy  (Bakan et al. , 2011)  (open-source Python package for protein structural dynamics). ANM nodes were the C α atoms. For each entry in the XLdb database, ten decoy conformations were generated along the first non-trivial mode, with a maximum amplitude of oscillation of 2 Å.
Binary Classification study The binary classification scheme serves to predict if two residues can be cross-linked based on several features. It served us to rigorously compare the quality of predictions based on the different features. For this, we analyzed all lysine pairs from a set of 53 different PDB structures for which experimental XL-MS data were available in the XLdb database  (Kahraman et al. , 2013)  (see XLdb) that were evaluated as compatible by a simple distance criterion. This database contains cross-linking data using BS3/DSS cross-linkers. Size, hydrophobicity, and charge of the cross-linker previous to the cross-linking event were not considered in our analysis. Therefore, due to the identical conformation of DSS and BS3 after being cross-linked, we equally handle both types of cross-links.
In summary, we analyzed lysine pairs with C β -C β distance shorter than 28. 42 Å, which is the maximum length that an XL conformation formed by two lysine side chains (13. 44 Å each), and a BS3/DSS cross-linker (14. 98 Å), can adopt. Database entries with high inconsistency between experimental data (list of cross-links from XLdb) and structural data (PDB structures) were discarded. More precisely, for all systems, a ROC plot per feature was computed. Then, systems with an AUC below 0. 75 for all features were removed. In other words, we ended up discarding database entries for which structure and experimental data were not in agreement for any feature. During the binary classification analysis, the same number of compatible lysine pairs with an experimentally detected XL (EyXL) and compatible lysine pairs without an experimentally detected XL (EnXL) were selected to ensure balanced population sets.
XLdb XLdb  (Kahraman et al. , 2013)  is a cross-link database, containing 506 intra-protein and 62 inter-protein cross-links from 14 publications. For all the entries in the database, either DSS or BS3 cross-linkers were used, which implies that all the database cross-links are between lysine residues. All these cross-links can be mapped to 56 different PDB structures. All the structures and the list of residue pairs cross-linked used are publicly available in the NRGXL repository (see below).
NRGXL implementation NRGXL is implemented in Python and is freely available as a python package. The most recent version can be found here: https://gitlab. pasteur. fr/bougui/NRGXL. NRGXL is also accessible as a web server at: https: //nrgxl. pasteur. fr. Compatible lysine pairs from the static model were splitted into 3 groups based on the number of residues of the protein were they belong  (0-2000, 2000-4000, 4000-6000 residues). Then, we computed an AUC per classifier (Eulidean, SASD or energy) for each defined group. 1  with the closest-to-(0,1) threshold selection alternative  (Unal, 2017)  and with the inclusion of three new statistical measures (sensitivity, specificity and negative prediced value) per optimal cutting point method. Closest-to-(0,1) approach utilise a ROC curve to calculate a feature threshold.
We explore the interaction between these disentangled characters by comparing the embedding reconstruction performance to a baseline case where no shared information is extracted. We use synthetic datasets with different levels of interdependency between feature and network characters and show (i) that shallow embeddings relying on shared information perform better than the corresponding reference with unshared information, (ii) that this performance gap increases with the correlation between network and feature structure, and (iii) that our embedding is able to capture joint information of structure and features. Our method can be relevant for the analysis and prediction of any featured network structure ranging from online social systems to network medicine.
Related Work The advent of increasing computational power coupled with the continuous release and ubiquity of large graph-structured datasets has triggered a surge of research in the field of network embeddings. The main motivation behind this trend is to be able to convert a graph into a low-dimensional space where its structural information and properties are maximally preserved  [38]. The aim is to extract unseen or hard to obtain properties of the network, either directly or by feeding the learned representations to a downstream inference pipeline.
Graph embedding survey: from matrix factorisation to deep learning In early work, low-dimensional node embeddings were learned for graphs constructed from non-relational data by relying on matrix factorisation techniques. By assuming that the input data lies on a low dimensional manifold, such methods sought to reduce the dimensionality of the data while preserving its structure, and did so by factorising graph Laplacian eigenmaps  [39]  or node proximity matrices  [40]. More recent work has attempted to develop embedding architectures that can use deep learning techniques to compute node representations. DeepWalk  [41] , for instance, computes node co-occurrence statistics by sampling the input graph via truncated random walks, and adopts a SkipGram neural language model to maximise the probability of observing the neighbourhood of a node given its embedding.
Such models consistently outperform earlier techniques on different benchmarks and have opened several interesting lines of research in fields ranging from drug design  [49]  to particle physics  [50]. Most of the more recent approaches mentioned above can incorporate node features (either because they rely on them centrally, or as an add-on). However, with the exception of DANE  [33] , they mostly do so by assuming that node features are an additional source of information, which is congruent with the network structure (e. g. multi-task learning with shared weights  [34] , or fusing both information types together  [35] ). That assumption may not hold in many complex datasets, and it seems important to explore what type of embeddings can be constructed when we lift it, considering different levels of congruence between a network and the features of its nodes.
Encoder We are given an undirected unweighted feautured graph G = (V, E), with N = |V| nodes, each node having a D-dimensional feature vector. Loosely following the notations of  [30] , we note A the graph's N × N adjacency matrix (diagonal elements set to 0), X the N × D matrix of node features, and X i the D-dimensional feature vector of a node i. The encoder part of our model is where F -dimensional node embeddings are generated. It computes µ and σ, two N × F matrices, which parametrise a stochastic embedding of each node: µ = GCN µ (X, A) and log σ = GCN σ (X, A). Here we use two graph-convolutional layers for each parameter set, with shared weights at the first layer and parameter-specific weights at the second layer: GCN α (X, A) = Â ReLU( ÂXW enc 0 )W enc 1,α In this equation, W enc 0 and W enc 1,α are the weight matrices for the linear transformations of each layer's input; ReLU refers to a rectified linear unit  [51] ; and following the formalism introduced in  [24] , Â is the standard normalised adjacency matrix with added self-connections, defined as: Â = D-1 2 Ã D-1 2 Ã = A + I N Dii = j Ãij 1 The implementation of our model is available online at github.
We have F A + F AX + F X = F. We further introduce the following notation for the restriction of the embedding of node i to a set of dedicated dimensions {a,. , b} 7  : ξ i,a:b = (ξ ij ) j∈{a,. ,b} This extends to the full matrix of embeddings similarly: ξ a:b = (ξ ij ) i∈{1,. ,N },j∈{a,. ,b} Using these notations we adapt the decoder to reconstruct adjacency and features as follows: A ij |ξ i,1:F A +F AX , ξ j,1:F A +F AX ∼ Ber(MLB(ξ 1:F A +F AX ) ij ) X i |ξ i,F A +1:F ∼ Multinomial(1, MLP(ξ F A +1:F ) i ) In other words, adjacency matrix reconstruction relies on F A + F AX embedding dimensions, feature reconstruction relies on F X + F AX dimensions, and F AX overlapping dimensions are shared between the two. Our reasoning is that for datasets where the dependency between features and network structure is strong, shallow models with higher overlap value will perform better than models with the same total embedding dimensions F and less overlap, or will perform on par with models that have more total embedding dimensions and less overlap.
Standard benchmarks Finally we compare the performance of our architecture to other well-known embedding methods, namely spectral clustering (SC)  [56] , DeepWalk (DW)  [41] , the vanilla non-variational and variational Graph Auto-Encoders (GAE and VGAE)  [30] , and GraphSAGE  [26]  which we look at in more detail. We do so on two tasks: (i) the link prediction task introduced by  [30]  and (ii) a node classification task, both on the Cora, CiteSeer and PubMed datasets, which are regularly used as citation network benchmarks in the literature  [57, 58]. Note that neither SC nor DW support feature information as an input. The Cora and CiteSeer datasets are citation networks made of respectively 2708 and 3312 machine learning articles, each assigned to a small number of document classes (7 for Cora, 6 for CiteSeer), with a bag-of-words feature vector for each article (respectively 1433 and 3703 words).
Link prediction The link prediction task consists in training a model on a version of the datasets where part of the edges has been removed, while node features are left intact. A test set is formed by randomly sampling 15% of the edges combined with the same number of random disconnected pairs (non-edges). Subsequently the model is trained on the remaining dataset where 15% of the real edges are missing. We pick hyperparameters such that the restriction of our model to VGAE would match the hyperparameters used by  [30]. That is a 32-dimensions intermediate layer in the encoder and the two intermediate layers in the decoder, and 16 embedding dimensions for each reconstruction task (F A + F AX = F X + F AX = 16). We call the zero-overlap and the full-overlap versions of this model AN2VEC-0 and AN2VEC-16 respectively.
Method We follow the test procedure laid out by  [30] : we train for 200 epochs using the Adam optimiser  [54]  with a learning rate of. 01, initialise weights following  [55] , and repeat each condition 10 times. The µ parameter of each node's embedding is then used for link prediction (i. e. the parameter is put through the decoder directly without sampling), for which we report area under the ROC curve and average precision scores in Table  1. 9 We argue that AN2VEC-0 and AN2VEC-16 should have somewhat poorer performance than VGAE. These models are required to reconstruct an additional output, which is not directly used to the link prediction task at hand. First results confirmed our intuition. However, we found that the shallow decoder models AN2VEC-S-0 and AN2VEC-S-16 perform consistently better than the vanilla VGAE for Cora and CiteSeer while their deep counterparts (AN2VEC-0 and AN2VEC-16) outperforms VGAE for all datasets.
As neither AN2VEC-0 nor AN2VEC-16 exhibited over-fitting, this behaviour is surprising and calls for further explorations which are beyond the scope of this paper (in particular, this may be specific to the link prediction task). Nonetheless, the higher performance of AN2VEC-S-0 and AN2VEC-S-16 over the vanilla VGAE on Cora and CiteSeer confirms that including feature reconstruction in the constraints of node embeddings is capable of increasing link prediction performance when feature and structure are not independent (consistent with  [33, 35, 34] ). An illustration of the embeddings produced by AN2VEC-S-16 on Cora is shown in Figure  4. On the other hand, performance of AN2VEC-S-0 on PubMed is comparable with GAE and VGAE, while AN2VEC-S-16 has slightly lower performance. The fact that lower overlap models perform better on this dataset indicates that features and structure are less congruent here than in Cora or CiteSeer (again consistent with the comparisons found in  [34] ).
Despite this, an advantage of the embeddings produced by the AN2VEC-S-16 model is that they encode both the network structure and the node features, and can therefore be used for downstream tasks involving both types of information. We further explore the behaviour of the model for different sizes of the training set, ranging from 10% to 90% of the edges in each dataset (reducing the training set accordingly), and compare the behaviour of AN2VEC to GraphSAGE. To make the comparison meaningful we train two variants of the two-layer GraphSAGE model with mean aggregators and no bias vectors: one with an intermediate layer of 32 dimensions and an embedding layer of 16 dimensions (roughly equivalent in dimensions to the full overlap AN2VEC models), the second with an intermediate layer of 64 dimensions and an embedding layer of 32 dimensions (roughly equivalent to no overlap in AN2VEC).
Both layers use neighbourhood sampling, 10 neighbours for the first layer and 5 for the second. Similarly to the shallow AN2VEC decoder, each pair of node embeddings is reduced by inner product and a sigmoid activation, yielding a scalar prediction between 0 and 1 for each possible edge. The model is trained on minibatches of 50 edges and non-edges (edges generated with random walks of length 5), learning rate 0. 001, and 4 total epochs. Note that on Cora, one epoch represents about 542 minibatches, 10  such that 4 epochs represent about 2166 gradient updates; thus with a learning rate of 0. 001, we remain comparable to the 200 full batches with learning rate 0. 01 used to train AN2VEC. Figure  5  plots the AUC produced by AN2VEC and GraphSAGE for different training set sizes and different embedding sizes (and overlaps, for AN2VEC), for each dataset.