dataesr/scientific-paragraphs-categorization huggingface dataset:
Interestingly, analysis of HSV-1 pUL32 sequences from 44 strains retrieved from the Genbank ® reveals more diffuse polymorphisms than in HCMV-pUL52. In general, the polymorphism is like that of pUL52, and these results were found with sequence alignment. Indeed, sequence alignment in the N-terminal part and in the middle part showed a lot of gaps and variability between residues. It appears that these two variable regions identified in HSV-1 and HCMV have the same location. It is unlikely that an essential function would be associated with these regions. Four motifs CXXC were described in pUL32; three of them are located in the most conserved regions. One of the four motifs is found in a region rich in polymorphisms (VRII), and this motif is not conserved in beta-and gamma-herpesviruses. In addition, the mutation of UL32-C291A does not lead to a loss of viral replication.
The statistics of the V(D)J recombination process described by the probability of generating a given receptor sequence σ, Pgen(σ), are inferred using the IGOR software  [8]. Pgen(σ) acts as a baseline for the selection model. We then infer selection factors Q, which act as weights that modulate the initial distribution Pgen(σ). We infer two types of selection weights: linear in log space (using the SONIA software  [9] ) and non-linear weights using a deep neural network, in the soNNia software presented here. Non-linear selection weights are more flexible than linear ones. (C) Pipeline of the algorithm: Pgen is inferred from unproductive sequences using IGOR. Selection factors for both the linear and non-linear models are inferred from productive sequences by maximizing their log-likelihood L, which involves a normalization term calculated by sampling unselected sequences generated by the OLGA software  [10].
Neural network models of TCR and BCR selection Previous work has inferred biophysically informed models of V(D)J recombination underlying the generation of TCRs and BCRs  [8, 30]. In brief, these models are parametrized according to the probabilities by which different V-, D-, J-genes are used and base pairs are inserted in or deleted from the CDR3 junctions to generate a receptor sequence. We infer the parameters of these models using the IGoR software  [8]  from unproductive receptor sequences, which are generated, but due to a frameshift or insertion of stop codons are not expressed, and hence, are not subject to functional selection. The inferred models are used to characterize the generation probability of a receptor sequence P gen , and to synthetically generate an ensemble of pre-selection receptors  [10].
This baseline set is often generated by sampling from a previously inferred generation model P gen , using the IGoR software  [8]. Note that this expression becomes exact as the number of generated sequences approaches infinity. In soNNia we divide the sequence features f into three categories: (i) (V,J) usage, (ii) CDR3 length, and (iii) CDR3 amino acid composition encoded by a 20 × 50 binary matrix that specifies the identity of an amino acid and its relative position within a 25 amino acid range from both the 5' and the 3' ends of the CDR3, equivalent to the left-right encoding of the SONIA model  [9]  (Fig. 1D ). Input from each of the three categories are first propagated through their own network. Outputs from these three networks are then combined and transformed through a dense layer.
This choice of architecture reduces the number of parameters in the DNN and makes the contributions of the three categories (which have different dimensions) comparable; see Methods and Figs. S1,S3 and S4 for details on the architecture of the DNN. The baseline ensemble G, which we have described as being generated from the P gen model (Fig. 1 C ), can in principle be replaced by any dataset, including empirical ones, at no additional computational cost, for selection inference with soNNia. This is especially useful when the goal is to only compare the selection models associated with different sub-repertoires with distinct functions. We will use this feature of soNNia to learn selection coefficients of subsets relative to an empirically constructed generic functional repertoire. In that case, the inferred selection factors Q only reflect differential selection relative to the generic baseline.
Importantly, this approach enables us to infer differential selection without having to infer a common underlying generation model P gen for the sub-repertoires. Once two soNNia models have been learned from two distinct datasets, their statistics may be compared by computing a sequencedependence log-likelihood ratio r(x) = log Q 1 (x)/Q 2 (x) predicting the preference of a sequence for a subset over the other. This log-likelihood ratio can be used as a functional classifier for receptor repertoires (Fig. 1 E ). (Fig. 1 B ) by inferring selection on TCRβ repertoires from a large cohort of 743 individuals from ref. [33]. Our goal is to characterize selection on functional receptors irrespective of their phenotype. To avoid biases caused by expansions of particular receptors in different individuals, we pool the unique nucleotide sequences of receptors from all individuals and construct a universal donor.
Multiplicity of an amino-acid sequence in this universal donor indicates the number of independent recombination events that have led to that receptor (in different individuals, or in the same individual by convergent recombination). We randomly split the pooled dataset into a training and a test set of equal sizes and trained both a SONIA and a soNNia selection model on the training set (Methods and Fig. 1 C ). Our inference is highly stable, and the selection models are reproducible when trained on subsets of the training data (see Methods and Fig. S2 ). To assess the performance of our selection models, we compared their inferred probabilities P post (x) with the observed frequencies of the receptor sequences P data (x) in the test set (Fig. 2A  and B ). Prediction accuracy can be quantified through the Pearson correlation between the two log-frequencies, or through their Kullback-Leibler divergence D KL (P data |P post ) (Methods and Fig.
Intra-and inter-chain interactions in TCRs and BCRs T-cell receptors are disulfide-linked membrane-bound proteins made of variable α and β chains, and expressed as part of a complex that interact with pathogens. Similarly, B-cell receptors and antibodies are made up of a heavy and two major groups (κ and λ) of light chains. Previous work has identified low but consistent correlations between features of αβ chain pairs in T-cell receptors, with the largest contributions between V α , V β and J α , V β  [34] [35] [36] [37] [38]. In B-cells, preferences for receptor features within heavy and light chains have been studied separately  [39, 40]  but inter-chain correlations have not been systematically investigated. We first aimed to quantify dependencies between chains by re-analyzing recently published single-cell datasets: TCR αβ pairs of unfractionated repertoires from ref.
4D ), which is a much harder task since these subsets are very similar (Fig. 4A ). The accuracy of the inference depends on the size of the unfractionated data, with a mean expected error that falls below 1% for datasets with size 10 4 or larger for the CD8 + /CD4 + mixture (red and orange lines in Fig. 4E ). Our method uses a theoretically grounded maximum likelihood approach, which includes all the features captured by the soNNia model. Nonetheless, a simple linear selection model with only V-and J-gene usage and CDR3 length information (blue line in Fig. 4E ), analogous to the method used in ref. [47] , reliably infers the composition of the mixture repertoire. Additional information about amino acid usage in the linear SO-NIA model results in moderate but significant improvement (orange line).
Classification of TCRs targeting distinct antigenic epitopes Recognition of a pathogenic epitope by a TCR is mediated through molecular interactions between the two proteins. The strength of this interaction depends on the complementarity of a TCR against an antigen presented by a MHC molecule on the T-cell surface. Recent growth of data on paired TCRs and their target epitopes  [26, 48]  has led to the development of machine learning methods for TCR-epitope mapping  [25] [26] [27] [28] [29]. A TCR-epitope map is a classification problem that determines whether a TCR binds to a specific epitope. We use our selectionbased classifier (eq. 5) to address this problem. We determine the target ensemble P r post from the training set of TCRs associated with a given epitope (positive data), and the alternative ensemble P r post from a set of generic unfractionated TCRs (negative data).
The precision-recall curves in Fig. 5D-F  show that TCRex and the logistic classifier can equally well classify the data, and moderately outperform the selection-based classifier. While both the logistic classifier and TCRex are optimized for classification tasks, the selection-based classifier is a generative model trained to infer the receptor distribution of interest (positive set) and identify its distinguishing features from the baseline (negative set). As a result, selection-based classification underperforms in the low-data regime, for which fitting a reliable distribution is difficult (e. g. for the SARS-CoV-2 epitope model, with only N = 315 positive examples). By contrast, the logistic classifier finds a hyperplane that best separates the two sets, and therefore, is better suited for classification tasks, and may be trained on smaller datasets.
Thus characterizing the distributions of positive and negative examples is more data demanding than mere classification. For this reason pure classifiers are generally expected to perform better, but lack the ability to sample new data. Our analysis complements the collection of proposed classifiers by adding a generative alternative that is grounded on the biophysical process of T-cell generation and selection. This model is simple and interpretable, and performs well with large amounts of data. The epitope discrimination task discussed here and in previous work focuses on predicting TCR specificity to one specific epitope. A long-term goal would be to predict the affinity of any TCR-epitope pair. However, currently available databases  [26, 48]  do not contain sufficiently diverse epitopes to train models that would generalize to unseen epitopes  [54].
A further complication is that multiple TCR specificity motifs may co-exist even for a single epitope  [29, 55] , which cannot be captured by linear models  [56]. Progress will be made possible by a combination of high-throughput experiments assaying many TCR-epitope pairs  [57] , and machine learning based techniques such as soNNia. In summary, we show that nonlinear features captured by soNNia capture more information about the initial and peripheral selection process than linear models. However, deep neural network methods such as soNNia suffer from the drawback of being data hungry, and show their limitations in practical applications where data are scarce. Nonetheless, with the rapid growth of functionally annotated datasets, we expect soNNia to be more readily used for inference of nonlinear selection on immune receptor sequences.
Methods Data description. In this work we used different datasets to evaluate selection on T-and B-cell receptor features. 1. To quantify the accuracy of the soNNia model (Fig. 2 ), we used the TCRβ repertoires from a large cohort of 743 individuals from ref. [33]. We pool the unique nucleotide sequences of receptors from all individuals and construct a universal donor totalling 9 × 10 7 sequences. We randomly split the pooled dataset into a training and a test set of equal sizes. We then subsampled the training set to 10 7 to reduce the computational cost of inference. 2. To characterize selection on paired chain receptors (Fig. 3 ), we analyzed TCR αβ pairs of unfractionated repertoires from ref. [37]  (totalling 5 × 10 5 receptors), and BCR of naive cells from ref. [41]  totalling 22 × 10 3 and 28 × 10 3 receptors for the Hλ and Hκ repertoires, respectively.
This divergence is symmetric and only depends on the relative differences of selection factors between functional sub-repertoires, and not on the baseline model. 5. Filter sequences with CDR3 amino acid length smaller than a maximum value (set to 30 in this paper) 6. Remove sequences with small read counts (optional). For the analysis of Fig. 2  we analysed data from  [1]. We first applied the standard pipeline. In addition we excluded TCRs with gene TRBJ2-5 which is badly annotated by the Adaptive pipeline  [2]  and removed a cluster of artefact sequences, which was previously identified in  [3]  and corresponds to the consensus sequence CFFKQKTAYEQYF. For the analysis of Fig. 3  we analysed data from  [4]  and  [5]. Dataset from  [4]  was obtained already pre-processed directly from the authors, while pre-processed dataset from  [5]  is part of the supplementary material of the corresponding paper.
The soNNia standard pipeline is then applied to both datasets, independently for each chain, and a pair is accepted only if it passes both filtering steps. For α TCR datasets, sequences carrying the following rare genes were removed due to their rarity in the out-of-frame dataset: TRAJ33, TRAJ38, TRAJ24, TRAV19. For the analysis of Figs. 4  and 5  we analysed data from  [6]  and  [7] , to which we applied our standard pre-processing pipeline.
Generation model The generation model relies on previously published models described in  [8, 9, 10]. Briefly, the model is defined by the probability distributions of the various events involved in the VDJ recombination process: V, D, and J gene usage, and number of deletions and insertions at each junction. The model is learned from non-productive sequences using the IGoR software  [9]. For BCR, only a few nonproductive sequences were available, and so we instead started from the default IGoR models learned elsewhere  [9] , and re-inferred only the V gene usage distribution for the heavy chain, and VJ joint gene distribution for light chains, keeping all other parameters fixed. Amino-acid sequence probability computation and generation is done with the OLGA software, which relies on a dynamic programming approach.
soNNia model inference Given a sample of data sequences D = {x i } N D i=1 and a baseline G = {x i } N G i=1 we want to maximize the average log-likelihood: L(θ) = log P θ post D = 1 N D N D i=1 log P θ post (x i ) = 1 N D N D i=1 [log Q θ (x i ) + log P gen (x i )] -log Z θ = log Q θ D + log P gen D -log Q θ G , (S1) where Z θ = Q θ G = N -1 G N G i=1 Q θ (x i ). The P gen term in the last equation is parameter independent and can thus be discarded in the inference. When an empirical baseline is used, P gen is replaced by P emp (x) = N -1 G N G i=1 δ x,x i. Otherwise, the baseline G is sampled from the P gen model, which we learn from nonproductive sequences using the IGoR software  [9]. The above likelihood is implemented in the soNNia inference procedure (linear and non-linear case) with the Keras  [13]  package.
The model is invariant with respect to the transformation Q(x) → cQ(x) and Z → Z/c, where c is an arbitrary constant, so we fix dynamically the gauge Z = 1. We lift this degeneracy by adding the penalty Γ(θ) = (Z θ -1) 2 , and minimize -L sonia (θ)+γΓ(θ) with γ = 1 as a loss function. In our implementation batch sizes between 10 3 -10 4 sequences produced a reliable inference. L2 and L1 regularization on kernel weights are also applied. Hyperparameters were chosen using a validation dataset of size 10 % of training data. The inference converges after around 100 epochs and the network does not overfit (Fig. S2A ). To test the stability of our inference, we evaluated the P post values of generated sequences, based on two models trained on subsets of the initial training data, and show that the P post estimated are highly reproducible between these selection models (Fig  S2B ).
• Entropy of paired receptor repertoires To quantify diversity of immune receptors associated with paired chains, we estimated the entropy H(P post ) = -log 2 P post Ppost = -Q log 2 P post Pgen (S10) of the paired chain models by sampling 10 6 sequences from the generation model P gen (Table  S1 ). For comparison, we evaluated the entropy of single chain repertoires, using models inferred for each chain separately. We also evaluated the entropy of V and J gene features using the observed marginal probabilities of these features in the data. For example, the entropy associated with V-genes in the heavy chain repertoire can be calculated as H(P V H post ) = -i P (V i H ) log 2 P (V i H ) , where P (V i H ) is the marginal probability for the i th V-gene in a heavy chain (H) dataset. Errors in estimating Entropy H (Table  S1 ), the Kullback-Leibler divergences D KL (Fig.
2 ) and Jensen-Shannon divergences D JS (P post , P post ) (Figs. 4, S7) are evaluated by computing the standard deviation of the above quantities using subsampled datasets of size one fifth of the original data. Here we assume that P post (r) = 1 Z P gen (r)Q(r) or P post (r) = 1 Z P emp (r)Q(r), with P gen or P emp as baselines, respectively (eq. 1). The aminoacid input is encoded using an embedding layer, called EmbedViaMatrix and then processed by a tanh non-linearity, called Activation layer. The Flatten layer turns the encoded matrix in the corresponding flattened array where each row of the matrix is concatenated to the successive one. A dense layer is then applied to reduce its dimensionality. The other two inputs are also processed through a dense feed-forward layer to reduce their corresponding dimensionality.
FigureS5: Transfer learning consists of a 2-step inference (left): in the first step we infer a deep neural network on a bigger data set G, and in the second second step we re-infer a subsection of the network, or an additional layer on a smaller dataset, which is the real target. In our specific application, the big data G is the unfractioned repertoire from ref. [7]  (P emp (x) = N -1.
D 2 FIG. 1 : 21 FIG. 1: Inference of functional selection models for immune receptor repertoires. (A) T cell receptor α and β chains are stochastically rearranged through a process called V(D)J recombination. Successfully rearranged receptors undergo selection for binding to self-pMHCs. Receptors that bind too weakly or too strongly are rejected, while intermediately binding ones exit the thymus and enter peripheral circulation. Development of B-cell receptors follows similar stages of stochastic recombination and selection. (B)We model these two processes independently. The statistics of the V(D)J recombination process described by the probability of generating a given receptor sequence σ, Pgen(σ), are inferred using the IGOR software[8]. Pgen(σ) acts as a baseline for the selection model.
We then infer selection factors Q, which act as weights that modulate the initial distribution Pgen(σ). We infer two types of selection weights: linear in log space (using the SONIA software[9]) and non-linear weights using a deep neural network, in the soNNia software presented here. Non-linear selection weights are more flexible than linear ones. (C) Pipeline of the algorithm: Pgen is inferred from unproductive sequences using IGOR. Selection factors for both the linear and non-linear models are inferred from productive sequences by maximizing their log-likelihood L, which involves a normalization term calculated by sampling unselected sequences generated by the OLGA software[10]. (D) In both selection models the amino acid composition of the CDR3 is encoded by its relative distance from the left and right borders (left-right encoding).
Cells and Reagents Primary HSF were obtained from ScienCell Research Laboratories (ScienCell, 4700; Clinisciences, Carlsbad, CA, USA). Cells were cultured in Modified Eagle's Medium (MEM eagle, PAN Biotech P0408500, Aidenbach, Germany) supplemented with 10% heat-inactivated fetal bovine serum (FBS, PAN Biotech, 3302 P290907, Aidenbach, Germany), 2 mM L-glutamine (Biochrom AG, K0282, Berlin, Germany), 0. 1 mg/mL penicillinstreptomycin (PAN Biotech, P0607100), 1 mM sodium pyruvate (PAN Biotech, P0443100) and 0. 5 µg/mL amphotericin B (PAN Biotech, P0601001). ONNV was provided by the CNR (Centre national de référence) Arbovirus of Marseille (France). Polyinosinic-polycytidylic acid (PIC) was purchased from Amersham Biosciences, Little Chalfont, UK (catalog number: 27-4732-01) and recombinant human IL-1β and TNF-α from Peprotech, Rocky Hill, CT, USA (catalog number: 200-01B and 300-01A, respectively).
7 µL of enzyme mix and 1. 3 µL of primers mix at a final concentration of 250 nM. The specific primers used are listed in Table  1. The RT-qPCR program was executed with the QuantStudio 3 Real-Time PCR System with the following steps: a reverse transcription at 42 • C for 5 min and 40 cycles comprising a denaturation step at 95 • C for 5 s, an annealing step at 58 • C for 15 s and an extension step at 72 • C for 15 s. Fluorescence data were collected at 520 nm. PCR reactions were validated by analyzing the melting curve of each pair of primers with the QuantStudio Design and Analysis Software v1. 5. 1 (ThermoFisher Scientific, Waltham, MA, USA). Relative gene expression was calculated using GAPDH as a housekeeping gene and the fold change versus control (set to 1) was calculated for each gene.
6-fold and 3-fold, respectively (Figure  S4A, B ). Of note, COX-2 expression was not significantly upregulated by TNF-α (data not shown). As the antagonistic pathways NF-κB and p53 predominantly modulate inflammation in synovial fibroblasts in the context of rheumatoid arthritis  [26, 29] , RT-qPCR experiments As the antagonistic pathways NF-κB and p53 predominantly modulate inflammation in synovial fibroblasts in the context of rheumatoid arthritis  [26, 29] , RT-qPCR experiments were carried out to assess the expression of these two transcription factors. IL-1β markedly increased NF-κB1 gene expression (by 8-fold) at 6 h of treatment and this effect was completely blunted over time (at 24 h). CPT-11 did not modulate NF-κB1 expression in control cells but down-regulated it by 1. 5-fold and in a significant manner in cells exposed to IL-1β (Figure  6a ).
Study Design and Settings. We performed a retrospective study from April 4 to June 1, 2018, in Centre Pasteur du Cameroun (CPC), Yaoundé, the national reference laboratory in biology and public health, and Member of the Pasteur Institute International Network. As a nationwide reference laboratory, CPC analyzes specimen coming from Yaoundé and other regions of the country. We analyzed bacteriological routine data, collected during two distinct one-year periods (2010 and 2017), and stored in the laboratory data informa-tion system (GLIMS). We initially aimed to perform an upto-date ten-year comparison (2007-2017) of the various AMR features targeted, 2017 being the more recent year with completed data. Unfortunately, GLIMS was put in place in July 2009, making 2010 the first year with comprehensive stored data.
Data Management and Analysis. From GLIMS, we extracted all bacteriology analysis stored in 2010 and 2017 into Excel® tables without patients' name. The crude global database underwent the abovementioned filters (Figure  1 ). The resulting table was then uploaded to R version 3. 4. 3, for data management and analysis. Variables were renamed and harmonized, and new variables were created to fit with our 3 main outcomes: (i) Global Acquired Resistance. The susceptibility profile was processed to obtain a binary variable (absence or presence of acquired resistance). There was no acquired resistance (wild strain) if a given bacterium was still fully susceptible (S) to all antibiotics (ATB) naturally active in vitro. There was acquired resistance (resistant strain) if the pathogen had been found intermediate (I) or resistant (R) for at least one of these ATBs (ii) Specific Resistance.