dataesr/scientific-paragraphs-categorization huggingface dataset:
Size-exclusion chromatography coupled to multi-angle light scattering The molecular weights of the constructs were determined by size exclusion chromatography coupled to a multi angle light scattering detector (SEC-MALS). The mass measurements were performed on a Dionex UltiMate3000 HPLC system equipped with a 3 angles miniDAWN TREOS static light scattering detector (Wyatt Technology). The sample volumes of 100 µl at a concentration of 40 µM, were applied to a Superose 6 10/300 GL column (GE Healthcare) previously equilibrated with 20 mM HEPES pH 7. 5, 300 mM NaCl, 2 mM TCEP at a flow rate of 0. 5 ml/min. The data were analyzed using the ASTRA 6. 1 software package (Wyatt technology), using the refractive index of the buffer as a baseline and the refractive index increment for protein dn/dc = 0.
Analytical Ultra Centrifugation Sedimentation velocity data of the N-terminal RBCC domain of KAP1 (0. 5 mg/mL) were obtained using a Beckman Optima XL-I analytical ultracentrifuge (Beckman Coulter) with an absorbance optical system. Epon double-sector centerpieces containing 200 µl of protein solution and 300 µl of sample buffer were centrifuged at 55000 rpm and 20°C and data acquired with a radial increment of 0. 003 cm with no delay between scans. The sedimentation of the protein was monitored at 292 nm and 280 nm. Data analysis was performed with Sedfit 87. Buffer viscosity (0. 01002 cp), density (1. 04 g/cm3) and protein partial specific volumes (0. 72 g/cm) were estimated using the program Sednterp  88.
Small-Angle X-Ray Scattering data collection and analysis To exclude sample aggregation, the proteins were analyzed by size-exclusion chromatography in line with small-angle X-Ray scattering (SEC-SAXS). The data were collected at the European Synchrotron Radiation Facility (ESRF), beamline 29 (BM29) at a wavelength of 0. 99 Å with a sample to detector distance of 2. 867 m and a PILATUS 1M detector, covering a momentum transfer of 0. 0025 < q > 0. 6 Å -1 [q = 4πsin (θ)/λ]. Measurements were made at 18°C. The samples were applied to a Superose 6 10/300 GL column (GE Healthcare) at a concentration of 9-12 mg/ml and run at a flow rate of 0. 75 ml/min in 20 mM HEPES, pH 7. 5, 500 mM NaCl, 10% (v/v) glycerol, 2 mM TCEP. During the elution, 2160 scattering measurements were taken with 1s time-frames.
The in-house software BsxCuBE (Biosaxs Customized Beamline Environment) connected to a data processing pipeline (EDNA)  89  was used to control the real time data display (two dimensional and one dimensional) and to provide the first automatic data processing up to a preliminary ab initio model. SAXS data were analyzed using the ATSAS package version 2. 8. 3  90  and ScÅtter  91. For each sample, using Chromixs 92 , an elution profile was generated with the integrated intensities plotted versus the recorded frame number. Using Chromixs, 30 buffer frames were averaged and used to (i) subtract the buffer average from each frame of the sample peak selected and (ii) calculate the corresponding Radius of Gyration (R g ). The subtracted peak region was selected in Chromixs and averaged to generate the final scattering curve used for subsequent analysis.
The scattering curves were initially viewed in PRIMUS  93  where the R g was obtained from the slope of the Guinier plot within the region defined by q min < q < q max where q max < 1. 3/R g and q min is the lowest angle data point included by the program. When one dimension of a scattering particle is greater than the other two (e. g. a rod particle), "Rod" Guinier analysis in PRIMUS can be used to calculate the radius of gyration of its cross-section, R gc (Supplementary Figure  3 ). The P(r) function, the distribution of the intra-atomic distances (r) in the particle, was generated using the indirect transform program GNOM  94. The maximum distance (D max ) was selected by letting the P(r) curve decay smoothly to zero. As our molecules are rod-like and flexible, the R g was also estimated from the P(r) function, such that, unlike the Guinier R g estimation, the P(r) R g calculation takes the whole scattering curve into account.
Ab initio models were produced using GASBOR  64  imposing a P1 symmetry with prolate (elongated protein) anisotropy until a c 2 of 1 was reached. DATPOROD, DATMOW and DATVC within the ATSAS package were used to estimate the Porod Volume (Vp) and the concentration-independent estimate of the MW for the proteins. The final figures were generated using VMD  95  , PyMOL (Schrödinger, LLC) and Chimera  96.
Molecular modelling and flexible SAXS data fitting The preliminary KAP1 model was created using existing, homologous structures of the RING, B-box and coiled-coil α-helical domains, and the solved NMR structure of the PHD-Br domain (PDB: 2RO1)  40. Specifically, the SWISS Model server  97  was used to construct the RBCC domain using the following templates: the RING domain was based on the dimer of Rad18 (PDB: 2Y43)  98  , the B-box1 was based on the Bbox domain of MuRF1 (PDB: 3DDT)  99  , the B-box 2 was based on the B-box domain of TRIM54 (PDB: 3Q1D), and the coiled-coil domain was based on the coiled-coil of TRIM69 (PDB: 4NQJ)  60. To model the TSS domain in the central part of KAP1 we used the Robetta server  100  , while MODELLER v9. 14  101  was used for the HP1binding domain, assigning as a template the structure of EMSY protein in complex with HP1 (PDB: 2FMM)  102  and the structure of the small subunit of the mammalian mitoribosome (PDB: 5AJ3)  103.
Isothermal titration calorimetry ITC experiments were performed using a MicroCal PEAQ ITC from Malvern. KAP1 FL and HP1 FL were buffer exchanged into 20 mM HEPES pH 7. 5, 300 mM NaCl, 2 mM TCEP and concentrated to 1. 5 and 5 mg/ml (15 and 200 µM), respectively. HP1 FL containing solution was injected into the KAP1 solution (2 μl of HP1 per injection at an interval of 180 sec, a total of 19 injections into the cell volume of 300 μl, with stirring speed of 800 rpm, at 25° C). HP1 FL was also injected into buffer to determine the unspecific heat of dilution. However, subtracting this experimental heat of dilution was not sufficient to get a good fit, so the last injections were used to better estimate the heat of dilution and subtract a straight line from our data. The experimental data were fitted to a theoretical titration curve ("one set of sites" model) using software supplied by Microcal, with n (number of binding sites per monomer), ΔH (binding enthalpy, kcal/mol) and K d (dissociation constant, M), as adjustable parameters.
Acknowledgements We thank the  EPFL Protein Production and Structure Core Facility  for providing the equipment for the biophysical characterization of the protein complexes;  Natacha Olieric  and  Bruno Correia  for the use of their MALS machines;  Cy Jeffries  for his assistance with SAXS data preparation and analysis;  Martin Moncrieffe  for his help with AUC data analysis;  Luciano Abriata  and  Florence Pojer  for helpful discussions;  BM29  staff scientists for their assistance in SAXS data collection; EPFL for funding.
Similar results with other SGLT2is have been reported subsequently by both randomized clinical trials and observational studies  (4, 5). SGLT2 receptors are located in the proximal renal tubule and are responsible for 90% of glucose reabsorption into the bloodstream. Inhibition of SGLT2 receptors causes glucosuria and reduces glycemic levels  (6). However, control of glycemic levels in response to SGLTis alone seems insufficient to explain the reported cardiovascular benefits. In fact, the EMPA-TROPISM (Are the "Cardiac Benefits" of Empagliflozin Independent of Its Hypoglycemic Activity? ) trial (NCT03485222) is an ongoing trial that aims to elucidate whether the benefits obtained in the EMPA-REG OUTCOME trial were mediated, at least in part, by a glucose- Iborra-Egea et al. J A C C : B A S I C T O T R A N S L A T I O N A L S C I E N C E V O L.
Details about how therapeutic performance mapping system (TPMS) (Anaxomics, Barcelona, Spain) machine learning analyses work have been reported previously  (13). Briefly, TPMS incorporated massive amounts of biological information relevant to both the disease(s) and the drug under study, drawn from curated reports and from public and private databases (Figure  1 ). Next, TPMS adopted the gene expression dynamics stored in the truth table as a reference framework to shape all the biological information. Finally, TPMS generated mathematical models to form complex relationships between datasets or to find patterns in the data  (14, 15). The system built an ANN, which was trained with 1,000 iterations. During this process, the models generated different MoAs, which progressively increased in accuracy by correcting itself with already-known clinical and biomedical information, and cross-validation with independent sets of data (Figure  1 , Supplemental Video 1).
G e n e r a t i o n o f m a t h e m a t i c a l m o d e l s. Finally, artificial neural network (ANN)-based analyses were used to identify the most relevant pathophysiological processes implicated in HF and DM. To train the mathematical models, this large collection of wellestablished pathophysiological signals and clinical information relevant to the pathologies under study was reviewed. The molecular description of these input-output signals was investigated as follows: 1. Model inputs, for example, information about drugs, pathologies, and protein/gene relationships provided by public databases to assess inhibition or activation of 1 or more nodes of the protein network (their targets) triggering a perturbation through the system. 2. Model outputs, for example, experimental RNAseq data (upregulated or downregulated genes/proteins) and clinical information.
application. A key differentiating feature of deep learning compared with other subtypes of artificial intelligence is the ability to operate without external guidance to draw conclusions. The neural network is not designed by humans, rather, the number of layers is determined by the data itself. Deep ANNs have used primarily supervised learning, with training from known patterns and labeled input data (truth table) to prospect reliable insights in large datasets. At the same time, all the outputs from the algorithm are restricted by the current scientific and medical standards.
Flow Chart Depicting the Machine Learning Approach Used in This Study C o m p i l a t i o n o f t r a n s c r i p t o m i c d a t a. Next, an RNA sequencing (RNAseq) dataset (public data avail- able in Gene Expression Omnibus [GEO], accession GSE57345 and subseries GSE57338 and GSE26887) of myocardial tissue data derived from HFrEF patients with (10) or without DM (11) was thoroughly analyzed to model the molecular effects of empagliflozin in each scenario. Gene information was mapped one-to- one to proteins for introduction into the protein network, which included 280 proteins of interest (Supplemental Table 1). This collection of experi- mental pathophysiological signals was used as a list of principles, termed a "truth table," which apply to a spe- cific condition (i. e. , some proteins are overexpressed only in HF patients with DM) (Figure full-length articles in PubMed was conducted to obtain information about HF and DM.
its effects in HF and reduced ejection fraction (HFrEF) in Moreover, empagliflozin pharmacological data and patients with or without DM (Figure 1). Machine molecular relationships described for HF and DM learning-driven pathways were validated in an in vivo were extracted by integrating massive, publicly available empagliflozin-treated post-infarct HF (MI-HF) rat model. databases such as Drugbank, Reactome, MINT, and Bio- METHODS Grid. All this information accounted for more than 350,000 interactions (Figure 1). Briefly, by integrating MACHINE LEARNING IN SILICO ANALYSES. Molecular published data, a set of molecular profiles characterizing characterization of HF and DM pathology and drug HF and DM were defined, which were used to build the treatments. First, extensive and careful review of protein network and the mathematical model.
Biological data Age and length data from 13056 fish (6322 males and 6734 females) identified as S. norvegicus were extracted from the database held by the Institute of Marine Research of Norway. Samples were collected with bottom, shrimp and pelagic trawls, gillnets and longlines by research and commercial vessels in the Norwegian and Barents Seas between 1992 and 2015. Specimens were obtained following the IMR manual for sampling fish and crustaceans  (Mjanger et al. , 2016). Data for each individual included geographical coordinates, date and time of sampling, fishing gear used, depth and duration of sampling, fish length, age, sex and maturity stage. For bottom trawls, sampling depth was set to ocean floor depth at the sampling location. For pelagic nets and trawls, sampling depth was defined as the mean of the gear deployment depth recorded during fishing.
Model with depth effect In the single-level model where the VBGF parameters vary between depth intervals, mean length at maximum age for the females/males is estimated to be 53/51 cm. Individuals show a pattern of increasing length with increasing depths, to the exception of the depth interval of 300-375 m where a slight decrease can be observed. In the last depth interval there is an important increase of more than 20 cm (Figure  3A ). The data displayed in Figure  3B  are the results of statistical growth modelling for female individuals. Within each depth interval, the variability not explained by the model amounts to 33%. The hyper-parameters L and k are negatively correlated. The model with depth effects shows definitely an increase with depth, with an important increase in the deepest layer.
Acknowledgements Thanks to the IMR and commercial vessels for the sampling collection. We wish to give our gratitude to  Kjell Nedreas  and  Arne Storaker  from  Institute of Marine Research (IMR) in Bergen , for their technical work and the rapidity with which they responded to our requests and to  Torild Johansen  from  IMR in Tromsø  for her knowledge and work in the field of genetics. Many thanks also to  Professor Gilles Yoccoz  and  Uffe  for their constructive comments and useful remarks improving our work.
Addressing Answer Selection (AS) tasks with complex neural networks typically requires a large amount of annotated data to increase the accuracy of the models. In this work, we are interested in simple models that can potentially give good performance on datasets with no or few annotations. First, we propose new unsupervised baselines that leverage distributed word and sentence representations. Second, we compare the ability of our neural architectures to learn from few annotated examples in a weakly supervised scheme and we demonstrate how these methods can benefit from a pre-training on an external dataset. With an emphasis on results reproducibility, we show that our simple methods can reach or approach state-of-the-art performances on four common AS datasets.
Introduction Large-scale Question Answering (QA) tasks have recently received a substantial amount of attention. In this paper, we are interested in the QA subtask of Answer Selection (AS), where the goal is to retrieve the correct answers to a question amongst a set of candidate answers. While a vari-ety of AS datasets exists, it appears that these are typically smaller than open domain Question Answering corpus such as SQuAD  [1]. Most efforts to adress the AS task are directed towards supervised end-to-end training of complex neural architectures. Existing models are typically based on convolutional  [2]  or recurrent neural networks  [3] , with hundred of thousands of parameters, and a set of hyperparameters typically tuned for the target dataset. Additionally, they require a large amount of annotated data to perform end-to-end training.
Going one step further, we present a domain adaptation approach, where we pre-train our models on a large external QA dataset and fine-tune the models on the target dataset, similar to the work by Min et al. [9]. We assess the behaviour of our model when varying the amount of training data and observe the ability of some architecture to achieve low-shot learning. Finally, we show that our methods can reach approach state-of-the-art performance when all the training data is used. Importantly, we make a major effort to have fully reproducible results from the initial dataset, as it is established that the deep learning community is affected by a reproducibility crisis due to poorly controlled environment settings  [14]. We demonstrate on four publicly available datasets that our methods reach near state-of-the-art results.
Unsupervised approaches The unsupervised approaches proposed in this section aim to reach acceptable performances in the case where there are no annotations available for training. First, we propose two methods relying on pre-trained word embeddings that associate a vector for each word in the vocabulary  [12]. Second, we proposed to leverage a pre-trained language model, the Universal Sentence Encoder  [13]. The model relies on attention modules (Transformer  [15] ) to encode the input sentence into a 512-dimensional vector. Finally, we compare our approaches to the BM25 information retrieval baseline  [16]. BM25: The score for each candidate answer is given by the 1 https://github. com/manuvallee/qa_matching BM25 weights of question words in the answer: score(Q, A) = n ∑︁ i=1 (︂ IDF(Q i ).
Domain adaptation To deal with the scarce annotation of our target datasets, we explore how pre-training our models on an external dataset with the same task could improve the performance. This well-known approach is what we call here domain adaptation. Following a recent study on domain adaptation for AS task  [9] , we pre-train our models on an external dataset and fine-tune them on the target dataset. Importantly, pre-training a model on an external dataset allows deploying the model as such, without further finetuning on the target dataset. This approach could be of major interest when no annotated data is available.
Experimental data We evaluate our methods on four publicly available AS datasets. WikiQA: Open domain questions extracted from Bing requests. The answers are sentences from a Wikipedia para-graph. There is only one sentence per candidate answer. Following recent work, we only consider questions that have correct answers for evaluation  [10]. TrecQA: Dataset created from the Text REtrieval Conference (TREC) question answering tracks (8-13)  [18]. Following the literature  [19]  , we use the CLEAN version, where all the questions with only positive or only negative answers are removed. Semeval -Task 3 -English subtask: Questions and answers from a life forum in Qatar  [20]. Note that the candidate answers are potentially composed of several sentences. Yahoo: 10k Questions extracted from Yahoo! L5 142k Nonfactoid questions.
We use the subset and the corresponding train/dev/test splits described in  [21]. Importantly, Semeval and Yahoo datasets are community question answering where the questions and answers are generated by the user. As a consequence, candidate answers are complex multi-sentence texts with detailed information. The CQA datasets therefore present a domain shift with WikiQA and TrecQA, which are open domain QA with encyclopedia sourced answers. In Table  1 , we report some basics statistics of the four datasets. We note that the training set is relatively small for all datasets, considering the size of recent question answering corpus such as SQuAD (90k training questions).
External resources and pre-trained models The first external resource is the training data used to compute the word embedding. The generic w2v embedding that is trained on the Google News dataset (about 100 bil-lion words) and publicly available². The word2vec embedding can alternatively be trained on a large set of unannotated data from the same source as the target corpus. These specific embeddings are built from Wikipedia for WikiQA and TrecQA datasets, and from the target corpus for Yahoo answers and SemEval. We used the skip-gram w2v embedding approach with a window of 8 tokens and a minimum word count of 10. Second, we propose to employ the pre-trained USE model to compute an embedding for the question and for the answer  [13]. It produces a 512 dimensional vector for a given sentence.
Pre-processing For the USE representation, no pre-processing is necessary because the USE model was trained with raw text as input. For the other methods we apply the following pipeline: switching to lower case, deleting diacritics (glyph added to a letter, or basic glyph), adding spaces around the punctuation, removing confusing signs (coma and dots in numbers, smileys), and replacing URLs by a "_url_" tag. We also remove stop-words as defined in the NLTK English list³. We used the data format proposed in the original release of the datasets, and the pre-processing steps were the same across datasets.
Networks parameters The hyper-parameters of our neural networks are fixed a priori and kept constant across datasets. For both architectures, the hidden layer has a number of cells equal to the dimensionality of the input layer. For example, with the siamese architecture and a USE representation (d=512), the hidden layer has 512 cells, whereas with the Concat architecture, the number of cells would be 512×4 = 2048. Each hidden layer is followed by a dropout layer with a rate of 0. 3. The only parameter optimized across datasets is the margin for the pair-wise approach. The optimal values for the development data splits are 0. 05 for SQuAD-T, 0. 01 for WikiQA, 0. 02 for SemEval, 0. 1 for TrecQA and 0. 5 for Yahoo. The optimization is conducted with Adam and its default parameters proposed in the original paper  [22].
Evaluation The evaluation is performed with the official SemEval scorer to compute the metrics⁴. Note that we evaluate all the question sets, including when there is no correct answer for a given question. To ensure reproducible results, each experiment is repeated 10 times with different question subsets, and we report the mean of each metric over the 10 runs. To be consistent with the literature, we report the Mean Average Precision (MAP) for WikiQA, Semeval and TrecQA and the precision at one (P@1) for the Yahoo dataset. We consider that two results are significantly different when the p-value computed with the Wilcoxon test is smaller than 0. 05. Besides performance obtained using the full training dataset, we are interested in the relationship between the amount of training data and the performance of our methods.
Unsupervised methods In Table  2  (top and middle rows), we report the results obtained with the IR baseline and the unsupervised methods on the four datasets. We reach acceptable performance with fully unsupervised baselines, and two methods stand out from the others. The Hungarian approach with a specific word embedding performs the best on three datasets (TrecQA, SemEval and Yahoo), with a significant difference with the second best (Hungarian with generic embedding). On the Yahoo dataset, there is no significant difference between generic and specific embedding, neither with the Hungarian nor the average embedding approach. We note that the USE representation performs best only on the WikiQA dataset, with a significant difference with the Hungarian method.