dataesr/scientific-paragraphs-categorization huggingface dataset:
With a type II-b InAs/InAs1-xSbx band offset  [25]  where electrons are confined in the binary layer (InAs) while holes are confined in the alloy one (InAsSb), the quantized miniband energies of the strain balanced InAs/InAs1-xSbx T2SL have been calculated with nextnano 3 commercial software  [17]. From these data, it appears that x = 0. 35 and 5 ≤ p (nm) ≤ 6 are required to reach, at 150 K, a cut-off wavelength (λco) around 5 µ m together with a wave function overlap between 55% and 66% (Figure  1b ). An important feature of the Ga-free T2SL material structure is the possibility of implementing it in a nBn  [18]  or pBn  [21]  MWIR barrier detector structure as absorbing layer (AL) associated with a high band gap barrier material. The main objective of the barrier structure is to reduce the contribution of the Shockley-Read-Hall (SRH) recombination current, thus the generationrecombination (GR) current to the dark current of the detector.
For that, it is required to control the electric field zone by confining it in the barrier material instead of in the absorbing zone structure. Therefore, SRH processes occur in the high band gap material, instead of the IR absorbing layer. In addition, the barrier layer (BL) plays a similar role as the space charge zone in the pn structure by blocking the majority carriers and allowing the transfer of the minority ones. Consequently, this kind of device is called a bariode. When the bariode is correctly designed, GR dark current is eliminated and the dark current is diffusion current (Jdiff) limited whatever the operation temperature, improving the performances of the detector compared to a pn junction photodiode  [5, 6]. The diffusion current density is given by Equation (3): With a type II-b InAs/InAs 1-x Sb x band offset  [25]  where electrons are confined in the binary layer (InAs) while holes are confined in the alloy one (InAsSb), the quantized miniband energies of the strain balanced InAs/InAs 1-x Sb x T2SL have been calculated with nextnano 3 commercial software  [17].
12 nm, that is to say, less than one monolayer in the case of Sb-based materials. PL measurements were performed from 50 to 250 K. PL spectra, presented in Figure  6 , display a shift in PL peak from 4. 9 µ m to 5 µ m in the temperature range (77-150 K). Such a trend strengthens the choice of the InAs/InAsSb T2SL period and antimony composition suitable for addressing MWIR broadband domain. At 77 K, the full width at half maximum (FWHM) is equal to 36 meV, higher than the one reported by E. H. Steenbergen et al. at around 30 meV  [26]. Using data from Figure  6a , the bandgap energy Eg(T), given by Equation (  5 ), was plotted as a function of temperature (Figure  6b ). Eg(T) = Epeak(T) -KBT/2 (5) The continuous red curve in Figure  6b  represents the Einstein Oscillator model used by Webster et al.
Background: Positional weight matrix (PWM) is a de facto standard model to describe transcription factor (TF) DNA binding specificities. PWMs inferred from in vivo or in vitro data are stored in many databases and used in a plethora of biological applications. This calls for comprehensive benchmarking of public PWM models with large experimental reference sets. Results: Here we report results from all-against-all benchmarking of PWM models for DNA binding sites of human TFs on a large compilation of in vitro (HT-SELEX, PBM) and in vivo (ChIP-seq) binding data. We observe that the best performing PWM for a given TF often belongs to another TF, usually from the same family. Occasionally, binding specificity is correlated with the structural class of the DNA binding domain, indicated by good cross-family performance measures.
Overview of current TF motif databases Starting with the first experiments characterizing TF specificity by low-throughput studies, numerous attempts were initiated to aggregate and systematize the resulting data, mostly, by collecting and aligning the sequences of the binding sites, producing the PWM models, and providing them in the form of motif collections. One of the first and most widely known databases was TRANSFAC  [6] , which probably still remains the largest, although proprietary, collection of PWMs from curated low-throughput data related to mammalian TFs. Open-access databases such as JASPAR  [7]  had notably less volume until the rise of high-throughput data, which significantly reduced the need for literature mining but put an emphasis on the curation of the PWMs and application of more sophisticated motif discovery tools.
Nowadays, most of the major motif collections, from species-centric (e. g. , FlyFactorSurvey  [8] , YeTFaSCo  [9] , Plant Cistrome Database  [10] , HOCOMOCO  [11] ), or technology-specific (e. g. , UniPROBE  [12] ) to all-inclusive resources (e. g. , CIS-BP  [13] , FootprintDB  [14] , iRegulon  [15] , and HOMER  [16] ), all use one or another form of the PWM as the primary (and most often the only) motif representation.
Novelty and objectives of this study A limitation of all previous studies was that TF binding motifs were only benchmarked against experimental data for the same TF (the TF to which the PWM is assigned in the motif database). This approach ignores the fact that TFs from the same family often have identical or very similar binding specificities, and thus, as a consequence, many different PWMs may predict binding sites to multiple factors very well. Moreover, some motifs from public databases may represent the binding specificity of multimeric complexes involving other factors, and the identity of these other factors may be unknown, or not explicitly indicated by the authors of the PWM resource. For these and other reasons, we advocate an all-against-all (TF-versus-experiment) benchmarking approach and, for the first time, exemplify the merits of such an approach with results from a large-scale benchmarking study.
The potential benefits of all-against-all benchmarking are manifold, including: Identification of best performing PWMs for each factor, irrespective of PWM annotation in the source database Addressing the question of whether closely related members of the same TF structural family share the same binding specificity. This question can only be answered correctly if the motif comparison is coupled to the performance evaluation because otherwise, a difference in the motif accuracy (quality) could be misinterpreted as a difference in binding specificity. Coping with redundancy. Thousands of PWMs for human TFs are nowadays available from public databases. Eliminating suboptimal PWMs reduces this number to a few hundred while substantially increasing the average performance of the remaining matrices.
Only peak lists containing at least 5000 peaks were considered. The benchmarking with SELEX data was done twice, with top-score cut-offs of 10% and 50%, in order to account for different degrees in true binding site enrichment of individual SELEX libraries. In total, this study produced a database of over 18 million performance values for experiment-PWM combinations, which we offer as a public resource. All results presented further below are based on this resource.
Benchmarking reveals similar binding specificity across many but not all TF structural families The ability of a motif to recognize peaks from a ChIPseq experiment targeted at another TF of the same structural TF family is a long-disputed issue  [28]. To quantitatively assess this family-related cross-binding, we grouped the PWMs with regard to the structural class of the respective TF's DNA binding domain as recorded in the CIS-BP  [13]  and TFClass databases  [29]. CIS-BP provides a more compact view of the TF families, while TFClass allows separating particular subfamilies, which are of interest in huge and diverse families such as zinc finger TFs. ReMap ChIP-Seq peak lists included only human TFs data, and HT-SELEX and PBM included human and mouse data to increase coverage of TF families.
In all cases, there is a clear signal at the diagonal showing that PWMs present in the databases indeed recognize TFBS of the corresponding TFs or at least of the TFs from the same CIS-BP family. Thus, on the global scale, the benchmark results agree with expectations based on DNA binding domain classification. Interestingly, there are families, e. g. , "C2H2 zinc finger factors," the TFs of which show almost negligible average family-wide AUC ROC on the corresponding data sets, both in vivo and in vitro, which agrees with their known diversity of DNA binding specificities. The results are consistent with the use of the 50% threshold in HT-SELEX benchmarks (Additional file 1: Figure  S2A ). The PBM-based heatmap (Fig. 2c ) is sparse due to lower representation of TF families in the available experiments.
The two number series are highly concordant with a correlation coefficient of 0. 78. The alternative approach to reduce redundancy is exemplified by the widely used methods for identifying representative motifs by clustering. Motif clustering is a  usual procedure in many applications. In motif activity response analysis  [31] , TFs with similar binding specificities bring about mathematical difficulties, so it is necessary to reduce the set of considered motifs to a subset of dissimilar motifs. As a result of motif clustering, a representative motif is often selected or constructed for a set of similar known motifs. Binding specificities of untested TFs can be predicted by the similarity of DNA binding domains  [32]. For motif clustering, we used a benchmark-blind approach similar to that used in  [33] , with PWMs from HOCOMOCO and JASPAR databases.
The qualifier "filtered" relates to the numbers obtained when we only considered experiments for which at least one matrix achieved a ROC AUC value > 0. 75 (ChIP-seq, HT-SELEX) or a Pearson correlation coefficient > 0. 35 (PBM). The first three bar plots show the numbers for individual motif collections analyzed separately, whereas the last plot at the bottom shows the numbers obtained when all three collections were considered simultaneously surprisingly not for ELK1, where it is outperformed by JASPAR MA0765. 1 assigned to ETV5 (AUC ROC 0. 9998). The extremely high-performance values for most HT-SELEX data sets suggest that the majority of ETS members have indistinguishable DNA binding specificity. Also with PBM data, this matrix shows good performance only for ETS family members.
It also performs well on HT-SELEX data for POU5F1 and on PBM data for SOX-related proteins. The matrix on the right side shows good performance on ChIP-seq data for general promoter-binding factors (NFY, SP1/2) and (with somewhat lower AUC ROC values) for Nanog itself. Unfortunately, the HT-SELEX and PBM collections do not include data for Nanog. However, the fact that the top-ranked HT-SELEX experiment, and 4 out of 10 top-ranked PBM experiments are assigned to a TF from the same homeodomain subfamily as Nanog (NKrelated), speaks in favor of the hypothesis that this matrix represents the intrinsic DNA binding specificity of Nanog. In contrast, the majority of the top-ranked.
Discussion We have generated the first data set of all-against-all performance measures for three human PWM libraries and three reference experiment collections, one in vivo and two in vitro. The results presented here are only the first deliverables of an ongoing benchmarking initiative, which we plan to extend to more PWM resources and non-human model organisms in the near future. Nevertheless, clear trends have already become apparent, which have important implications for the research field. As suspected (but never formally proven by systematic testing), many human TFs have indistinguishable binding specificity, at least at the resolution of current experimental techniques. In fact, in a majority of cases, the best matrix for a given TF was not derived from experiments targeted at the same TF but very often from another member of the same family.
The performance measure returned is a central enrichment score for the read-density around top-scoring motif matches from a whole genome scan. This protocol has the principle advantage that it starts closer to the raw data, bypassing the peak finding step, which adds an additional source of variation influencing the performance score. As an alternative to our current peak-list-based method, we may in the future consider the motif-enrichment-based approach proposed in  [21] , which we find conceptually appealing. There is also room for improvement for the existing protocols. For the ChIP-seq peak-list-based method, there may be better choices for the negative control set. For instance, one could try to match positive and negative genomic regions in terms of properties such as chromatin accessibility or nucleosome occupancy (if data are available for the corresponding type).
For HT-SELEX data, an automatic and objective way of determining the optimal top score threshold would be desirable, to make results from libraries with varying levels of binding site enrichment more comparable. A potential criticism of our study design relates to potential circularity and over-fitting effects arising when a PWM is benchmarked on the data, from which it was originally derived. Unfortunately, the matrices from public databases are not necessarily linked to source data. Even CIS-BP matrices annotated as being derived from HT-SELEX or PBM cannot easily be traced to individual experiments when multiple experiments have been carried out for the same TF. Thus, we cannot know which performance values may be inflated by over-fitting effects. However, we think this problem is not so critical for this study.
For most TFs, there are multiple test sets in our collection. Users of our benchmarking results can thus assess the quality of a given matrix based on the robustness of its performance across multiple data sets. Furthermore, the majority of best performing matrices for a given experiment are assigned to a different TF in the source databases. In all these cases, the best performing matrices were obviously derived from a different experiment, indicating that over-fitting is not a major source of bias in this study. The benchmarking of the representative motifs obtained from benchmarking-blind motif clustering underlines the risks and challenges of reducing the PWM library complexity by straightforward clustering PWMs by similarity and selecting the representative members as cluster centers.
Conclusions We have carried out a comprehensive all-against-all PWM-to-experiment benchmarking study, resulting in the computation of more than 18 million performance measure values. We make these numbers available as a public resource, in a format that can easily be imported into statistical analysis software, such as the R environment, for further exploratory analysis. We have shown by selected examples and statistical plots that our resource is useful for selecting an optimal matrix for a particular purpose, as well as interpreting motif matches reported by a DNA sequence analysis program. The PWM performance spectrum for a given ChIP-seq experiment sheds light on the various TF-to-target site recruitment processes active in a particular cell type under particular conditions. We consider extending this resource in the future but are already confident that it will be of great value in its present state.
Origin and preprocessing of PWM libraries Base probability matrices from JASPAR Core Vertebrate (2018) and HOCOMOCO Human (v11 FULL) were downloaded in MEME database format from the MEME suite web server. CIS-BP matrices were directly downloaded from the original web site. The CIS-BP collection used in this work contains all matrices directly inferred from experiments with human TFs except those matrices imported from JASPAR and HOCOMOCO. Matrices were manually mapped to gene symbols and TF families from TFclass  [29]  and CIS-BP. The annotations of matrices (including basic features such as length and GC content) and respective TFs are provided in Additional file 6. The original matrices were slightly modified by adding a correction term of 0. 0001 to each matrix element, followed by renormalization of the position-specific probability distributions.
Benchmarking with HT-SELEX data Library sequences described in  [26]  and  [27]  were downloaded from the European Nucleotide Archive ENA  [39]  in FASTQ format. The source files contain the DNA sequences of the random inserts without barcodes or primers. FASTQ files were converted into FASTA format. Only sequences exclusively composed of A, C, G, and T and having the length indicated by the library name were retained. Note that the sequences from the two studies were derived from the same series of experiments. Libraries representing different cycles from the same experiment were pooled. Duplicates were then eliminated from the pooled libraries (assuming that they were PCR copies from the same founder molecules). A random subset of one million sequences was extracted from libraries containing more than a million sequences, in order to reduce benchmarking computing time.
As an alternative to input (zero-cycle) sequences, we also generated negative control sequences by mononucleotide shuffling of the inserts but leaving the primer and barcode sequences unchanged (see below). We extended the random insert sequences provided in the source files with sequences that were physically present during the binding experiments. According to  [26] , the 5′ and 3′ flanking primer sequences were as follows: 5′ TCCATCACGAATGATACGGCGACCACCGAA CACTCTTTCCCTACACGACGCTCTTCCGATC 3′ ATCGTATGCCGTCTTCTGCTTGCCGACTCCG The barcodes vary from experiment to experiment. After flanking with barcode and primers on both sides, sequences were truncated to include only the 20 bp adjacent to the random insert on each flank. For instance, random insert sequences from experiment ELK3_ TCGGGG20NGGT_AG (barcodes TCCGGGG and GGT, as indicated by the name) were extended in the following way: ACGCTCTTCCGATCTCGGGGNNNNNNNNNN NNNNNNNNNNGGTATCGTATGCCGTCTTCT As the source files containing the ReMap peak lists and SELEX libraries have informative and intuitively understandable names, they were thus used as experiment identifiers in this article.
Benchmarking with PBM data Protein-binding microarray (PBM) data for human and mouse were downloaded from UniPROBE (Hume et al. [12] ) as "Normalized Probe Data. " The files for each experiment contain one column with normalized intensities per probe and the actual probe sequence including a fixed linker sequence. To obtain correlation values for an input PWM (obtained from JASPAR, HOCOMOCO, or CIS-BP and processed as described previously) with respect to measured log intensities, the following procedure was adhered to: For each probe sequence, the first 41 base pairs were extracted, excluding part of the fixed linker sequence. Within this sequence, the PWM was applied in a sliding window approach to obtain perposition probabilities, which were aggregated into a log sum occupancy score, i.
Benchmarking protocol availability The ChIP-seq and HT-SELEX-based benchmarking protocols are publicly available via a web interface at https://ccg. epfl. ch/pwmtools/. The ChIP-seq, HT-SELEX-based, and PBM-based protocols are available as docker images from https://github. com/autosome-ru/motif_benchmarks [40]. Aggregate rank score and identification of the best performing matrix for a gene All rows corresponding to a given gene were extracted from a complete table containing performance measures (ROC AUC or correlation coefficients) for combinations of experiments (rows) and TF motif matrices (columns). The numbers in a given row were first converted into ranks. The overall performance of a matrix for multiple experiments for the same gene was then computed as the geometric mean over the ranks.
Analysis of the association between the basic motif features and the performance values Each point underlying the density plot corresponds to a single PWM of a particular TF. To normalize for different numbers of data sets per TF, the AUC ROC (for ChIP-seq and SELEX) and correlation (for PBM) values were calculated by averaging the corresponding values over all data sets for the TF corresponding to the PWM under consideration. The Python seaborn package was used for visualization. The complete benchmarking results are available from https://github. com/autosome-ru/motif_benchmarking_ data  [43].
Availability of data and materials JASPAR matrices: http://jaspar. genereg. net HOCOMOCO matrices: http://hocomoco11. autosome. ru/ CIS-BP matrices: http://cisbp. ccbr. utoronto. ca/ ReMap peak lists: http://remap. cisreg. eu HT-SELEX data: https://www. ebi. ac. uk/ena/data/view/PRJEB3289 (+PRJEB14744) PBM data: http://thebrain. bwh. harvard. edu/uniprobe/ Docker implementation of benchmarking protocols: https://github. com/ autosome-ru/motif_benchmarks  [40]. All-against-all benchmarking results: https://github. com/autosome-ru/motif_ benchmarking_data  [43].
A standardized classification of SVD in the aortic position has recently been proposed. 9, 51, 52 The reasons for valve replacement and the type of intrinsic structural valve anomalies (ie, calcification, pannus formation, tears, perforations) were specified by the surgeon at the time of surgery and the data prospectively collected. Some cases of early failure or intermediate longevity in the new generation of bovine bioprostheses were explanted during this period (1985-1998) and analyzed separately.
Data Collection A prospective database for bioprosthetic heart valves has been in place since 1985 and has identified several risks for SVD. The type of SVD was reported for 50 of 963 explanted bioprostheses. Patient blood group information was obtained retrospectively from the blood bank (66%) or patients' records (33%). If the information was not available in the blood bank, patients' medical records were consulted for patients with the highest (≥13 years) and lowest longevity (≤7 years). Since 1998, all PBs with a longevity of ≥13 years have been systematically sent to the laboratory for further analysis. This enabled us to go back to a patient's name and chart. Since 1985, the new generations of BBs have almost completely replaced PBs, so that for the year 2014, we have at least 29 years of follow-up for PBs.
Study variables The main outcome variable was the interval between valve implantation and explantation (longevity) and the type of SVD. The main risk factor of interest was the patient's blood type (ABO and rhesus) and was retrospectively collected. Other known classical risk factors for structural degeneration of bioprostheses, and for which data were prospectively collected at replacement, were as follows: patient's age at the moment of implantation, sex, valve location and number of bioprostheses implanted initially. Some additional risk factors in bioprosthesis degeneration, especially in the aortic position, have also been reported recently and include factors that increase hemodynamic stress (larger body surface area, small prosthesis size, prosthesis-patient mismatch, left ventricle hypertrophy) and cardiovascular risk factors such as smoking, hypertension, metabolic syndrome, diabetes mellitus, dyslipidemia  9 52.
Data collection A prospective database for bioprosthetic heart valves has been developed since 1985. Thus, from 1985 to 1998, data was collected prospectively and valve information recorded in the operating room. Data on the type of bioprosthesis alteration was prospectively reported by the surgeon in the operating room, so that we now have information for the main criteria (pannus/calcification/tears/perforation) for most degenerative bioprostheses (913 out of 963). Other additional information collected included factors contributing to bioprosthesis replacement (e. g. thrombosis, endocarditis, non-dysfunction valve), date of implantation, longevity, site of implantation and number, origin (bovine or porcine) and type of bioprosthesis. Information on the patient's blood group was obtained from the blood bank (2/3 of cases) and patients' records (1/3 of cases).