nils-herrmann/clean_scirepeval huggingface dataset:
The Principled Design of Large-Scale Recursive Neural Network Architectures--DAG-RNNs and the Protein Structure Prediction Problem

We describe a general methodology for the design of large-scale recursive neural network architectures (DAG-RNNs) which comprises three fundamental steps: (1) representation of a given domain using suitable directed acyclic graphs (DAGs) to connect visible and hidden node variables; (2) parameterization of the relationship between each variable and its parent variables by feedforward neural networks; and (3) application of weight-sharing within appropriate subsets of DAG connections to capture stationarity and control model complexity. Here we use these principles to derive several specific classes of DAG-RNN architectures based on lattices, trees, and other structured graphs. These architectures can process a wide range of data structures with variable sizes and dimensions. While the overall resulting models remain probabilistic, the internal deterministic dynamics allows efficient propagation of information, as well as training by gradient descent, in order to tackle large-scale problems. These methods are used here to derive state-of-the-art predictors for protein structural features such as secondary structure (1D) and both fine- and coarse-grained contact maps (2D). Extensions, relationships to graphical models, and implications for the design of neural architectures are briefly discussed. The protein prediction servers are available over the Web at: www.igb.uci.edu/tools.htm .
Differential analysis of DNA microarray gene expression data

Here, we review briefly the sources of experimental and biological variance that affect the interpretation of high‐dimensional DNA microarray experiments. We discuss methods using a regularized t‐test based on a Bayesian statistical framework that allow the identification of differentially regulated genes with a higher level of confidence than a simple t‐test when only a few experimental replicates are available. We also describe a computational method for calculating the global false‐positive and false‐negative levels inherent in a DNA microarray data set. This method provides a probability of differential expression for each gene based on experiment‐wide false‐positive and ‐negative levels driven by experimental error and biological variance.
Simple jury predicts protein secondary structure best

. Affiliations 1 CUBIC, Columbia Univ., Dept. of Biochemistry and Molecular Biophysics, 650 West 168th Street, New York, NY 10032, USA 2 Univ. of California, Irvine, Dept. of Information and Computer Science, Institute of Genomics and Bioinformatics, Irvine, CA92697, USA 3 European Bioinformatics Institute, Genome Campus, Hinxton, Cambs CB10 1SD, England 4 The Sanger Centre, Wellcome Trust Genome Campus, Hinxton, Cambs CB10 1SA, England 5 Columbia Univ., Dept. of Chemistry, 3000 Broadway MC 3167, New York, NY 10027, USA 6 Dept. of Biological Sciences,  274348, Uxbridge, Middlesex UB8 3PH, England 7 Computer Engineering, Univ. of California, Santa Cruz, Santa Cruz, CA 95064, USA 8 The Univ. of Wales Aberystwyth, Dept. of Computer Science, Penglais, Ceredigion, SY23 3DB, Wales, UK * Corresponding author: rost@columbia.edu, http://cubic.bioc.columbia.edu/
Global Gene Expression Profiling in Escherichia coli K12

The work presented here is a first step toward a long term goal of systems biology, the complete elucidation of the gene regulatory networks of a living organism. To this end, we have employed DNA microarray technology to identify genes involved in the regulatory networks that facilitate the transition of Escherichia coli cells from an aerobic to an anaerobic growth state. We also report the identification of a subset of these genes that are regulated by a global regulatory protein for anaerobic metabolism, FNR. Analysis of these data demonstrated that the expression of over one-third of the genes expressed during growth under aerobic conditions are altered when E. coli cells transition to an anaerobic growth state, and that the expression of 712 (49%) of these genes are either directly or indirectly modulated by FNR. The results presented here also suggest interactions between the FNR and the leucine-responsive regulatory protein (Lrp) regulatory networks. Because computational methods to analyze and interpret high dimensional DNA microarray data are still at an early stage, and because basic issues of data analysis are still being sorted out, much of the emphasis of this work is directed toward the development of methods to identify differentially expressed genes with a high level of confidence. In particular, we describe an approach for identifying gene expression patterns (clusters) obtained from multiple perturbation experiments based on a subset of genes that exhibit high probability for differential expression values.
LineUp: statistical detection of chromosomal homology with application to plant comparative genomics.

The identification of homologous regions between chromosomes forms the basis for studies of genome organization, comparative genomics, and evolutionary genomics. Identification of these regions can be based on either synteny or colinearity, but there are few methods to test statistically for significant evidence of homology. In the present study, we improve a preexisting method that used colinearity as the basis for statistical tests. Improvements include computational efficiency and a relaxation of the colinearity assumption. Two algorithms perform the method: FullPermutation, which searches exhaustively for runs of markers, and FastRuns, which trades faster run times for exhaustive searches. The algorithms described here are available in the LineUp package (http://www.igb.uci.edu/ approximately baldig/lineup). We explore the performance of both algorithms on simulated data and also on genetic map data from maize (Zea mays ssp. mays). The method has reasonable power to detect a homologous region; for example, in >90% of simulations, both algorithms detect a homologous region of 10 markers buried in a random background, even when the homologous regions have diverged by numerous inversion events. The methods were applied to four maize molecular maps. All maps indicate that the maize genome contains extensive regions of genomic duplication and multiplication. Nonetheless, maps differ substantially in the location of homologous regions, probably reflecting the incomplete nature of genetic map data. The variation among maps has important implications for evolutionary inference from genetic map data.
Prediction of Protein Topologies Using GIOHMMs and GRNNs

We develop and test new machine learning methods for the prediction of topological representations of protein structures in the form of coarseor ne-grained contact or distance maps that are translation and rotation invariant. The methods are based on generalized input-output hidden Markov models (GIOHMMs) and generalized recursive neural networks (GRNNs). The methods are used to predict topology directly in the ne-grained case and, in the coarsegrained case, indirectly by rst learning how to score candidate graphs and then using the scoring function to search the space of possible con gurations. Computer simulations show that the predictors achieve state-of-the-art performance.
Appendix A: Mathematical Complements

A.1 Functions and Integration Analog signals are modeled by measurable functions. We first give the main theorems of Lebesgue integration. A function f is said to be integrable if ∫ +∞ −∞ |f(t)| dt < +∞. The space of integrable functions is written L(R). Two functions f1 and f2 are equal in L(R) if ∫ +∞ −∞ |f1(t)− f2(t)| dt = 0. This means that f1(t) and f2(t) can differ only on a set of points of measure 0. We say that they are almost everywhere equal. The Fatou lemma gives an inequality when taking a limit under the Lebesgue integral of positive functions. Lemma A.1 (Fatou). Let {fn}n∈N be a family of positive functions fn(t) ! 0. If limn→+∞ fn(t) = f(t) almost everywhere then ∫ +∞ −∞ f(t) dt " lim n→+∞ ∫ +∞ −∞ fn(t) dt.
Modeling the Internet and the Web: Probabilistic Method and Algorithms

1.1 Probability and Learning from a Bayesian Perspective. 1.2 Parameter Estimation from Data. 1.3 Mixture Models and the Expectation Maximization Algorithm.    Exploiting Hyperlinks. 4.8 Document Clustering. 4.9 Information Extraction. 4.10 Exercises. 5 Link Analysis. 5.1 Early Approaches to Link Analysis. 5.2 Nonnegative Matrices and Dominant Eigenvectors.  7.2 Web Data and Measurement Issues. 7.3 Empirical Client--Side Studies of Browsing Behavior. 7.4 Probabilistic Models of Browsing Behavior.  Modeling and Understanding Search Engine Querying. 7.6 Exercises. 8 Commerce on the Web: Models and Applications.  8.2 Customer Data on theWeb. 8.3 Automated Recommender Systems. 8.4 Networks and Recommendations. 8.5 Web Path Analysis for Purchase Prediction. 8.6 Exercises.
A Machine-Learning Strategy for Protein Analysis

or function (or both) is largely unknown. Proteomics is the protein counterpart to genomics, the large-scale analysis of complete genomes. Proteomes contain a cell’s total protein expression at a given time. Proteome analysis not only deals with determining protein-encoding genes’ sequence and function, but is also strongly concerned with the precise biochemical state of each protein in its posttranslational form—that is, the form it takes after it has been translated from its original DNA encoding. Traditional experimental techniques for determining a protein’s structure and function, such as x-ray diffraction or Nuclear Magnetic Resonance methods, remain slow and laborious, and do not scale up to current sequencing speeds. Furthermore, using experiments to determine how proteins function is a daunting task: Protein interactions are complex, and their native operating environments are very specific, which can be difficult to replicate in the laboratory. Researchers are developing many new highthroughput experimental techniques for proteomics applications, including mass spectrometry and protein chips. Still, given proteins’ fundamental importance to biology, biotechnology, and medicine, we must continue developing computer methods that can rapidly sift through massive amounts of data and help determine the structure and function of all the proteins in a given genome. We’re applying machine-learning methods to proteomic problems, and have developed a novel strategy for completely predicting protein 3D coordinates. The strategy has three stages: structural features prediction, topology prediction, and coordinate prediction. Here, we offer an overview of the domain and our machine-learning techniques, and describe the software suite we’ve developed, which is available at http://promoter.ics.uci.edu/BRNN-PRED/.
Modeling and optimization of UWB communication networks through a flexible cost function

The traditional design of communication networks has rarely been able to focus on the optimization of global network properties. Ultra-wideband (UWB) radio is emerging as an attractive physical layer for wireless communication networks offering new opportunities for the principled design and optimization of network properties. We develop a framework for the principled design of UWB wireless networks based on a flexible cost function that can be tailored and scaled to a wide range of networks and applications, ranging from sensor networks to voice and data wireless networks. The function comprises cost terms associated with transmission, connection setup, interference, and quality-of-service. Multihop routing strategies are associated with admissible paths of minimal cost that are computable in linear time. The cost function together with the overall level of requests determine the dynamics of the connections and the equilibrium topology of the network. We report simulation results in the case of simple ring and square lattice networks.
Improving the prediction of protein secondary structure in three and eight classes using recurrent neural networks and profiles

Secondary structure predictions are increasingly becoming the workhorse for several methods aiming at predicting protein structure and function. Here we use ensembles of bidirectional recurrent neural network architectures, PSI‐BLAST‐derived profiles, and a large nonredundant training set to derive two new predictors: (a) the second version of the SSpro program for secondary structure classification into three categories and (b) the first version of the SSpro8 program for secondary structure classification into the eight classes produced by the DSSP program. We describe the results of three different test sets on which SSpro achieved a sustained performance of about 78% correct prediction. We report confusion matrices, compare PSI‐BLAST to BLAST‐derived profiles, and assess the corresponding performance improvements. SSpro and SSpro8 are implemented as web servers, available together with other structural feature predictors at: http://promoter.ics.uci.edu/BRNN‐PRED/. Proteins 2002;47:228–235.
Global gene expression profiling in Escherichia coli K12. The effects of leucine-responsive regulatory protein.

Leucine-responsive regulatory protein (Lrp) is a global regulatory protein that affects the expression of multiple genes and operons in bacteria. Although the physiological purpose of Lrp-mediated gene regulation remains unclear, it has been suggested that it functions to coordinate cellular metabolism with the nutritional state of the environment. The results of gene expression profiles between otherwise isogenic lrp(+) and lrp(-) strains of Escherichia coli support this suggestion. The newly discovered Lrp-regulated genes reported here are involved either in small molecule or macromolecule synthesis or degradation, or in small molecule transport and environmental stress responses. Although many of these regulatory effects are direct, others are indirect consequences of Lrp-mediated changes in the expression levels of other global regulatory proteins. Because computational methods to analyze and interpret high dimensional DNA microarray data are still an early stage, much of the emphasis of this work is directed toward the development of methods to identify differentially expressed genes with a high level of confidence. In particular, we describe a Bayesian statistical framework for a posterior estimate of the standard deviation of gene measurements based on a limited number of replications. We also describe an algorithm to compute a posterior estimate of differential expression for each gene based on the experiment-wide global false positive and false negative level for a DNA microarray data set. This allows the experimenter to compute posterior probabilities of differential expression for each individual differential gene expression measurement.
Prediction of contact maps by GIOHMMs and recurrent neural networks using lateral propagation from all four cardinal corners

Accurate prediction of protein contact maps is an important step in computational structural proteomics. Because contact maps provide a translation and rotation invariant topological representation of a protein, they can be used as a fundamental intermediary step in protein structure prediction.



We develop a new set of flexible machine learning architectures for the prediction of contact maps, as well as other information processing and pattern recognition tasks. The architectures can be viewed as recurrent neural network implemantations of a class of Bayesian networks we call generalized input-output HMMs (GIOHMMs). For the specific case of contact maps, contextual information is propagated laterally through four hidden planes, one for each cardinal corner. We show that these architectures can be trained from examples and yield contact map predictors that outperform previously reported methods. While several extensions and improvements are in progress, the current version can accurately predict 60.5% of contacts at a distance cutoff of 8 A and 45% of distant contacts at 10 A, for proteins of length up to 300.
Distribution patterns of over-represented k-mers in non-coding yeast DNA

Over-represented k-mers in genomic DNA regions are often of particular biological interest. For example, over-represented k-mers in co-regulated families of genes are associated with the DNA binding sites of transcription factors. To measure over-representation, we introduce a statistical background model based on single-mismatches, and apply it to the pooled 500 bp ORF Upstream Regions (USRs) of yeast. More importantly, we investigate the context and spatial distribution of over-represented k-mers in yeast USRs.



Single and double-stranded spatial distributions of most over-represented k-mers are highly non-random, and predominantly cluster into a small number of classes that are robust with respect to over-representation measures. Specifically, we show that the three most common distribution patterns can be related to DNA structure, function, and evolution and correspond to: (a) homologous ORF clusters associated with sharply localized distributions; (b) regulatory elements associated with a symmetric broad hill-shaped distribution in the 50-200 bp USR; and (c) runs of As, Ts, and ATs associated with a broad hill-shaped distribution also in the 50-200 bp USR, with extreme structural properties. Analysis of over-representation, homology, localization, and DNA structure are essential components of a general data-mining approach to finding biologically important k-mers in raw genomic DNA and understanding the 'lexicon' of regulatory regions.
Prediction of Protein Topologies Using Generalized IOHMMS and RNNs

We develop and test new machine learning methods for the prediction of topological representations of protein structures in the form of coarse- or fine-grained contact or distance maps that are translation and rotation invariant. The methods are based on generalized input-output hidden Markov models (GIOHMMs) and generalized recursive neural networks (GRNNs). The methods are used to predict topology directly in the fine-grained case and, in the coarse-grained case, indirectly by first learning how to score candidate graphs and then using the scoring function to search the space of possible configurations. Computer simulations show that the predictors achieve state-of-the-art performance.
DNA Microarrays and Gene Expression - From Experiments to Data Analysis and Modeling

Preface 1. A brief history of genomics 2. DNA array formats 3. DNA array readout methods 4. Gene expression profiling experiments: problems, pitfalls and solutions 5. Statistical analysis of array data: inferring changes 6. Statistical analysis of array data: dimensionality reduction, clustering, and regulatory regions 7. Survey of current DNA array applications 8. Systems biology: overview of regulatory, metabolic and signaling networks.
New Machine Learning Methods for the Prediction of Protein Topologies

Protein structures are translation and rotation invariant. In protein structure prediction, it is therefore important to be able to assess and predict intermediary topological representations, such as distance or contact maps, that are translation and rotation invariant. Here we develop several new machine learning methods for the prediction and assessment of fine-grained and coarse topological representations of proteins. In particular, we introduce a general class of graphical model architectures together with the corresponding neural network implementations. These architectures can be viewed as Bayesian network generalizations of input-output hidden Markov models (GIOHMMs), involving an input layer, an output layer, and a hidden layer supported by one or several directed acyclic graphs. The corresponding generalized recursive neural network (GRNN) architectures are derived by preserving the graphical structures of the GIOHMMs, but replacing the conditional probability tables with learnable deterministic functions. Two methods are proposed for the prediction of protein topological structures. The first method uses a GIOHMM organized into six horizontal layers: one input plane, four hidden planes, and one output plane that directly represents the adjacency matrix of the contact map (or the distance matrix). Each hidden plane is associated with one of the four cardinal corners towards which all the edges of the corresponding lattice are oriented. The corresponding GRNNs are used to construct a fine-grained contact map predictor. The second method uses a GIOHMM approach to learn a graph scoring function which, in turn, is used to efficiently search the space of possible configurations. The corresponding GRNNs are used to construct a coarse-grained contact map predictor. Computer simulations show that the predictors for both tasks achieve stateof-the-art performance.
A Computational Theory of Surprise

While eminently successful for the transmission of data, Shannon’s theory of information does not address semantic and subjective dimensions of data, such as relevance and surprise. We propose an observer-dependent computational theory of surprise where surprise is defined by the relative entropy between the prior and the posterior distributions of an observer. Surprise requires integration over the space of models in contrast with Shannon’s entropy, which requires integration over the space of data. We show how surprise can be computed exactly in a number of discrete and continuous cases using distributions from the exponential family with conjugate priors. We show that during sequential Bayesian learning, surprise decreases like 1/N and study how surprise differs and complements Shannon’s definition of information.
Why are complementary DNA strands symmetric?

Over sufficiently long windows, complementary strands of DNA tend to have the same base composition. A few reports have indicated that this first-order parity rule extends at higher orders to oligonucleotide composition, at least in some organisms or taxa. However, the scientific literature falls short of providing a comprehensive study of reverse-complement symmetry at multiple orders and across the kingdom of life. It also lacks a characterization of this symmetry and a convincing explanation or clarification of its origin.



We develop methods to measure and characterize symmetry at multiple orders, and analyze a wide set of genomes, encompassing single- and double-stranded RNA and DNA viruses, bacteria, archae, mitochondria, and eukaryota. We quantify symmetry at orders 1 to 9 for contiguous sequences and pools of coding and non-coding upstream regions, compare the observed symmetry levels to those predicted by simple statistical models, and factor out the effect of lower-order distributions. We establish the universality and variability range of first-order strand symmetry, as well as of its higher-order extensions, and demonstrate the existence of genuine high-order symmetric constraints. We show that ubiquitous reverse-complement symmetry does not result from a single cause, such as point mutation or recombination, but rather emerges from the combined effects of a wide spectrum of mechanisms operating at multiple orders and length scales.