nils-herrmann/clean_scirepeval huggingface dataset:
On the convergence of a clustering algorithm for protein-coding regions in microbial genomes

As the number of fully sequenced prokaryotic genomes continues to grow rapidly, computational methods for reliably detecting protein-coding regions become even more important. Audic and Claverie (1998) 10031, have proposed a clustering algorithm for protein-coding regions in microbial genomes. The algorithm is based on three Markov models of order k associated with subsequences extracted from a given genome. The parameters of the three Markov models are recursively updated by the algorithm which, in simulations, always appear to converge to a unique stable partition of the genome. The partition corresponds to three kinds of regions: (1) coding on the direct strand, (2) coding on the complementary strand, (3) non-coding.



Here we provide an explanation for the convergence of the algorithm by observing that it is essentially a form of the expectation maximization (EM) algorithm applied to the corresponding mixture model. We also provide a partial justification for the uniqueness of the partition based on identifiability. Other possible variations and improvements are briefly discussed.
Matching protein beta-sheet partners by feedforward and recurrent neural networks.

Predicting the secondary structure (alpha-helices, beta-sheets, coils) of proteins is an important step towards understanding their three dimensional conformations. Unlike alpha-helices that are built up from one contiguous region of the polypeptide chain, beta-sheets are more complex resulting from a combination of two or more disjoint regions. The exact nature of these long distance interactions remains unclear. Here we introduce two neural-network based methods for the prediction of amino acid partners in parallel as well as anti-parallel beta-sheets. The neural architectures predict whether two residues located at the center of two distant windows are paired or not in a beta-sheet structure. Variations on these architecture, including also profiles and ensembles, are trained and tested via five-fold cross validation using a large corpus of curated data. Prediction on both coupled and non-coupled residues currently approaches 84% accuracy, better than any previously reported method.
Matching Protein b-Sheet Partners by Feedforward and Recurrent Neural Networks

Predicting the secondary structure (a-helices, j3sheets, coils) of proteins is an important step towards understanding their three dimensional conformations. Unlike a-helices that are built up from one contiguous region of the polypeptide chain, r-sheets are more complex resulting from a combination of two or more disjoint regions. The exact nature of these long distance interactions remains unclear. Here we introduce two neural-network based methods for the prediction of amino acid partners in parallel as well as antiparallel j3-sheets. The neural architectures predict whether two residues located at the center of two distant windows are paired or not in a r-sheet structure. Variations on these architecture, including also profiles and ensembles, are trained and tested via five-fold cross validation using a large corpus of curated data. Prediction on both coupled and non-coupled residues currently approaches 84% accuracy, better than any previously reported method.
Analysis of Yeast's ORF Upstream Regions by Parallel Processing, Microarrays, and Computational Methods

We use a network of workstations to compute all pairwise alignments of the 500 bp upstream regions of 6,225 yeast ORFs (Open Reading Frames). We correlate the alignments with DNA microarray expression data f~om budding yeast cells over an oxidative stress time course. We confirm on a genomic scale that, in general, genes with extremely similar upstream regions have similar activity levels, even when located on different chromosomes. As the difference in upstream regions increases, the correlation rapidly drops towards zero. Divergent ORFs with overlapping upstream regions do not seem to be correlated in any way. The pairwise alignments coupled with the expression data, together with other computational techniques, suggest a few new putative regulatory binding sites that can be tested experimentally. Finally, we investigate the inherent symmetry present in the two strands of the yeast genome. We show that it extends at least all the way up to 9-mers and is likely to result from different evolutionary pressures operating at different length scales. * and Depaxtment of Biological Chemistry, College of Medicine, University of California, Irvine. To whom all correspondence should be addressed. 190 HAMPSON
Protein β-Sheet Partner Prediction by Neural Networks

Predicting the secondary structure (α-helices, β-sheets, coils) of proteins is an important step towards understanding their three dimensional conformations. Unlike α-helices that are built up from one contiguous region of the polypeptide chain, β-sheets are more complex resulting from a combination several disjoint regions. The exact nature of these long distance interactions remains unclear. Here we introduce a neural-network based method for the prediction of amino acid partners in parallel as well as anti-parallel β-sheets. The neural architecture predicts whether two residues located at the center of two distant windows are paired or not in a β-sheet structure. The distance between the windows is a third essential input into the architecture. Variations on this architecture are trained using a large corpus of curated data. Prediction on both coupled and non-coupled residues currently exceeds 83% accuracy, well above any previously reported method. Unlike standard secondary structure prediction methods, the use of multiple alignment (profiles) in our case seems to degrade the performance, probably as a result of intra-chain correlation effects.
Bidirectional IOHMMs and Recurrent Neural Networks for Protein Secondary Structure Prediction

Prediction of protein secondary structure (SS) is one of the classical problems in bioinformatics that are best solved using computational prediction methods based on machine learning. Current state-of-the-art predictors are based on feedforward artificial neural networks fed by a fixed-width window of amino acids, centered on the predicted residue. Using a fixed-width small window offers the advantage of architectural simplicity and allows controlling parameter overfitting. On the other hand, relevant information is also contained in distant portions of the proteins and current methods cannot exploit this information. In this chapter, we describe two alternative architectures based on noncausal (bidirectional) dynamics. These architectures can be seen as generalizations of input-output hidden Markov models or recurrent neural networks. Unlike their conventional counterparts, their outputs depend on both upstream and downstream information. This novel algorithmic idea is a first step towards architectures capable of making predictions based on variable ranges of dependencies.
Software Foundation Libraries for Intelligent Systems

We describe a set of object-oriented software foundation libraries for the design and modeling of complex and intelligent systems. The basic thesis underlying the libraries is that the language required to represent and design all such systems in a scalable way is the language of sparse recursive graphs with the corresponding local message passing propagation algorithms. We provide some formal arguments to support this thesis going from decision theory, to Bayesian statistics, to probabilistic graphical models. The current version of the libraries consist of 6 different modules: ObjectCost, ObjectStat, ObjectNet, ObjectComp, ObjectData, and ObjectGraph. The libraries support most of the standard models used in artificial intelligence and machine learning, such as neural networks, hidden Markov models, and Bayesian networks. The libraries are being used in real data mining applications and products, and are actively being developed and extended.
Exploiting the past and the future in protein secondary structure prediction

Predicting the secondary structure of a protein (alpha-helix, beta-sheet, coil) is an important step towards elucidating its three-dimensional structure, as well as its function. Presently, the best predictors are based on machine learning approaches, in particular neural network architectures with a fixed, and relatively short, input window of amino acids, centered at the prediction site. Although a fixed small window avoids overfitting problems, it does not permit capturing variable long-rang information.



We introduce a family of novel architectures which can learn to make predictions based on variable ranges of dependencies. These architectures extend recurrent neural networks, introducing non-causal bidirectional dynamics to capture both upstream and downstream information. The prediction algorithm is completed by the use of mixtures of estimators that leverage evolutionary information, expressed in terms of multiple alignments, both at the input and output levels. While our system currently achieves an overall performance close to 76% correct prediction--at least comparable to the best existing systems--the main emphasis here is on the development of new algorithmic ideas.


AVAILABILITY
The executable program for predicting protein secondary structure is available from the authors free of charge.
Structural basis for triplet repeat disorders: a computational analysis

Over a dozen major degenerative disorders, including myotonic distrophy, Huntington's disease and fragile X syndrome, result from unstable expansions of particular trinucleotides. Remarkably, only some of all the possible triplets, namely CAG/CTG, CGG/CCG and GAA/TTC, have been associated with the known pathological expansions. This raises some basic questions at the DNA level. Why do particular triplets seem to be singled out? What is the mechanism for their expansion and how does it depend on the triplet itself? Could other triplets or longer repeats be involved in other diseases?



Using several different computational models of DNA structure, we show that the triplets involved in the pathological repeats generally fall into extreme classes. Thus, CAG/CTG repeats are particularly flexible, whereas GCC, CGG and GAA repeats appear to display both flexible and rigid (but curved) characteristics depending on the method of analysis. The fact that (1) trinucleotide repeats often become increasingly unstable when they exceed a length of approximately 50 repeats, and (2) repeated 12-mers display a similar increase in instability above 13 repeats, together suggest that approximately 150 bp is a general threshold length for repeat instability. Since this is about the length of DNA wrapped up in a single nucleosome core particle, we speculate that chromatin structure may play an important role in the expansion mechanism. We furthermore suggest that expansion of a dodecamer repeat, which we predict to have very high flexibility, may play a role in the pathogenesis of the neurodegenerative disorder multiple system atrophy (MSA).



pfbaldi@ics.uci.edu, yves@netid.com, brunak@cbs.dtu.dk, gorm@cbs.dtu.dk.
DNA structure in human RNA polymerase II promoters.

The fact that DNA three-dimensional structure is important for transcriptional regulation begs the question of whether eukaryotic promoters contain general structural features independently of what genes they control. We present an analysis of a large set of human RNA polymerase II promoters with a very low level of sequence similarity. The sequences, which include both TATA-containing and TATA-less promoters, are aligned by hidden Markov models. Using three different models of sequence-derived DNA bendability, the aligned promoters display a common structural profile with bendability being low in a region upstream of the transcriptional start point and significantly higher downstream. Investigation of the sequence composition in the two regions shows that the bendability profile originates from the sequential structure of the DNA, rather than the general nucleotide composition. Several trinucleotides known to have high propensity for major groove compression are found much more frequently in the regions downstream of the transcriptional start point, while the upstream regions contain more low-bendability triplets. Within the region downstream of the start point, we observe a periodic pattern in sequence and bendability, which is in phase with the DNA helical pitch. The periodic bendability profile shows bending peaks roughly at every 10 bp with stronger bending at 20 bp intervals. These observations suggest that DNA in the region downstream of the transcriptional start point is able to wrap around protein in a manner reminiscent of DNA in a nucleosome. This notion is further supported by the finding that the periodic bendability is caused mainly by the complementary triplet pairs CAG/CTG and GGC/GCC, which previously have been found to correlate with nucleosome positioning. We present models where the high-bendability regions position nucleosomes at the downstream end of the transcriptional start point, and consider the possibility of interaction between histone-like TAFs and this area. We also propose the use of this structural signature in computational promoter-finding algorithms.
Computational Applications of DNA Structural Scales

We study from a computational standpoint several different physical scales associated with structural features of DNA sequences, including dinucleotide scales such as base stacking energy and propeller twist, and trinucleotide scales such as bendability and nucleosome positioning. We show that these scales provide an alternative or complementary compact representation of DNA sequences. As an example we construct a strand invariant representation of DNA sequences. The scales can also be used to analyze and discover new DNA structural patterns, especially in combinations with hidden Markov models (HMMs). The scales are applied to HMMs of human promoter sequences revealing a number of significant differences between regions upstream and downstream of the transcriptional start point. Finally we show, with some qualifications, that such scales are by and large independent, and therefore complement each other.
Probabilistic Models of Neuronal Spike Trains

This paper looks et time series associated with neuronal signals. It provides a brief overview of some of the main problems and applications in this area, and develops a general probabilistic approach for the study of spike trains. Here we first motivate the problematic by looking specifically at two of the main techniques used by neuroscientists : multi unit recording, and compartmental modeling. We then briefly review the genaral Bayesian framework for inference and probabilistic modeling. Probabilistic models of spike trains of increasing complexity are introduced and demonstrated in the following sections.
Hybrid Modeling, HMM/NN Architectures, and Protein Applications

We describe a hybrid modeling approach where the parameters of a model are calculated and modulated by another model, typically a neural network (NN), to avoid both overfitting and underfitting. We develop the approach for the case of Hidden Markov Models (HMMs), by deriving a class of hybrid HMM/NN architectures. These architectures can be trained with unified algorithms that blend HMM dynamic programming with NN backpropagation. In the case of complex data, mixtures of HMMs or modulated HMMs must be used. NNs can then be applied both to the parameters of each single HMM, and to the switching or modulation of the models, as a function of input or context. Hybrid HMM/NN architectures provide a flexible NN parameterization for the control of model structure and complexity. At the same time, they can capture distributions that, in practice, are inaccessible to single HMMs. The HMM/NN hybrid approach is tested, in its simplest form, by constructing a model of the immunoglobulin protein family. A hybrid model is trained, and a multiple alignment derived, with less than a fourth of the number of parameters used with previous single HMMs.
Naturally occurring nucleosome positioning signals in human exons and introns.

We describe the structural implications of a periodic pattern found in human exons and introns by hidden Markov models. We show that exons (besides the reading frame) have a specific sequential structure in the form of a pattern with triplet consensus non-T(A/T)G, and a minimal periodicity of roughly ten nucleotides. The periodic pattern is also present in intron sequences, although the strength per nucleotide is weaker. Using two independent profile methods based on triplet bendability parameters from DNase I experiments and nucleosome positioning data, we show that the pattern in multiple alignments of internal exon and intron sequences corresponds to a periodic "in phase" bending potential towards the major groove of the DNA. The nucleosome positioning data show that the consensus triplets (and their complements) have a preference for locations on a bent double helix where the major groove faces inward and is compressed. The in-phase triplets are located adjacent to GCC/GGC triplets known to have the strongest bias in their positioning on the nuclesome. Analysis of mRNA sequences encoding proteins with known tertiary structure exclude the possibility that the pattern is a consequence of the previously well-known periodicity caused by the encoding of alpha-helices in proteins. Finally, we discuss the relation between the bending potential of coding and non-coding regions and its impact on the translational positioning of nucleosomes and the recognition of genes by the transcriptional machinery.
Modular, Hierarchical Learning By Artificial Neural Networks

Modular and hierarchical approach to supervised learning by artificial neural networks leads to neural networks more structured than neural networks in which all neurons fully interconnected. These networks utilize general feedforward flow of information and sparse recurrent connections to achieve dynamical effects. The modular organization, sparsity of modular units and connections, and fact that learning is much more circumscribed are all attractive features for designing neural-network hardware. Learning streamlined by imitating some aspects of biological neural networks.
Characterization of Prokaryotic and Eukaryotic Promoters Using Hidden Markov Models

In this paper we utilize hidden Markov models (HMMs) and information theory to analyze prokaryotic and eukaryotic promoters. We perform this analysis with special emphasis on the fact that promoters are divided into a number of different classes, depending on which polymerase-associated factors that bind to them. We find that HMMs trained on such subclasses of Escherichia coli promoters (specifically, the so-called sigma 70 and sigma 54 classes) give an excellent classification of unknown promoters with respect to sigma-class. HMMs trained on eukaryotic sequences from human genes also model nicely all the essential well known signals, in addition to a potentially new signal upstream of the TATA-box. We furthermore employ a novel technique for automatically discovering different classes in the input data (the promoters) using a system of self-organizing parallel HMMs. These self-organizing HMMs have at the same time the ability to find clusters and the ability to model the sequential structure in the input data. This is highly relevant in situations where the variance in the data is high, as is the case for the subclass structure in for example promoter sequences.
Protein Modeling with Hybrid Hidden Markov Model/Neural Network Architectures

Hidden Markov Models (HMMs) are useful in a number of tasks in computational molecular biology, and in particular to model and align protein families. We argue that HMMs are somewhat optimal within a certain modeling hierarchy. Single first order HMMs, however, have two potential limitations: a large number of unstructured parameters, and a built-in inability to deal with long-range dependencies. Hybrid HMM/Neural Network (NN) architectures attempt to overcome these limitations. In hybrid HMM/NN, the HMM parameters are computed by a NN. This provides a reparametrization that allows for flexible control of model complexity, and incorporation of constraints. The approach is tested on the immunoglobulin family. A hybrid model is trained, and a multiple alignment derived, with less than a fourth of the number of parameters used with previous single HMMs. To capture dependencies, however, one must resort to a larger hybrid model class, where the data is modeled by multiple HMMs. The parameters of the HMMs, and their modulation as a function of input or context, is again calculated by a NN.
Substitution Matrices and Hidden Markov Models

Hidden Markov models (HMMs) provide a general framework for expressing primary sequence consensus. HMMs can effectively be used to model and align protein families, and to search data bases. HMMs, however, have a large number of parameters. When only few sequences are available for model fitting, additional prior information must be incorporated into the models. We derive a simple algorithm that directly incorporates prior information provided by substitution matrices into the HMM learning procedure.
Periodic Sequence Patterns in Human Exons

We analyse the sequential structure of human exons and their flanking introns by hidden Markov models. Together, models of donor site regions, acceptor site regions and flanked internal exons, show that exons--besides the reading frame--hold a specific periodic pattern. The pattern, which has the consensus: non-T(A/T)G and a minimal periodicity of roughly 10 nucleotides, is not a consequence of the nucleotide statistics in the three codon positions, nor of the well known nucleosome positioning signal. We discuss the relation between the pattern and other known sequence elements responsible for the intrinsic bending or curvature of DNA.
Learning in linear neural networks: a survey

Networks of linear units are the simplest kind of networks, where the basic questions related to learning, generalization, and self-organization can sometimes be answered analytically. We survey most of the known results on linear networks, including: 1) backpropagation learning and the structure of the error function landscape, 2) the temporal evolution of generalization, and 3) unsupervised learning algorithms and their properties. The connections to classical statistical ideas, such as principal component analysis (PCA), are emphasized as well as several simple but challenging open questions. A few new results are also spread across the paper, including an analysis of the effect of noise on backpropagation networks and a unified view of all unsupervised algorithms.
Gradient descent learning algorithm overview: a general dynamical systems perspective

Gives a unified treatment of gradient descent learning algorithms for neural networks using a general framework of dynamical systems. This general approach organizes and simplifies all the known algorithms and results which have been originally derived for different problems (fixed point/trajectory learning), for different models (discrete/continuous), for different architectures (forward/recurrent), and using different techniques (backpropagation, variational calculus, adjoint methods, etc.). The general approach can also be applied to derive new algorithms. The author then briefly examines some of the complexity issues and limitations intrinsic to gradient descent learning. Throughout the paper, the author focuses on the problem of trajectory learning.