nils-herrmann/clean_scirepeval huggingface dataset:
Computing with Innnite Networks

For neural networks with a wide class of weight-priors, it can be shown that in the limit of an innnite number of hidden units the prior over functions tends to a Gaussian process. In this paper analytic forms are derived for the covariance function of the Gaussian processes corresponding to networks with sigmoidal and Gaussian hidden units. This allows predictions to be made eeciently using networks with an innnite number of hidden units, and shows that, somewhat paradoxically, it may be easier to compute with innnite networks than nite ones.
GTM: A Principled Alternative to the Self-Organizing Map

The Self-Organizing Map (SOM) algorithm has been extensively studied and has been applied with considerable success to a wide variety of problems. However, the algorithm is derived from heuristic ideas and this leads to a number of significant limitations. In this paper, we consider the problem of modelling the probability density of data in a space of several dimensions in terms of a smaller number of latent, or hidden, variables. We introduce a novel form of latent variable model, which we call the GTM algorithm (for Generative Topographic Mapping), which allows general non-linear transformations from latent space to data space, and which is trained using the EM (expectation-maximization) algorithm. Our approach overcomes the limitations of the SOM, while introducing no significant disadvantages. We demonstrate the performance of the GTM algorithm on simulated data from flow diagnostics for a multi-phase oil pipeline.
Computing with Infinite Networks

For neural networks with a wide class of weight-priors, it can be shown that in the limit of an infinite number of hidden units the prior over functions tends to a Gaussian process. In this paper analytic forms are derived for the covariance function of the Gaussian processes corresponding to networks with sigmoidal and Gaussian hidden units. This allows predictions to be made efficiently using networks with an infinite number of hidden units, and shows that, somewhat paradoxically, it may be easier to compute with infinite networks than finite ones.
Using Generative Models for Handwritten Digit Recognition

We describe a method of recognizing handwritten digits by fitting generative models that are built from deformable B-splines with Gaussian "ink generators" spaced along the length of the spline. The splines are adjusted using a novel elastic matching procedure based on the expectation maximization algorithm that maximizes the likelihood of the model generating the data. This approach has many advantages: 1) the system not only produces a classification of the digit but also a rich description of the instantiation parameters which can yield information such as the writing style; 2) the generative models can perform recognition driven segmentation; 3) the method involves a relatively small number of parameters and hence training is relatively easy and fast; and 4) unlike many other recognition schemes, it does not rely on some form of pre-normalization of input images, but can handle arbitrary scalings, translations and a limited degree of image rotation. We have demonstrated that our method of fitting models to images does not get trapped in poor local minima. The main disadvantage of the method is that it requires much more computation than more standard OCR techniques.
Gaussian Processes for Regression

The Bayesian analysis of neural networks is difficult because a simple prior over weights implies a complex prior distribution over functions. In this paper we investigate the use of Gaussian process priors over functions, which permit the predictive Bayesian analysis for fixed values of hyperparameters to be carried out exactly using matrix operations. Two methods, using optimization and averaging (via Hybrid Monte Carlo) over hyperparameters have been tested on a number of challenging problems and have produced excellent results.
EM Optimization of Latent-Variables Density Models

There is currently considerable interest in developing general nonlinear density models based on latent, or hidden, variables. Such models have the ability to discover the presence of a relatively small number of underlying 'causes' which, acting in combination, give rise to the apparent complexity of the observed data set. Unfortunately, to train such models generally requires large computational effort. In this paper we introduce a novel latent variable algorithm which retains the general non-linear capabilities of previous models but which uses a training procedure based on the EM algorithm. We demonstrate the performance of the model on a toy problem and on data from flow diagnostics for a multi-phase oil pipeline.
On the relationship between Bayesian error bars and the input data density

We investigate the dependence of Bayesian error bars on the distribution of data in input space. For generalized linear regression models we derive an upper bound on the error bars which shows that, in the neighbourhood of the data points, the error bars are substantially reduced from their prior values. For regions of high data density we also show that the contribution to the output variance due to the uncertainty in the weights can exhibit an approximate inverse proportionality to the probability density. Empirical results support these conclusions.
Note on rejection sampling and exact sampling with the Metropolised independence sampler

This short note shows a close relationship between standard rejection sampling and exact sampling by coupling from the past applied to a Metropolised independence sampler. Little background is assumed, but  provides a clear review of all required material. I now know that this idea, first presented as a ten-minute tea-time talk, is probably a duplicate of an unavailable work , and is closely related to a paper by Jun S. Liu , who provides a much more detailed analysis. Perhaps this exposition will be of interest to some readers.
Implementation of Dasher, an information efficient input mechanism

Dasher is a novel information efficient text entry system using a language model to offer predictions to the user without constraining the range of words which can be written. In this paper, we discuss the implementation of Dasher and how it is well suited to providing input on keyboardless devices and for disabled users. We will also look at how its integration with the GNOME accessibility framework allows Dasher to be not just a text entry system, but a comprehensive application control system.
Hot Coupling: A Particle Approach to Inference and Normalization on Pairwise Undirected Graphs

This paper presents a new sampling algorithm for approximating functions of variables representable as undirected graphical models of arbitrary connectivity with pairwise potentials, as well as for estimating the notoriously difficult partition function of the graph. The algorithm fits into the framework of sequential Monte Carlo methods rather than the more widely used MCMC, and relies on constructing a sequence of intermediate distributions which get closer to the desired one. While the idea of using “tempered” proposals is known, we construct a novel sequence of target distributions where, rather than dropping a global temperature parameter, we sequentially couple individual pairs of variables that are, initially, sampled exactly from a spanning tree of the variables. We present experimental results on inference and estimation of the partition function for sparse and densely-connected graphs.
Fast Computational Methods for Visually Guided Robots

This paper proposes numerical algorithms for reducing the computational cost of semi-supervised and active learning procedures for visually guided mobile robots from O(M3to O(M), while reducing the storage requirements from M2to M . This reduction in cost is essential for real-time interaction with mobile robots. The considerable speed ups are achieved using Krylov subspace methods and the fast Gauss transform. Although these state-of-the-art numerical algorithms are known, their application to semi-supervised learning, active learning and mobile robotics is new and should be of interest and great value to the robotics community. We apply our fast algorithms to interactive object recognition on Sony’s ERS-7 Aibo. We provide comparisons that clearly demonstrate remarkable improvements in computational speed.
From Fields to Trees

We present new MCMC algorithms for computing the posterior distributions and expectations of the unknown variables in undirected graphical models with regular structure. For demonstration purposes, we focus on Markov Random Fields (MRFs). By partitioning the MRFs into non-overlapping trees, it is possible to compute the posterior distribution of a particular tree exactly by conditioning on the remaining tree. These exact solutions allow us to construct efficient blocked and Rao-Blackwellised MCMC algorithms. We show empirically that tree sampling is considerably more efficient than other partitioned sampling schemes and the naive Gibbs sampler, even in cases where loopy belief propagation fails to converge. We prove that tree sampling exhibits lower variance than the naive Gibbs sampler and other naive partitioning schemes using the theoretical measure of maximal correlation. We also construct new information theory tools for comparing different MCMC schemes and show that, under these, tree sampling is more efficient.
Precise Temporal Modulation in the Response of the SOS DNA Repair Network in Individual Bacteria

The SOS genetic network is responsible for the repair/bypass of DNA damage in bacterial cells. While the initial stages of the response have been well characterized, less is known about the dynamics of the response after induction and its shutoff. To address this, we followed the response of the SOS network in living individual Escherichia coli cells. The promoter activity (PA) of SOS genes was monitored using fluorescent protein-promoter fusions, with high temporal resolution, after ultraviolet irradiation activation. We find a temporal pattern of discrete activity peaks masked in studies of cell populations. The number of peaks increases, while their amplitude reaches saturation, as the damage level is increased. Peak timing is highly precise from cell to cell and is independent of the stage in the cell cycle at the time of damage. Evidence is presented for the involvement of the umuDC operon in maintaining the pattern of PA and its temporal precision, providing further evidence for the role UmuD cleavage plays in effecting a timed pause during the SOS response, as previously proposed. The modulations in PA we observe share many features in common with the oscillatory behavior recently observed in a mammalian DNA damage response. Our results, which reveal a hitherto unknown modulation of the SOS response, underscore the importance of carrying out dynamic measurements at the level of individual living cells in order to unravel how a natural genetic network operates at the systems level.
Single-Nucleosome Mapping of Histone Modifications in S. cerevisiae

Covalent modification of histone proteins plays a role in virtually every process on eukaryotic DNA, from transcription to DNA repair. Many different residues can be covalently modified, and it has been suggested that these modifications occur in a great number of independent, meaningful combinations. Published low-resolution microarray studies on the combinatorial complexity of histone modification patterns suffer from confounding effects caused by the averaging of modification levels over multiple nucleosomes. To overcome this problem, we used a high-resolution tiled microarray with single-nucleosome resolution to investigate the occurrence of combinations of 12 histone modifications on thousands of nucleosomes in actively growing S. cerevisiae. We found that histone modifications do not occur independently; there are roughly two groups of co-occurring modifications. One group of lysine acetylations shows a sharply defined domain of two hypo-acetylated nucleosomes, adjacent to the transcriptional start site, whose occurrence does not correlate with transcription levels. The other group consists of modifications occurring in gradients through the coding regions of genes in a pattern associated with transcription. We found no evidence for a deterministic code of many discrete states, but instead we saw blended, continuous patterns that distinguish nucleosomes at one location (e.g., promoter nucleosomes) from those at another location (e.g., over the 3′ ends of coding regions). These results are consistent with the idea of a simple, redundant histone code, in which multiple modifications share the same role.
Ab Initio Prediction of Transcription Factor Targets Using Structural Knowledge

Current approaches for identification and detection of transcription factor binding sites rely on an extensive set of known target genes. Here we describe a novel structure-based approach applicable to transcription factors with no prior binding data. Our approach combines sequence data and structural information to infer context-specific amino acid–nucleotide recognition preferences. These are used to predict binding sites for novel transcription factors from the same structural family. We demonstrate our approach on the Cys2His2 Zinc Finger protein family, and show that the learned DNA-recognition preferences are compatible with experimental results. We use these preferences to perform a genome-wide scan for direct targets of Drosophila melanogaster Cys2His2 transcription factors. By analyzing the predicted targets along with gene annotation and expression data we infer the function and activity of these proteins.
Quantitative kinetic analysis of the bacteriophage lambda genetic network.

The lysis-lysogeny decision of bacteriophage lambda has been a paradigm for a developmental genetic network, which is composed of interlocked positive and negative feedback loops. This genetic network is capable of responding to environmental signals and to the number of infecting phages. An interplay between CI and Cro functions suggested a bistable switch model for the lysis-lysogeny decision. Here, we present a real-time picture of the execution of lytic and lysogenic pathways with unprecedented temporal resolution. We monitor, in vivo, both the level and function of the CII and Q gene regulators. These activators are cotranscribed yet control opposite developmental pathways. Conditions that favor the lysogenic response show severe delay and down-regulation of Q activity, in both CII-dependent and CII-independent ways. Whereas CII activity correlates with its protein level, Q shows a pronounced threshold before its function is observed. Our quantitative analyses suggest that by regulating CII and CIII, Cro plays a key role in the ability of the lambda genetic network to sense the difference between one and more than one phage particles infecting a cell. Thus, our results provide an improved framework to explain the longstanding puzzle of the decision process.
Predicting Transcription Factor Binding Sites Using Structural Knowledge

Current approaches for identification and detection of transcription factor binding sites rely on an extensive set of known target genes. Here we describe a novel structure-based approach applicable to transcription factors with no prior binding data. Our approach combines sequence data and structural information to infer context-specific amino acid-nucleotide recognition preferences. These are used to predict binding sites for novel transcription factors from the same structural family. We apply our approach to the Cys2His2 Zinc Finger protein family, and show that the learned DNA-recognition preferences are compatible with various experimental results. To demonstrate the potential of our algorithm, we use the learned preferences to predict binding site models for novel proteins from the same family. These models are then used in genomic scans to find putative binding sites of the novel proteins.
A Gamma mixture model better accounts for among site rate heterogeneity

Variation of substitution rates across nucleotide and amino acid sites has long been recognized as a characteristic of molecular sequence evolution. Evolutionary models that account for this rate heterogeneity usually use a gamma density function to model the rate distribution across sites. This density function, however, may not fit real datasets, especially when there is a multimodal distribution of rates. Here, we present a novel evolutionary model based on a mixture of gamma density functions. This model better describes the among-site rate variation characteristic of molecular sequence evolution. The use of this model may improve the accuracy of various phylogenetic methods, such as reconstructing phylogenetic trees, dating divergence events, inferring ancestral sequences and detecting conserved sites in proteins.



Using diverse sets of protein sequences we show that the gamma mixture model better describes the stochastic process underlying protein evolution. We show that the proposed gamma mixture model fits protein datasets significantly better than the single-gamma model in 9 out of 10 datasets tested. We further show that using the gamma mixture model improves the accuracy of model-based prediction of conserved residues in proteins.


AVAILABILITY
C++ source codes are available from the authors upon request.
CIS: compound importance sampling method for protein-DNA binding site p-value estimation.

A key aspect of transcriptional regulation is the binding of transcription factors to sequence-specific binding sites that allow them to modulate the expression of nearby genes. Given models of such binding sites, one can scan regulatory regions for putative binding sites and construct a genome-wide regulatory network. In such genome-wide scans, it is crucial to control the amount of false positive predictions. Recently, several works demonstrated the benefits of modeling dependencies between positions within the binding site. Yet, computing the statistical significance of putative binding sites in this scenario remains a challenge.



We present a general, accurate and efficient method for computing p-values of putative binding sites that is applicable to a large class of probabilistic binding site and background models. We demonstrate the accuracy of the method on synthetic and real-life data.


AVAILABILITY
The procedure for scanning DNA sequences and computing the statistical significance of putative binding site scores is available upon request at http://compbio.cs.huji.ac.il/CIS/ CONTACT: nir@cs.huji.ac.il.
Towards an Integrated Protein-Protein Interaction Network

Protein-protein interactions play a major role in most cellular processes. Thus, the challenge of identifying the full repertoire of interacting proteins in the cell is of great importance, and has been addressed both experimentally and computationally. Today, large scale experimental studies of interacting proteins, while partial and noisy, allow us to characterize properties of interacting proteins and develop predictive algorithms. Most existing algorithms, however, ignore possible dependencies between interacting pairs, and predict them independently of one another. In this study, we present a computational approach that overcomes this drawback by predicting protein-protein interactions simultaneously. In addition, our approach allows us to integrate various protein attributes and explicitly account for uncertainty of assay measurements. Using the language of relational Markov Random Fields, we build a unified probabilistic model that includes all of these elements. We show how we can learn our model properties efficiently and then use it to predict all unobserved interactions simultaneously. Our results show that by modeling dependencies between interactions, as well as by taking into account protein attributes and measurement noise, we achieve a more accurate description of the protein interaction network. Furthermore, our approach allows us to gain new insights into the properties of interacting proteins.
Discovering Motif Interactions in Gene Promoter Regions

We seek to understand the rules governing the organization of cis-regulatory elements in promoter regions of saccharomyces cerevisiae(yeast). Starting with a library of known motifs, we construct a Hidden Markov Model where states correspond either to positions within different motifs or to background states. By learning the transition probabilities of this HMM we uncover preferences in ordering and orientations between cis-regulatory elements.
Learning Hidden Variable Networks: The Information Bottleneck Approach

A central challenge in learning probabilistic graphical models is dealing with domains that involve hidden variables. The common approach for learning model parameters in such domains is the expectation maximization (EM) algorithm. This algorithm, however, can easily get trapped in sub-optimal local maxima. Learning the model structure is even more challenging. The structural EM algorithm can adapt the structure in the presence of hidden variables, but usually performs poorly without prior knowledge about the cardinality and location of the hidden variables. In this work, we present a general approach for learning Bayesian networks with hidden variables that overcomes these problems. The approach builds on the information bottleneck framework of Tishby et al. (1999). We start by proving formal correspondence between the information bottleneck objective and the standard parametric EM functional. We then use this correspondence to construct a learning algorithm that combines an information-theoretic smoothing term with a continuation procedure. Intuitively, the algorithm bypasses local maxima and achieves superior solutions by following a continuous path from a solution of, an easy and smooth, target function, to a solution of the desired likelihood function. As we show, our algorithmic framework allows learning of the parameters as well as the structure of a network. In addition, it also allows us to introduce new hidden variables during model selection and learn their cardinality. We demonstrate the performance of our procedure on several challenging real-life data sets.
Quantitative kinetic analysis of the bacteriophage λ genetic network

The lysis–lysogeny decision of bacteriophage λ has been a paradigm for a developmental genetic network, which is composed of interlocked positive and negative feedback loops. This genetic network is capable of responding to environmental signals and to the number of infecting phages. An interplay between CI and Cro functions suggested a bistable switch model for the lysis–lysogeny decision. Here, we present a real-time picture of the execution of lytic and lysogenic pathways with unprecedented temporal resolution. We monitor, in vivo, both the level and function of the CII and Q gene regulators. These activators are cotranscribed yet control opposite developmental pathways. Conditions that favor the lysogenic response show severe delay and down-regulation of Q activity, in both CII-dependent and CII-independent ways. Whereas CII activity correlates with its protein level, Q shows a pronounced threshold before its function is observed. Our quantitative analyses suggest that by regulating CII and CIII, Cro plays a key role in the ability of the λ genetic network to sense the difference between one and more than one phage particles infecting a cell. Thus, our results provide an improved framework to explain the longstanding puzzle of the decision process.