nils-herrmann/clean_scirepeval huggingface dataset:
Biomarker discovery in microarray gene expression data with Gaussian processes

In clinical practice, pathological phenotypes are often labelled with ordinal scales rather than binary, e.g. the Gleason grading system for tumour cell differentiation. However, in the literature of microarray analysis, these ordinal labels have been rarely treated in a principled way. This paper describes a gene selection algorithm based on Gaussian processes to discover consistent gene expression patterns associated with ordinal clinical phenotypes. The technique of automatic relevance determination is applied to represent the significance level of the genes in a Bayesian inference framework.



The usefulness of the proposed algorithm for ordinal labels is demonstrated by the gene expression signature associated with the Gleason score for prostate cancer data. Our results demonstrate how multi-gene markers that may be initially developed with a diagnostic or prognostic application in mind are also useful as an investigative tool to reveal associations between specific molecular and cellular events and features of tumour physiology. Our algorithm can also be applied to microarray data with binary labels with results comparable to other methods in the literature.
Modeling genetic regulatory networks using gene expression profiling and state space models

We describe a Bayesian network approach to infer genetic regulatory interactions from microarray gene expression data. This problem was introduced in Chapter 7 and an alternative Bayesian network approach was presented in Chapter 8. Our approach is based on a linear dynamical system, which renders the inference problem tractable: the E-step of the EM algorithm draws on the well-established Kalman smoothing algorithm. While the intrinsic linearity constraint makes our approach less suitable for modeling non-linear genetic interactions than the approach of Chapter 8, it has two important advantages over the method of Chapter 8. First, our approach works with continuous expression levels, which avoids the information loss inherent in a discretization of these signals. Second, we include hidden states to allow for the effects that cannot be measured in a microarray experiment, for example: the effects of genes that have not been included on the microarray, levels of regulatory proteins, and the effects of mRNA and protein degradation.
A note on the evidence and Bayesian Occam's razor

In his thesis, MacKay (1991) introduced figure 1, explaining how Bayes rule provides an automatic “Occam’s razor” effect, penalizing unnecessarily complex models. This figure has been adopted by several authors in the same schematic form. Here, after briefly reviewing necessary material, we compute a realization of the plot for a toy data modeling problem. We discuss interesting aspects of this plot and their implications for understanding model complexity.
Sparse Gaussian Processes using Pseudo-inputs

We present a new Gaussian process (GP) regression model whose co-variance is parameterized by the the locations of M pseudo-input points, which we learn by a gradient based optimization. We take M ≪ N, where N is the number of real data points, and hence obtain a sparse regression method which has O(M2N) training cost and O(M2) prediction cost per test case. We also find hyperparameters of the covariance function in the same joint optimization. The method can be viewed as a Bayesian regression model with particular input dependent noise. The method turns out to be closely related to several other sparse GP approaches, and we discuss the relation in detail. We finally demonstrate its performance on some large data sets, and make a direct comparison to other sparse GP methods. We show that our method can match full GP performance with small M, i.e. very sparse solutions, and it significantly outperforms other approaches in this regime.
A Bayesian approach to reconstructing genetic regulatory networks with hidden factors

We have used state-space models (SSMs) to reverse engineer transcriptional networks from highly replicated gene expression profiling time series data obtained from a well-established model of T cell activation. SSMs are a class of dynamic Bayesian networks in which the observed measurements depend on some hidden state variables that evolve according to Markovian dynamics. These hidden variables can capture effects that cannot be directly measured in a gene expression profiling experiment, for example: genes that have not been included in the microarray, levels of regulatory proteins, the effects of mRNA and protein degradation, etc.



We have approached the problem of inferring the model structure of these state-space models using both classical and Bayesian methods. In our previous work, a bootstrap procedure was used to derive classical confidence intervals for parameters representing 'gene-gene' interactions over time. In this article, variational approximations are used to perform the analogous model selection task in the Bayesian context. Certain interactions are present in both the classical and the Bayesian analyses of these regulatory networks. The resulting models place JunB and JunD at the centre of the mechanisms that control apoptosis and proliferation. These mechanisms are key for clonal expansion and for controlling the long term behavior (e.g. programmed cell death) of these cells.


AVAILABILITY
Supplementary data is available at http://public.kgi.edu/wild/index.htm and Matlab source code for variational Bayesian learning of SSMs is available at http://www.cse.ebuffalo.edu/faculty/mbeal/software.html.
Bayesian hierarchical clustering

We present a novel algorithm for agglomerative hierarchical clustering based on evaluating marginal likelihoods of a probabilistic model. This algorithm has several advantages over traditional distance-based agglomerative clustering algorithms. (1) It defines a probabilistic model of the data which can be used to compute the predictive distribution of a test point and the probability of it belonging to any of the existing clusters in the tree. (2) It uses a model-based criterion to decide on merging clusters rather than an ad-hoc distance metric. (3) Bayesian hypothesis testing is used to decide which merges are advantageous and to output the recommended depth of the tree. (4) The algorithm can be interpreted as a novel fast bottom-up approximate inference method for a Dirichlet process (i.e. countably infinite) mixture model (DPM). It provides a new lower bound on the marginal likelihood of a DPM by summing over exponentially many clusterings of the data in polynomial time. We describe procedures for learning the model hyperpa-rameters, computing the predictive distribution, and extensions to the algorithm. Experimental results on synthetic and real-world data sets demonstrate useful properties of the algorithm.
U-Likelihood and U-Updating Algorithms: Statistical Inference in Latent Variable Models

In this paper we consider latent variable models and introduce a new $\mathcal{U}$-likelihood concept for estimating the distribution over hidden variables. One can derive an estimate of parameters from this distribution. Our approach differs from the Bayesian and Maximum Likelihood (ML) approaches. It gives an alternative to Bayesian inference when we don't want to define a prior over parameters and gives an alternative to the ML method when we want a better estimate of the distribution over hidden variables. As a practical implementation, we present a $\mathcal{U}$-updating algorithm based on the mean field theory to approximate the distribution over hidden variables from the $\mathcal{U}$-likelihood. This algorithm captures some of the correlations among hidden variables by estimating reaction terms. Those reaction terms are found to penalize the likelihood. We show that the $\mathcal{U}$-updating algorithm becomes the EM algorithm as a special case in the large sample limit. The useful behavior of our method is confirmed for the case of mixture of Gaussians by comparing to the EM algorithm.
Bayesian Sets

Inspired by "Google™ Sets", we consider the problem of retrieving items from a concept or cluster, given a query consisting of a few items from that cluster. We formulate this as a Bayesian inference problem and describe a very simple algorithm for solving it. Our algorithm uses a model-based concept of a cluster and ranks items using a score which evaluates the marginal probability that each item belongs to a cluster containing the query items. For exponential family models with conjugate priors this marginal probability is a simple function of sufficient statistics. We focus on sparse binary data and show that our score can be evaluated exactly using a single sparse matrix multiplication, making it possible to apply our algorithm to very large datasets. We evaluate our algorithm on three datasets: retrieving movies from EachMovie, finding completions of author sets from the NIPS dataset, and finding completions of sets of words appearing in the Grolier encyclopedia. We compare to Google™ Sets and show that Bayesian Sets gives very reasonable set completions.
Protein secondary structure prediction using sigmoid belief networks to parameterize segmental semi-Markov models

�� � �� �� �� � � � � � � �� �� �� � �� �� �� �� �� �� �� �� �� �� �� �� � � � � � � � � � � � � � � � � � � � � � � � � � � � � ������� ������� ��� ���� ����� ���� ���� ���� ���� ���� ���
Predictive automatic relevance determination by expectation propagation

In many real-world classification problems the input contains a large number of potentially irrelevant features. This paper proposes a new Bayesian framework for determining the relevance of input features. This approach extends one of the most successful Bayesian methods for feature selection and sparse learning, known as Automatic Relevance Determination (ARD). ARD finds the relevance of features by optimizing the model marginal likelihood, also known as the evidence. We show that this can lead to overfitting. To address this problem, we propose Predictive ARD based on estimating the predictive performance of the classifier. While the actual leave-one-out predictive performance is generally very costly to compute, the expectation propagation (EP) algorithm proposed by Minka provides an estimate of this predictive performance as a side-effect of its iterations. We exploit this in our algorithm to do feature selection, and to select data points in a sparse Bayesian kernel classifier. Moreover, we provide two other improvements to previous algorithms, by replacing Laplace's approximation with the generally more accurate EP, and by incorporating the fast optimization algorithm proposed by Faul and Tipping. Our experiments show that our method based on the EP estimate of predictive performance is more accurate on test data than relevance determination by optimizing the evidence.
The Status of Structural Genomics Defined Through the Analysis of Current Targets and Structures

Structural genomics--large-scale macromolecular 3-dimenional structure determination--is unique in that major participants report scientific progress on a weekly basis. The target database (TargetDB) maintained by the Protein Data Bank (http://targetdb.pdb.org) reports this progress through the status of each protein sequence (target) under consideration by the major structural genomics centers worldwide. Hence, TargetDB provides a unique opportunity to analyze the potential impact that this major initiative provides to scientists interested in the sequence-structure-function-disease paradigm. Here we report such an analysis with a focus on: (i) temporal characteristics--how is the project doing and what can we expect in the future? (ii) target characteristics--what are the predicted functions of the proteins targeted by structural genomics and how biased is the target set when compared to the PDB and to predictions across complete genomes? (iii) structures solved--what are the characteristics of structures solved thus far and what do they contribute? The analysis required a more extensive database of structure predictions using different methods integrated with data from other sources. This database, associated tools and related data sources are available from http://spam.sdsc.edu.
An Introduction to Variational Methods for Graphical Models

This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields). We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, which exploit laws of large numbers to transform the original graphical model into a simplified graphical model in which inference is efficient. Inference in the simpified model provides bounds on probabilities of interest in the original model. We describe a general framework for generating variational transformations based on convex duality. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.
Clustering Protein Sequence and Structure Space with Infinite Gaussian Mixture Models

We describe a novel approach to the problem of automatically clustering protein sequences and discovering protein families, subfamilies etc., based on the theory of infinite Gaussian mixtures models. This method allows the data itself to dictate how many mixture components are required to model it, and provides a measure of the probability that two proteins belong to the same cluster. We illustrate our methods with application to three data sets: globin sequences, globin sequences with known three-dimensional structures and G-protein coupled receptor sequences. The consistency of the clusters indicate that our method is producing biologically meaningful results, which provide a very good indication of the underlying families and subfamilies. With the inclusion of secondary structure and residue solvent accessibility information, we obtain a classification of sequences of known structure which both reflects and extends their SCOP classifications. A supplementray web site containing larger versions of the figures is available at http://public.kgi.edu/approximately wid/PSB04/index.html
Factorial Hidden Markov Models

Hidden Markov models (HMMs) have proven to be one of the most widely used tools for learning probabilistic models of time series data. In an HMM, information about the past is conveyed through a single discrete variable—the hidden state. We discuss a generalization of HMMs in which this state is factored into multiple state variables and is therefore represented in a distributed manner. We describe an exact algorithm for inferring the posterior probabilities of the hidden state variables given the observations, and relate it to the forward–backward algorithm for HMMs and to algorithms for more general graphical models. Due to the combinatorial nature of the hidden state representation, this exact algorithm is intractable. As in other intractable systems, approximate inference can be carried out using Gibbs sampling or variational methods. Within the variational framework, we present a structured approximation in which the the state variables are decoupled, yielding a tractable algorithm for learning the parameters of the model. Empirical comparisons suggest that these approximations are efficient and provide accurate alternatives to the exact methods. Finally, we use the structured approximation to model Bach's chorales and show that factorial HMMs can capture statistical structure in this data set which an unconstrained HMM cannot.
Analysis of the synergies underlying complex hand manipulation

Coupling of actuators into motor synergies has been observed repeatedly, and is traditionally interpreted as a strategy for simplifying complex coordination problems. This view implies a small number of task-independent synergies. We have shown that optimal feedback control also gives rise to synergies in the absence of any simplification; the structure and number of such optimal synergies depends on the task. To compare these hypotheses, we recorded hand postures from a range of complex manipulation task. The structure of the synergies we extracted (via PCA) was task-dependent, and their number significantly exceeded previous observations in a simpler grasping task. Our results lend support to an optimal control explanation rather than a "simplicity" explanation.
Nonparametric Transforms of Graph Kernels for Semi-Supervised Learning

We present an algorithm based on convex optimization for constructing kernels for semi-supervised learning. The kernel matrices are derived from the spectral decomposition of graph Laplacians, and combine labeled and unlabeled data in a systematic fashion. Unlike previous work using diffusion kernels and Gaussian random field kernels, a nonparametric kernel approach is presented that incorporates order constraints during optimization. This results in flexible kernels and avoids the need to choose among different parametric forms. Our approach relies on a quadratically constrained quadratic program (QCQP), and is computationally feasible for large datasets. We evaluate the kernels on real datasets using support vector machines, with encouraging results.
Modeling T-cell activation using gene expression profiling and state-space models

We have used state-space models to reverse engineer transcriptional networks from highly replicated gene expression profiling time series data obtained from a well-established model of T-cell activation. State space models are a class of dynamic Bayesian networks that assume that the observed measurements depend on some hidden state variables that evolve according to Markovian dynamics. These hidden variables can capture effects that cannot be measured in a gene expression profiling experiment, e.g. genes that have not been included in the microarray, levels of regulatory proteins, the effects of messenger RNA and protein degradation, etc.



Bootstrap confidence intervals are developed for parameters representing 'gene-gene' interactions over time. Our models represent the dynamics of T-cell activation and provide a methodology for the development of rational and experimentally testable hypotheses.


AVAILABILITY
Supplementary data and Matlab computer source code will be made available on the web at the URL given below.



http://public.kgi.edu/~wild/LDS/index.htm
A graphical model for protein secondary structure prediction

In this paper, we present a graphical model for protein secondary structure prediction. This model extends segmental semi-Markov models (SSMM) to exploit multiple sequence alignment profiles which contain information from evolutionarily related sequences. A novel parameterized model is proposed as the likelihood function for the SSMM to capture the segmental conformation. By incorporating the information from long range interactions in ß-sheets, this model is capable of carrying out inference on contact maps. The numerical results on benchmark data sets show that incorporating the profiles results in substantial improvements and the generalization performance is promising.
Classical and Bayesian approaches to reconstructing genetic regulatory networks

1 Computer Science and Engineering Department, SUNY at Buffalo, NY, USA mbeal@cse.buffalo.edu 2 Department of Computational and Molecular Biology, University of Southern California, USA rangelc@usc.edu 3 School of Biosciences, University of Birmingham, UK f.falciani@bham.ac.uk 4 Gatsby Computational Neuroscience Unit, University College London, UK zoubin@gatsby.ucl.ac.uk 5 Keck Graduate Institute of Applied Life Sciences, CA, USA david wild@kgi.edu
Bayes rule in perception, action and cognition

Cognition and intelligent behaviour are fundamentally tied to the ability to survive in an uncertain and changing environment. Our only access to knowledge about the world is through our senses which provide information that is usually corrupted by random fluctuations, termed noise, and may provide ambiguous information about the possible states of the environment. Moreover, when we act on the world through our motor system, the commands we send to our muscles are also corrupted by variability or noise. This combined sensory and motor variability limits the precision with which we can perceive and act on the world. Here we will review the framework of Bayesian decision theory, which has emerged as a principled approach to handle uncertain information in an attempt to behave optimally, and how this framework can be used to understand sensory, motor and cognitive processes. Bayesian Theory is named after Thomas Bayes  who was an 18th century English Presbyterian minister. He is only known to have published two works during his life, of which only one dealt with mathematics in which he defended the logical foundation of Isaac Newton's methods against contemporary criticism. After Bayes death, his friend Richard Price found an interesting mathematical proof among Bayes' papers and sent the paper to the Editor of the Philosophical Transactions of the Royal Society stating " I now send you an essay which I have found among the papers of our deceased friend Mr Bayes, and which, in my opinion, has great merit.... " The paper was subsequently published in 1764 as " Essay towards solving a problem in the doctrine of chances. " In the latter half of the 20 th century Bayesian approaches have become a mainstay of statistics and a more general framework has now emerged, termed Bayesian decision theory (BDT). Bayesian decision theory incorporates two main components: Bayesian statistics and decision theory: Bayesian statistics Bayesian statistics is a framework for making inferences based on uncertain information. The fundamental idea is that probabilities can be used to represent the degree of belief in different propositions and therefore the rules of probability can be used to update beliefs based on new information. For example, imagine you are a doctor and see a patient who you believe may have a particular disease. The strength of this belief can be represented by a real number between 0 and 1, which reflects the probability that we …
Propagating uncertainty in POMDP value iteration with Gaussian processes

In this paper, we describe the general approach of trying to solve Partially Observable Markov Decision Processes with approximate value iteration. Methods based on this approach have shown promise for tackling larger problems where exact methods are doomed, but we explain how most of them suffer from the fundamental problem of ignoring information about the uncertainty of their estimates. We then suggest a new method for value iteration which uses Gaussian processes to form a Bayesian representation of the uncertain POMDP value function. We evaluate this method on several standard POMDPs and obtain promising