nils-herrmann/clean_scirepeval huggingface dataset:
Bayesian deconvolution of poissonian point sources

In this article, we address the problem of Bayesian de-convolution of point sources with Poisson statistics. A high level Bayesian approach is proposed to solve this problem. The original image is modeled list of an unknown number of points sources with unknow parameters. A prior distribution reflecting our degree of belief is introduced on all these unknown parameters, including the number of sources. All Bayesian inference relies on the posterior distribution. This latter admitting no analytical expression, we estimate it using an original Reversible Jump Markov Chain Monte Carlo method. The algorithm developed is tested over real data. It displays satisfactory results compared to traditional low level Bayesian approaches.
Stochastic algorithms for Bayesian model selection of AR processes

In this paper we address the problem of determining the dimensions of an autoregressive process in a Bayesian framework under various assumptions, including stationarity of the process. Solving this problem requires the ability to solve integration and/or optimization problems of complicated posterior distributions. We thus propose efficient stochastic algorithms based on Markov chain Monte Carlo methods. Their convergence is established and computer simulations are provided, demonstrating the efficiency of the approach adopted.
Reversible jump Markov chain Monte Carlo for Bayesian deconvolution of point sources

In this article, we address the problem of Bayesian deconvolution of point sources in nuclear imaging under the assumption of Poissonian statistics. The observed image is the result of the convolution by a known point spread function of an unknown number of point sources with unknown parameters. To detect the number of sources and estimate their parameters we follow a Bayesian approach. However, instead of using a classical low level prior model based on Markov random fields, we prose a high-level model which describes the picture as a list of its constituent objects, rather than as a list of pixels on which the data are recorded. More precisely, each source is assumed to have a circular Gaussian shape and we set a prior distribution on the number of sources, on their locations and on the amplitude and width deviation of the Gaussian shape. This high-level model has far less parameters than a Markov random field model as only s small number of sources are usually present. The Bayesian model being defined, all inference is based on the resulting posterior distribution. This distribution does not admit any closed-form analytical expression. We present here a Reversible Jump MCMC algorithm for its estimation. This algorithm is tested on both synthetic and real data.
Markov chain Monte Carlo methods for tracking a maneuvering target in clutter

We address the problem of tracking a maneuvering target in a cluttered environment. We propose three algorithms based on stochastic sampling methods to solve the following combinatorial optimization problems: (a) data association, and (b) maneuver detection. The first proposed algorithm is a data augmentation (DA) scheme, that yields conditional mean state estimates of the maneuvering target in clutter. The second proposed scheme is a simulated annealing (SA) version of DA that computes the joint MAP state sequence estimates of the target state, the measurement to target associations and the input maneuvering control sequences. Finally, a SA Metropolis-Hastings DA scheme is designed to yield the MAP state sequence estimate of the measurement to target associations and the input maneuvering control sequences. The cost per iteration is linear in the data length. Furthermore, theoretical convergence results of the three proposed Markov chain Monte Carlo algorithms towards the desired estimates, have been obtained.
State estimation of jump Markov linear systems via stochastic sampling algorithms

We present three algorithms based on stochastic sampling methods for state estimation of jump Markov linear systems. The cost per iteration is linear in the data length. The first proposed algorithm is a data augmentation (DA) scheme that yields conditional mean state estimates. The second proposed scheme is a stochastic annealing (SA) version of DA that computes the joint MAP sequence estimate of the finite and continuous states. Finally, a Metropolis-Hastings DA scheme based on SA is designed to yield the MAP estimate of the finite state Markov chain, is proposed. Convergence results of the three above mentioned stochastic algorithms are obtained.
Instantaneous frequency estimation: Bayesian approaches versus reassignment-application to gravitational waves

Three new methods of instantaneous frequency estimation are introduced and compared in view of characterizing gravitational waves. Two methods are Bayesian and can be formulated as solutions of an ill-posed inverse problem with two different stochastic regularizations. Using either a state-space model for the time-frequency data or a compound non-uniform Bernoulli-Gauss model for the instantaneous frequency. The third method uses a reassignment technique applied to a spectrogram. In each case, averages based on different windowings permit to enhance the signal-to-noise ratio, leading to accurate results even below 0 dB.
Fully Bayesian analysis of conditionally linear Gaussian state space models

In this paper, we use the Gibbs sampler to carry out Bayesian inference on conditionally linear Gaussian state space models. In a Bayesian framework, the Gibbs sampler is a powerful iterative procedure which can be seen as a stochastic analogue of the EM algorithm. To use it, it is necessary to sample from complex multivariate densities. An efficient algorithm is derived. An application to Bernoulli-Gauss processes deconvolution is given for which very satisfactory results are obtained. For this example, the geometric convergence of the algorithm is established.
Fully Bayesian analysis of Hidden Markov models

In this paper, we present in an unified framework some applications of stochastic simulation techniques, the Markov chain Monte Carlo methods, to perform Bayesian inference for a very wide class of hidden Markov models. Efficient implementation of the Gibbs sampler based on finite dimensional optimal filters is described. An improved version of this algorithm is also presented. Two problems of great practical interest in signal processing are addressed: blind deconvolution of Bernoulli-Gauss processes and blind equalization of a channel. In simulations, we obtain very satisfactory results.
Estimating Ratios of Normalizing Constants Using Linked Importance Sampling

Ratios of normalizing constants for two distributions are needed in both Bayesian statistics, where they are used to compare models, and in statistical physics, where they correspond to differences in free energy. Two approaches have long been used to estimate ratios of normalizing constants. The `simple importance sampling' (SIS) or `free energy perturbation' method uses a sample drawn from just one of the two distributions. The `bridge sampling' or `acceptance ratio' estimate can be viewed as the ratio of two SIS estimates involving a bridge distribution. For both methods, difficult problems must be handled by introducing a sequence of intermediate distributions linking the two distributions of interest, with the final ratio of normalizing constants being estimated by the product of estimates of ratios for adjacent distributions in this sequence. Recently, work by Jarzynski, and independently by Neal, has shown how one can view such a product of estimates, each based on simple importance sampling using a single point, as an SIS estimate on an extended state space. This `Annealed Importance Sampling' (AIS) method produces an exactly unbiased estimate for the ratio of normalizing constants even when the Markov transitions used do not reach equilibrium. In this paper, I show how a corresponding `Linked Importance Sampling' (LIS) method can be constructed in which the estimates for individual ratios are similar to bridge sampling estimates. I show empirically that for some problems, LIS estimates are much more accurate than AIS estimates found using the same computation time, although for other problems the two methods have similar performance. Linked sampling methods similar to LIS are useful for other purposes as well.
Classification with Bayesian Neural Networks

I submitted entries for the two classification problems — “Catalysis” and “Gatineau” — in the Evaluating Predictive Uncertainty Challenge. My entry for Catalysis was the best one; my entry for Gatineau was the third best, behind two similar entries by Nitesh Chawla. 
 
The Catalysis dataset was later revealed to be about predicting a property of yeast proteins from expression levels of the genes encoding them. The nature of the Gatineau dataset has not been revealed, for proprietary reasons. The two datasets are similar in number of input variables that are available for predicting the binary outcome (617 for Catalysis, 1092 for Gatineau). They differ substantially in the number of cases available for training (1173 for Catalysis, 5176 for Gatineau) and in the fractions of cases that are in the two classes (43%/57% for Catalysis, 9%/91% for Gatineau).
The Short-Cut Metropolis Method

I show how one can modify the random-walk Metropolis MCMC method in such a way that a sequence of modified Metropolis updates takes little computation time when the rejection rate is outside a desired interval. This allows one to effectively adapt the scale of the Metropolis proposal distribution, by performing several such "short-cut" Metropolis sequences with varying proposal stepsizes. Unlike other adaptive Metropolis schemes, this method converges to the correct distribution in the same fashion as the standard Metropolis method.
Taking Bigger Metropolis Steps by Dragging Fast Variables

I show how Markov chain sampling with the Metropolis-Hastings algorithm can be modied so as to take bigger steps when the distribution being sampled from has the characteristic that its density can be quickly recomputed for a new point if this point diers from a previous point only with respect to a subset of \fast" variables. I show empirically that when using this method, the eciency of sampling for the remaining \slow" variables can approach what would be possible using Metropolis updates based on the marginal distribution for the slow variables.
Improving Classification When a Class Hierarchy is Available Using a Hierarchy-Based Prior

We introduce a new method for building classification models when we have prior knowledge of how the classes can be arranged in a hierarchy, based on how easily they can be distinguished. The new method uses a Bayesian form of the multinomial logit (MNL, a.k.a. “softmax”) model, with a prior that introduces correlations between the parameters for classes that are nearby in the tree. We compare the performance on simulated data of the new method, the ordinary MNL model, and a model that uses the hierarchy in different way. We also test the new method on a document labelling problem, and find that it performs better than the other methods, particularly when the amount of training data is small.
Multiple Alignment of Continuous Time Series

Multiple realizations of continuous-valued time series from a stochastic process often contain systematic variations in rate and amplitude. To leverage the information contained in such noisy replicate sets, we need to align them in an appropriate way (for example, to allow the data to be properly combined by adaptive averaging). We present the Continuous Profile Model (CPM), a generative model in which each observed time series is a non-uniformly subsampled version of a single latent trace, to which local rescaling and additive noise are applied. After unsupervised training, the learned trace represents a canonical, high resolution fusion of all the replicates. As well, an alignment in time and scale of each observation to this trace can be found by inference in the model. We apply CPM to successfully align speech signals from multiple speakers and sets of Liquid Chromatography-Mass Spectrometry proteomic data.
Improving Asymptotic Variance of MCMC Estimators: Non-reversible Chains are Better

I show how any reversible Markov chain on a finite state space that is irreducible, and hence suitable for estimating expectations with respect to its invariant distribution, can be used to construct a non-reversible Markov chain on a related state space that can also be used to estimate these expectations, with asymptotic variance at least as small as that using the reversible chain (typically smaller). The non-reversible chain achieves this improvement by avoiding (to the extent possible) transitions that backtrack to the state from which the chain just came. The proof that this modification cannot increase the asymptotic variance of an MCMC estimator uses a new technique that can also be used to prove Peskun's (1973) theorem that modifying a reversible chain to reduce the probability of staying in the same state cannot increase asymptotic variance. A non-reversible chain that avoids backtracking will often take little or no more computation time per transition than the original reversible chain, and can sometime produce a large reduction in asymptotic variance, though for other chains the improvement is slight. In addition to being of some practical interest, this construction demonstrates that non-reversible chains have a fundamental advantage over reversible chains for MCMC estimation. Research into better MCMC methods may therefore best be focused on non-reversible chains.
A Split-Merge Markov chain Monte Carlo Procedure for the Dirichlet Process Mixture Model

This article proposes a split-merge Markov chain algorithm to address the problem of inefficient sampling for conjugate Dirichlet process mixture models. Traditional Markov chain Monte Carlo methods for Bayesian mixture models, such as Gibbs sampling, can become trapped in isolated modes corresponding to an inappropriate clustering of data points. This article describes a Metropolis-Hastings procedure that can escape such local modes by splitting or merging mixture components. Our algorithm employs a new technique in which an appropriate proposal for splitting or merging components is obtained by using a restricted Gibbs sampling scan. We demonstrate empirically that our method outperforms the Gibbs sampler in situations where two or more components are similar in structure.
Slice Sampling

Markov chain sampling methods that adapt to characteristics of the distribution being sampled can be constructed using the principle that one can ample from a distribution by sampling uniformly from the region under the plot of its density function. A Markov chain that converges to this uniform distribution can be constructed by alternating uniform sampling in the vertical direction with uniform sampling from the horizontal "slice" defined by the current vertical position, or more generally, with some update that leaves the uniform distribution over this slice invariant. Such "slice sampling" methods are easily implemented for univariate distributions, and can be used to sample from a multivariate distribution by updating each variable in turn. This approach is often easier to implement than Gibbs sampling and more efficient than simple Metropolis updates, due to the ability of slice sampling to adaptively choose the magnitude of changes made. It is therefore attractive for routine and automated use. Slice sampling methods that update all variables simultaneously are also possible. These methods can adaptively choose the magnitudes of changes made to each variable, based on the local properties of the density function. More ambitiously, such methods could potentially adapt to the dependencies between variables by constructing local quadratic approximations. Another approach is to improve sampling efficiency by suppressing random walks. This can be done for univariate slice sampling by "overrelaxation," and for multivariate slice sampling by "reflection" from the edges of the slice.
Markov Chain Sampling for Non-linear State Space Models Using Embedded Hidden Markov Models

I describe a new Markov chain method for sampling from the distribution of the state sequences in a non-linear state space model, given the observation sequence. This method updates all states in the sequence simultaneously using an embedded Hidden Markov model (HMM). An update begins with the creation of a ``pool'' of K states at each time, by applying some Markov chain update to the current state. These pools define an embedded HMM whose states are indexes within this pool. Using the forward-backward dynamic programming algorithm, we can then efficiently choose a state sequence at random with the appropriate probabilities from the exponentially large number of state sequences that pass through states in these pools. I show empirically that when states at nearby times are strongly dependent, embedded HMM sampling can perform better than Metropolis methods that update one state at a time.
Transferring Prior Information Between Models Using Imaginary Data

Bayesian modeling is limited by our ability to formulate prior distributions that adequately represent our actual prior beliefs | a task that is especially diicult for realistic models with many interacting parameters. I show here how a prior distribution formulated for a simpler, more easily understood model can be used to modify the prior distribution of a more complex model. This is done by generating imaginary data from the simpler \donor" model, which is conditioned on in the more complex \recipient" model, eeectively transferring the donor model's well-speciied prior information to the recipient model. Such prior information transfers are also useful when comparing two complex models for the same data. Bayesian model comparison based on the Bayes factor is very sensitive to the prior distributions for each model's parameters, with the result that the wrong model may be favoured simply because the prior for the right model was not carefully formulated. This problem can be alleviated by modifying each model's prior to potentially incorporate prior information transferred from the other model. I discuss how these techniques can be implemented by simple Monte Carlo and by Markov chain Monte Carlo with annealed importance sampling. Demonstrations on models for two-way contingency tables and on graphical models for categorical data show that prior information transfer can indeed overcome deeciencies in prior speciication for complex models.
Improving Markov chain Monte Carlo Estimators by Coupling to an Approximating Chain

We show how large improvements in the accuracy of MCMC estimates for posterior expectations can sometimes be obtained by coupling a Markov chain that samples from the posterior distribution with a chain that samples from a Gaussian approximation to the posterior. Use of this method requires a coupling scheme that produces high correlation between the two chains. An eecient estimator can then be constructed that exploits this correlation, provided an accurate value for the expectation under the Gaussian approximation can be found, which for simple functions can be done analytically. Good coupling schemes are available for many Markov chain samplers, including Gibbs sampling with standard conditional distributions. For many moderate-dimensional problems, the improvement in accuracy using this method will be much greater than the overhead from simulating a second chain.
Annealed importance sampling

Simulated annealing—moving from a tractable distribution to a distribution of interest via a sequence of intermediate distributions—has traditionally been used as an inexact method of handling isolated modes in Markov chain samplers. Here, it is shown how one can use the Markov chain transitions for such an annealing sequence to define an importance sampler. The Markov chain aspect allows this method to perform acceptably even for high-dimensional problems, where finding good importance sampling distributions would otherwise be very difficult, while the use of importance weights ensures that the estimates found converge to the correct values as the number of annealing runs increases. This annealed importance sampling procedure resembles the second half of the previously-studied tempered transitions, and can be seen as a generalization of a recently-proposed variant of sequential importance sampling. It is also related to thermodynamic integration methods for estimating ratios of normalizing constants. Annealed importance sampling is most attractive when isolated modes are present, or when estimates of normalizing constants are required, but it may also be more generally useful, since its independent sampling allows one to bypass some of the problems of assessing convergence and autocorrelation in Markov chain samplers.
Inference for Belief Networks Using Coupling From the Past

Inference for belief networks using Gibbs sampling produces a distribution for unobserved variables that differs from the correct distribution by a (usually) unknown error, since convergence to the right distribution occurs only asymptotically. The method of "coupling from the past" samples from exactly the correct distribution by (conceptually) running dependent Gibbs sampling simulations from every possible starting state from a time far enough in the past that all runs reach the same state at time t = 0. Explicitly considering every possible state is intractable for large networks, however. We propose a method for layered noisy-or networks that uses a compact, but often imprecise, summary of a set of states. This method samples from exactly the correct distribution, and requires only about twice the time per step as ordinary Gibbs sampling, but it may require more simulation steps than would be needed if chains were tracked exactly.
Markov Chain Sampling Methods for Dirichlet Process Mixture Models

This article reviews Markov chain methods for sampling from the posterior distribution of a Dirichlet process mixture model and presents two new classes of methods. One new approach is to make Metropolis—Hastings updates of the indicators specifying which mixture component is associated with each observation, perhaps supplemented with a partial form of Gibbs sampling. The other new approach extends Gibbs sampling for these indicators by using a set of auxiliary parameters. These methods are simple to implement and are more efficient than previous ways of handling general Dirichlet process mixture models with non-conjugate priors.
On Deducing Conditional Independence from d-Separation in Causal Graphs with Feedback (Research Note)

Pearl and Dechter (1996) claimed that the d-separation criterion for conditional independence in acyclic causal networks also applies to networks of discrete variables that have feedback cycles, provided that the variables of the system are uniquely determined by the random disturbances. I show by example that this is not true in general. Some condition stronger than uniqueness is needed, such as the existence of a causal dynamics guaranteed to lead to the unique solution.