nils-herrmann/clean_scirepeval huggingface dataset:
Expectation propagation for signal detection in flat-fading channels

In this paper, we propose a new Bayesian receiver for signal detection in flat- fading channels. First, the detection prob- lem is formulated as an inference problem in a hybrid dynamic system. Then, an ex- pectation propagation algorithm is proposed to address the inference problem. Our sim- ulations demonstrate that the new receiver achieves accurate detection without the aid of any training symbols or decision feedbacks. Due to the increased use of various wireless ser- vices in our daily life, adaptive signal detection in flat Rayleigh fading channels has become an impor- tant research topic. Different approaches have been proposed to address this problem (5, 2, 1, 61. The most accurate approach so far employs sequential Monte Carlo filters and smoothers 11, 61. In this ap proach, signal detection and decoding is treated as an estimation problem in a hybrid dynamic system that has both continuous channel coefficients and discrete uncoded information bits. Exact estimation is intractable, so sequential Monte Carlo is used to get an approximation. This approach gives excel- lent performance in simulations (l, 61. However, the computational cost is prohibitive in practice. In this paper, we develop an efficient expectation propagation (EP) algorithm for hybrid dynamic sys- tems and apply it to adaptive detection in flat-fading channels. EP (4) is a powerful extension of belief propagation (BP). BP has been used in digital com- munications, such as Turbo decoding (3). However, BP can handle only discrete distributions or con- tinuous distributions in the exponential family. In contrast, EP not only iteratively propagates the in- formation in the dynamic system, but also projects exact messages into the exponential family. Con- sequently, EP can efficiently approximate Bayesian
Bayesian Color Constancy with Non-Gaussian Models

We present a Bayesian approach to color constancy which utilizes a non-Gaussian probabilistic model of the image formation process. The parameters of this model are estimated directly from an uncalibrated image set and a small number of additional algorithmic parameters are chosen using cross validation. The algorithm is empirically shown to exhibit RMS error lower than other color constancy algorithms based on the Lambertian surface reflectance model when estimating the illuminants of a set of test images. This is demonstrated via a direct performance comparison utilizing a publicly available set of real world test images and code base.
A Useful Distribution for Fitting Discrete Data: Revival of the COM-Poisson

A useful discrete distribution (the Conway-Maxwell-Poisson, or COM-Poisson) is revived and its statistical and probabilistic properties are introduced and explored. The COMPoisson is a two-parameter extension of the Poisson distribution that generalizes some wellknown discrete distributions (Poisson, Bernoulli, Geometric). It also leads to the generalization of distributions derived from these discrete distributions (viz. the Binomial and Negative Binomial). We describe three methods for estimating the COM-Poisson’s parameters: The first is a fast simple weighted least squares method, which leads to estimates that are sufficiently accurate for practical purposes. The second method, using maximum likelihood, can be used to refine the initial estimates. This method requires iterations and is more computationally intensive. The third estimation method is Bayesian. Using the conjugate prior, the posterior density of the COM-Poisson parameters is easily computed. The COM-Poisson is a flexible distribution that can account for over/under dispersion commonly encountered in count data. We also explore two sets of real world data demonstrating the flexibility and elegance of the COM-Poisson in fitting count data which do not seem to follow the Poisson distribution.
Bayesian inference, entropy, and the multinomial distribution

Instead of maximum-likelihood or MAP, Bayesian inference encourages the use of predictive densities and evidence scores. This is illustrated in the context of the multinomial distribution, where predictive estimates are often used but rarely described as Bayesian. By using an entropy approximation to the evidence, many Bayesian quantities can be expressed in information-theoretic terms. For example, testing whether two samples come from the same distribution or testing whether two variables are independent boils down to a mutual information score (with appropriate smoothing). The same analysis can be applied to discrete Markov chains to get a test for Markovianity.
The ‘summation hack’ as an outlier model

The ‘summation hack’ is the ad-hoc replacement of a product by a sum in a probabilistic expression. This hack is usually explained as a device to cope with outliers, with no formal derivation. This note shows that the hack does make sense probabilistically, and can be best thought of as replacing an outlier-sensitive likelihood with an outlier-tolerant one. This interpretation exposes the hack as an assumption about the outliers, allowing us to determine when it makes sense to use the hack.
Novelty and redundancy detection in adaptive filtering

This paper addresses the problem of extending an adaptive information filtering system to make decisions about the novelty and redundancy of relevant documents. It argues that relevance and redundance should each be modelled explicitly and separately. A set of five redundancy measures are proposed and evaluated in experiments with and without redundancy thresholds. The experimental results demonstrate that the cosine similarity metric and a redundancy measure based on a mixture of language models are both effective for identifying redundant documents.
Expectation-Propogation for the Generative Aspect Model

The generative aspect model is an extension of the multinomial model for text that allows word probabilities to vary stochastically across documents. Previous results with aspect models have been promising, but hindered by the computational difficulty of carrying out inference and learning. This paper demonstrates that the simple variational methods of Blei et al. (2001) can lead to inaccurate inferences and biased learning for the generative aspect model. We develop an alternative approach that leads to higher accuracy at comparable cost. An extension of Expectation-Propagation is used for inference and then embedded in an EM algorithm for learning. Experimental results are presented for both synthetic and real data sets.
Learning distance measures from labeled data — An overview

Canonical distance measure Fit generative models to the classes. The distance measure is the probability that two individuals were sampled from different generative models. Model are often fit separately for each dimension, corresponding to a naive Bayes assumption. There are many papers on this. See (Blanzieri & Ricci, 1999) and references therein. Similar to, but more general than, relevance weighting approaches below.
Bayesian model averaging is not model combination

In a recent paper, Domingos (2000) compares Bayesian model averaging (BMA) to other model combination methods on some benchmark data sets, is surprised that BMA performs worst, and suggests that BMA may be flawed. These results are actually not surprising, especially in light of an earlier paper by Domingos (1997) where it was shown that model combination works by enriching the space of hypotheses, not by approximating a Bayesian model average. And the only flaw with BMA is the belief that it is an algorithm for model combination, when it is not.
Bayesian spectrum estimation of unevenly sampled nonstationary data

Spectral estimation methods typically assume stationarity and uniform spacing between samples of data. The non-stationarity of real data is usually accommodated by windowing methods, while the lack of uniformly-spaced samples is typically addressed by methods that “fill in” the data in some way. This paper presents a new approach to both of these problems: We use a non-stationary Kalman filter within a Bayesian framework to jointly estimate all spectral coefficients instantaneously. The new method works regardless of how the signal samples are spaced. We illustrate the method on several data sets, showing that it provides more accurate estimation than the Lomb-Scargle method and several classical spectral estimation methods.
Psychophysical experiments on the PicHunter image retrieval system

Psychophysical experiments were conducted on PicHunter, a content-based image retrieval (CBIR) experimental prototype with the following properties: (1) Based on a model of how users respond, it uses Bayes's rule to predict what target users want, given their actions. (2) It possesses an extremely simple user interface. (3) It employs an entropy-based scheme to improve convergence. (4) It introduces a paradigm for assessing the performance of CBIR systems. Experiments 1-3 studied human judgment of image similarity to obtain data for the model. Experiment 4 studied the importance of using: (a) semantic information, (b) memory of earlier input and (c) relative and absolute judgments of similarity. Experiment 5 tested an approach that we propose for comparing performances of CBIR systems objectively. Finally, experiment 6 evaluated the most informative display-updating scheme that is based on entropy minimization, and confirmed earlier simulation results. These experiments represent one of the first attempts to quantify CBIR performance based on psychophysical studies, and they provide valuable data for improving CBIR algorithms. Even though they were designed with PicHunter in mind, their results can be applied to any CBIR system and, more generally, to any system that involves judgment of image similarity by humans.
Expectation Propagation for approximate Bayesian inference

This paper presents a new deterministic approximation technique in Bayesian networks. This method, "Expectation Propagation," unifies two previous techniques: assumed-density filtering, an extension of the Kalman filter, and loopy belief propagation, an extension of belief propagation in Bayesian networks. Loopy belief propagation, because it propagates exact belief states, is useful for a limited class of belief networks, such as those which are purely discrete.  Propagation approximates the belief states by only retaining expectations, such as mean and varitmce, and iterates until these expectations are consistent throughout the network. This makes it applicable to hybrid networks with discrete and continuous nodes. Experiments with Gaussian mixture models show Expectation Propagation to be donvincingly better than methods with similar computational cost: Laplace's method, variational Bayes, and Monte Carlo.  Propagation also provides an efficient algorithm for training Bayes point machine classifiers.
Inferring a Gaussian distribution

A common question in statistical modeling is “which out of a continuum of models are likely to have generated this data?” For the Gaussian class of models, this question can be answered completely and exactly. This paper derives the exact posterior distribution over the mean and variance of the generating distribution, i.e. p(m, V|X), as well as the marginals p(m|X) and p(V|X). It also derives p(X|Gaussian), the probability that the data came from any Gaussian whatsoever. From this we can get the posterior predictive density p(x|X), which has the most practical importance. The analysis is done for noninformative priors and for arbitrary conjugate priors. The presentation borrows from MacKay (1995). The paper concludes with a simulated classification experiment demonstrating the advantage of the Bayesian method over maximumlikelihood and unbiased estimation.
Using lower bounds to approxi-mate integrals

Variational lower bounds are a simple and efficient way to approximate Bayesian integrals. By bounding the integrand at every point, we obtain a bound on the integral value. Variational Bayes (Attias, 1999) and convex duality methods (Jaakkola & Jordan 1999a; 1999b) are of this type, and—as shown in this paper—so is the approximation method of Cheeseman & Stutz (1996). In each case, EM is used to maximize the bound. The EM updates are derived for Gaussian mixtures and multinomial mixtures with conjugate priors. Experimental results show the lower bound method to be less accurate than Laplace’s method, but often simpler.
A family of algorithms for approximate Bayesian inference

One of the major obstacles to using Bayesian methods for pattern recognition has been its computational expense. This thesis presents an approximation technique that can perform Bayesian inference faster and more accurately than previously possible. This method, “Expectation Propagation,” unifies and generalizes two previous techniques: assumed-density filtering, an extension of the Kalman filter, and loopy belief propagation, an extension of belief propagation in Bayesian networks. The unification shows how both of these algorithms can be viewed as approximating the true posterior distribution with simpler distribution, which is close in the sense of KL-divergence.  Propagation exploits the best of both algorithms: the generality of assumed-density filtering and the accuracy of loopy belief propagation. 
Loopy belief propagation, because it propagates exact belief states, is useful for limited types of belief networks, such as purely discrete networks.  Propagation approximates the belief states with expectations, such as means and variances, giving it much wider scope.  Propagation also extends belief propagation in the opposite direction—propagating richer belief states which incorporate correlations between variables. 
This framework is demonstrated in a variety of statistical models using synthetic and real-world data. On Gaussian mixture problems, Expectation Propagation is found, for the same amount of computation, to be convincingly better than rival approximation techniques: Monte Carlo, Laplace's method, and variational Bayes. For pattern recognition, Expectation Propagation provides an algorithm for training Bayes Point Machine classifiers that is faster and more accurate than any previously known. The resulting classifiers outperform Support Vector Machines on several standard datasets, in addition to having a comparable training time.  Propagation can also be used to choose an appropriate feature set for classification, via Bayesian model selection.
Document image decoding using Iterated Complete Path search with subsampled heuristic scoring

It has been shown that the computation time of document image decoding can be significantly reduced by employing heuristics in the search for the best decoding of a text line. In the Iterated Complete Path (ICP) method, template matches are performed only along the best path found by dynamic programming on each iteration. When the best path stabilizes, the decoding is optimal and no more template matches need to be performed. In this way, only a tiny fraction of potential template matches must be evaluated, and the computation time is typically dominated by the evaluation of the initial heuristic upper bound for each template at each location in the image. The time to compute this bound depends on the resolution at which the matching scores are found. At lower resolution, the heuristic computation is reduced, but because a weaker bound is used, the number of Viterbi iterations is increased. We present the optimal (lowest upper-bound) heuristic for any degree of subsampling of multilevel template and/or interpolation, for use in text line decoding with ICP. The optimal degree of subsampling depends on image quality, but it is typically found that a small amount of template subsampling is effective in reducing the overall decoding time.
The EP energy function and minimization schemes

j fj(x)τj) (3) The ti(x) must be positive but they do not have to be proper densities. However p(x), which is not being approximated, does need to be a proper density. It doesn’t have to be the actual “prior” in the problem; you could choose it to be 1, for example, if x has finite domain. ) does not have to be in an exponential family, but it will make your life easier if it is. The EP (dual) energy function is min ν max λ (n− 1) log ∫
Automatic Choice of Dimensionality for PCA

A central issue in principal component analysis (PCA) is choosing the number of principal components to be retained. By interpreting PCA as density estimation, we show how to use Bayesian model selection to estimate the true dimensionality of the data. The resulting estimate is simple to compute yet guaranteed to pick the correct dimensionality, given enough data. The estimate involves an integral over the Steifel manifold of k-frames, which is difficult to compute exactly. But after choosing an appropriate parameterization and applying Laplace's method, an accurate and practical estimator is obtained. In simulations, it is convincingly better than cross-validation and other proposed algorithms, plus it runs much faster.
The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments

This paper presents the theory, design principles, implementation and performance results of PicHunter, a prototype content-based image retrieval (CBIR) system. In addition, this document presents the rationale, design and results of psychophysical experiments that were conducted to address some key issues that arose during PicHunter's development. The PicHunter project makes four primary contributions to research on CBIR. First, PicHunter represents a simple instance of a general Bayesian framework which we describe for using relevance feedback to direct a search. With an explicit model of what users would do, given the target image they want, PicHunter uses Bayes's rule to predict the target they want, given their actions. This is done via a probability distribution over possible image targets, rather than by refining a query. Second, an entropy-minimizing display algorithm is described that attempts to maximize the information obtained from a user at each iteration of the search. Third, PicHunter makes use of hidden annotation rather than a possibly inaccurate/inconsistent annotation structure that the user must learn and make queries in. Finally, PicHunter introduces two experimental paradigms to quantitatively evaluate the performance of the system, and psychophysical experiments are presented that support the theoretical claims.
Document image decoding using iterated complete path search

The computation time of Document Image Decoding can be significantly reduced by employing heuristics in the search for the best decoding of a text line. By using a cheap upper bound on template match scores, up to 99.9% of the potential template matches can be avoided. In the Iterated Complete Path method, template matches are performed only along the best path found by dynamic programming on each iteration. When the best path stabilizes, the decoding is optimal and no more template matches need be performed. Computation can be further reduced in this scheme by exploiting the incremental nature of the Viterbi iterations. Because only a few trellis edge weights have changed since the last iteration, most of the backpointers do not need to be updated. We describe how to quickly identify these backpointers, without forfeiting optimality of the path. Together these improvements provide a 30x speedup over previous implementations of Document Image Decoding.
Distance measures as prior probabilities

Many learning algorithms, especially nonparametric ones, use distance measures as a source of prior knowledge about the domain. This paper shows how the work of Baxter and Yianilos provides a formal equivalence between distance measures and prior probability distributions in Bayesian inference. The prior distribution applies either to how the data was generated or to the shape of the discrimination boundary. This perspective is useful for extending distance-based algorithms to new feature spaces and especially for learning distance measures on those spaces.
Old and New Matrix Algebra Useful for Statistics

The partials with respect to the numerator are laid out according to the shape of Y while the partials with respect to the denominator are laid out according to the transpose of X. For example, dy/dx is a column vector while dy/dx is a row vector (assuming x and y are column vectors—otherwise it is flipped). Each of these derivatives can be tediously computed via partials, but this section shows how they instead can be computed with matrix manipulations. The material is based on Magnus and Neudecker (1988).
From Hidden Markov Models to Linear Dynamical Systems

Hidden Markov Models (HMMs) and Linear Dynamical Systems (LDSs) are based on the same assumption: a hidden state variable, of which we can make noisy measurements, evolves with Markovian dynamics. Both have the same independence diagram and consequently the learning and inference algorithms for both have the same structure. The only diierence is that the HMM uses a discrete state variable with arbitrary dynamics and arbitrary measurements while the LDS uses a continuous state variable with linear-Gaussian dynamics and measurements. We show how the forward-backward equations for the HMM, specialized to linear-Gaussian assumptions, lead directly to Kalman ltering and Rauch-Tung-Streibel smoothing. We also investigate the most general possible modeling assumptions which lead to eecient recursions in the case of continuous state variables.
Psychophysical studies of the performance of an image database retrieval system

We describe psychophysical experiments conducted to study PicHunter, a content-based image retrieval (CBIR) system. Experiment 1 studies the importance of using (a) semantic information, (2) memory of earlier input and (3) relative, rather than absolute, judgements of image similarity. The target testing paradigm is used in which a user must search for an image identical to a target. We find that the best performance comes from a version of PicHunter that uses only semantic cues, with memory and relative similarity judgements. Second best is use of both pictorial and semantic cues, with memory and relative similarity judgements. Most reports of CBIR systems provide only qualitative measures of performance based on how similar retrieved images are to a target. Experiment 2 puts PicHunter into this context with a more rigorous test. We first establish a baseline for our database by measuring the time required to find an image that is similar to a target when the images are presented in random order. Although PicHunter's performance is measurably better than this, the test is weak because even random presentation of images yields reasonably short search times. This casts doubt on the strength of results given in other reports where no baseline is established.
An optimized interaction strategy for Bayesian relevance feedback

A new algorithm and systematic evaluation is presented for searching a database via relevance feedback. It represents a new image display strategy for the PicHunter system. The algorithm takes feedback in the form of relative judgments ("item A is more relevant than item B") as opposed to the stronger assumption of categorical relevance judgments ("item A is relevant but item B is not"). It also exploits a learned probabilistic model of human behavior to make better use of the feedback it obtains. The algorithm can be viewed as an extension of indexing schemes like the k-d tree to a stochastic setting, hence the name "stochastic-comparison search." In simulations, the amount of feedback required for the new algorithm scales like log/sub 2/ |D|, where |D| is the size of the database, while a simple query-by-example approach scales like |D|/sup /spl alpha//, where /spl alpha/<1 depends on the structure of the database. This theoretical advantage is reflected by experiments with real users on a database of 1500 stock photographs.