nils-herrmann/clean_scirepeval huggingface dataset:
Renewal Strings for Cleaning Astronomical Databases

Large astronomical databases obtained from sky surveys such as the SuperCOSMOS Sky Surveys (SSS) invariably suffer from spurious records coming from artefactual effects of the telescope, satellites and junk objects in orbit around earth and physical defects on the photographic plate or CCD. Though relatively small in number these spurious records present a significant problem in many situations where they can become a large proportion of the records potentially of interest to a given astronomer. We have developed renewal strings, a probabilistic technique combining the Hough transform, renewal processes and hidden Markov models which has proven highly effective in this context. The methods are applied to the SSS data to develop a dataset of spurious object detections, along with confidence measures, which can allow this unwanted data to be removed from consideration. These methods are general and can be adapted to any other astronomical survey data.
Image Modeling with Position-Encoding Dynamic Trees

This paper describes the position-encoding dynamic tree (PEDT). The PEDT is a probabilistic model for images that improves on the dynamic tree by allowing the positions of objects to play a part in the model. This increases the flexibility of the model over the dynamic tree and allows the positions of objects to be located and manipulated. This paper motivates and defines this form of probabilistic model using the belief network formalism. A structured variational approach for inference and learning in the PEDT is developed, and the resulting variational updates are obtained, along with additional implementation considerations that ensure the computational cost scales linearly in the number of nodes of the belief network. The PEDT model is demonstrated and compared with the dynamic tree and fixed tree. The structured variational learning method is compared with mean field approaches.
Scientific DataMining, Integration, and Visualization

This report summarises the workshop on Scientific Data Mining, Integration and Visualization (SDMIV)
held at the e-Science Institute, Edinburgh (eSI[1] ) on 24-25 October 2002, and presents a set of
recommendations arising from the discussion that took place there. The aims of the workshop were threefold:
(A) To inform researchers in the SDMIV communities of the infrastructural advances being made
by computing initiatives, such as the Grid; (B) To feed back requirements from the SDMIV areas to
those developing the computational infrastructure; and (C) To foster interaction among all these
communities, since the coordinated efforts of all of them will be required to realise the potential for
scientific knowledge extraction offered by e-science initiatives worldwide.
DME Handout: Support Vector Machines

Support Vector Machines (SVMs) are a relatively new concept in supervised learning, but since the publication of  in 1995 they have been applied to a wide variety of problems. In many ways the application of SVMs to almost any learning problem mirrors the enthusiasm (and fashionability) that was observed for neural networks in the second half of the 1980’s. The ingredients of the SVM had, in fact, been around for a decade or so, but they were not put together until the early 90’s. The two key ideas of support vector machines are
Dynamic Structure Super-Resolution

The problem of super-resolution involves generating feasible higher resolution images, which are pleasing to the eye and realistic, from a given low resolution image. This might be attempted by using simple filters for smoothing out the high resolution blocks or through applications where substantial prior information is used to imply the textures and shapes which will occur in the images. In this paper we describe an approach which lies between the two extremes. It is a generic unsupervised method which is usable in all domains, but goes beyond simple smoothing methods in what it achieves. We use a dynamic tree-like architecture to model the high resolution data. Approximate conditioning on the low resolution image is achieved through a mean field approach.
UAI '00: Proceedings of the 16th Conference in Uncertainty in Artificial Intelligence

It sounds good when knowing the uai 97 proceedings of the 13th conference in uncertainty in artificial intelligence in this website. This is one of the books that many people looking for. In the past, many people ask about this book as their favourite book to read and collect. And now, we present hat you need quickly. It seems to be so happy to offer you this famous book. It will not become a unity of the way for you to get amazing benefits at all. But, it will serve something that will let you get the best time and moment to spend for reading the book.
Dynami Positional Trees for Stru tural Image Analysis

We onsider the problem of stru tural image analysis and in parti ular the inferen e of s ene properties from image data. We are espe ially on erned with image de omposition, that is obtaining the hara teristi parts of an image and the relationships between them. The omponents of an image are not independent of ea h other; ertain obje ts are expe ted to o ur together, and obje ts are made up of di erent sub omponents. One way of thinking of this problem is by analogy with parsing a language; we are interested in parsing images. However, the important hara teristi s and stru ture in an image is signi antly di erent from linguisti data.
Dynamic Trees: A Structured Variational Method Giving Efficient Propagation Rules

Dynamic trees are mixtures of tree structured belief networks. They solve some of the problems of fixed tree networks at the cost of making exact inference intractable. For this reason approximate methods such as sampling or mean field approaches have been used. However, mean field approximations assume a factorised distribution over node states. Such a distribution seems unlikely in the posterior, as nodes are highly correlated in the prior. Here a structured variational approach is used, where the posterior distribution over the non-evidential nodes is itself approximated by a dynamic tree. It turns out that this form can be used tractably and efficiently. The result is a set of update rules which can propagate information through the network to obtain both a full variational approximation, and the relevant marginals. 
 
The propagation rules are more efficient than the mean field approach and give noticeable quantitative and qualitative improvement in the inference. The marginals calculated give better approximations to the posterior than loopy propagation on a small toy problem.
Truncated covariance matrices and Toeplitz methods in Gaussian processes

Gaussian processes are a limit extension of neural networks. Standard Gaussian process techniques use a squared exponential covariance function. Here, the use of truncated covariances is proposed. Such covariances have compact support. Their use speeds up matrix inversion and increases precision. Furthermore they allow the use of speedy, memory efficient Toeplitz inversion for high dimensional grid based Gaussian process predictors.
Efficient Covariance Matrix Methods for Bayesian Gaussian Processes and Hopfield Neural Networks

Covariance matrices are important in many areas of neural modelling. In Hop eld networks they are used to form the weight matrix which controls the autoassociative properties of the network. In Gaussian processes, which have been shown to be the in nite neuron limit of many regularised feedforward neural networks, covariance matrices control the form of Bayesian prior distribution over function space. This thesis examines interesting modi cations to the standard covariance matrix methods to increase functionality or e ciency of these neural techniques. Firstly the problem of adapting Gaussian process priors to perform regression on switching regimes is tackled. This involves the use of block covariance matrices and Gibbs sampling methods. Then the use of Toeplitz methods is proposed for Gaussian process regression where sampling positions can be chosen. A comparison is made between Hop eld weight matrices, and sample covariances. This allows work on sample covariances to be used to estimate the eigenvalue structure of standard Hop eld weight matrices. This structure of Hop eld weight matrices enables the development of e cient incremental and local learning techniques for Hop eld networks. A new Hop eld learning rule is introduced. It is shown both empirically and mathematically that such learning methods give a higher capacity than the usual Hebb rule. Furthermore it is shown that this rule is more able to deal with the storage of correlated patterns. The basins of attraction of this learning rule are examined empirically. It is found that they are larger, rounder and more evenly distributed than those of the Hebb rule. Lastly we show using iterated function sequence methods that the learning rule acts as a palimpsest (or forgetful) learning rule, and does not su er from catastrophic forgetting. 3
Palimpsest memories: a new high-capacity forgetful learning rule for Hopfield networks

Palimpsest or forgetful learning rules for attractor neural networks do not suuer from catastrophic forgetting. Instead they selectively forget older memories in order to store new patterns. Standard palimpsest learning algorithms have a capacity of up to 0:05n, where n is the size of the network. Here a new learning rule is introduced. This rule is local and incremental. It is shown that it has palimpsest properties, and it has a palimpsest capacity of about 0:25n, much higher than the capacity of standard palimpsest schemes. It is shown that the algorithm acts as an iterated function sequence on the space of matrices, and this is used to illustrate the performance of the learning rule.
Increasing the Capacity of a Hopfield Network without Sacrificing Functionality

Hopfield networks are commonly trained by one of two algorithms. The simplest of these is the Hebb rule, which has a low absolute capacity of n/(2ln n), where n is the total number of neurons. This capacity can be increased to n by using the pseudo-inverse rule. However, capacity is not the only consideration. It is important for rules to be local (the weight of a synapse depends ony on information available to the two neurons it connects), incremental (learning a new pattern can be done knowing only the old weight matrix and not the actual patterns stored) and immediate (the learning process is not a limit process). The Hebbian rule is all of these, but the pseudo-inverse is never incremental, and local only if not immediate. The question addressed by this paper is, ‘Can the capacity of the Hebbian rule be increased without losing locality, incrementality or immediacy?’
Object categorization by learned universal visual dictionary

This paper presents a new algorithm for the automatic recognition of object classes from images (categorization). Compact and yet discriminative appearance-based object class models are automatically learned from a set of training images. The method is simple and extremely fast, making it suitable for many applications such as semantic image retrieval, Web search, and interactive image editing. It classifies a region according to the proportions of different visual words (clusters in feature space). The specific visual words and the typical proportions in each object are learned from a segmented training set. The main contribution of this paper is twofold: i) an optimally compact visual dictionary is learned by pair-wise merging of visual words from an initially large dictionary. The final visual words are described by GMMs. ii) A novel statistical measure of discrimination is proposed which is optimized by each merge operation. High classification accuracy is demonstrated for nine object classes on photographs of real objects viewed under general lighting conditions, poses and viewpoints. The set of test images used for validation comprise: i) photographs acquired by us, ii) images from the Web and iii) images from the recently released Pascal dataset. The proposed algorithm performs well on both texture-rich objects (e.g. grass, sky, trees) and structure-rich ones (e.g. cars, bikes, planes)
Bayesian Conditional Random Fields

We propose Bayesian Conditional Random Fields (BCRFs) for classifying interdependent and structured data, such as sequences, images or webs. BCRFs are a Bayesian approach to training and inference with conditional random fields, which were previously trained by maximizing likelihood (ML) (Lafferty et al., 2001). Our framework eliminates the problem of overfitting, and offers the full advantages of a Bayesian treatment. Unlike the ML approach, we estimate the posterior distribution of the model parameters during training, and average over this posterior during inference. We apply an extension of EP method, the power EP method, to incorporate the partition function. For algorithmic stability and accuracy, we flatten the approximation structures to avoid two-level approximations. We demonstrate the superior prediction accuracy of BCRFs over conditional random fields trained with ML or MAP on synthetic and real datasets.
A useful distribution for fitting discrete data: revival of the Conway–Maxwell–Poisson distribution

A useful discrete distribution (the Conway–Maxwell–Poisson distribution) is revived and its statistical and probabilistic properties are introduced and explored. This distribution is a two‐parameter extension of the Poisson distribution that generalizes some well‐known discrete distributions (Poisson, Bernoulli and geometric). It also leads to the generalization of distributions derived from these discrete distributions (i.e. the binomial and negative binomial distributions). We describe three methods for estimating the parameters of the Conway–Maxwell–Poisson distribution. The first is a fast simple weighted least squares method, which leads to estimates that are sufficiently accurate for practical purposes. The second method, using maximum likelihood, can be used to refine the initial estimates. This method requires iterations and is more computationally intensive. The third estimation method is Bayesian. Using the conjugate prior, the posterior density of the parameters of the Conway–Maxwell–Poisson distribution is easily computed. It is a flexible distribution that can account for overdispersion or underdispersion that is commonly encountered in count data. We also explore two sets of real world data demonstrating the flexibility and elegance of the Conway–Maxwell–Poisson distribution in fitting count data which do not seem to follow the Poisson distribution.
Diagram structure recognition by Bayesian conditional random fields

Hand-drawn diagrams present a complex recognition problem. Elements of the diagram are often individually ambiguous, and require context to be interpreted. We present a recognition method based on Bayesian conditional random fields (BCRFs) that jointly analyzes all drawing elements in order to incorporate contextual cues. The classification of each object affects the classification of its neighbors. BCRFs allow flexible and correlated features, and take both spatial and temporal information into account. BCRFs estimate the posterior distribution of parameters during training, and average predictions over the posterior for testing. As a result of model averaging, BCRFs avoid the overfitting problems associated with maximum likelihood training. We also incorporate automatic relevance determination (ARD), a Bayesian feature selection technique, into BCRFs. The result is significantly lower error rates compared to ML- and MAP-trained CRFs.
Discriminative models, not discriminative training

Suppose you are given a dataset of pairs (x, c) where c is a class variable and x is a vector of features. Given a new x, you want to predict its class. The generative i.i.d. approach to this problem posits a model family p(x, c | θ) = p(x | c, λ)p(c | π) (1) and chooses the best parameters θ = {λ, π} by maximizing (or integrating over) the joint distribution (where D denotes the data): p(D, θ) = p(θ) ∏
Structured Region Graphs: Morphing EP into GBP

GBP and EP are two successful algorithms for approximate probabilistic inference, which are based on different approximation strategies. An open problem in both algorithms has been how to choose an appropriate approximation structure. We introduce "structured region graphs," a formalism which marries these two strategies, reveals a deep connection between them, and suggests how to choose good approximation structures. In this formalism, each region has an internal structure which defines an exponential family, whose sufficient statistics must be matched by the parent region. Reduction operators on these structures allow conversion between EP and GBP free energies. Thus it is revealed that all EP approximations on discrete variables are special cases of GBP, and conversely that some well-known GBP approximations, such as overlapping squares, are special cases of EP. Furthermore, region graphs derived from EP have a number of good structural properties, including maxent-normality and overall counting number of one. The result is a convenient framework for producing high-quality approximations with a user-adjustable level of complexity.
Divergence measures and message passing

This paper presents a unifying view of messagepassing algorithms, as methods to approximate a complex Bayesian network by a simpler network with minimum information divergence. In this view, the difference between mean-field methods and belief propagation is not the amount of structure they model, but only the measure of loss they minimize (‘exclusive’ versus ‘inclusive’ Kullback-Leibler divergence). In each case, message-passing arises by minimizing a localized version of the divergence, local to each factor. By examining these divergence measures, we can intuit the types of solution they prefer (symmetry-breaking, for example) and their suitability for different tasks. Furthermore, by considering a wider variety of divergence measures (such as alpha-divergences), we can achieve different complexity and performance goals.
Vision texture for annotation

This paper demonstrates a new application of computer vision to digital libraries — the use of texture forannotation, the description of content. Vision-based annotation assists the user in attaching descriptions to large sets of images and video. If a user labels a piece of an image aswater, a texture model can be used to propagate this label to other “visually similar” regions. However, a serious problem is that no single model has been found that is good enough to match reliably human perception of similarity in pictures. Rather than using one model, the system described here knows several texture models, and is equipped with the ability to choose the one that “best explains” the regions selected by the user for annotating. If none of these models suffices, then it creates new explanations by combining models. Examples of annotations propagated by the system on natural scenes are given. The system provides an average gain of four to one in label prediction for a set of 98 images.
Exemplar-based Likelihoods Using the PDF Projection Theorem

Consider a vision task where we want to classify shapes by matching them to exemplars. To make the system modular and capable of handling prior knowledge, we want this process to be probabilistic. That is, we want a likelihood p(x|c) which is high when the observation x matches the exemplar c, and low otherwise. We also want the likelihood to be normalized so that comparisons between p(x|c = 1) and p(x|c = 2) make sense. Suppose our matching score is f(x, c), e.g. the chamfer distance between x’s edge map and the exemplar edge map. One common approach is to define
Power EP

This note describes power EP, an extension of Expectation Propagation (EP) that makes the computations more tractable. In this way, power EP is applicable to a wide variety of models, much more than EP. Instead of minimizing KL-divergence at each step, power EP minimizes α-divergence. This minimization turns out to be equivalent to minimizing KL-divergence with the exact distribution raised to a power. By choosing this power to cancel exponents, the problem may be substantially simplified. The resulting approximation is not the same as regular EP, but in practice is still very good, and allows tackling problems which are intractable under regular EP.
A comparison of numerical optimizers for logistic regression

Logistic regression is a workhorse of statistics and is closely related to method s used in Machine Learning, including the Perceptron and the Support Vector Machine. This note compares eight different algorithms for computing the maximum a-posteriori parameter estimate. A full derivation of each algorithm is given. In particular, a new derivation of Iterative Scaling is given which applies more generally than the conventional one. A new derivation is also given for the Modified Iterative Scaling algorithm of Collins et al. (2002). Most of the alg orithms operate in the primal space, but can also work in dual space. All algorithms are compared in terms of computational complexity by experiments on large data sets. The fastest algorithms turn out to be conjugate gradient ascent and quasi-Newton algorithms, which far outstrip Iterative Scaling and its variants.
Tree-structured Approximations by Expectation Propagation

Approximation structure plays an important role in inference on loopy graphs. As a tractable structure, tree approximations have been utilized in the variational method of Ghahramani & Jordan (1997) and the sequential projection method of Frey et al. (2000). However, belief propagation represents each factor of the graph with a product of single-node messages. In this paper, belief propagation is extended to represent factors with tree approximations, by way of the expectation propagation framework. That is, each factor sends a "message" to all pairs of nodes in a tree structure. The result is more accurate inferences and more frequent convergence than ordinary belief propagation, at a lower cost than variational trees or double-loop algorithms.
Building Statistical Models by Visualization

Practical applications of statistical methods usually involve assumptions about the domain, such as independence, normality, linearity, and the choice of variables. I will describe a set of graphical tools which simplify the process of learning about a domain and checking model assumptions. Some of these tools have been around for years, but vastly underused by statistical learning researchers. Using real-world examples, I will illustrate how visualization can be used for model selection, identifying exceptional cases, and interpreting the results of learning algorithms.