nils-herrmann/clean_scirepeval huggingface dataset:
Robust Probabilistic Inference in Distributed Systems

Probabilistic inference problems arise naturally in distributed systems such as sensor networks and teams of mobile robots. Inference algorithms that use message passing are a natural fit for distributed systems, but they must be robust to the failure situations that arise in real-world settings, such as unreliable communication and node failures. Unfortunately, the popular sum--product algorithm can yield very poor estimates in these settings because the nodes' beliefs before convergence can be arbitrarily different from the correct posteriors. In this paper, we present a new message passing algorithm for probabilistic inference which provides several crucial guarantees that the standard sum--product algorithm does not. Not only does it converge to the correct posteriors, but it is also guaranteed to yield a principled approximation at any point before convergence. In addition, the computational complexity of the message passing updates depends only upon the model, and is independent of the network topology of the distributed system. We demonstrate the approach with detailed experimental results on a distributed sensor calibration task using data from an actual sensor network deployment.
Thin Junction Tree Filters for Simultaneous Localization and Mapping

The Simultaneous Localization and Mapping problem is a fundamental problem in mobile robotics: while a robot navigates in an unknown environment, it must incrementally build a map of its surroundings and localize itself within that map. Traditional approaches to the problem are based upon Kalman filters, but suffer from complexity issues: first, the belief state grows quadratically in the size of the map; and second, the filtering operation can take time quadratic in the size of the map. I present a linear-space filter that maintains a tractable approximation of the belief state as a thin junction tree. The junction tree grows under measurement and motion updates and is periodically "thinned" to remain tractable. The time complexity of the filter operation is linear in the size of the map. I also present simple enhancements that permit constant-time approximate filtering.
Junction tree algorithms for solving sparse linear systems

In this technical report we present a self-contained introduction to algorithms that solve sparse linear systems by message passing on a junction tree (a.k.a. clique tree). When solving the systemAx = b whereA is a sparse n×nmatrix, these methods require O(n · w3) time and O(n · w2) space, where w is the treewidth of A’s sparsity graph. These algorithms are useful for parallel or distributed solutions to linear systems, or to efficiently solve sets of similar linear systems.
Sample Propagation

Rao–Blackwellization is an approximation technique for probabilistic inference that flexibly combines exact inference with sampling. It is useful in models where conditioning on some of the variables leaves a simpler inference problem that can be solved tractably. This paper presents Sample Propagation, an efficient implementation of Rao–Blackwellized approximate inference for a large class of models. Sample Propagation tightly integrates sampling with message passing in a junction tree, and is named for its simple, appealing structure: it walks the clusters of a junction tree, sampling some of the current cluster's variables and then passing a message to one of its neighbors. We discuss the application of Sample Propagation to conditional Gaussian inference problems such as switching linear dynamical systems.
Maximum Entropy Probabilistic Logic

Recent research has shown there are two types of uncertainty that can be expressed in first-order logic— propositional and statistical uncertainty—and that both types can be represented in terms of probability spaces. However, these efforts have fallen short of providing a general account of how to design probability measures for these spaces; as a result, we lack a crucial component of any system that reasons under these types of uncertainty. In this paper, we describe an automatic procedure for defining such measures in terms of a probabilistic knowledge base. In particular, we employ the principle of maximum entropy to select measures that are consistent with our knowledge and that make the fewest assumptions in doing so. This approach yields models of first-order uncertainty that are principled, intuitive, and economical in their representation.
Linear-time inference in Hierarchical HMMs

The hierarchical hidden Markov model (HHMM) is a generalization of the hidden Markov model (HMM) that models sequences with structure at many length/time scales [FST98]. Unfortunately, the original inference algorithm is rather complicated, and takes O(T3) time, where is the length of the sequence, making it impractical for many domains. In this paper, we show how HHMMs are a special kind of dynamic Bayesian network (DBN), and thereby derive a much simpler inference algorithm, which only takes O(T) time. Furthermore, by drawing the connection between HHMMs and DBNs, we enable the application of many standard approximation techniques to further speed up inference.
Maximum-Entropy Probabilistic Logics

I present probabilistic extensions to propositional and first-order logic. The general approach is to define a probability distribution over possible worlds; this distribution is chosen as the unique distribution that agrees with a set of probability constraints over a set of logical sentences and that has maximal entropy. Such distributions are well studied in the Statistics and Machine Learning literature, and using them not only results in simple, intuitive models, but makes available standard inference and learning algorithms.
Grammatical Bigrams

Unsupervised learning algorithms have been derived for several statistical models of English grammar, but their computational complexity makes applying them to large data sets intractable. This paper presents a probabilistic model of English grammar that is much simpler than conventional models, but which admits an efficient EM training algorithm. The model is based upon grammatical bigrams, i.e., syntactic relationships between pairs of words. We present the results of experiments that quantify the representational adequacy of the grammatical bigram model, its ability to generalize from labelled data, and its ability to induce syntactic structure from large amounts of raw text.
PREDATOR: an OR-DBMS with enhanced data types

We are witnessing an explosion in the volume and complexity of digital information that people want to access and analyze. Much of this data is “multi-media” like images, video, audio, and documents. Several other kinds of complex and semi-structured data are also important, including geographical objects, chemical and biological structures, mathematical entities like matrices and equations, and financial data like time-series. Databaae systems must efficiently support queries over such richly structured datw otherwise, they will fast become “roadklll on the information super-highway” [DeW95]. PREDATOR is an object-relational DBMS being developed at Cornell University. It uses a novel “Enhanced ADT” (E-ADT) technology to meet this challenge. EADTs add semantics to the complex data types, resulting in dramatically improved performance, while retaining the extensibility of the database system. The demonstration shows the following features of PREDATOR: (a) standard relational query processing capability through SQL queries, (b) extensibility with complex data types (images, audio, video, documents), (c) content-baaed feature extraction, indexing and retrieval using path indexes, (d) extensible optimization of E-ADT expressions by reordering and merging, (e) WWW/Javabased user interface that supports extensible querying and display mechanisms. These capabilities are demonstrated individually, aa well aa in the context of the following applications : (1) a multimedia digital library application involving thousands of images from museums of rare art, (2) a GIS application demonstrating raater, point and polygon data, and multi-dimensional indexing. The PREDATOR system will be publicly awailable in summer 1997, along with source code. The goal of the releaae of this software is for it to be widely used for research and educational purposes. We will provide extensive webbased documentation of both the usage and the internal design details. No other freely available research system provides these advanced features and capabilities; consequently, we hope that PREDATOR will be used aa a test-bed for a variety of research ideaa.
Generative model based polyphonic music transcription

We present a model for simultaneous tempo and polyphonic pitch tracking. Our model, a form of dynamic Bayesian network (Murphy, K.P., 2002), embodies a transparent and computationally tractable approach to this acoustic analysis problem. An advantage of our approach is that it places emphasis on modeling the sound generation procedure. It provides a clear framework in which both high level (cognitive) prior information on music structure can be coupled with low level (acoustic physical) information in a principled manner to perform the analysis. The model is readily extensible to more complex sound generation processes.
Application of Cluster Variation Method to Genetic Linkage Analysis

In this paper we discuss the application of the Cluster Variation Method to the problem of genetic linkage analysis. The objective of genetic linkage analysis is to link an observed affection status (phenotype) of all individuals in a pedigree to the inheritance pattern observed certain locations on the chromosome. The Bayesian approach introduced by [7] is computationally intractable for large pedigrees and we propose to apply the Cluster Variation Method to approximately compute quantities such as the likelihood of the data. We obtain good results on an important subproblem, the computation of the intractable probability distribution of the inheritance pattern. These results encourage the incorporation of the disease model into the inheritance model. 1 Genetic Linkage Analysis One of the major questions in genetics is to link an observed disease to a certain region on the chromosome. When a disease is thought to be (partially) genetic, scientists collect a large data base of pedigree data from families in which usually a few affected individuals occur. Most or all of the family members are genotyped with markers, a technique which can be used to reconstruct the way genetic information has passed along the generations (very similar to tests for fathership). The objective of genetic linkage analysis is to link an observed affection status (phenotype) of all individuals in a pedigree to the inheritance pattern observed certain locations on the chromosome. The affection status can be directly observed for each individual in the pedigree. The disease model that determines the relation between affection status and genotype may be unknown as well as the number of locations on the chromosome that cause the disease. Through DNA-analysis there is information about the inheritance pattern, i.e. information about which genes were passed on from parents to children. For reconstructing the inheritance pattern in a pedigree, there exist algorithms, which are exact and make use of all available marker data. They are either exponential in the number of locations on the chromosome taken into account [1] or the size of the pedigree . For each locus on the chromosome, the inheritance pattern within a pedigree is denoted by the inheritance vector , which contains two binary valued variables for each non-founder1. The subscript denotes the location at the chromosomes. The binary vari1Non-founders are individuals in the pedigree whose parents are also in the pedigree.
A Dynamical Bayesian Network for Tempo and Polyphonic Pitch Tracking

We present a model for simultaneous tempo and polyphonic pitch tracking. Both these acoustic analysis tasks are difficult and, arguably, no satisfactory solution currently exists for polyphonic pitch tracking. Our model, a form of Dynamical Bayesian Network, embodies a transparent and computationally tractable approach to this acoustic analysis problem. An advantage of our approach is that it places emphasis on modelling the sound generation procedure. It provides a clear framework in which both high level (cognitive) prior information on music structure can be coupled with low level (acoustic physical) information in a principled manner to perform the analysis. The model is readily extensible to more complex sound generation pro-
Computational neuro-science

2 Neural information processing is noisy 6 2.1 Poisson Processes . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1.1 Interval distribution . . . . . . . . . . . . . . . . . . . 10 2.1.2 Integration reduces CV . . . . . . . . . . . . . . . . . . 12 2.1.3 Refractoriness . . . . . . . . . . . . . . . . . . . . . . . 12 2.2 ISI distributions .  First passage times . . . . . . . . . . . . . . . . . . . . . . . . 15 2.3.1 Diffusion . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.3.2 First passage time distribution . . . . . . . . . . . . . . 17 2.3.3 Laplace transformation . . . . . . . . . . . . . . . . . . 18 2.3.4 Scale invariance . . . . . . . . . . . . . . . . . . . . . . 20 2.3.5 Approximate scale invariance. . . . . . . . . . . . . . . 20 2.4 Integrate and Fire Neuron . . . . . . . . . . . . . . . . . . . .  25 2.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Tempo Tracking Rhythm by Sequential Monte

We present a probabilistic generative model for timing deviations in expressive music. performance. The structure of the proposed model is equivalent to a switching state space model. We formulate two well known music recognition problems, namely tempo tracking and automatic transcription (rhythm quantization) as filtering and maximum a posteriori (MAP) state estimation tasks. The inferences are carried out using sequential Monte Carlo integration (particle filtering) techniques. For this purpose, we have derived a novel Viterbi algorithm for Rao-Blackwellized particle filters, where a subset of the hidden variables is integrated out. The resulting model is suitable for realtime tempo tracking and transcription and hence useful in a number of music applications such as adaptive automatic accompaniment and score typesetting.
A p proxim ate A lgorith m s for N eural-B ayesian A pproaches.

A b stract We describe two specific examples of neural-Bayesian approaches for complex mod­ eling tasks: survival analysis and multitask learning. In both cases, we can come up with reasonable priors on the parameters of the neural network. As a result, the Bayesian approaches improve their (maximum likelihood) frequentist counter­ parts dramatically. By illustrating their application on the models under study, we review and compare algorithms that can be used for Bayesian inference: Laplace approximation, variational algorithms, Monte Carlo sampling, and empirical Bayes.
On tempo tracking: Tempogram Representation and Kalman filtering

We formulate tempo tracking in a Bayesian framework where a tempo tracker is modeled as a stochastic dynamical system. The tempo is modeled as a hidden state variable of the system and is estimated by a Kalman filter. The Kalman filter operates on a Tempogram, a wavelet-like multiscale expansion of a real performance. An important advantage of our approach is that it is possible to formulate both off-line or real-time algorithms. The simulation results on a systematically collected set of MIDI piano performances of Yesterday and Michelle by the Beatles shows accurate tracking of approximately 90% of the beats.
Stimulus segmentation in a stochastic neural network with exogenous signals

Segmentation by synchrony of firing is investigated in stochastic neural networks with binary neurons. The network is trained by presenting sparsely coded patterns with a Hebbian-type learning rule. Retrieval of these patterns and synchrony of firing is investigated by presenting one or multiple patterns simultaneously to the network. For stimuli consisting of several superimposed patterns the model correctly predicts a high covariance of firing (indicating synchrony) for neurons which are excited by the same pattern in the stimulus. The model gives a negative covariance for neurons which are stimulated by different patterns (indicating absence of synchrony). To obtain useful covariance levels, noise levels should neither be too large, since this would induce complete random firing, nor be too small, since this would minimize the chance to switch between firing assemblies. Furthermore, the strength of the multipattern input may neither be too large, as this would overrule the memory function of the network, nor so small that the desired firing assemblies are never attained.
An Introduction to Stochastic Neural Networks an Introduction to Stochastic Neural Net- Works

x1  How does the brain compute? Particularly in the last hundred years have we gathered an enormous amount of experimental ndings that shed some light on this question. The picture that has emerged is that the neuron is the central computing element of the brain which performs a non-linear input to output mapping between its synaptic inputs and its spiky output. The neurons are connected by synaptic junctions, thus forming a neural network. A central question is how such a neural network implements brain functions such as vision, audition and motor control. These questions are to a certain extend premature, because our knowledge of the functioning of the neuron and the synaptic process itself is only partial and much remains to be discovered. Nevertheless, it is interesting to see what emergent behavior arises in a network of very simple neurons. The pioneering work in this direction was done by McCulloch and Pitts 1) in the '40s. Taking the thresholding property of neurons to the extreme, they proposed that neurons perform logical operations on their inputs, such as AND and OR. One can show that a network of such neurons, when properly wired, can perform any logical function and is equivalent to a Turing machine. When considering neural networks, an important distinction is between feed-forward networks and recurrent networks. In feed-forward networks, the neurons can be labeled such that each neuron only receives input from neurons with lower label. Thus, one can identify input neurons, which receive no input from other neurons and whose activity depends only on the sensory stimulus,
Neural Networks: Best Practice in Europe: Proceedings of the Stichting Neurale Netwerken Conference 1997

After the explosion of research on neural networks in the eighties, the nineties have seen a boom in industrial applications of neural networks. In contrast to the large output of publications in international neural network journals, which reflects the increase in neural network research, the large number of successful applications are less accessible. To make a survey of successful applications in industry in Europe, a project called SIENA was initiated with support from the European Community. The aim of SIENA was to assess the business impact of neural networks. Data were collected on both the supplier side and the end-user side of the market. In addition, case studies of successful "money-making" applications using neural networks were gathered. This book contains detailed descriptions of some of the applications.
Neural Networks: Artificial Intelligence and Industrial Applications

Synchronized oscillatory activity in the visual system has been proposed as a temporal label of feature-linking, by assuming that spatial segmentation coding of a visual scene is based on synchronized oscillations within a segment's neural representation and on desynchronized signals among different segments. In an extended hypothesis it was stated that segmentation may be based on any type of synchronization and desynchronization, being rhythmical or non-rhythmical, stimulus-dominated or internally generated. This was supported by experimental evidence obtained by multiple microelectrode recordings from the visual cortex of cats and monkeys. It includes stimulus specific synchronization and facilitation at zero-delay correlation during short stimulus-locked responses and during sustained oscillations (35-90 Hz). We now extend the synchronization hypothesis to spatio-temporal segmentation coding by suggesting that the synchronized single or repetetive interruptions of excitations define temporally precise internal representations. By applying this to natural vision, we can argue that during phases of slowly changing retinal images (like ocular fixation and smooth pursuit) oscillations may prevent perceptual "smearing" by interrupting the flow of visual information repetetively. However, when a visual object suddenly changes its position, ongoing oscillations should immediately be interrupted by the single excitation-inhibition cycle evoked by the objects displacement. Such stimulus-locked cycles might acts as "reset" signals of prior, and evoke new, segmentations of an object against the remaining scene. Experimental evidence in support of this hypothesis includes our observations on suppression of oscillatory by stimuluslocked responses and on changes of oscillation frequencies with velocity, size and intensity of a stimulus.
A latent mixed membership model for relational data

Modeling relational data is an important problem for modern data analysis and machine learning. In this paper we propose a Bayesian model that uses a hierarchy of probabilistic assumptions about the way objects interact with one another in order to learn latent groups, their typical interaction patterns, and the degree of membership of objects to groups. Our model explains the data using a small set of parameters that can be reliably estimated with an efficient inference algorithm. In our approach, the set of probabilistic assumptions may be tailored to a specific application domain in order to incorporate intuitions and/or semantics of interest. We demonstrate our methods on simulated data and we successfully apply our model to a data set of protein-to-protein interactions.
Inferring regulatory networks using a hierarchical Bayesian graphical Gaussian model

In this paper, we propose a new formalism based on graphical Gaussian model (GGM) to infer genetic regulatory networks. A hierarchical Bayesian prior for the precision matrix of the GGM is introduced to impose a bias toward sparse graph structure. We show that the MAP estimation of the undirected graph can be readily obtained by a variant of the well-known Lasso regression algorithm. Then we integrate the estimated graph with the “CHIp-Chip” protein-binding location data to infer the regulatory networks using a post-processing algorithm. Compared to extant Bayesian network (BN) models for similar tasks, our formalism captures statistical dependencies among genes that are more prevalent and plausible in the biological system. Our approach is also capable of modeling partial correlations between mRNA levels and therefore goes beyond clustering-based approaches. We applied our method to an expression microarray data (more than 6000 genes) together with a genome-wide location analysis data (more than 100 TFs). Evaluated on the consistency with the GO annotations, our method achieves a significantly better performance than clustering and BN learning algorithms in discovering genetic regulatory modules.