OpenRaiser/Intern-Atlas huggingface dataset:
An Empirical Evaluation of Possible Variations of Lazy Propagation
Venue: arXiv cs.AI 2012
Year: 2012

As real-world Bayesian networks continue to grow larger and more complex, it is important to investigate the possibilities for improving the performance of existing algorithms of probabilistic inference. Motivated by examples, we investigate the dependency of the performance of Lazy propagation on the message computation algorithm. We show how Symbolic Probabilistic Inference (SPI) and Arc-Reversal (AR) can be used for computation of clique to clique messages in the addition to the traditional use of Variable Elimination (VE). In addition, the paper resents the results of an empirical evaluation of the performance of Lazy propagation using VE, SPI, and AR as the message computation algorithm. The results of the empirical evaluation show that for most networks, the performance of inference did not depend on the choice of message computation algorithm, but for some randomly generated networks the choice had an impact on both space and time performance. In the cases where the choice had an impact, AR produced the best results.
Active Model Selection
Venue: arXiv cs.LG 2012
Year: 2012

Classical learning assumes the learner is given a labeled data sample, from which it learns a model. The field of Active Learning deals with the situation where the learner begins not with a training sample, but instead with resources that it can use to obtain information to help identify the optimal model. To better understand this task, this paper presents and analyses the simplified "(budgeted) active model selection" version, which captures the pure exploration aspect of many active learning problems in a clean and simple problem formulation. Here the learner can use a fixed budget of "model probes" (where each probe evaluates the specified model on a random indistinguishable instance) to identify which of a given set of possible models has the highest expected accuracy. Our goal is a policy that sequentially determines which model to probe next, based on the information observed so far. We present a formal description of this task, and show that it is NPhard in general. We then investigate a number of algorithms for this task, including several existing ones (eg, "Round-Robin", "Interval Estimation", "Gittins") as well as some novel ones (e.g., "Biased-Robin"), describing first their approximation properties and then their empirical performance on various problem instances. We observe empirically that the simple biased-robin algorithm significantly outperforms the other algorithms in the case of identical costs and priors.
An Extended Cencov-Campbell Characterization of Conditional Information Geometry
Venue: arXiv cs.LG 2012
Year: 2012

We formulate and prove an axiomatic characterization of conditional information geometry, for both the normalized and the nonnormalized cases. This characterization extends the axiomatic derivation of the Fisher geometry by Cencov and Campbell to the cone of positive conditional models, and as a special case to the manifold of conditional distributions. Due to the close connection between the conditional I-divergence and the product Fisher information metric the characterization provides a new axiomatic interpretation of the primal problems underlying logistic regression and AdaBoost.
PAC-learning bounded tree-width Graphical Models
Venue: arXiv cs.LG 2012
Year: 2012

We show that the class of strongly connected graphical models with treewidth at most k can be properly efficiently PAC-learnt with respect to the Kullback-Leibler Divergence. Previous approaches to this problem, such as those of Chow ([1]), and Ho gen ([7]) have shown that this class is PAC-learnable by reducing it to a combinatorial optimization problem. However, for k > 1, this problem is NP-complete ([15]), and so unless P=NP, these approaches will take exponential amounts of time. Our approach differs significantly from these, in that it first attempts to find approximate conditional independencies by solving (polynomially many) submodular optimization problems, and then using a dynamic programming formulation to combine the approximate conditional independence information to derive a graphical model with underlying graph of the tree-width specified. This gives us an efficient (polynomial time in the number of random variables) PAC-learning algorithm which requires only polynomial number of samples of the true distribution, and only polynomial running time.
Identifying Conditional Causal Effects
Venue: arXiv cs.AI 2012
Year: 2012

This paper concerns the assessment of the effects of actions from a combination of nonexperimental data and causal assumptions encoded in the form of a directed acyclic graph in which some variables are presumed to be unobserved. We provide a procedure that systematically identifies cause effects between two sets of variables conditioned on some other variables, in time polynomial in the number of variables in the graph. The identifiable conditional causal effects are expressed in terms of the observed joint distribution.
Factored Latent Analysis for far-field tracking data
Venue: arXiv cs.LG 2012
Year: 2012

This paper uses Factored Latent Analysis (FLA) to learn a factorized, segmental representation for observations of tracked objects over time. Factored Latent Analysis is latent class analysis in which the observation space is subdivided and each aspect of the original space is represented by a separate latent class model. One could simply treat these factors as completely independent and ignore their interdependencies or one could concatenate them together and attempt to learn latent class structure for the complete observation space. Alternatively, FLA allows the interdependencies to be exploited in estimating an effective model, which is also capable of representing a factored latent state. In this paper, FLA is used to learn a set of factored latent classes to represent different modalities of observations of tracked objects. Different characteristics of the state of tracked objects are each represented by separate latent class models, including normalized size, normalized speed, normalized direction, and position. This model also enables effective temporal segmentation of these sequences. This method is data-driven, unsupervised using only pairwise observation statistics. This data-driven and unsupervised activity classi- fication technique exhibits good performance in multiple challenging environments.
Sequential Information Elicitation in Multi-Agent Systems
Venue: arXiv cs.GT 2012
Year: 2012

We introduce the study of sequential information elicitation in strategic multi-agent systems. In an information elicitation setup a center attempts to compute the value of a function based on private information (a-k-a secrets) accessible to a set of agents. We consider the classical multi-party computation setup where each agent is interested in knowing the result of the function. However, in our setting each agent is strategic,and since acquiring information is costly, an agent may be tempted not spending the efforts of obtaining the information, free-riding on other agents' computations. A mechanism which elicits agents' secrets and performs the desired computation defines a game. A mechanism is 'appropriate' if there exists an equilibrium in which it is able to elicit (sufficiently many) agents' secrets and perform the computation, for all possible secret vectors.We characterize a general efficient procedure for determining an appropriate mechanism, if such mechanism exists. Moreover, we also address the existence problem, providing a polynomial algorithm for verifying the existence of an appropriate mechanism.
Predictive State Representations: A New Theory for Modeling Dynamical Systems
Venue: arXiv cs.AI 2012
Year: 2012

Modeling dynamical systems, both for control purposes and to make predictions about their behavior, is ubiquitous in science and engineering. Predictive state representations (PSRs) are a recently introduced class of models for discrete-time dynamical systems. The key idea behind PSRs and the closely related OOMs (Jaeger's observable operator models) is to represent the state of the system as a set of predictions of observable outcomes of experiments one can do in the system. This makes PSRs rather different from history-based models such as nth-order Markov models and hidden-state-based models such as HMMs and POMDPs. We introduce an interesting construct, the systemdynamics matrix, and show how PSRs can be derived simply from it. We also use this construct to show formally that PSRs are more general than both nth-order Markov models and HMMs/POMDPs. Finally, we discuss the main difference between PSRs and OOMs and conclude with directions for future work.
Evidence-invariant Sensitivity Bounds
Venue: arXiv cs.AI 2012
Year: 2012

The sensitivities revealed by a sensitivity analysis of a probabilistic network typically depend on the entered evidence. For a real-life network therefore, the analysis is performed a number of times, with different evidence. Although efficient algorithms for sensitivity analysis exist, a complete analysis is often infeasible because of the large range of possible combinations of observations. In this paper we present a method for studying sensitivities that are invariant to the evidence entered. Our method builds upon the idea of establishing bounds between which a parameter can be varied without ever inducing a change in the most likely value of a variable of interest.
Variational Chernoff Bounds for Graphical Models
Venue: arXiv cs.LG 2012
Year: 2012

Recent research has made significant progress on the problem of bounding log partition functions for exponential family graphical models. Such bounds have associated dual parameters that are often used as heuristic estimates of the marginal probabilities required in inference and learning. However these variational estimates do not give rigorous bounds on marginal probabilities, nor do they give estimates for probabilities of more general events than simple marginals. In this paper we build on this recent work by deriving rigorous upper and lower bounds on event probabilities for graphical models. Our approach is based on the use of generalized Chernoff bounds to express bounds on event probabilities in terms of convex optimization problems; these optimization problems, in turn, require estimates of generalized log partition functions. Simulations indicate that this technique can result in useful, rigorous bounds to complement the heuristic variational estimates, with comparable computational cost.
Hybrid Influence Diagrams Using Mixtures of Truncated Exponentials
Venue: arXiv cs.AI 2012
Year: 2012

Mixtures of truncated exponentials (MTE) potentials are an alternative to discretization for representing continuous chance variables in influence diagrams. Also, MTE potentials can be used to approximate utility functions. This paper introduces MTE influence diagrams, which can represent decision problems without restrictions on the relationships between continuous and discrete chance variables, without limitations on the distributions of continuous chance variables, and without limitations on the nature of the utility functions. In MTE influence diagrams, all probability distributions and the joint utility function (or its multiplicative factors) are represented by MTE potentials and decision nodes are assumed to have discrete state spaces. MTE influence diagrams are solved by variable elimination using a fusion algorithm.
A Robust Specification Theory for Modal Event-Clock Automata
Venue: arXiv cs.LO 2012
Year: 2012

In a series of recent work, we have introduced a general framework for quantitative reasoning in specification theories. The contribution of this paper is to show how this framework can be applied to yield a robust specification theory for timed specifications.
Refinement for Transition Systems with Responses
Venue: arXiv cs.LO 2012
Year: 2012

Motivated by the response pattern for property specifications and applications within flexible workflow management systems, we report upon an initial study of modal and mixed transition systems in which the must transitions are interpreted as must eventually, and in which implementations can contain may behaviors that are resolved at run-time. We propose Transition Systems with Responses (TSRs) as a suitable model for this study. We prove that TSRs correspond to a restricted class of mixed transition systems, which we refer to as the action-deterministic mixed transition systems. We show that TSRs allow for a natural definition of deadlocked and accepting states. We then transfer the standard definition of refinement for mixed transition systems to TSRs and prove that refinement does not preserve deadlock freedom. This leads to the proposal of safe refinements, which are those that preserve deadlock freedom. We exemplify the use of TSRs and (safe) refinements on a small medication workflow.
Transfer Function Synthesis without Quantifier Elimination
Venue: arXiv cs.LO 2012
Year: 2012

Traditionally, transfer functions have been designed manually for each operation in a program, instruction by instruction. In such a setting, a transfer function describes the semantics of a single instruction, detailing how a given abstract input state is mapped to an abstract output state. The net effect of a sequence of instructions, a basic block, can then be calculated by composing the transfer functions of the constituent instructions. However, precision can be improved by applying a single transfer function that captures the semantics of the block as a whole. Since blocks are program-dependent, this approach necessitates automation. There has thus been growing interest in computing transfer functions automatically, most notably using techniques based on quantifier elimination. Although conceptually elegant, quantifier elimination inevitably induces a computational bottleneck, which limits the applicability of these methods to small blocks. This paper contributes a method for calculating transfer functions that finesses quantifier elimination altogether, and can thus be seen as a response to this problem. The practicality of the method is demonstrated by generating transfer functions for input and output states that are described by linear template constraints, which include intervals and octagons.
Securing Electronic Transactions to Support E-Commerce
Venue: arXiv cs.CR 2012
Year: 2012

Many reports regarding online fraud in varieties media create skepticism for conducting transactions online, especially through an open network such as the Internet, which offers no security whatsoever. Therefore, encryption technology is vitally important to support secure e-commerce on the Internet. Two well-known encryption representing symmetric and asymmetric cryptosystems as well as their applications are discussed in this paper. Encryption is a key technology to secure electronic transactions. However, there are several challenges such as crytoanalysis or code breaker as well as US export restrictions on encryption. The future threat is the development of quantum computers, which makes the existing encryption technology cripple.
Assessment of SAR Image Filtering using Adaptive Stack Filters
Venue: arXiv cs.CV 2012
Year: 2012

Stack filters are a special case of non-linear filters. They have a good performance for filtering images with different types of noise while preserving edges and details. A stack filter decomposes an input image into several binary images according to a set of thresholds. Each binary image is then filtered by a Boolean function, which characterizes the filter. Adaptive stack filters can be designed to be optimal; they are computed from a pair of images consisting of an ideal noiseless image and its noisy version. In this work we study the performance of adaptive stack filters when they are applied to Synthetic Aperture Radar (SAR) images. This is done by evaluating the quality of the filtered images through the use of suitable image quality indexes and by measuring the classification accuracy of the resulting images.
Quantum-like Tests for Contextual Querying
Venue: arXiv cs.IR 2012
Year: 2012

Tests are essential in Information Retrieval (IR), in order to evaluate the effectiveness of a query. Tests intended to exhibit the sense of words in con-text were undertaken and linked with Quantum Mechanics (QM). Poll tests were undertaken on heterogeneous media such as music and polysemy in foreign languages. Interference effects are shown in the results. Bell inequality was used leading to a significant spread in the results of the poll tests but without violating the classical limit. Then an automatic pertinence measure tool on texts has been developed using the HAL algorithm using an orthonormal vector decomposition model. In this case the spread in the values can lead to the violation of the Bell inequality even beyond Cirel'son bound.
Approximating minimum-cost edge-covers of crossing biset-families
Venue: arXiv cs.DS 2012
Year: 2012

An ordered pair $\hat{S}=(S,S^+)$ of subsets of $V$ is called a {\em biset} if $S \subseteq S^+$; $(V-S^+,V-S)$ is the co-biset of $\hat{S}$. Two bisets $\hat{X},\hat{Y}$ intersect if $X \cap Y \neq \emptyset$ and cross if both $X \cap Y \neq \emptyset$ and $X^+ \cup Y^+ \neq V$. The intersection and the union of two bisets $\hat{X},\hat{Y}$ is defined by $\hat{X} \cap \hat{Y} = (X \cap Y, X^+ \cap Y^+)$ and $\hat{X} \cup \hat{Y} = (X \cup Y,X^+ \cup Y^+)$. A biset-family ${\cal F}$ is crossing (intersecting) if $\hat{X} \cap \hat{Y}, \hat{X} \cup \hat{Y} \in {\cal F}$ for any $\hat{X},\hat{Y} \in {\cal F}$ that cross (intersect). A directed edge covers a biset $\hat{S}$ if it goes from $S$ to $V-S^+$. We consider the problem of covering a crossing biset-family ${\cal F}$ by a minimum-cost set of directed edges. While for intersecting ${\cal F}$, a standard primal-dual algorithm computes an optimal solution, the approximability of the case of crossing ${\cal F}$ is not yet understood, as it includes several NP-hard problems, for which a poly-logarithmic approximation was discovered only recently. Let us say that a biset-family ${\cal F}$ is $k$-regular if $\hat{X} \cap \hat{Y}, \hat{X} \cup \hat{Y} \in {\cal F}$ for any $\hat{X},\hat{Y} \in {\cal F}$ with $|V-(X \cup Y)| \geq k+1$ that intersect. In this paper we obtain an $O(\log |V|)$-approximation algorithm for arbitrary crossing ${\cal F}$; if in addition both ${\cal F}$ and the family of co-bisets of ${\cal F}$ are $k$-regular, our ratios are: $O(\log \frac{|V|}{|V|-k})$ if $|S^+ \setminus S|=k$ for all $\hat{S} \in {\cal F}$, and $O(\frac{|V|}{|V|-k} \log \frac{|V|}{|V|-k})$ if $|S^+ \setminus S| \leq k$ for all $\hat{S} \in {\cal F}$. Using these generic algorithms, we derive approximation algorithms for some network design problems.
Equivalence between Priority Queues and Sorting in External Memory
Venue: arXiv cs.DS 2012
Year: 2012

A priority queue is a fundamental data structure that maintains a dynamic ordered set of keys and supports the followig basic operations: insertion of a key, deletion of a key, and finding the smallest key. The complexity of the priority queue is closely related to that of sorting: A priority queue can be used to implement a sorting algorithm trivially. Thorup \cite{thorup2007equivalence} proved that the converse is also true in the RAM model. In particular, he designed a priority queue that uses the sorting algorithm as a black box, such that the per-operation cost of the priority queue is asymptotically the same as the per-key cost of sorting. In this paper, we prove an analogous result in the external memory model, showing that priority queues are computationally equivalent to sorting in external memory, under some mild assumptions. The reduction provides a possibility for proving lower bounds for external sorting via showing a lower bound for priority queues.
DAMS: Distributed Adaptive Metaheuristic Selection
Venue: arXiv cs.NE 2012
Year: 2012

We present a distributed generic algorithm called DAMS dedicated to adaptive optimization in distributed environments. Given a set of metaheuristic, the goal of DAMS is to coordinate their local execution on distributed nodes in order to optimize the global performance of the distributed system. DAMS is based on three-layer architecture allowing node to decide distributively what local information to communicate, and what metaheuristic to apply while the optimization process is in progress. The adaptive features of DAMS are first addressed in a very general setting. A specific DAMS called SBM is then described and analyzed from both a parallel and an adaptive point of view. SBM is a simple, yet efficient, adaptive distributed algorithm using an exploitation component allowing nodes to select the metaheuristic with the best locally observed performance, and an exploration component allowing nodes to detect the metaheuristic with the actual best performance. The efficiency of BSM-DAMS is demonstrated through experimentations and comparisons with other adaptive strategies (sequential and distributed).
Iterative Design of L_p Digital Filters
Venue: arXiv cs.IT 2012
Year: 2012

The design of digital filters is a fundamental process in the context of digital signal processing. The purpose of this paper is to study the use of $\lp$ norms (for $2 < p < \infty$) as design criteria for digital filters, and to introduce a set of algorithms for the design of Finite (FIR) and Infinite (IIR) Impulse Response digital filters based on the Iterative Reweighted Least Squares (IRLS) algorithm. The proposed algorithms rely on the idea of breaking the $\lp$ filter design problem into a sequence of approximations rather than solving the original $\lp$ problem directly. It is shown that one can efficiently design filters that arbitrarily approximate a desired $\lp$ solution (for $2 < p < \infty$) including the commonly used $l_\infty$ (or minimax) design problem. A method to design filters with different norms in different bands is presented (allowing the user for better control of the signal and noise behavior per band). Among the main contributions of this work is a method for the design of {\it magnitude} $\lp$ IIR filters. Experimental results show that the algorithms in this work are robust and efficient, improving over traditional off-the-shelf optimization tools. The group of proposed algorithms form a flexible collection that offers robustness and efficiency for a wide variety of digital filter design applications.
Minimum-Length Scheduling with Finite Queues: Solution Characterization and Algorithmic Framework
Venue: arXiv cs.IT 2012
Year: 2012

We consider a set of transmitter-receiver pairs, or links, that share a common channel and address the problem of emptying backlogged queues at the transmitters in minimum time. The problem amounts to determining activation subsets of links and their time durations to form a minimum-length schedule. The problem of scheduling has been studied under various formulations before. In this paper, we present fundamental insights and solution characterizations that include: (i) showing that the complexity of the problem remains high for any continuous and increasing rate function, (ii) formulating and proving sufficient and necessary optimality conditions of two base scheduling strategies that correspond to emptying the queues using "one-at-a-time" or "all-at-once" strategies, (iii) presenting and proving the tractability of the special case in which the transmission rates are functions only of the cardinality of the link activation sets. These results are independent of physical-layer system specifications and are valid for any form of rate function. We then develop an algorithmic framework. The framework encompasses exact as well as sub-optimal, but fast, scheduling algorithms, all under a unified principle design. Through computational experiments we finally investigate the performance of several specific algorithms.