OpenRaiser/Intern-Atlas huggingface dataset:
Computing Posterior Probabilities of Structural Features in Bayesian Networks
Venue: arXiv cs.LG 2012
Year: 2012

We study the problem of learning Bayesian network structures from data. Koivisto and Sood (2004) and Koivisto (2006) presented algorithms that can compute the exact marginal posterior probability of a subnetwork, e.g., a single edge, in O(n2n) time and the posterior probabilities for all n(n-1) potential edges in O(n2n) total time, assuming that the number of parents per node or the indegree is bounded by a constant. One main drawback of their algorithms is the requirement of a special structure prior that is non uniform and does not respect Markov equivalence. In this paper, we develop an algorithm that can compute the exact posterior probability of a subnetwork in O(3n) time and the posterior probabilities for all n(n-1) potential edges in O(n3n) total time. Our algorithm also assumes a bounded indegree but allows general structure priors. We demonstrate the applicability of the algorithm on several data sets with up to 20 variables.
Products of Hidden Markov Models: It Takes N>1 to Tango
Venue: arXiv cs.LG 2012
Year: 2012

Products of Hidden Markov Models(PoHMMs) are an interesting class of generative models which have received little attention since their introduction. This maybe in part due to their more computationally expensive gradient-based learning algorithm,and the intractability of computing the log likelihood of sequences under the model. In this paper, we demonstrate how the partition function can be estimated reliably via Annealed Importance Sampling. We perform experiments using contrastive divergence learning on rainfall data and data captured from pairs of people dancing. Our results suggest that advances in learning and evaluation for undirected graphical models and recent increases in available computing power make PoHMMs worth considering for complex time-series modeling tasks.
Virtual Vector Machine for Bayesian Online Classification
Venue: arXiv cs.LG 2012
Year: 2012

In a typical online learning scenario, a learner is required to process a large data stream using a small memory buffer. Such a requirement is usually in conflict with a learner's primary pursuit of prediction accuracy. To address this dilemma, we introduce a novel Bayesian online classi cation algorithm, called the Virtual Vector Machine. The virtual vector machine allows you to smoothly trade-off prediction accuracy with memory size. The virtual vector machine summarizes the information contained in the preceding data stream by a Gaussian distribution over the classi cation weights plus a constant number of virtual data points. The virtual data points are designed to add extra non-Gaussian information about the classi cation weights. To maintain the constant number of virtual points, the virtual vector machine adds the current real data point into the virtual point set, merges two most similar virtual points into a new virtual point or deletes a virtual point that is far from the decision boundary. The information lost in this process is absorbed into the Gaussian distribution. The extra information provided by the virtual points leads to improved predictive accuracy over previous online classification algorithms.
Quantifying the Strategyproofness of Mechanisms via Metrics on Payoff Distributions
Venue: arXiv cs.GT 2012
Year: 2012

Strategyproof mechanisms provide robust equilibrium with minimal assumptions about knowledge and rationality but can be unachievable in combination with other desirable properties such as budget-balance, stability against deviations by coalitions, and computational tractability. In the search for maximally-strategyproof mechanisms that simultaneously satisfy other desirable properties, we introduce a new metric to quantify the strategyproofness of a mechanism, based on comparing the payoff distribution, given truthful reports, against that of a strategyproof "reference" mechanism that solves a problem relaxation. Focusing on combinatorial exchanges, we demonstrate that the metric is informative about the eventual equilibrium, where simple regretbased metrics are not, and can be used for online selection of an effective mechanism.
Improving Compressed Counting
Venue: arXiv cs.DS 2012
Year: 2012

Compressed Counting (CC) [22] was recently proposed for estimating the ath frequency moments of data streams, where 0 < a <= 2. CC can be used for estimating Shannon entropy, which can be approximated by certain functions of the ath frequency moments as a -> 1. Monitoring Shannon entropy for anomaly detection (e.g., DDoS attacks) in large networks is an important task. This paper presents a new algorithm for improving CC. The improvement is most substantial when a -> 1--. For example, when a = 0:99, the new algorithm reduces the estimation variance roughly by 100-fold. This new algorithm would make CC considerably more practical for estimating Shannon entropy. Furthermore, the new algorithm is statistically optimal when a = 0.5.
Monolingual Probabilistic Programming Using Generalized Coroutines
Venue: arXiv cs.PL 2012
Year: 2012

Probabilistic programming languages and modeling toolkits are two modular ways to build and reuse stochastic models and inference procedures. Combining strengths of both, we express models and inference as generalized coroutines in the same general-purpose language. We use existing facilities of the language, such as rich libraries, optimizing compilers, and types, to develop concise, declarative, and realistic models with competitive performance on exact and approximate inference. In particular, a wide range of models can be expressed using memoization. Because deterministic parts of models run at full speed, custom inference procedures are trivial to incorporate, and inference procedures can reason about themselves without interpretive overhead. Within this framework, we introduce a new, general algorithm for importance sampling with look-ahead.
A Sampling-Based Approach to Computing Equilibria in Succinct Extensive-Form Games
Venue: arXiv cs.GT 2012
Year: 2012

A central task of artificial intelligence is the design of artificial agents that act towards specified goals in partially observed environments. Since such environments frequently include interaction over time with other agents with their own goals, reasoning about such interaction relies on sequential game-theoretic models such as extensive-form games or some of their succinct representations such as multi-agent influence diagrams. The current algorithms for calculating equilibria either work with inefficient representations, possibly doubly exponential inthe number of time steps, or place strong assumptions on the game structure. In this paper,we propose a sampling-based approach, which calculates extensive-form correlated equilibria with small representations without placing such strong assumptions. Thus, it is practical in situations where the previous approaches would fail. In addition, our algorithm allows control over characteristics of the target equilibrium, e.g., we can ask for an equilibrium with high social welfare. Our approach is based on a multiplicativeweight update algorithm analogous to AdaBoost, and Markov chain Monte Carlo sampling. We prove convergence guarantees and explore the utility of our approach on several moderately sized multi-player games.
Correlated Non-Parametric Latent Feature Models
Venue: arXiv cs.LG 2012
Year: 2012

We are often interested in explaining data through a set of hidden factors or features. When the number of hidden features is unknown, the Indian Buffet Process (IBP) is a nonparametric latent feature model that does not bound the number of active features in dataset. However, the IBP assumes that all latent features are uncorrelated, making it inadequate for many realworld problems. We introduce a framework for correlated nonparametric feature models, generalising the IBP. We use this framework to generate several specific models and demonstrate applications on realworld datasets.
Mean Field Variational Approximation for Continuous-Time Bayesian Networks
Venue: arXiv cs.AI 2012
Year: 2012

Continuous-time Bayesian networks is a natural structured representation language for multicomponent stochastic processes that evolve continuously over time. Despite the compact representation, inference in such models is intractable even in relatively simple structured networks. Here we introduce a mean field variational approximation in which we use a product of inhomogeneous Markov processes to approximate a distribution over trajectories. This variational approach leads to a globally consistent distribution, which can be efficiently queried. Additionally, it provides a lower bound on the probability of observations, thus making it attractive for learning tasks. We provide the theoretical foundations for the approximation, an efficient implementation that exploits the wide range of highly optimized ordinary differential equations (ODE) solvers, experimentally explore characterizations of processes for which this approximation is suitable, and show applications to a large-scale realworld inference problem.
On Smoothing and Inference for Topic Models
Venue: arXiv cs.LG 2012
Year: 2012

Latent Dirichlet analysis, or topic modeling, is a flexible latent variable framework for modeling high-dimensional sparse count data. Various learning algorithms have been developed in recent years, including collapsed Gibbs sampling, variational inference, and maximum a posteriori estimation, and this variety motivates the need for careful empirical comparisons. In this paper, we highlight the close connections between these approaches. We find that the main differences are attributable to the amount of smoothing applied to the counts. When the hyperparameters are optimized, the differences in performance among the algorithms diminish significantly. The ability of these algorithms to achieve solutions of comparable accuracy gives us the freedom to select computationally efficient approaches. Using the insights gained from this comparative study, we show how accurate topic models can be learned in several seconds on text corpora with thousands of documents.
Are visual dictionaries generalizable?
Venue: arXiv cs.CV 2012
Year: 2012

Mid-level features based on visual dictionaries are today a cornerstone of systems for classification and retrieval of images. Those state-of-the-art representations depend crucially on the choice of a codebook (visual dictionary), which is usually derived from the dataset. In general-purpose, dynamic image collections (e.g., the Web), one cannot have the entire collection in order to extract a representative dictionary. However, based on the hypothesis that the dictionary reflects only the diversity of low-level appearances and does not capture semantics, we argue that a dictionary based on a small subset of the data, or even on an entirely different dataset, is able to produce a good representation, provided that the chosen images span a diverse enough portion of the low-level feature space. Our experiments confirm that hypothesis, opening the opportunity to greatly alleviate the burden in generating the codebook, and confirming the feasibility of employing visual dictionaries in large-scale dynamic environments.
Analyses of Cardinal Auctions
Venue: arXiv cs.GT 2012
Year: 2012

We study cardinal auctions for selling multiple copies of a good, in which bidders specify not only their bid or how much they are ready to pay for the good, but also a cardinality constraint on the number of copies that will be sold via the auction. We perform first known Price of Anarchy type analyses with detailed comparison of the classical Vickrey-Clarke-Groves (VCG) auction and one based on minimum pay property (MPP) which is similar to Generalized Second Price auction commonly used in sponsored search. Without cardinality constraints, MPP has the same efficiency (total value to bidders) and at least as much revenue (total income to the auctioneer) as VCG; this also holds for certain other generalizations of MPP (e.g., prefix constrained auctions, as we show here). In contrast, our main results are that, with cardinality constraints, (a) equilibrium efficiency of MPP is 1/2 of that of VCG and this factor is tight, and (b) in equilibrium MPP may collect as little as 1/2 the revenue of VCG. These aspects arise because in presence of cardinality constraints, more strategies are available to bidders in MPP, including bidding above their value, and this makes analyses nontrivial.
Optimal Listing of Cycles and st-Paths in Undirected Graphs
Venue: arXiv cs.DS 2012
Year: 2012

We present the first optimal algorithm for the classical problem of listing all the cycles in an undirected graph. We exploit their properties so that the total cost is the time taken to read the input graph plus the time to list the output, namely, the edges in each of the cycles. The algorithm uses a reduction to the problem of listing all the paths from a vertex s to a vertex t which we also solve optimally.
User Association for Load Balancing in Heterogeneous Cellular Networks
Venue: arXiv cs.IT 2012
Year: 2012

For small cell technology to significantly increase the capacity of tower-based cellular networks, mobile users will need to be actively pushed onto the more lightly loaded tiers (corresponding to, e.g., pico and femtocells), even if they offer a lower instantaneous SINR than the macrocell base station (BS). Optimizing a function of the long-term rates for each user requires (in general) a massive utility maximization problem over all the SINRs and BS loads. On the other hand, an actual implementation will likely resort to a simple biasing approach where a BS in tier j is treated as having its SINR multiplied by a factor A_j>=1, which makes it appear more attractive than the heavily-loaded macrocell. This paper bridges the gap between these approaches through several physical relaxations of the network-wide optimal association problem, whose solution is NP hard. We provide a low-complexity distributed algorithm that converges to a near-optimal solution with a theoretical performance guarantee, and we observe that simple per-tier biasing loses surprisingly little, if the bias values A_j are chosen carefully. Numerical results show a large (3.5x) throughput gain for cell-edge users and a 2x rate gain for median users relative to a max received power association.
Visibly pushdown automata on trees: universality and u-universality
Venue: arXiv cs.FL 2012
Year: 2012

An automaton is universal if it accepts every possible input. We study the notion of u-universality, which asserts that the automaton accepts every input starting with u. Universality and u-universality are both EXPTIME-hard for non-deterministic tree automata. We propose efficient antichain-based techniques to address these problems for visibly pushdown automata operating on trees. One of our approaches yields algorithms for the universality and u-universality of hedge automata.
Efficiency-Revenue Trade-offs in Auctions
Venue: arXiv cs.GT 2012
Year: 2012

When agents with independent priors bid for a single item, Myerson's optimal auction maximizes expected revenue, whereas Vickrey's second-price auction optimizes social welfare. We address the natural question of trade-offs between the two criteria, that is, auctions that optimize, say, revenue under the constraint that the welfare is above a given level. If one allows for randomized mechanisms, it is easy to see that there are polynomial-time mechanisms that achieve any point in the trade-off (the Pareto curve) between revenue and welfare. We investigate whether one can achieve the same guarantees using deterministic mechanisms. We provide a negative answer to this question by showing that this is a (weakly) NP-hard problem. On the positive side, we provide polynomial-time deterministic mechanisms that approximate with arbitrary precision any point of the trade-off between these two fundamental objectives for the case of two bidders, even when the valuations are correlated arbitrarily. The major problem left open by our work is whether there is such an algorithm for three or more bidders with independent valuation distributions.
A Comparative Study of Collaborative Filtering Algorithms
Venue: arXiv cs.IR 2012
Year: 2012

Collaborative filtering is a rapidly advancing research area. Every year several new techniques are proposed and yet it is not clear which of the techniques work best and under what conditions. In this paper we conduct a study comparing several collaborative filtering techniques -- both classic and recent state-of-the-art -- in a variety of experimental contexts. Specifically, we report conclusions controlling for number of items, number of users, sparsity level, performance criteria, and computational complexity. Our conclusions identify what algorithms work well and in what conditions, and contribute to both industrial deployment collaborative filtering algorithms and to the research community.
Capacity and Spectral Efficiency of Interference Avoiding Cognitive Radio with Imperfect Detection
Venue: arXiv cs.IT 2012
Year: 2012

In this paper, we consider a model in which the unlicensed or the Secondary User (SU) equipped with a Cognitive Radio (CR) (together referred to as CR) interweaves its transmission with that of the licensed or the Primary User (PU). In this model, when the CR detects the PU to be (i) busy it does not transmit and; (ii) PU to be idle it transmits. Two situations based on CR's detection of PU are considered, where the CR detects PU (i) perfectly - referred to as the "ideal case" and; (ii) imperfectly - referred to as "non ideal case". For both the cases we bring out the rate region, sum capacity of PU and CR and spectral efficiency factor - the ratio of sum capacity of PU and CR to the capacity of PU without CR. We consider the Rayleigh fading channel to provide insight to our results. For the ideal case we study the effect of PU occupancy on spectral efficiency factor. For the non ideal case, in addition to the effect of occupancy, we study the effect of false alarm and missed detection on the rate region and spectral efficiency factor. We characterize the set of values of false alarm and missed detection probabilities for which the system benefits, in the form of admissible regions. We show that false alarm has a more profound effect on the spectral efficiency factor than missed detection. We also show that when PU occupancy is small, the effects of both false alarm and missed detection decrease. Finally, for the standard detection techniques viz. energy detection, matched filter and magnitude squared coherence, we show that that the matched filter performs best followed by magnitude squared coherence followed by energy detection with respect to spectral efficiency factor.
Tracking Topology Dynamicity for Link Prediction in Intermittently Connected Wireless Networks
Venue: arXiv cs.NI 2012
Year: 2012

Through several studies, it has been highlighted that mobility patterns in mobile networks are driven by human behaviors. This effect has been particularly observed in intermittently connected networks like DTN (Delay Tolerant Networks). Given that common social intentions generate similar human behavior, it is relevant to exploit this knowledge in the network protocols design, e.g. to identify the closeness degree between two nodes. In this paper, we propose a temporal link prediction technique for DTN which quantifies the behavior similarity between each pair of nodes and makes use of it to predict future links. We attest that the tensor-based technique is effective for temporal link prediction applied to the intermittently connected networks. The validity of this method is proved when the prediction is made in a distributed way (i.e. with local information) and its performance is compared to well-known link prediction metrics proposed in the literature.
Some Combinatorial Operators in Language Theory
Venue: arXiv cs.FL 2012
Year: 2012

Multitildes are regular operators that were introduced by Caron et al. in order to increase the number of Glushkov automata. In this paper, we study the family of the multitilde operators from an algebraic point of view using the notion of operad. This leads to a combinatorial description of already known results as well as new results on compositions, actions and enumerations.
Efficient Parallel Computation of Nearest Neighbor Interchange Distances
Venue: arXiv cs.DS 2012
Year: 2012

The nni-distance is a well-known distance measure for phylogenetic trees. We construct an efficient parallel approximation algorithm for the nni-distance in the CRCW-PRAM model running in O(log n) time on O(n) processors. Given two phylogenetic trees T1 and T2 on the same set of taxa and with the same multi-set of edge-weights, the algorithm constructs a sequence of nni-operations of weight at most O(log n) \cdot opt, where opt denotes the minimum weight of a sequence of nni-operations transforming T1 into T2 . This algorithm is based on the sequential approximation algorithm for the nni-distance given by DasGupta et al. (2000). Furthermore, we show that the problem of identifying so called good edge-pairs between two weighted phylogenies can be computed in O(log n) time on O(n log n) processors.
Efficient Expression Templates for Operator Overloading-based Automatic Differentiation
Venue: arXiv cs.MS 2012
Year: 2012

Expression templates are a well-known set of techniques for improving the efficiency of operator overloading-based forward mode automatic differentiation schemes in the C++ programming language by translating the differentiation from individual operators to whole expressions. However standard expression template approaches result in a large amount of duplicate computation, particularly for large expression trees, degrading their performance. In this paper we describe several techniques for improving the efficiency of expression templates and their implementation in the automatic differentiation package Sacado. We demonstrate their improved efficiency through test functions as well as their application to differentiation of a large-scale fluid dynamics simulation code.
Untyping Typed Algebras and Colouring Cyclic Linear Logic
Venue: arXiv cs.LO 2012
Year: 2012

We prove "untyping" theorems: in some typed theories (semirings, Kleene algebras, residuated lattices, involutive residuated lattices), typed equations can be derived from the underlying untyped equations. As a consequence, the corresponding untyped decision procedures can be extended for free to the typed settings. Some of these theorems are obtained via a detour through fragments of cyclic linear logic, and give rise to a substantial optimisation of standard proof search algorithms.
The Good, the Bad, and the Odd: Cycles in Answer-Set Programs
Venue: arXiv cs.AI 2012
Year: 2012

Backdoors of answer-set programs are sets of atoms that represent clever reasoning shortcuts through the search space. Assignments to backdoor atoms reduce the given program to several programs that belong to a tractable target class. Previous research has considered target classes based on notions of acyclicity where various types of cycles (good and bad cycles) are excluded from graph representations of programs. We generalize the target classes by taking the parity of the number of negative edges on bad cycles into account and consider backdoors for such classes. We establish new hardness results and non-uniform polynomial-time tractability relative to directed or undirected cycles.