OpenRaiser/Intern-Atlas huggingface dataset:
Modelling Information Incorporation in Markets, with Application to Detecting and Explaining Events
Venue: arXiv cs.AI 2012
Year: 2012

We develop a model of how information flows into a market, and derive algorithms for automatically detecting and explaining relevant events. We analyze data from twenty-two "political stock markets" (i.e., betting markets on political outcomes) on the Iowa Electronic Market (IEM). We prove that, under certain efficiency assumptions, prices in such betting markets will on average approach the correct outcomes over time, and show that IEM data conforms closely to the theory. We present a simple model of a betting market where information is revealed over time, and show a qualitative correspondence between the model and real market data. We also present an algorithm for automatically detecting significant events and generating semantic explanations of their origin. The algorithm operates by discovering significant changes in vocabulary on online news sources (using expected entropy loss) that align with major price spikes in related betting markets.
Academic Ranking with Web Mining and Axiomatic Analysis
Venue: arXiv cs.DL 2012
Year: 2012

Academic ranking is a public topic, such as for universities, colleges, or departments, which has significant educational, administrative and social effects. Popular ranking systems include the US News & World Report (USNWR), the Academic Ranking of World Universities (ARWU), and others. The most popular observables for such ranking are academic publications and their citations. However, a rigorous, quantitative and thorough methodology has been missing for this purpose. With modern web technology and axiomatic bibliometric analysis, here we perform a feasibility study on Microsoft Academic Search metadata and obtain the first-of-its-kind ranking results for American departments of computer science. This approach can be extended for fully automatic intuitional and college ranking based on comprehensive data on Internet.
The Mysterious Optimality of Naive Bayes: Estimation of the Probability in the System of "Classifiers"
Venue: arXiv cs.CV 2012
Year: 2012

Bayes Classifiers are widely used currently for recognition, identification and knowledge discovery. The fields of application are, for example, image processing, medicine, chemistry (QSAR). But by mysterious way the Naive Bayes Classifier usually gives a very nice and good presentation of a recognition. It can not be improved considerably by more complex models of Bayes Classifier. We demonstrate here a very nice and simple proof of the Naive Bayes Classifier optimality, that can explain this interesting fact.The derivation in the current paper is based on arXiv:cs/0202020v1
Tight Bounds on the Redundancy of Huffman Codes
Venue: arXiv cs.IT 2012
Year: 2012

In this paper we study the redundancy of Huffman codes. In particular, we consider sources for which the probability of one of the source symbols is known. We prove a conjecture of Ye and Yeung regarding the upper bound on the redundancy of such Huffman codes, which yields in a tight upper bound. We also derive a tight lower bound for the redundancy under the same assumption. We further apply the method introduced in this paper to other related problems. It is shown that several other previously known bounds with different constraints follow immediately from our results.
Lecture notes on the lambda calculus
Venue: arXiv cs.LO 2013
Year: 2013

This is a set of lecture notes that developed out of courses on the lambda calculus that I taught at the University of Ottawa in 2001 and at Dalhousie University in 2007 and 2013. Topics covered in these notes include the untyped lambda calculus, the Church-Rosser theorem, combinatory algebras, the simply-typed lambda calculus, the Curry-Howard isomorphism, weak and strong normalization, polymorphism, type inference, denotational semantics, complete partial orders, and the language PCF.
Optimal Strategies in Perfect-Information Stochastic Games with Tail Winning Conditions
Venue: arXiv cs.GT 2013
Year: 2013

We prove that optimal strategies exist in every perfect-information stochastic game with finitely many states and actions and a tail winning condition.
On the Internal Topological Structure of Plane Regions
Venue: arXiv cs.AI 2013
Year: 2013

The study of topological information of spatial objects has for a long time been a focus of research in disciplines like computational geometry, spatial reasoning, cognitive science, and robotics. While the majority of these researches emphasised the topological relations between spatial objects, this work studies the internal topological structure of bounded plane regions, which could consist of multiple pieces and/or have holes and islands to any finite level. The insufficiency of simple regions (regions homeomorphic to closed disks) to cope with the variety and complexity of spatial entities and phenomena has been widely acknowledged. Another significant drawback of simple regions is that they are not closed under set operations union, intersection, and difference. This paper considers bounded semi-algebraic regions, which are closed under set operations and can closely approximate most plane regions arising in practice.
Towards a mathematical theory of meaningful communication
Venue: arXiv cs.IT 2013
Year: 2013

Despite its obvious relevance, meaning has been outside most theoretical approaches to information in biology. As a consequence, functional responses based on an appropriate interpretation of signals has been replaced by a probabilistic description of correlations between emitted and received symbols. This assumption leads to potential paradoxes, such as the presence of a maximum information associated to a channel that would actually create completely wrong interpretations of the signals. Game-theoretic models of language evolution use this view of Shannon's theory, but other approaches considering embodied communicating agents show that the correct (meaningful) match resulting from agent-agent exchanges is always achieved and natural systems obviously solve the problem correctly. How can Shannon's theory be expanded in such a way that meaning -at least, in its minimal referential form- is properly incorporated? Inspired by the concept of {\em duality of the communicative sign} stated by the swiss linguist Ferdinand de Saussure, here we present a complete description of the minimal system necessary to measure the amount of information that is consistently decoded. Several consequences of our developments are investigated, such the uselessness of an amount of information properly transmitted for communication among autonomous agents.
Nonsingular Efficient Modeling of Rotations in 3-space using three components
Venue: arXiv cs.MS 2013
Year: 2013

This article introduces yet another representation of rotations in 3-space. The rotations form a 3-dimensional projective space, which fact has not been exploited in Computer Science. We use the four affine patches of this projective space to parametrize the rotations. This affine patch representation is more compact than quaternions (which require 4 components for calculations), encompasses the entire rotation group without singularities (unlike the Euler angles and rotation vector approaches), and requires only ratios of linear or quadratic polynomials for basic computations (unlike the Euler angles and rotation vector approaches which require transcendental functions). As an example, we derive the differential equation for the integration of angular velocity using this affine patch representation of rotations. We remark that the complexity of this equation is the same as the corresponding quaternion equation, but has advantages over the quaternion approach e.g. renormalization to unit length is not required, and state space has no dead directions.
On the Achievability of Cramér-Rao Bound In Noisy Compressed Sensing
Venue: arXiv cs.IT 2013
Year: 2013

Recently, it has been proved in Babadi et al. that in noisy compressed sensing, a joint typical estimator can asymptotically achieve the Cramer-Rao lower bound of the problem.To prove this result, this paper used a lemma,which is provided in Akcakaya et al,that comprises the main building block of the proof. This lemma is based on the assumption of Gaussianity of the measurement matrix and its randomness in the domain of noise. In this correspondence, we generalize the results obtained in Babadi et al by dropping the Gaussianity assumption on the measurement matrix. In fact, by considering the measurement matrix as a deterministic matrix in our analysis, we find a theorem similar to the main theorem of Babadi et al for a family of randomly generated (but deterministic in the noise domain) measurement matrices that satisfy a generalized condition known as The Concentration of Measures Inequality. By this, we finally show that under our generalized assumptions, the Cramer-Rao bound of the estimation is achievable by using the typical estimator introduced in Babadi et al.
Construction of Slepian-Wolf Source Code and Broadcast Channel Code Based on Hash Property
Venue: arXiv cs.IT 2013
Year: 2013

The aim of this paper is to prove theorems for the Slepian-Wolf source coding and the broadcast channel coding (independent messages and no common message) based on the the notion of a stronger version of the hash property for an ensemble of functions. Since an ensemble of sparse matrices has a strong hash property, codes using sparse matrices can realize the achievable rate region. Furthermore, extensions to the multiple source coding and multiple output broadcast channel coding are investigated.
Systems Theoretic Techniques for Modeling, Control, and Decision Support in Complex Dynamic Systems
Venue: arXiv cs.SY 2013
Year: 2013

We discuss the problems of modeling, control, and decision support in complex dynamic systems from a general system theoretic point of view. The main characteristics of complex systems and of system approach to complex system study are considered. We provide an overview and analysis of known existing paradigms and methods of mathematical modeling and simulation of complex systems, which support the processes of control and decision making. Then we continue with the general dynamic modeling and simulation technique for complex hierarchical systems functioning in control loop. Architectural and structural models of computer information system intended for simulation and decision support in complex systems are presented.
Parallel Evolutionary Computation in Very Large Scale Eigenvalue Problems
Venue: arXiv cs.DC 2013
Year: 2013

The history of research on eigenvalue problems is rich with many outstanding contributions. Nonetheless, the rapidly increasing size of data sets requires new algorithms for old problems in the context of extremely large matrix dimensions. This paper reports on a new method for finding eigenvalues of very large matrices by a synthesis of evolutionary computation, parallel programming, and empirical stochastic search. The direct design of our method has the added advantage that it could be adapted to extend many algorithmic variants of solutions of generalized eigenvalue problems to improve the accuracy of our algorithms. The preliminary evaluation results are encouraging and demonstrate the method's efficiency and practicality.
Fat Polygonal Partitions with Applications to Visualization and Embeddings
Venue: arXiv cs.CG 2013
Year: 2013

Let $\mathcal{T}$ be a rooted and weighted tree, where the weight of any node is equal to the sum of the weights of its children. The popular Treemap algorithm visualizes such a tree as a hierarchical partition of a square into rectangles, where the area of the rectangle corresponding to any node in $\mathcal{T}$ is equal to the weight of that node. The aspect ratio of the rectangles in such a rectangular partition necessarily depends on the weights and can become arbitrarily high. We introduce a new hierarchical partition scheme, called a polygonal partition, which uses convex polygons rather than just rectangles. We present two methods for constructing polygonal partitions, both having guarantees on the worst-case aspect ratio of the constructed polygons; in particular, both methods guarantee a bound on the aspect ratio that is independent of the weights of the nodes. We also consider rectangular partitions with slack, where the areas of the rectangles may differ slightly from the weights of the corresponding nodes. We show that this makes it possible to obtain partitions with constant aspect ratio. This result generalizes to hyper-rectangular partitions in $\mathbb{R}^d$. We use these partitions with slack for embedding ultrametrics into $d$-dimensional Euclidean space: we give a $\mathop{\rm polylog}(Δ)$-approximation algorithm for embedding $n$-point ultrametrics into $\mathbb{R}^d$ with minimum distortion, where $Δ$ denotes the spread of the metric, i.e., the ratio between the largest and the smallest distance between two points. The previously best-known approximation ratio for this problem was polynomial in $n$. This is the first algorithm for embedding a non-trivial family of weighted-graph metrics into a space of constant dimension that achieves polylogarithmic approximation ratio.
On the Doubt about Margin Explanation of Boosting
Venue: arXiv cs.LG 2013
Year: 2013

Margin theory provides one of the most popular explanations to the success of \texttt{AdaBoost}, where the central point lies in the recognition that \textit{margin} is the key for characterizing the performance of \texttt{AdaBoost}. This theory has been very influential, e.g., it has been used to argue that \texttt{AdaBoost} usually does not overfit since it tends to enlarge the margin even after the training error reaches zero. Previously the \textit{minimum margin bound} was established for \texttt{AdaBoost}, however, \cite{Breiman1999} pointed out that maximizing the minimum margin does not necessarily lead to a better generalization. Later, \cite{Reyzin:Schapire2006} emphasized that the margin distribution rather than minimum margin is crucial to the performance of \texttt{AdaBoost}. In this paper, we first present the \textit{$k$th margin bound} and further study on its relationship to previous work such as the minimum margin bound and Emargin bound. Then, we improve the previous empirical Bernstein bounds \citep{Maurer:Pontil2009,Audibert:Munos:Szepesvari2009}, and based on such findings, we defend the margin-based explanation against Breiman's doubts by proving a new generalization error bound that considers exactly the same factors as \cite{Schapire:Freund:Bartlett:Lee1998} but is sharper than \cite{Breiman1999}'s minimum margin bound. By incorporating factors such as average margin and variance, we present a generalization error bound that is heavily related to the whole margin distribution. We also provide margin distribution bounds for generalization error of voting classifiers in finite VC-dimension space.
Near-Optimal Bayesian Active Learning with Noisy Observations
Venue: arXiv cs.LG 2013
Year: 2013

We tackle the fundamental problem of Bayesian active learning with noise, where we need to adaptively select from a number of expensive tests in order to identify an unknown hypothesis sampled from a known prior distribution. In the case of noise-free observations, a greedy algorithm called generalized binary search (GBS) is known to perform near-optimally. We show that if the observations are noisy, perhaps surprisingly, GBS can perform very poorly. We develop EC2, a novel, greedy active learning algorithm and prove that it is competitive with the optimal policy, thus obtaining the first competitiveness guarantees for Bayesian active learning with noisy observations. Our bounds rely on a recently discovered diminishing returns property called adaptive submodularity, generalizing the classical notion of submodular set functions to adaptive policies. Our results hold even if the tests have non-uniform cost and their noise is correlated. We also propose EffECXtive, a particularly fast approximation of EC2, and evaluate it on a Bayesian experimental design problem involving human subjects, intended to tease apart competing economic theories of how people make decisions under uncertainty.
An Optimized Divide-and-Conquer Algorithm for the Closest-Pair Problem in the Planar Case
Venue: arXiv cs.CG 2013
Year: 2013

We present an engineered version of the divide-and-conquer algorithm for finding the closest pair of points, within a given set of points in the XY-plane. For this version of the algorithm we show that only two pairwise comparisons are required in the combine step, for each point that lies in the 2 delta-wide vertical slab. The correctness of the algorithm is shown for all Minkowski distances with p>=1. We also show empirically that, although the time complexity of the algorithm is still O(n lg n), the reduction in the total number of comparisons leads to a significant reduction in the total execution time, for inputs with size sufficiently large.
Vertex Cover Kernelization Revisited: Upper and Lower Bounds for a Refined Parameter
Venue: arXiv cs.DS 2013
Year: 2013

An important result in the study of polynomial-time preprocessing shows that there is an algorithm which given an instance (G,k) of Vertex Cover outputs an equivalent instance (G',k') in polynomial time with the guarantee that G' has at most 2k' vertices (and thus O((k')^2) edges) with k' <= k. Using the terminology of parameterized complexity we say that k-Vertex Cover has a kernel with 2k vertices. There is complexity-theoretic evidence that both 2k vertices and Theta(k^2) edges are optimal for the kernel size. In this paper we consider the Vertex Cover problem with a different parameter, the size fvs(G) of a minimum feedback vertex set for G. This refined parameter is structurally smaller than the parameter k associated to the vertex covering number vc(G) since fvs(G) <= vc(G) and the difference can be arbitrarily large. We give a kernel for Vertex Cover with a number of vertices that is cubic in fvs(G): an instance (G,X,k) of Vertex Cover, where X is a feedback vertex set for G, can be transformed in polynomial time into an equivalent instance (G',X',k') such that |V(G')| <= 2k and |V(G')| <= O(|X'|^3). A similar result holds when the feedback vertex set X is not given along with the input. In sharp contrast we show that the Weighted Vertex Cover problem does not have a polynomial kernel when parameterized by the cardinality of a given vertex cover of the graph unless NP is in coNP/poly and the polynomial hierarchy collapses to the third level.
Quasirandom Rumor Spreading
Venue: arXiv cs.DS 2013
Year: 2013

We propose and analyze a quasirandom analogue of the classical push model for disseminating information in networks ("randomized rumor spreading"). In the classical model, in each round each informed vertex chooses a neighbor at random and informs it, if it was not informed before. It is known that this simple protocol succeeds in spreading a rumor from one vertex to all others within O(log n) rounds on complete graphs, hypercubes, random regular graphs, Erdos-Renyi random graph and Ramanujan graphs with probability 1-o(1). In the quasirandom model, we assume that each vertex has a (cyclic) list of its neighbors. Once informed, it starts at a random position on the list, but from then on informs its neighbors in the order of the list. Surprisingly, irrespective of the orders of the lists, the above-mentioned bounds still hold. In some cases, even better bounds than for the classical model can be shown.
The degree structure of Weihrauch-reducibility
Venue: arXiv cs.LO 2013
Year: 2013

We answer a question by Vasco Brattka and Guido Gherardi by proving that the Weihrauch-lattice is not a Brouwer algebra. The computable Weihrauch-lattice is also not a Heyting algebra, but the continuous Weihrauch-lattice is. We further investigate the existence of infinite infima and suprema, as well as embeddings of the Medvedev-degrees into the Weihrauch-degrees.
The Capacity Region of the MIMO Interference Channel and its Reciprocity to Within a Constant Gap
Venue: arXiv cs.IT 2013
Year: 2013

The capacity region of the 2-user multi-input multi-output (MIMO) Gaussian interference channel (IC) is characterized to within a constant gap that is independent of the channel matrices for the general case of the MIMO IC with an arbitrary number of antennas at each node. An achievable rate region and an outer bound to the capacity region of a class of interference channels were obtained in previous work by Telatar and Tse as unions over all possible input distributions. In contrast to that previous work on the MIMO IC, a simple and an explicit achievable coding scheme are obtained here and shown to have the constant-gap-to-capacity property and in which the sub-rates of the common and private messages of each user are explicitly specified for each achievable rate pair. The constant-gap-to-capacity results are thus proved in this work by first establishing explicit upper and lower bounds to the capacity region. A reciprocity result is also proved which is that the capacity of the reciprocal MIMO IC is within a constant gap of the capacity region of the forward MIMO IC.
Theory of Atomata
Venue: arXiv cs.FL 2013
Year: 2013

We show that every regular language defines a unique nondeterministic finite automaton (NFA), which we call "átomaton", whose states are the "atoms" of the language, that is, non-empty intersections of complemented or uncomplemented left quotients of the language. We describe methods of constructing the átomaton, and prove that it is isomorphic to the reverse automaton of the minimal deterministic finite automaton (DFA) of the reverse language. We study "atomic" NFAs in which the right language of every state is a union of atoms. We generalize Brzozowski's double-reversal method for minimizing a deterministic finite automaton (DFA), showing that the result of applying the subset construction to an NFA is a minimal DFA if and only if the reverse of the NFA is atomic. We prove that Sengoku's claim that his method always finds a minimal NFA is false.
A system of relational syllogistic incorporating full Boolean reasoning
Venue: arXiv cs.LO 2013
Year: 2013

We present a system of relational syllogistic, based on classical propositional logic, having primitives of the following form: Some A are R-related to some B; Some A are R-related to all B; All A are R-related to some B; All A are R-related to all B. Such primitives formalize sentences from natural language like `All students read some textbooks'. Here A and B denote arbitrary sets (of objects), and R denotes an arbitrary binary relation between objects. The language of the logic contains only variables denoting sets, determining the class of set terms, and variables denoting binary relations between objects, determining the class of relational terms. Both classes of terms are closed under the standard Boolean operations. The set of relational terms is also closed under taking the converse of a relation. The results of the paper are the completeness theorem with respect to the intended semantics and the computational complexity of the satisfiability problem.