pinecone/core-2020-05-10-deduplication huggingface dataset:
A Constant-Factor Approximation Algorithm for thek-MST Problem

AbstractGiven an undirected graph with nonnegative edge costs and an integerk, thek-MST problem is that of finding a tree of minimum cost onknodes. This problem is known to be NP-hard. We present a simple approximation algorithm that finds a solution whose cost is less than 17 times the cost of the optimum. This improves upon previous performance ratios for this problem −O(k) due to Raviet al.,O(log2k) due to Awerbuchet al., and the previous best bound ofO(logk) due to Rajagopalan and Vazirani. Given any 0<α<1, we first present a bicriteria approximation algorithm that outputs a tree onp⩾αkvertices of total cost at most 2pL/(1−α)k, whereLis the cost of the optimalk-MST. The running time of the algorithm isO(n2log2n) on ann-node graph. We then show how to use this algorithm to derive a constant factor approximation algorithm for thek-MST problem. The main subroutine in our algorithm is an approximation algorithm of Goemans and Williamson for the prize-collecting Steiner tree problem
Characterizing Linear Size Circuits in Terms of Privacy

AbstractIn this paper we prove a perhaps unexpected relationship between the complexity class of the boolean functions that have linear size circuits andn-party private protocols. Specifically, letfbe a boolean function. We show thatfhas a linear size circuit if and only iffhas a 1-privaten-party protocol in which the total number of random bits used byallplayers is constant. From the point of view of complexity theory, our result gives a characterization of the class of linear size circuits in terms of another class of a very different nature. From the point of view of privacy, this result provides 1-private protocols that use a constant number of random bits, for many important functions for which no such protocol was previously known. On the other hand, our result suggests that proving, for anyNPfunction, that it has no 1-private constant-random protocol, might be difficult
Lower Bounds on the Multiparty Communication Complexity

AbstractWe derive a general technique for obtaining lower bounds on the multiparty communication complexity of boolean functions. We extend the two-party method based on a crossing sequence argument introduced by Yao to the multiparty communication model. We use our technique to derive optimal lower and upper bounds of some simple boolean functions. Lower bounds for the multiparty model have been a challenge since (D. Dolev and T. Feder,in“Proceedings, 30th IEEE FOCS, 1989,” pp. 428–433), where only an upper bound on the number of bits exchanged by a deterministic algorithm computing a boolean functionf(x1, …, xn) was derived, namely of the order (k0C0)(k1C1)2, up to logarithmic factors, wherek1andC1are the number of processors accessed and the bits exchanged in a nondeterministic algorithm forf, andk0andC0are the analogous parameters for the complementary function 1−f. We show thatC0⩽n(1+2C1) andD⩽n(1+2C1), whereDis the number of bits exchanged by a deterministic algorithm computingf. We also investigate the power of a restricted multiparty communication model in which the coordinator is allowed to send at most one message to each party
Counting Protocols for Reliable End-to-End Transmission

AbstractWe present and analyze the performance of two newcounting protocols. Counting protocols use bounded headers yet provide a reliable FIFO channel in a computer network in which packets may be lost or delivered out of order. Using the classic alternating bit protocol as a basis, we derive two counting protocols: (i) theone-bit protocolwhich uses one bit headers and sends one packet per message under ideal conditions, but performs extremely poorly in networks with realistic loss rates and (ii) themode protocolwhich uses multiple-bit headers and whose performance improves as more bits are used in the header
On Interpolating Arithmetic Read-Once Formulas with Exponentiation

AbstractA formula is a read-once formula if each variable appears at most once in it. An arithmetic read-once formula (AROF) with exponentiation is one in which the operations are addition, substraction, multiplication, division and exponentiation to an arbitrary integer. We present a polynomial time algorithm for interpolating AROF withexponentiationusing randomized substitutions. Interpolating AROF without exponentiation is studied in (Bshouty, Hancock, and Hellerstein,SIAM J. Comput.24, No. 4, 1995). To add the exponentiation operation to the basis we develop a new technique
On the Learnability and Usage of Acyclic Probabilistic Finite Automata

AbstractWe propose and analyze a distribution learning algorithm for a subclass ofacyclic probalistic finite automata(APFA). This subclass is characterized by a certain distinguishability property of the automata's states. Though hardness results are known for learning distributions generated by general APFAs, we prove that our algorithm can efficiently learn distributions generated by the subclass of APFAs we consider. In particular, we show that the KL-divergence between the distribution generated by the target source and the distribution generated by our hypothesis can be made arbitrarily small with high confidence in polynomial time. We present two applications of our algorithm. In the first, we show how to model cursively written letters. The resulting models are part of a complete cursive handwriting recognition system. In the second application we demonstrate how APFAs can be used to build multiple-pronunciation models for spoken words. We evaluate the APFA-based pronunciation models on labeled speech data. The good performance (in terms of the log-likelihood obtained on test data) achieved by the APFAs and the little time needed for learning suggests that the learning algorithm of APFAs might be a powerful alternative to commonly used probabilistic models
A Game of Prediction with Expert Advice

AbstractWe consider the following problem. At each point of discrete time the learner must make a prediction; he is given the predictions made by a pool of experts. Each prediction and the outcome, which is disclosed after the learner has made his prediction, determine the incurred loss. It is known that, under weak regularity, the learner can ensure that his cumulative loss never exceedscL+alnn, wherecandaare some constants,nis the size of the pool, andLis the cumulative loss incurred by the best expert in the pool. We find the set of those pairs (c, a) for which this is true
Prediction, Learning, Uniform Convergence, and Scale-Sensitive Dimensions

AbstractWe present a new general-purpose algorithm for learning classes of [0, 1]-valued functions in a generalization of the prediction model and prove a general upper bound on the expected absolute error of this algorithm in terms of a scale-sensitive generalization of the Vapnik dimension proposed by Alon, Ben-David, Cesa-Bianchi, and Haussler. We give lower bounds implying that our upper bounds cannot be improved by more than a constant factor in general. We apply this result, together with techniques due to Haussler and to Benedek and Itai, to obtain new upper bounds on packing numbers in terms of this scale-sensitive notion of dimension. Using a different technique, we obtain new bounds on packing numbers in terms of Kearns and Schapire's fat-shattering function. We show how to apply both packing bounds to obtain improved general bounds on the sample complexity of agnostic learning. For eachε>0, we establish weaker sufficient and stronger necessary conditions for a class of [0, 1]-valued functions to be agnostically learnable to withinεand to be anε-uniform Glivenko–Cantelli class
Specification and Simulation of Statistical Query Algorithms for Efficiency and Noise Tolerance

AbstractA recent innovation in computational learning theory is the statistical query (SQ) model. The advantage of specifying learning algorithms in this model is that SQ algorithms can be simulated in the probably approximately correct (PAC) model, both in the absenceandin the presence of noise. However, simulations of SQ algorithms in the PAC model have non-optimal time and sample complexities. In this paper, we introduce a new method for specifying statistical query algorithms based on a type ofrelative errorand provide simulations in the noise-free and noise-tolerant PAC models which yield more efficient algorithms. Requests for estimates of statistics in this new model take the following form: “Return an estimate of the statistic within a 1±μfactor, or return ⊥, promising that the statistic is less thanθ.” In addition to showing that this is a very natural language for specifying learning algorithms, we also show that this new specification is polynomially equivalent to standard SQ, and thus, known learnability and hardness results for statistical query learning are preserved. We then give highly efficient PAC simulations of relative error SQ algorithms. We show that the learning algorithms obtained by simulating efficient relative error SQ algorithms both in the absence of noise and in the presence of malicious noise have roughly optimal sample complexity. We also show that the simulation of efficient relative error SQ algorithms in the presence of classification noise yields learning algorithms at least as efficient as those obtained through standard methods, and in some cases improved, roughly optimal results are achieved. The sample complexities for all of these simulations are based on thedνmetric, which is a type of relative error metric useful for quantities which are small or even zero. We show that uniform convergence with respect to thedνmetric yields “uniform convergence” with respect to (μ, θ) accuracy. Finally, while we show that manyspecificlearning algorithms can be written as highly efficient relative error SQ algorithms, we also show, in fact, thatallSQ algorithms can be written efficiently by proving general upper bounds on the complexity of (μ, θ) queries as a function of the accuracy parameterε. As a consequence of this result, we give general upper bounds on the complexity of learning algorithms achieved through the use of relative error SQ algorithms and the simulations described above
Learning with Unreliable Boundary Queries

AbstractWe introduce a model for learning from examples and membership queries in situations where the boundary between positive and negative examples is somewhat ill-defined. In our model, queries near the boundary of a target concept may receive incorrect or “don't care” responses, and the distribution of examples has zero probability mass on the boundary region. The motivation behind our model is that in many cases the boundary between positive and negative examples is complicated or “fuzzy.” However, one may still hope to learn successfully, because the typical examples that one sees do not come from that region. We present several positive results in this new model. We show how to learn the intersection of two arbitrary halfspaces when membership queries near the boundary may be answered incorrectly. Our algorithm is an extension of an algorithm of Baum (1990, 1991) that learns the intersection of two halfspaces whose bounding planes pass through the origin in the PAC-with-membership-queries model. We also describe algorithms for learning several subclasses of monotone DNF formulas
Covering Cubes by Random Half Cubes, with Applications to Binary Neural Networks

AbstractLetQnbe the (hyper)cube {−1, 1}n. This paper is concerned with the following question: How many vectors must be chosen uniformly and independently at random fromQnbefore every vector inQnitself has negative inner product with at least one of the random vectors? For any fixedε>0, a simple expectation argument shows that for all sufficiently largen, (1+ε)nrandom vectors suffice with high probability. In this paper we prove that there areε, ρ>0 such that (1−ε)nrandom vectors are also enough and such that at leastρnrandom vectors are necessary. This problem is partially motivated by neural network problems. Neural networks are being used to solve a growing number of difficult problems such as speech recognition, handwriting recognition, and protein structure prediction. Recently, for both theoretical and practical reasons, neural networks with binary weights (binary neural networks) have attracted much attention. In spite of considerable analysis based on statistical mechanics, the following two basic questions about binary neural networks have remained unanswered. Q1.Is there a positive constantρsuch that for all sufficiently largenthere is a binary neural network ofnneurons which can separateρn(unbiased) random patterns with probability close to 1? Q2.Is it possible for a binary neural network ofnneurons to separate (1−o(1))nrandom patterns with probability greater than some positive constant? (Hereo(1) goes to 0 asngoes to infinity.) This question is raised because no binary neural network ofnneurons can separate more than n random patterns with probability close to 1. Our results yield the answers “YES” to Q1 and “NO” to Q2
Sequences, Datalog, and Transducers

AbstractThis paper develops a query language for sequence databases, such as genome databases and text databases. The language, calledSequence Datalog, extends classical Datalog with interpreted function symbols for manipulating sequences. It has both a clear operational and declarative semantics, based on a new notion called theextended active domainof a database. The extended domain contains all the sequences in the database and all their subsequences. This idea leads to a clear distinction between safe and unsafe recursion over sequences: safe recursion stays inside the extended active domain, while unsafe recursion does not. By carefully limiting the amount of unsafe recursion, the paper develops a safe and expressive subset of Sequence Datalog. As part of the development, a new type of transducer is introduced, called ageneralized sequence transducer. Unsafe recursion is allowed only within these generalized transducers. Generalized transducers extend ordinary transducers by allowing them to invoke other transducers as “subroutines.” Generalized transducers can be implemented in Sequence Datalog in a straightforward way. Moreover, their introduction into the language leads to simple conditions that guarantee safety and finiteness. This paper develops two such conditions. The first condition expresses exactly the class ofptimesequence functions, and the second expresses exactly the class of elementary sequence functions
Safe Locking Policies for Dynamic Databases

AbstractYannakakis showed that a locking policy is not safe if and only if it allows a canonical nonserializable schedule of transactions in which all transactions except one are executed serially (Yannakakis, 1982). In the present paper, we study the generalization of this result to a dynamic database, that is, a database that may undergo insertions and deletions of entities. We illustrate the utility of this generalization by applying it to obtain correctness proofs of three locking policies that handle dynamic databases
Algebras for Querying Text Regions: Expressive Power and Optimization

AbstractThere is a significant amount of interest in combining and extending database and information retrieval technologies to manage textual data. The challenge is becoming more relevant due to increased availability of documents in digital form. Document data has a natural hierarchical structure, which may be made explicit due to the use of markup conventions (as with SGML). An important aspect of managing structured and semistructured textual data consists of supporting the efficient retrieval of text components based both on their content and on their structure. In this paper we study issues related to the expressive power and optimization of a class of algebras that support combining string (or pattern) searches with queries on the hierarchical structure of the text. Theregion algebrastudied is a set-at-a-time algebra for manipulatingtext regions(substrings of the text) that supports finding out nesting and ordering properties of the text regions. This algebra is part of the language in use in commercial text retrieval systems and can form the basis for supporting SQL-like access to textual data. By presenting a close relationship between the region algebra and the monadic first order theory of finite binary trees, we show that queries in the algebra can be optimized, in the sense that equivalence to less expensive expressions can be tested. This optimization can be difficult (co-NP-hard in the general case), but there is an important class of queries that can be optimized in polynomial time. On the negative side, we show that the language is incapable of capturing some important properties of the text structure, related to the nesting and ordering of text regions. We conclude by suggesting possible extensions to increase the expressive power of the language and consider one such example
Arity Bounds in First-Order Incremental Evaluation and Definition of Polynomial Time Database Queries

AbstractAfter inserting a tuple into or deleting a tuple from a database, a first-order incremental evaluation system (or a “foies”) for a database query derives the new query answer by using a first-order query on the new database, the old answer, and perhaps some stored auxiliary relations. Moreover, the auxiliary relations must also be maintained similarly using first-order queries. In this paper we measure the space needed by foies in terms of the maximal arity of the auxiliary relations and present results on the existence and nonexistence of such space-restricted foies for a variety of graph queries, including counting and transitive closure. We construct space efficient foies for these queries and show that the arity bounds are tight using Ehrenfeucht–Fraı̈ssé games. In particular, for transitive closure over undirected graphs, the minimum arity bound of its foies is shown to be exactly two; this resolves an open problem raised by Patnaik and Immerman in 1994. For the general case, we show that the arity-based hierarchy is strict for all arities. The strictness proof uses queries with input relations having arities much larger than the auxiliary relations. It is still open whether the hierarchy remains strict for arities two or greater when the input relations of queries have arities bounded by a fixed number, such as two, or by the arities of the auxiliary relations. Finally, we consider a variation of foies where the cost of storing the answer to the query is also “charged.” We show that the arity hierarchy in this case is also strict. The positions of queries in the two arity-based hierarchies can differ; we give the positions in this hierarchy of the queries considered for the other arity hierarchy
Increasing the Resilience of Distributed and Replicated Database Systems

AbstractThis paper presents a new atomic commitment protocol,enhanced three phase commit(E3PC), thatalwaysallows a quorum in the system to make progress. Previously suggested quorum-based protocols (e.g., the quorum-basedthree phase commit(3PC) (Skeen, 1982)), allow a quorum to make progress in case of one failure. If failures cascade, however, and the quorum in the system is “lost” (i.e., at a given time no quorum component exists), a quorum can later become connected and still remain blocked. With our protocol, a connected quorum never blocks. E3PC is based on the quorum-based 3PC (Skeen, 1982), and it does not require more time or communication than 3PC. We describe how this protocol can be exploited in a replicated database setting, making the databasealwaysavailable to a majority of the sites
A Note on Batch and Incremental Learnability

AbstractAccording to Gold's criterion of identification in the limit, a learner, presented with data about a concept, is allowed to make a finite number of incorrect hypotheses before converging to a correct hypothesis. If, on the other hand, the learner is allowed to make only one conjecture which has to be correct, the resulting criterion of success is known as finite identification Identification in the limit may be viewed as an idealized model for incremental learning whereas finite identification may be viewed as an idealized model for batch learning. The present paper establishes a surprising fact that the collections of recursively enumerable languages that can be finite identified (batch learned in the ideal case) from both positive and negative data can also be identified in the limit (incrementally learned in the ideal case) from only positive data. It is often difficult to extract insights about practical learning systems from abstract theorems in inductive inference. However, this result may be seen as carrying a moral for the design of learning systems, as it yields, in theidealcase of no inaccuracies, an algorithm for converting batch systems that learn from both positive and negative data into incremental systems that learn from only positive data without any loss in learning power. This is achieved by the incremental system simulating the batch system in incremental fashion and using the heuristic of “localized closed-world assumption” to generate negative data
Learning with Restricted Focus of Attention

AbstractWe consider learning tasks in which the learner faces restrictions on the amount of information he can extract from each example he encounters. We introduce a formal framework for the analysis of such scenarios. We call this framework RFA (restricted focus of attention) learning. Although it is a natural refinement of the PAC learning model, some of the fundamental PAC-learning results and techniques fail in the RFA paradigm; learnability in the RFA model is no longer characterized by the VC dimension, and many PAC learning algorithms are not applicable in the RFA setting. Hence, the RFA formulation reflects the need for new techniques and tools to cope with some fundamental constraints of realistic learning problems. In this work we also present some paradigms and algorithms that may serve as a first step toward answering this need. Two main types of restrictions are considered here: In the more stringent one, calledk-RFA, onlykof thenattributes of each example are revealed to the learner, while in the more permissive one, calledk-wRFA, the restriction is made on the size of each observation (kbits), and no restriction is made on how the observations are extracted from the examples. For thek-RFA restriction we develop a general technique for composing efficientk-RFA algorithms and apply it to deduce, for instance, the efficientk-RFA learnability ofk-DNF formulas and the efficient 1-RFA learnability of axis-aligned rectangles in the Euclidean spaceRn. We also prove thek-RFA learnability of richer classes of Boolean functions (such ask-decision lists) with respect to a given distribution and the efficient (n−1)-RFA learnability (for fixedn), under product distributions, of classes of subsets ofRnwhich are defined by mild surfaces. For thek-wRFA restriction, we show that fork=O(logn), efficientk-wRFA learning is robust against classification noise. As a straight- forward application, we obtain a new simple noise-tolerant algorithm for the class ofk-decision lists by constructing an intuitivek-wRFA algorithm for this task