pinecone/core-2020-05-10-deduplication huggingface dataset:
Faster Shortest-Path Algorithms for Planar Graphs

AbstractWe give a linear-time algorithm for single-source shortest paths in planar graphs with nonnegative edge-lengths. Our algorithm also yields a linear-time algorithm for maximum flow in a planar graph with the source and sink on the same face. For the case where negative edge-lengths are allowed, we give an algorithm requiringO(n4/3log(nL)) time, whereLis the absolute value of the most negative length. This algorithm can be used to obtain similar bounds for computing a feasible flow in a planar network, for finding a perfect matching in a planar bipartite graph, and for finding a maximum flow in a planar graph when the source and sink are not on the same face. We also give parallel and dynamic versions of these algorithms
Optimal Parallel Suffix Tree Construction

AbstractAnO(m)-work,O(m)-space,O(log4m)-time CREW-PRAM algorithm for constructing the suffix tree of a stringsof lengthmdrawn from any fixed alphabet set is obtained. This is the first known work and space optimal parallel algorithm for this problem. It can be generalized to a string s drawn from any general alphabet set to perform inO(log4m) time andO(mlog|Σ|) work and space, after the characters inshave been sorted alphabetically, where |Σ| is the number of distinct characters ins. In this case too, the algorithm is work-optimal
Paging against a Distribution and IP Networking

AbstractIn this paper we consider the paging problem when the page request sequence is drawn from a distribution and we give an application to computer networking. In theIP-paging problemthe page interrequest times are chosen according to independent distributions. For this model we construct a very simple deterministic algorithm whose page fault rate is at most five times that of the best online algorithm (that knows the interrequest time distributions). We also show that many other natural algorithms for this problem do not have constant competitive ratio. Indistributional pagingthe interrequest time distributions may be dependent, and hence, any probabilistic model of page request sequences can be represented. We construct a simple randomized algorithm whose page fault rate is at most four times that of the best online algorithm. The IP-paging problem is motivation by the following application to data networks. Next generation wide area networks are very likely to use connection-oriented protocols such as Asynchronous Transfer Mode. For the existing investment in current IP networks such as the Internet to remain useful, we must devise mechanisms to carry IP traffic over connection-oriented networks. A basic issue is to devise holding policies for virtual circuits carrying datagrams; for some connection-oriented networks the holding policy problem is exactly IP-paging
Stochastic Complexity in Learning

AbstractThis is an expository paper on the latest results in the theory of stochastic complexity and the associated MDL principle with special interest in modeling problems arising in machine learning. As an illustration we discuss the problem of designing MDL decision trees, which are meant to improve the earlier designs in two ways: First, by use of the sharper formula for the stochastic complexity at the nodes the earlier found tendency of getting too small trees appears to be overcome. Second, a dynamic programming-based pruning algorithm is described for finding the optimal trees, which generalizes an algorithm described in R. Nohre (Ph.D. thesis Linkoping University, 1994)
Learning about the Parameter of the Bernoulli Model

AbstractWe consider the problem of learning as much information as possible about the parameterθof the Bernoulli model {Pθ∣θ∈[0, 1]} from the statistical datax∈{0, 1}n,n⩾1 being the sample size. Explicating this problem in terms of the Kolmogorov complexity and Rissanen's minimum description length principle, we construct a computable point estimator which (a) extracts from x all information it contains aboutθ, and (b) discards all sample noise inx. Our result is closely connected with Rissanen's theorem about the optimality of his scheme of coding statistical data
On-Line Maximum Likelihood Prediction with Respect to General Loss Functions

AbstractThis paper introduces a new family of deterministic and stochastic on-line prediction algorithms which work with respect to general loss functions and analyzes their behavior in terms of expected loss bounds. The algorithms useparametric probabilistic modelsregardless of the kind of loss function used. The key ideas of the algorithms are to iteratively estimate the probabilistic model using the maximum likelihood method and then to construct an optimal prediction function that minimizes the average of the loss taken with respect to the estimated probabilistic model. A future outcome is predicted using this optimal prediction function. We analyze the algorithms in the cases where the target distribution is (1)k-dimensional parametric andkis known, (2)k-dimensional parametric butkis unknown, and (3) nonparametric. For all the cases, we derive upper bounds on the expected instantaneous or cumulative losses for the algorithms with respect to a large family of loss functions satisfying the constraint introduced by Merhav and Feder. These loss bounds show new universal relations among the expected prediction accuracy, the indexes of the loss function, the complexity of the target rule, and the number of training examples
A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting

AbstractIn the first part of the paper we consider the problem of dynamically apportioning resources among a set of options in a worst-case on-line framework. The model we study can be interpreted as a broad, abstract extension of the well-studied on-line prediction model to a general decision-theoretic setting. We show that the multiplicative weight-update Littlestone–Warmuth rule can be adapted to this model, yielding bounds that are slightly weaker in some cases, but applicable to a considerably more general class of learning problems. We show how the resulting learning algorithm can be applied to a variety of problems, including gambling, multiple-outcome prediction, repeated games, and prediction of points in Rn. In the second part of the paper we apply the multiplicative weight-update technique to derive a new boosting algorithm. This boosting algorithm does not require any prior knowledge about the performance of the weak learning algorithm. We also study generalizations of the new boosting algorithm to the problem of learning functions whose range, rather than being binary, is an arbitrary finite set or a bounded segment of the real line
Characterizing Rational versus Exponential Learning Curves

AbstractWe consider the standard problem of learning a concept from random examples. Here alearning curveis defined to be the expected error of a learner's hypotheses as a function of training sample size. Haussler, Littlestone, and Warmuth have shown that, in the distribution-free setting, the smallest expected error a learner can achieve in the worst case over a class of conceptsCconverges rationally to zero error; i.e.,Θ(t−1) in the training sample sizet. However, Cohn and Tesauro have recently demonstrated thatexponentialconvergence can often be observed in experimental settings (i.e., average error decreasing aseΘ−t)). By addressing a simple non-uniformity in the original analysis this paper shows how the dichotomy between rational and exponential worst case learning curves can be recovered in the distribution-free theory. In particular, our results support the experimental findings of Cohn and Tesauro: for finite concept classes any consistent learner achieves exponential convergence, even in the worst case, whereas for continuous concept classes no learner can exhibit sub-rational convergence for every target concept and domain distribution. We also draw a precise boundary between rational and exponential convergence for simple concept chains—showing that somewhere-dense chains always force rational convergence in the worst case, while exponential convergence can always be achieved for nowhere-dense chains
Approximation and Learning of Convex Superpositions

AbstractWe present a fairly general method for constructing classes of functions of finite scale-sensitive dimension (the scale-sensitive dimension is a generalization of the Vapnik–Chervonenkis dimension to real-valued functions). The construction is as follows: start from a classFof functions of finite VC dimension, take the convex hull coFofF, and then take the closurecoFof coFin an appropriate sense. As an example, we study in more detail the case whereFis the class of threshold functions. It is shown thatcoFincludes two important classes of functions: •neural networks with one hidden layer and bounded output weights; •the so-calledΓclass of Barron, which was shown to satisfy a number of interesting approximation and closure properties. We also give an integral representation in the form of a “continuous neural network” which generalizes Barron's. It is shown that the existence of an integral representation is equivalent to bothL2andL∞approximability. A preliminary version of this paper was presented at EuroCOLT'95. The main difference with the conference version is the addition of Theorem 7, where we show that a key topological result fails when the VC dimension hypothesis is removed
Learning Distributions by Their Density Levels: A Paradigm for Learning without a Teacher

AbstractWe propose a mathematical model for learning the high-density areas of an unknown distribution from (unlabeled) random points drawn according to this distribution. While this type of a learning task has not been previously addressed in the computational learnability literature, we believe that this it a rather basic problem that appears in many practical learning scenarios. From a statistical theory standpoint, our model may be viewed as a restricted instance of the fundamental issue of inferring information about a probability distribution from the random samples it generates. From a computational learning angle, what we propose is a few framework of unsupervised concept learning. The examples provided to the learner in our model are not labeled (and are not necessarily all positive or all negative). The only information about their membership is indirectly disclosed to the student through the sampling distribution. We investigate the basic features of the proposed model and provide lower and upper bounds on the sample complexity of such learning tasks. We prove that classes whose VC-dimension is finite are learnable in a very strong sense, while on the other hand,ε-covering numbers of a concept class impose lower bounds on the sample size needed for learning in our models. One direction of the proof involves a reduction of the density-level learnability to PAC learning with respect to fixed distributions (as well as some fundamental statistical lower bounds), while the sufficiency condition is proved through the introduction of a generic learning algorithm
Learning Recursive Functions from Approximations

AbstractThis article investigates algorithmic learning, in the limit, of correct programs for recursive functionsffrom both input/output examples offand several interesting varieties ofapproximateadditional (algorithmic) information aboutf. Specifically considered, as such approximate additional information aboutf, are Rose's frequency computations forfand several natural generalizations from the literature, each generalization involving programs for restricted trees of recursive functions which havefas a branch. Considered as the types of trees are those with bounded variation, bounded width, and bounded rank. For the case of learning final correct programs for recursive functions, EX-learning, where the additional information involves frequency computations, an insightful and interestingly complex combinatorial characterization of learning power is presented as a function of the frequency parameters. For EX-learning (as well as for BC-learning, where a finalsequenceof correct programs is learned), for the cases of providing the types of additional information considered in this paper, the maximal probability is determined such that the entire class of recursive functions is learnable with that probability
Amortization Results for Chromatic Search Trees, with an Application to Priority Queues

AbstractThe intention in designing data structures with relaxed balance, such as chromatic search trees, is to facilitate fast updating on shared-memory asynchronous parallel architectures. To obtain this, the updating and rebalancing have been uncoupled, so extensive locking in connection with updates is avoided. In this paper, we prove that only an amortized constant amount of rebalancing is necessary after an update in a chromatic search tree. We also prove that the amount of rebalancing done at any particular level decreases exponentially, going from the leaves toward the root. These results imply that, in principle, a linear number of processes can access the tree simultaneously. We have included one interesting application of chromatic trees. Based on these trees, a priority queue with possibilities for a greater degree of parallelism than previous proposals can be implemented
Regular Sequence Operations and Their Use in Database Queries

AbstractOperations on sequences are a basic component of database queries that extract information from sequenced data. This paper introduces a family of regular sequence operations (called rs-operations) to be used in such queries. The family is based on a simple pattern matching mechanism using regular expressions as its patterns, and includes most of the “natural” operations on sequences. Properties of the family are examined. In particular, operations in the family are characterized by a mechanical device called generic a-transducer, and the expressive power of the family is studied through an investigation of finite generators of the operations. The usage of the rs-operations in database queries is illustrated through an extended relational data model. Two equivalent query languages, one algebraic and the other calculus, are given in the model. In these query languages, rs-operations are the only components used for dealing with sequences
Trading Independent for Synchronized Parallelism in Finite Copying Parallel Rewriting Systems

AbstractThe so-called family of finite copying parallel rewriting systems is considered in this work, including well-known generative devices and transducers as for instance the deterministic tree-walking transducers, the string generating context-free hypergraph grammars, and the multiple context-free grammars. Two parameters have been defined in the literature for all of the above systems, called the degree of synchronized parallelism and the degree of independent parallelism. When constant bounds are imposed on these parameters, the subclasses of languages generated by the above systems form a two-dimensional hierarchy. In this paper we investigate the interactions between these two parameters and establish new inclusion and separation results for subclasses of the hierarchy. More precisely, for a full half of the hierarchy we provide necessary and sufficient conditions to determine when a language subclass defined by an integer bound ofron the degree of independent parallelism is included in, includes, or is incomparable with a subclass defined by a bound ofr−1 on the same parameter. This means that, in the given range, we can exactly determine which increase in the degree of synchronized parallelism must be taken in order to compensate for a reduction of one unit in the degree of independent parallelism. This solves a question left open in the literature
Transforming a Single-Valued Transducer Into a Mealy Machine

AbstractThis article deals with the transformation of a single-valued finite transducer into a Mealy machine. The following results are obtained: (1) LetMbe a single-valued real-time (or “letter-to-word”) transducer withnstates, input alphabetΣ, and output alphabetΔwhich is equivalent to some Mealy machineM′. Then,Mcan be effectively transformed into such anM′ having at most 2n+1·min{#Σ, #Δ}n−1states. A similar result holds ifMis not real time. As an important side effect three “Mealy” properties are obtained which characterize the fact that the given transducerMis equivalent to some Mealy machine. (2) The upper bound in result (1) improves to 2n−1 ifMis known to be a letter-to-letter transducer. (3) For every integert⩾2 and every odd integern⩾3 there is a single-valued real-time transducerMwithnstates and input and output alphabets of cardinalitytsuch thatMis equivalent to some Mealy machineM′ and every suchM′ has at leastt(n−1)/2states. (4) Ift=3, then result (3) holds true with letter-to-letter transducers rather than real-time transducers and with a lower bound of 2(n−1)/2. (5) It is a PSPACE-complete problem to decide whether or not a given single-valued transducerMis equivalent to some Mealy machine. The problem remains PSPACE-complete ifMis known to be a letter-to-letter transducer
Maximal Machine Learnable Classes

AbstractA class of computable functions ismaximaliff it can be incrementally learned by some inductive inference machine (IIM), but no infinitely larger class of computable functions can be so learned. Rolf Wiehagen posed the question whether there exist such maximal classes. This question and many interesting variants are answered herein in the negative. Viewed positively, each IIM can be infinitely improved upon! Also discussed are the problems of algorithmically finding the improvements proved to exist
Efficient Oblivious Branching Programs for Threshold and Mod Functions

AbstractIn his survey paper on branching programs, Razborov asked the following question: Does every rectifier-switching network computing the majority ofnbits have sizen1+Ω(1)? We answer this question in the negative by constructing a simple oblivious branching program of sizeO[nlog3n/loglognlogloglogn] for computing any threshold function. This improves the previously best known upper bound ofO(n3/2) due to Lupanov. We also construct oblivious branching programs of sizeo(nlog4n) for computing general mod functions. All previously known constructions for computing general mod functions have sizeΩ(n3/2)
On the Design of Reliable Boolean Circuits That Contain Partially Unreliable Gates

AbstractWe investigate a model of gate failure for Boolean circuits in which a faulty gate is restricted to output one of its input values. For some types of gates, the model, which we call theshort-circuit model of gate failure, is weaker than the traditionalvon Neumann modelin which faulty gates always output precisely the wrong value. Our model has the advantage that it allows us to design Boolean circuits that can tolerate worst-case faults, as well as circuits that have arbitrarily high success probability in the case of random faults. Moreover, the short-circuit model captures a particular type of fault that commonly appears in practice, and it suggests a simple method for performing posttest alterations to circuits that have more severe types of faults. A variety of bounds on the size of fault-tolerant circuits are proved in the paper. Perhaps, the most important is a proof that anyk-fault-tolerant circuit for any input-sensitive function using any type of gates (even arbitrarily powerful, multiple-input gates) must have size at leastΩ(klogk/loglogk). Obtaining a tight bound on the size of a circuit for computing the AND of two values if up tokof the gates are faulty is one of the central questions left open in the paper
(De)randomized Construction of Small Sample Spaces in NC

AbstractKoller and Megiddo introduced the paradigm of constructing compact distributions that satisfy a given set of constraints and showed how it can be used to efficiently derandomize certain types of algorithms. In this paper, we significantly extend their results in two ways. First, we show how their approach can be applied to deal with more generalexpectation constraints. More importantly, we provide the firstparallel(NC) algorithm for constructing a compact distribution that satisfies the constraints up to a smallrelativeerror. This algorithm deals with constraints over any event that can be verified by finite automata, including allindependence constraintsas well as constraints over events relating to the parity or sum of a certain set of variables. Our construction relies on a new and independently interesting parallel algorithm for converting a solution to a linear system into an almost basic approximate solution to the same system. We use these techniques in the first NC derandomization of an algorithm for constructing large independent sets ind-uniform hypergraphs forarbitrary d. We also show how the linear programming perspective suggests new proof techniques which might be useful in general probabilistic analysis
An Efficient Membership-Query Algorithm for Learning DNF with Respect to the Uniform Distribution

AbstractWe present a membership-query algorithm for efficiently learning DNF with respect to the uniform distribution. In fact, the algorithm properly learns with respect to uniform the class TOP of Boolean functions expressed as a majority vote over parity functions. We also describe extensions of this algorithm for learning DNF over certain nonuniform distributions and for learning a class of geometric concepts that generalizes DNF. Furthermore, we show that DNF is weakly learnable with respect to uniform from noisy examples. Our strong learning algorithm utilizes one of Freund's boosting techniques and relies on the fact that boosting does not require a completely distribution-independent weak learner. The boosted weak learner is a nonuniform extension of a parity-finding algorithm discovered by Goldreich and Levin
Size-Estimation Framework with Applications to Transitive Closure and Reachability

AbstractComputing the transitive closure in directed graphs is a fundamental graph problem. We consider the more restricted problem of computing the number of nodes reachable from every node and the size of the transitive closure. The fastest known transitive closure algorithms run inO(min{mn,n2.38}) time, wherenis the number of nodes andmthe number of edges in the graph. We present anO(m) time randomized (Monte Carlo) algorithm that estimates, with small relative error, the sizes of all reachability sets and the transitive closure. Another ramification of our estimation scheme is a Õ(m) time algorithm for estimating sizes of neighborhoods in directed graphs with nonnegative edge lengths. Our size-estimation algorithms are much faster than performing the respective explicit computations
On Correctness of Nonserializable Executions

AbstractIn a number of application environments (e.g., computer aided design),serializability, the traditionally accepted notion of correctness has been found to be too restrictive, and a number of alternate criteria have been proposed in the literature. One such criterion ispredicate-wise serializability(PWSR) which requires only restrictions of schedules that access subsets of the database over which integrity constraints are defined to be serializable. In this paper, we identify restrictions on the structure of transaction programs, their concurrent execution, and their access characteristics under which PWSR schedules preserve database consistency
TAL Recognition inO(M(n2)) Time

AbstractWe propose anO(M(n2)) time algorithm for the recognition of tree adjoining languages (TALs), wherenis the size of the input string andM(k) is the time needed to multiply twok×kboolean matrices. Tree adjoining grammars are formalisms suitable for natural language processing and have received enormous attention in the past among not only natural language processing researchers but also algorithms designers. The first polynomial time algorithm for TAL parsing was proposed in 1986 and had a run time ofO(n6). Quite recently, anO(n3M(n)) algorithm has been proposed. The algorithm presented in this paper improves the run time of the recent result using an entirely different approach