pinecone/core-2020-05-10-deduplication huggingface dataset:
Which Problems Have Strongly Exponential Complexity?

AbstractFor several NP-complete problems, there have been a progression of better but still exponential algorithms. In this paper, we address the relative likelihood of sub-exponential algorithms for these problems. We introduce a generalized reduction that we call Sub-exponential Reduction Family (SERF) that preserves sub-exponential complexity. We show that Circuit-SAT is SERF-complete for all NP-search problems, and that for any fixed k⩾3, k-SAT, k-Colorability, k-Set Cover, Independent Set, Clique, and Vertex Cover, are SERF-complete for the class SNP of search problems expressible by second-order existential formulas whose first-order part is universal. In particular, sub-exponential complexity for any one of the above problems implies the same for all others.We also look at the issue of proving strongly exponential lower bounds for AC0, that is, bounds of the form 2Ω(n). This problem is even open for depth-3 circuits. In fact, such a bound for depth-3 circuits with even limited (at most nε) fan-in for bottom-level gates would imply a nonlinear size lower bound for logarithmic depth circuits. We show that with high probability even random degree 2 GF(2) polynomials require strongly exponential size for Σk3 circuits for k=o(loglogn). We thus exhibit a much smaller space of 2O(n2) functions such that almost every function in this class requires strongly exponential size Σk3 circuits. As a corollary, we derive a pseudorandom generator (requiring O(n2) bits of advice) that maps n bits into a larger number of bits so that computing parity on the range is hard for Σk3 circuits. Our main technical lemma is an algorithm that, for any fixed ε>0, represents an arbitrary k-CNF formula as a disjunction of 2εnk-CNF formulas that are sparse, that is, each disjunct has O(n) clauses
The Minimum Equivalent DNF Problem and Shortest Implicants

AbstractWe prove that the Minimum Equivalent DNF problem is ΣP2-complete, resolving a conjecture due to Stockmeyer. We also consider the complexity and approximability of a related optimization problem in the second level of the polynomial hierarchy, that of finding shortest implicants of a Boolean function. We show that when the input is given as a DNF, this problem is complete for a complexity class above coNP utilizing O(log2n)-limited nondeterminism. When the input is given as a formula or circuit, the problem is ΣP2-complete, and ΣP2-hard to approximate within factors of n1/2−ϵ and n1−ϵ, respectively
Perfect Information Leader Election in log*n+O(1) Rounds

AbstractIn the leader election problem, n players wish to elect a random leader. The difficulty is that some coalition of players may conspire to elect one of its own members. We adopt the perfect information model: all communication is by broadcast, and the bad players have unlimited computational power. Protocols proceed in rounds: although players are synchronized between rounds, within each round the bad players may wait to see the inputs of the good players. A protocol is called resilient if a good leader is elected with probability bounded away from 0. We give a simple, constructive leader election protocol that is resilient against coalitions of size βn, for any β<1/2. Our protocol takes log*n+O(1) rounds, each player sending at most logn bits per round. For any constant k, our protocol can be modified to take k rounds and offer resilience against coalitions of size εn/(log(k)n)3, were ε is a small enough constant and log(k) denotes the logarithm iterated k times. This is constructive for k⩾3. The primary component of the above protocols is a new collective sampling protocol: for a set S of large enough (polynomial) size, this protocol generates an element s∈S in a single round so that for any subset T⊂S, Pr[s∈T]⩽|T||S|α(1−β) for a constant α>0
Time–Space Tradeoffs for Branching Programs

AbstractWe obtain the first non-trivial time–space tradeoff lower bound for functions f:{0, 1}n→{0, 1} on general branching programs by exhibiting a Boolean function f that requires exponential size to be computed by any branching program of length (1+ε)n, for some constant ε>0. We also give the first separation result between the syntactic and semantic read-k models (A. Borodin et al., Comput. Complexity3 (1993), 1–18) for k>1 by showing that polynomial-size semantic read-twice branching programs can compute functions that require exponential size on any semantic read-k branching program. We also show a time–space tradeoff result on the more general R-way branching program model (Borodin et al., 1993): for any k, we give a function that requires exponential size to be computed by length knq-way branching programs, for some q=q(k). This result gives a similar tradeoff for RAMs, and thus provides the first nontrivial time–space tradedoff for decision problems in this model
Protocols for Asymmetric Communication Channels

AbstractIn this paper we examine the problem of sending an n-bit data item from a client to a server across an asymmetric communication channel. We demonstrate that there are scenarios in which a high-speed link from the server to the client can be used to greatly reduce the number of bits sent from the client to the server across a slower link. In particular, we assume that the data item is drawn from a probability distribution D that is known to the server but not to the client. We present several protocols in which the expected number of bits transmitted by the server and client are O(n) and O(H(D)+1), respectively, where H(D) is the binary entropy of D (and can range from 0 to n). These protocols are within a small constant factor of optimal in terms of the number of bits sent by the client. The expected number of rounds of communication between the server and client in the simplest of our protocols is O(H(D)). We also give a protocol for which the expected number of rounds is only O(1), but which requires more computational effort on the part of the server. A third technique provides a tradeoff between the computational effort and the number of rounds. These protocols are complemented by several lower bounds and impossibility results. We prove that all of our protocols are existentially optimal in terms of the number of bits sent by the server, i.e., there are distributions for which the total number of bits exchanged has to be at least n. In addition, we show that there is no protocol that is optimal for every distribution (as opposed to just existentially optimal) in terms of bits sent by the server. We demonstrate this by proving that it is undecidable to compute (even approximately), for an arbitrary distribution D, the expected number of bits that must be exchanged by the server and client on the distribution D
Randomness vs Time: Derandomization under a Uniform Assumption

AbstractWe prove that if BPP≠EXP, then every problem in BPP can be solved deterministically in subexponential time on almost every input (on every sampleable ensemble for infinitely many input sizes). This is the first derandomization result for BPP based on uniform, noncryptographic hardness assumptions. It implies the following gap in the deterministic average-case complexities of problems in BPP: either these complexities are always sub-exponential or they contain arbitrarily large exponential functions. We use a construction of a small “pseudorandom” set of strings from a “hard function” in EXP which is identical to that used in the analogous nonuniform results in L. Babai et al. (Comput. Complexity3 (1993), 307–318) and N. Nisan and A. Wigderson (J. Comput. System Sci.49 (1994), 149–167). However, previous proofs of correctness assume the “hard function” is not in P/poly. They give a non constructive argument that a circuit distinguishing the pseudorandom strings from truly random strings implies that a similarly sized circuit exists computing the “hard function.” Our main technical contribution is to show that, if the “hard function” has certain properties, then this argument can be made constructive
On Approximate Nearest Neighbors under l∞ Norm

AbstractThe nearest neighbor search (NNS) problem is the following: Given a set of n points P={p1, …, pn} in some metric space X, preprocess P so as to efficiently answer queries which require finding a point in P closest to a query point q∈X. The approximate nearest neighbor search (c-NNS) is a relaxation of NNS which allows to return any point within c times the distance to the nearest neighbor (called c-nearest neighbor). This problem is of major and growing importance to a variety of applications. In this paper, we give an algorithm for (4⌈log1+ρlog4d⌉+1)-NNS algorithm in ld∞ with O(dn1+ρlogO(1)n) storage and O(dlogO(1)n) query time. Moreover, we obtain an algorithm for 3-NNS for l∞ with nlogd+1 storage. The preprocessing time is close to linear in the size of the data structure. The algorithm can be also used (after simple modifications) to output the exact nearest neighbor in time bounded by O(dlogO(1)n) plus the number of (4⌈log1+ρlog4d⌉+1)-nearest neighbors of the query point. Building on this result, we also obtain an approximation algorithm for a general class of product metrics. Finally, we show that for any c<3 the c-NNS problem in l∞ is provably as hard as the subset query problem (also called the partial match problem). This indicates that obtaining a sublinear query time and subexponential (in d) space for c<3 might be hard
Efficient Approximation Algorithms for the Subset-Sums Equality Problem

AbstractWe investigate the problem of finding two nonempty disjoint subsets of a set of n positive integers, with the objective that the sums of the numbers in the two subsets be as close as possible. In two versions of this problem, the quality of a solution is measured by the ratio and the difference of the two partial sums, respectively. Answering a problem of G. J. Woeginger and Z. Yu (1992, Inform. Process. Lett.42, 299–302) in the affirmative, we give a fully polynomial-time approximation scheme for the case where the value to be optimized is the ratio between the sums of the numbers in the two sets. On the other hand, we show that in the case where the value of a solution is the positive difference between the two partial sums, the problem is not 2nk-approximable in polynomial time unless P=NP, for any constant k. In the positive direction, we give a polynomial time algorithm that finds two subsets for which the difference of the two sums does not exceed K/nΩ(logn), where K is the greatest number in the instance
Distance Realization Problems with Applications to Internet Tomography

AbstractIn recent years, a variety of graph optimization problems have arisen in which the graphs involved are much too large for the usual algorithms to be effective. In these cases, even though we are not able to examine the entire graph (which may be changing dynamically), we would still like to deduce various properties of it, such as the size of a connected component, the set of neighbors of a subset of vertices, etc. In this paper, we study a class of problems, called distance realization problems, which arise in the study of Internet data traffic models. Suppose we are given a set S of terminal nodes, taken from some (unknown) weighted graph. A basic problem is to reconstruct a weighted graph G including S, with possibly additional vertices, that realizes the given distance matrix for S. We will first show that this problem is not only difficult bu the solution is often unstable in the sense that even if all distances between nodes in S decrease, the solution can increase by a factor proportional to the size of S in the worst case. We then proceed to consider a weaker version of the realization problem that only requires the distances in G to upper bound the given distances. We will show that this weak realization problem is NP-complete and that its optimum solutions can be approximated to within a factor of 2. We also consider several variants of these problems and a number of heuristics are presented. These problems are of interest for monitoring large-scale networks and for supplementing network management techniques
Spectral Methods for Matrix Rigidity with Applications to Size–Depth Trade-offs and Communication Complexity

AbstractThe rigidity of a matrix measures the number of entries that must be changed in order to reduce its rank below a certain value. The known lower bounds on the rigidity of explicit matrices are very weak. It is known that stronger lower bounds would have important consequences in complexity theory. We consider some restricted variants of the rigidity problem over the complex numbers. Using spectral methods, we derive lower bounds on these variants. Two applications of such restricted variants are given. First, we show that our lower bound on a variant of rigidity implies lower bounds on size–depth trade-offs for arithmetic circuits with bounded coefficients computing linear transformations. These bounds generalize a result of Nisan and Wigderson. The second application is conditional; we show that it would suffice to prove lower bounds on certain restricted forms of rigidity to conclude several separation results such as separating the analogs of PH and PSPACE in the model of two-party communication complexity. Our results complement and strengthen a result of Razborov. We introduce a combinatorial complexity measure, called AC0-dimension, of sets of Boolean functions. While high rigidity implies large AC0-dimension, large AC0-dimension for explicit sets would already give explicit languages outside the analog of PH in two-party communication complexity. Moreover, the concept of AC0-dimension allows us to formulate interesting combinatorial problems which may be easier than rigidity and which would still have consequences to separation questions in communication complexity
Low Complexity Variants of the Arrow Distributed Directory

AbstractThis paper considers an enhancement to the arrow distributed directory protocol, introduced in [8]. The arrow protocol implements a directory service, allowing nodes to locate mobile objects in a distributed system, while ensuring mutual exclusion in the presence of concurrent requests. The arrow protocol makes use of a minimum spanning treeTm of the network, selected during system initialization, resulting in a worst-case overhead ration of (1+ stretch(Tm))/2. However, we observe that the arrow protocol is correct communicating over any spanning tree T of G. We show that the worst-case overhead ratio is minimized by the minimum stretch spanning tree and that the problem cannot be approximated within a factor better than (1+5)/2, unless P=NP. In contrast, other trees may be more suitable if one is interested in the average-case behavior of the network. We show that in the case where the distribution of the requests is fixed and known in advance, the expected communication is minimized using the minimum communication cost spanning tree (MCT). It is shown that the resulting MCT problem is a restricted case for which one can find a tree T over which the communication cost of the arrow protocol is at most 1.5 times the expected communication cost of an optimal protocol. We also show that even if the distribution of the requests is not fixed, or not known to the protocol in advance, then if the adversary is oblivious, one may use probabilistic approximation of metric space [2, 3] to ensure an expected overhead ratio of O(lognloglogn) in general and an expected ratio of O(logn) in the case of constant dimension Euclidean graphs
Hierarchical Graph Transformation

AbstractWhen graph transformation is used for programming purposes, large graphs should be structured in order to be comprehensible. In this paper, we present an approach for the rule-based transformation of hierarchically structured hypergraphs. In these graphs, distinguished hyperedges contain graphs that can be hierarchical again. Our framework extends the well-known double-pushout approach from flat to hierarchical graphs. In particular, we show how pushouts and pushout complements of hierarchical graphs and graph morphisms can be constructed recursively. Moreover, we make rules more expressive by introducing variables which allow us to copy and remove hierarchical subgraphs in a single rule application
A Polynomial-time Algorithm for the Bistable Roommates Problem

AbstractIn a recent paper, Weems introduced the bistable matching problem, and asked if a polynomial-time algorithm exists to decide the feasibility of the bistable roommates problem. We resolve this question in the affirmative using linear programming. In addition, we show that several (old and new) results for the bistable marriage and roommates problem become transparent using the polyhedral approach. This technique has been used recently by the authors to address classical stable matching problems
Estimation and Approximation Bounds for Gradient-Based Reinforcement Learning

AbstractWe model reinforcement learning as the problem of learning to control a partially observable Markov decision process (POMDP) and focus on gradient ascent approaches to this problem. In an earlier work (2001, J. Artificial Intelligence Res.14) we introduced GPOMDP, an algorithm for estimating the performance gradient of a POMDP from a single sample path, and we proved that this algorithm almost surely converges to an approximation to the gradient. In this paper, we provide a convergence rate for the estimates produced by GPOMDP and give an improved bound on the approximation error of these estimates. Both of these bounds are in terms of mixing times of the POMDP
A New Abstract Combinatorial Dimension for Exact Learning via Queries

AbstractWe introduce an abstract model of exact learning via queries that can be instantiated to all the query learning models currently in use, while being closer to them than previous unifying attempts. We present a characterization of those Boolean function classes learnable in this abstract model, in terms of a new combinatorial notion that we introduce, the abstract identification dimension. Then we prove that the particularization of our notion to specific known protocols such as equivalence, membership, and membership and equivalence queries results in exactly the same combinatorial notions currently known to characterize learning in these models, such as strong consistency dimension, extended teaching dimension, and certificate size. Our theory thus fully unifies all these characterizations. For models enjoying a specific property that we identify, the notion can be simplified while keeping the same characterizations. From our results we can derive combinatorial characterizations of all those other models for query learning proposed in the literature. We can also obtain the first polynomial-query learning algorithms for specific interesting problems such as learning DNF with proper subset and superset queries
Adaptive and Self-Confident On-Line Learning Algorithms

AbstractWe study on-line learning in the linear regression framework. Most of the performance bounds for on-line algorithms in this framework assume a constant learning rate. To achieve these bounds the learning rate must be optimized based on a posteriori information. This information depends on the whole sequence of examples and thus it is not available to any strictly on-line algorithm. We introduce new techniques for adaptively tuning the learning rate as the data sequence is progressively revealed. Our techniques allow us to prove essentially the same bounds as if we knew the optimal learning rate in advance. Moreover, such techniques apply to a wide class of on-line algorithms, including p-norm algorithms for generalized linear regression and Weighted Majority for linear regression with absolute loss. Our adaptive tunings are radically different from previous techniques, such as the so-called doubling trick. Whereas the doubling trick restarts the on-line algorithm several times using a constant learning rate for each run, our methods save information by changing the value of the learning rate very smoothly. In fact, for Weighted Majority over a finite set of experts our analysis provides a better leading constant than the doubling trick
Boosting Using Branching Programs

AbstractIt is known that decision tree learning can be viewed as a form of boosting. Given a weak learning hypothesis one can show that the training error of a decision tree declines as |T|−β where |T| is the size of the decision tree and β is a constant determined by the weak learning hypothesis. Here we consider the case of decision DAGs—decision trees in which a given node can be shared by different branches of the tree, also called branching programs (BP). Node sharing allows a branching program to be exponentially more compact than the corresponding decision tree. We show that under the same weak learning assumption used for decision tree learning there exists a greedy BP-growth algorithm whose training error is guaranteed to decline as 2−β|T|, where |T| is the size of the branching program and β is a constant determined by the weak learning hypothesis. Therefore, from the perspective of boosting theory, branching programs are exponentially more efficient than decision trees
The Computational Complexity of Densest Region Detection

AbstractWe investigate the computational complexity of the task of detecting dense regions of an unknown distribution from unlabeled samples of this distribution. We introduce a formal learning model for this task that uses a hypothesis class as it “anti-overfitting” mechanism. The learning task in our model can be reduced to a combinatorial optimization problem. We can show that for some constants, depending on the hypothesis class, these problems are NP-hard to approximate to within these constant factors. We go on and introduce a new criterion for the success of approximate optimization geometric problems. The new criterion requires that the algorithm competes with hypotheses only on the points that are separated by some margin μ from their boundaries. Quite surprisingly, we discover that for each of the two hypothesis classes that we investigate, there is a “critical value” of the margin parameter μ. For any value below the critical value the problems are NP-hard to approximate, while, once this value is exceeded, the problems become poly-time solvable
Relative Expected Instantaneous Loss Bounds

AbstractIn the literature a number of relative loss bounds have been shown for on-line learning algorithms. Here the relative loss is the total loss of the on-line algorithm in all trials minus the total loss of the best comparator that is chosen off-line. However, for many applications instantaneous loss bounds are more interesting where the learner first sees a batch of examples and then uses these examples to make a prediction on a new instance. We show relative expected instantaneous loss bounds for the case when the examples are i.i.d. with an unknown distribution. We bound the expected loss of the algorithm on the last example minus the expected loss of the best comparator on a random example. In particular, we study linear regression and density estimation problems and show how the leave-one-out loss can be used to prove instantaneous loss bounds for these cases. For linear regression we use an algorithm that is similar to a new on-line learning algorithm developed by Vovk. Recently a large number of relative total loss bounds have been shown that have the form O(lnT), where T is the number of trials/examples. Standard conversions of on-line algorithms to batch algorithms result in relative expected instantaneous loss bounds of the form O(lnTT). Our methods lead to O(1T) upper bounds. In many cases we give tight lower bounds