OpenRaiser/Intern-Atlas huggingface dataset:
The complexity of learning halfspaces using generalized linear methods
Venue: arXiv cs.LG 2014
Year: 2014

Many popular learning algorithms (E.g. Regression, Fourier-Transform based algorithms, Kernel SVM and Kernel ridge regression) operate by reducing the problem to a convex optimization problem over a vector space of functions. These methods offer the currently best approach to several central problems such as learning half spaces and learning DNF's. In addition they are widely used in numerous application domains. Despite their importance, there are still very few proof techniques to show limits on the power of these algorithms. We study the performance of this approach in the problem of (agnostically and improperly) learning halfspaces with margin $γ$. Let $\mathcal{D}$ be a distribution over labeled examples. The $γ$-margin error of a hyperplane $h$ is the probability of an example to fall on the wrong side of $h$ or at a distance $\leγ$ from it. The $γ$-margin error of the best $h$ is denoted $\mathrm{Err}_γ(\mathcal{D})$. An $α(γ)$-approximation algorithm receives $γ,ε$ as input and, using i.i.d. samples of $\mathcal{D}$, outputs a classifier with error rate $\le α(γ)\mathrm{Err}_γ(\mathcal{D}) + ε$. Such an algorithm is efficient if it uses $\mathrm{poly}(\frac{1}γ,\frac{1}ε)$ samples and runs in time polynomial in the sample size. The best approximation ratio achievable by an efficient algorithm is $O\left(\frac{1/γ}{\sqrt{\log(1/γ)}}\right)$ and is achieved using an algorithm from the above class. Our main result shows that the approximation ratio of every efficient algorithm from this family must be $\ge Ω\left(\frac{1/γ}{\mathrm{poly}\left(\log\left(1/γ\right)\right)}\right)$, essentially matching the best known upper bound.
Recovering the Optimal Solution by Dual Random Projection
Venue: arXiv cs.LG 2014
Year: 2014

Random projection has been widely used in data classification. It maps high-dimensional data into a low-dimensional subspace in order to reduce the computational cost in solving the related optimization problem. While previous studies are focused on analyzing the classification performance of using random projection, in this work, we consider the recovery problem, i.e., how to accurately recover the optimal solution to the original optimization problem in the high-dimensional space based on the solution learned from the subspace spanned by random projections. We present a simple algorithm, termed Dual Random Projection, that uses the dual solution of the low-dimensional optimization problem to recover the optimal solution to the original problem. Our theoretical analysis shows that with a high probability, the proposed algorithm is able to accurately recover the optimal solution to the original problem, provided that the data matrix is of low rank or can be well approximated by a low rank matrix.
Lemma for Linear Feedback Shift Registers and DFTs Applied to Affine Variety Codes
Venue: arXiv cs.IT 2014
Year: 2014

In this paper, we establish a lemma in algebraic coding theory that frequently appears in the encoding and decoding of, e.g., Reed-Solomon codes, algebraic geometry codes, and affine variety codes. Our lemma corresponds to the non-systematic encoding of affine variety codes, and can be stated by giving a canonical linear map as the composition of an extension through linear feedback shift registers from a Grobner basis and a generalized inverse discrete Fourier transform. We clarify that our lemma yields the error-value estimation in the fast erasure-and-error decoding of a class of dual affine variety codes. Moreover, we show that systematic encoding corresponds to a special case of erasure-only decoding. The lemma enables us to reduce the computational complexity of error-evaluation from O(n^3) using Gaussian elimination to O(qn^2) with some mild conditions on n and q, where n is the code length and q is the finite-field size.
Beyond the Vizing's bound for at most seven colors
Venue: arXiv cs.DS 2014
Year: 2014

Let $G=(V,E)$ be a simple graph of maximum degree $Δ$. The edges of $G$ can be colored with at most $Δ+1$ colors by Vizing's theorem. We study lower bounds on the size of subgraphs of $G$ that can be colored with $Δ$ colors. Vizing's Theorem gives a bound of $\fracΔ{Δ+1}|E|$. This is known to be tight for cliques $K_{Δ+1}$ when $Δ$ is even. However, for $Δ=3$ it was improved to $26/31|E|$ by Albertson and Haas [Parsimonious edge colorings, Disc. Math. 148, 1996] and later to $6/7|E|$ by Rizzi [Approximating the maximum 3-edge-colorable subgraph problem, Disc. Math. 309, 2009]. It is tight for $B_3$, the graph isomorphic to a $K_4$ with one edge subdivided. We improve previously known bounds for $Δ\in{3,...,7}$, under the assumption that for $Δ=3,4,6$ graph $G$ is not isomorphic to $B_3$, $K_5$ and $K_7$, respectively. For $Δ\geq 4$ these are the first results which improve over the Vizing's bound. We also show a new bound for subcubic multigraphs not isomorphic to $K_3$ with one edge doubled. In the second part, we give approximation algorithms for the Maximum k-Edge-Colorable Subgraph problem, where given a graph G (without any bound on its maximum degree or other restrictions) one has to find a k-edge-colorable subgraph with maximum number of edges. In particular, when G is simple for k=3,4,5,6,7 we obtain approximation ratios of 13/15, 9/11, 19/22, 23/27 and 22/25, respectively. We also present a 7/9-approximation for k=3 when G is a multigraph. The approximation algorithms follow from a new general framework that can be used for any value of k.
Stable XOR-based Policies for the Broadcast Erasure Channel with Feedback
Venue: arXiv cs.IT 2014
Year: 2014

In this paper we describe a network coding scheme for the Broadcast Erasure Channel with multiple unicast stochastic flows, in the case of a single source transmitting packets to $N$ users, where per-slot feedback is fed back to the transmitter in the form of ACK/NACK messages. This scheme performs only binary (XOR) operations and involves a network of queues, along with special rules for coding and moving packets among the queues, that ensure instantaneous decodability. The system under consideration belongs to a class of networks whose stability properties have been analyzed in earlier work, which is used to provide a stabilizing policy employing the currently proposed coding scheme. Finally, we show the optimality of the proposed policy for $N=4$ and i.i.d. erasure events, in the sense that the policy's stability region matches a derived outer bound (which coincides with the system's information-theoretic capacity region), even when a restricted set of coding rules is used.
Random Projections for Linear Support Vector Machines
Venue: arXiv cs.LG 2014
Year: 2014

Let X be a data matrix of rank ρ, whose rows represent n points in d-dimensional space. The linear support vector machine constructs a hyperplane separator that maximizes the 1-norm soft margin. We develop a new oblivious dimension reduction technique which is precomputed and can be applied to any input matrix X. We prove that, with high probability, the margin and minimum enclosing ball in the feature space are preserved to within ε-relative error, ensuring comparable generalization as in the original space in the case of classification. For regression, we show that the margin is preserved to ε-relative error with high probability. We present extensive experiments with real and synthetic data to support our theory.
TACT: A Transfer Actor-Critic Learning Framework for Energy Saving in Cellular Radio Access Networks
Venue: arXiv cs.NI 2014
Year: 2014

Recent works have validated the possibility of improving energy efficiency in radio access networks (RANs), achieved by dynamically turning on/off some base stations (BSs). In this paper, we extend the research over BS switching operations, which should match up with traffic load variations. Instead of depending on the dynamic traffic loads which are still quite challenging to precisely forecast, we firstly formulate the traffic variations as a Markov decision process. Afterwards, in order to foresightedly minimize the energy consumption of RANs, we design a reinforcement learning framework based BS switching operation scheme. Furthermore, to avoid the underlying curse of dimensionality in reinforcement learning, a transfer actor-critic algorithm (TACT), which utilizes the transferred learning expertise in historical periods or neighboring regions, is proposed and provably converges. In the end, we evaluate our proposed scheme by extensive simulations under various practical configurations and show that the proposed TACT algorithm contributes to a performance jumpstart and demonstrates the feasibility of significant energy efficiency improvement at the expense of tolerable delay performance.
The Convergence Guarantees of a Non-convex Approach for Sparse Recovery
Venue: arXiv cs.IT 2014
Year: 2014

In the area of sparse recovery, numerous researches hint that non-convex penalties might induce better sparsity than convex ones, but up until now those corresponding non-convex algorithms lack convergence guarantees from the initial solution to the global optimum. This paper aims to provide performance guarantees of a non-convex approach for sparse recovery. Specifically, the concept of weak convexity is incorporated into a class of sparsity-inducing penalties to characterize the non-convexity. Borrowing the idea of the projected subgradient method, an algorithm is proposed to solve the non-convex optimization problem. In addition, a uniform approximate projection is adopted in the projection step to make this algorithm computationally tractable for large scale problems. The convergence analysis is provided in the noisy scenario. It is shown that if the non-convexity of the penalty is below a threshold (which is in inverse proportion to the distance between the initial solution and the sparse signal), the recovered solution has recovery error linear in both the step size and the noise term. Numerical simulations are implemented to test the performance of the proposed approach and verify the theoretical analysis.
Performance Bounds on a Wiretap Network with Arbitrary Wiretap Sets
Venue: arXiv cs.IT 2014
Year: 2014

Consider a communication network represented by a directed graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$, where $\mathcal{V}$ is the set of nodes and $\mathcal{E}$ is the set of point-to-point channels in the network. On the network a secure message $M$ is transmitted, and there may exist wiretappers who want to obtain information about the message. In secure network coding, we aim to find a network code which can protect the message against the wiretapper whose power is constrained. Cai and Yeung \cite{cai2002secure} studied the model in which the wiretapper can access any one but not more than one set of channels, called a wiretap set, out of a collection $\mathcal{A}$ of all possible wiretap sets. In order to protect the message, the message needs to be mixed with a random key $K$. They proved tight fundamental performance bounds when $\mathcal{A}$ consists of all subsets of $\mathcal{E}$ of a fixed size $r$. However, beyond this special case, obtaining such bounds is much more difficult. In this paper, we investigate the problem when $\mathcal{A}$ consists of arbitrary subsets of $\mathcal{E}$ and obtain the following results: 1) an upper bound on $H(M)$; 2) a lower bound on $H(K)$ in terms of $H(M)$. The upper bound on $H(M)$ is explicit, while the lower bound on $H(K)$ can be computed in polynomial time when $|\mathcal{A}|$ is fixed. The tightness of the lower bound for the point-to-point communication system is also proved.
The Cost of Address Translation
Venue: arXiv cs.DS 2014
Year: 2014

Modern computers are not random access machines (RAMs). They have a memory hierarchy, multiple cores, and virtual memory. In this paper, we address the computational cost of address translation in virtual memory. Starting point for our work is the observation that the analysis of some simple algorithms (random scan of an array, binary search, heapsort) in either the RAM model or the EM model (external memory model) does not correctly predict growth rates of actual running times. We propose the VAT model (virtual address translation) to account for the cost of address translations and analyze the algorithms mentioned above and others in the model. The predictions agree with the measurements. We also analyze the VAT-cost of cache-oblivious algorithms.
Computing Consensus Curves
Venue: arXiv cs.CG 2014
Year: 2014

We consider the problem of extracting accurate average ant trajectories from many (possibly inaccurate) input trajectories contributed by citizen scientists. Although there are many generic software tools for motion tracking and specific ones for insect tracking, even untrained humans are much better at this task, provided a robust method to computing the average trajectories. We implemented and tested several local (one ant at a time) and global (all ants together) method. Our best performing algorithm uses a novel global method, based on finding edge-disjoint paths in an ant-interaction graph constructed from the input trajectories. The underlying optimization problem is a new and interesting variant of network flow. Even though the problem is NP-hard, we implemented two heuristics, which work very well in practice, outperforming all other approaches, including the best automated system.
Mechanism Design for Fair Division
Venue: arXiv cs.GT 2014
Year: 2014

We revisit the classic problem of fair division from a mechanism design perspective, using {\em Proportional Fairness} as a benchmark. In particular, we aim to allocate a collection of divisible items to a set of agents while incentivizing the agents to be truthful in reporting their valuations. For the very large class of homogeneous valuations, we design a truthful mechanism that provides {\em every agent} with at least a $1/e\approx 0.368$ fraction of her Proportionally Fair valuation. To complement this result, we show that no truthful mechanism can guarantee more than a $0.5$ fraction, even for the restricted class of additive linear valuations. We also propose another mechanism for additive linear valuations that works really well when every item is highly demanded. To guarantee truthfulness, our mechanisms discard a carefully chosen fraction of the allocated resources; we conclude by uncovering interesting connections between our mechanisms and known mechanisms that use money instead.
Achieving Optimal Throughput and Near-Optimal Asymptotic Delay Performance in Multi-Channel Wireless Networks with Low Complexity: A Practical Greedy Scheduling Policy
Venue: arXiv cs.NI 2014
Year: 2014

In this paper, we focus on the scheduling problem in multi-channel wireless networks, e.g., the downlink of a single cell in fourth generation (4G) OFDM-based cellular networks. Our goal is to design practical scheduling policies that can achieve provably good performance in terms of both throughput and delay, at a low complexity. While a class of $O(n^{2.5} \log n)$-complexity hybrid scheduling policies are recently developed to guarantee both rate-function delay optimality (in the many-channel many-user asymptotic regime) and throughput optimality (in the general non-asymptotic setting), their practical complexity is typically high. To address this issue, we develop a simple greedy policy called Delay-based Server-Side-Greedy (D-SSG) with a \lower complexity $2n^2+2n$, and rigorously prove that D-SSG not only achieves throughput optimality, but also guarantees near-optimal asymptotic delay performance. Specifically, we show that the rate-function attained by D-SSG for any delay-violation threshold $b$, is no smaller than the maximum achievable rate-function by any scheduling policy for threshold $b-1$. Thus, we are able to achieve a reduction in complexity (from $O(n^{2.5} \log n)$ of the hybrid policies to $2n^2 + 2n$) with a minimal drop in the delay performance. More importantly, in practice, D-SSG generally has a substantially lower complexity than the hybrid policies that typically have a large constant factor hidden in the $O(\cdot)$ notation. Finally, we conduct numerical simulations to validate our theoretical results in various scenarios. The simulation results show that D-SSG not only guarantees a near-optimal rate-function, but also empirically is virtually indistinguishable from delay-optimal policies.
Simultaneously Structured Models with Application to Sparse and Low-rank Matrices
Venue: arXiv cs.IT 2014
Year: 2014

The topic of recovery of a structured model given a small number of linear observations has been well-studied in recent years. Examples include recovering sparse or group-sparse vectors, low-rank matrices, and the sum of sparse and low-rank matrices, among others. In various applications in signal processing and machine learning, the model of interest is known to be structured in several ways at the same time, for example, a matrix that is simultaneously sparse and low-rank. Often norms that promote each individual structure are known, and allow for recovery using an order-wise optimal number of measurements (e.g., $\ell_1$ norm for sparsity, nuclear norm for matrix rank). Hence, it is reasonable to minimize a combination of such norms. We show that, surprisingly, if we use multi-objective optimization with these norms, then we can do no better, order-wise, than an algorithm that exploits only one of the present structures. This result suggests that to fully exploit the multiple structures, we need an entirely new convex relaxation, i.e. not one that is a function of the convex relaxations used for each structure. We then specialize our results to the case of sparse and low-rank matrices. We show that a nonconvex formulation of the problem can recover the model from very few measurements, which is on the order of the degrees of freedom of the matrix, whereas the convex problem obtained from a combination of the $\ell_1$ and nuclear norms requires many more measurements. This proves an order-wise gap between the performance of the convex and nonconvex recovery problems in this case. Our framework applies to arbitrary structure-inducing norms as well as to a wide range of measurement ensembles. This allows us to give performance bounds for problems such as sparse phase retrieval and low-rank tensor completion.
Graph Products Revisited: Tight Approximation Hardness of Induced Matching, Poset Dimension and More
Venue: arXiv cs.DM 2014
Year: 2014

Graph product is a fundamental tool with rich applications in both graph theory and theoretical computer science. It is usually studied in the form $f(G*H)$ where $G$ and $H$ are graphs, * is a graph product and $f$ is a graph property. For example, if $f$ is the independence number and * is the disjunctive product, then the product is known to be multiplicative: $f(G*H)=f(G)f(H)$. In this paper, we study graph products in the following non-standard form: $f((G\oplus H)*J)$ where $G$, $H$ and $J$ are graphs, $\oplus$ and * are two different graph products and $f$ is a graph property. We show that if $f$ is the induced and semi-induced matching number, then for some products $\oplus$ and *, it is subadditive in the sense that $f((G\oplus H)*J)\leq f(G*J)+f(H*J)$. Moreover, when $f$ is the poset dimension number, it is almost subadditive. As applications of this result (we only need $J=K_2$ here), we obtain tight hardness of approximation for various problems in discrete mathematics and computer science: bipartite induced and semi-induced matching (a.k.a. maximum expanding sequences), poset dimension, maximum feasible subsystem with 0/1 coefficients, unit-demand min-buying and single-minded pricing, donation center location, boxicity, cubicity, threshold dimension and independent packing.
Short lists for shortest descriptions in short time
Venue: arXiv cs.CC 2014
Year: 2014

Is it possible to find a shortest description for a binary string? The well-known answer is "no, Kolmogorov complexity is not computable." Faced with this barrier, one might instead seek a short list of candidates which includes a laconic description. Remarkably such approximations exist. This paper presents an efficient algorithm which generates a polynomial-size list containing an optimal description for a given input string. Along the way, we employ expander graphs and randomness dispersers to obtain an Explicit Online Matching Theorem for bipartite graphs and a refinement of Muchnik's Conditional Complexity Theorem. Our main result extends recent work by Bauwens, Mahklin, Vereschchagin, and Zimand.
Statistical mechanics approach to 1-bit compressed sensing
Venue: arXiv cs.IT 2014
Year: 2014

Compressed sensing is a technique for recovering a high-dimensional signal from lower-dimensional data, whose components represent partial information about the signal, utilizing prior knowledge on the sparsity of the signal. For further reducing the data size of the compressed expression, a scheme to recover the original signal utilizing only the sign of each entry of the linearly transformed vector was recently proposed. This approach is often termed the 1-bit compressed sensing. Here we analyze the typical performance of an L1-norm based signal recovery scheme for the 1-bit compressed sensing using statistical mechanics methods. We show that the signal recovery performance predicted by the replica method under the replica symmetric ansatz, which turns out to be locally unstable for modes breaking the replica symmetry, is in a good consistency with experimental results of an approximate recovery algorithm developed earlier. This suggests that the L1-based recovery problem typically has many local optima of a similar recovery accuracy, which can be achieved by the approximate algorithm. We also develop another approximate recovery algorithm inspired by the cavity method. Numerical experiments show that when the density of nonzero entries in the original signal is relatively large the new algorithm offers better performance than the abovementioned scheme and does so with a lower computational cost.
Clique versus Independent Set
Venue: arXiv cs.DM 2014
Year: 2014

Yannakakis' Clique versus Independent Set problem (CL-IS) in communication complexity asks for the minimum number of cuts separating cliques from stable sets in a graph, called CS-separator. Yannakakis provides a quasi-polynomial CS-separator, i.e. of size $O(n^{\log n})$, and addresses the problem of finding a polynomial CS-separator. This question is still open even for perfect graphs. We show that a polynomial CS-separator almost surely exists for random graphs. Besides, if H is a split graph (i.e. has a vertex-partition into a clique and a stable set) then there exists a constant $c_H$ for which we find a $O(n^{c_H})$ CS-separator on the class of H-free graphs. This generalizes a result of Yannakakis on comparability graphs. We also provide a $O(n^{c_k})$ CS-separator on the class of graphs without induced path of length k and its complement. Observe that on one side, $c_H$ is of order $O(|H| \log |H|)$ resulting from Vapnik-Chervonenkis dimension, and on the other side, $c_k$ is exponential. One of the main reason why Yannakakis' CL-IS problem is fascinating is that it admits equivalent formulations. Our main result in this respect is to show that a polynomial CS-separator is equivalent to the polynomial Alon-Saks-Seymour Conjecture, asserting that if a graph has an edge-partition into k complete bipartite graphs, then its chromatic number is polynomially bounded in terms of k. We also show that the classical approach to the stubborn problem (arising in CSP) which consists in covering the set of all solutions by $O(n^{\log n})$ instances of 2-SAT is again equivalent to the existence of a polynomial CS-separator.