OpenRaiser/Intern-Atlas huggingface dataset:
Compressed sensing of block-sparse positive vectors
Venue: arXiv cs.IT 2015
Year: 2015

In this paper we revisit one of the classical problems of compressed sensing. Namely, we consider linear under-determined systems with sparse solutions. A substantial success in mathematical characterization of an $\ell_1$ optimization technique typically used for solving such systems has been achieved during the last decade. Seminal works \cite{CRT,DOnoho06CS} showed that the $\ell_1$ can recover a so-called linear sparsity (i.e. solve systems even when the solution has a sparsity linearly proportional to the length of the unknown vector). Later considerations \cite{DonohoPol,DonohoUnsigned} (as well as our own ones \cite{StojnicCSetam09,StojnicUpper10}) provided the precise characterization of this linearity. In this paper we consider the so-called structured version of the above sparsity driven problem. Namely, we view a special case of sparse solutions, the so-called block-sparse solutions. Typically one employs $\ell_2/\ell_1$-optimization as a variant of the standard $\ell_1$ to handle block-sparse case of sparse solution systems. We considered systems with block-sparse solutions in a series of work \cite{StojnicCSetamBlock09,StojnicUpperBlock10,StojnicICASSP09block,StojnicJSTSP09} where we were able to provide precise performance characterizations if the $\ell_2/\ell_1$-optimization similar to those obtained for the standard $\ell_1$ optimization in \cite{StojnicCSetam09,StojnicUpper10}. Here we look at a similar class of systems where on top of being block-sparse the unknown vectors are also known to have components of the same sign. In this paper we slightly adjust $\ell_2/\ell_1$-optimization to account for the known signs and provide a precise performance characterization of such an adjustment.
Nonlinear continuous integral-derivative observer
Venue: arXiv cs.SY 2015
Year: 2015

In this paper, a high-order nonlinear continuous integral-derivative observer is presented based on finite-time stability and singular perturbation technique. The proposed integral-derivative observer can not only obtain the multiple integrals of a signal, but can also estimate the derivatives. Conditions are given ensuring finite-time stability for the presented integral-derivative observer, and the stability and robustness in time domain are analysed. The merits of the presented integral-derivative observer include its synchronous estimation of integrals and derivatives, finite-time stability, ease of parameters selection, sufficient stochastic noises rejection and almost no drift phenomenon. The theoretical results are confirmed by computational analysis and simulations.
Counting list matrix partitions of graphs
Venue: arXiv cs.CC 2015
Year: 2015

Given a symmetric D*D matrix M over {0,1,*}, a list M-partition of a graph G is a partition of G's vertices into D parts associated with the rows of M. The part of each vertex is chosen from a given list so that no edge of G maps to a 0 in M and no non-edge of G maps to a 1 in M. Many important graph-theoretic structures can be represented as list M-partitions, such as graph colourings, split graphs and homogeneous sets and pairs, which arise in the proofs of the weak and strong perfect graph conjectures. There has been quite a bit of work on determining for which matrices M computations involving list M-partitions are tractable. We focus on counting list M-partitions, given a graph G and a list for each vertex of G. We identify a set of "tractable" matrices and give an algorithm that counts list M-partitions in polynomial time for every (fixed) matrix M in this set. The algorithm uses data structures such as sparse-dense partitions and subcube decompositions to reduce each instance to a sequence of instances in which the lists restrict access to portions of M in which the interaction of 0s and 1s is controlled. We solve the resulting restricted instances by converting them into counting constraint satisfaction problems (#CSPs) which we solve using arc-consistency. For every matrix M for which our algorithm fails, we show that counting list M-partitions is #P-complete. Further, we give an explicit characterisation of the dichotomy theorem: counting list M-partitions is in FP if M has a structure called a derectangularising sequence; otherwise, counting list M-partitions is #P-hard. We show that the meta-problem of determining whether a given matrix has a derectangularising sequence is NP-complete. Finally, we show that lists can be used to encode cardinality restrictions in M-partitions problems and use this to give a polynomial-time algorithm for counting homogeneous pairs in graphs.
Deep Learning by Scattering
Venue: arXiv cs.LG 2015
Year: 2015

We introduce general scattering transforms as mathematical models of deep neural networks with l2 pooling. Scattering networks iteratively apply complex valued unitary operators, and the pooling is performed by a complex modulus. An expected scattering defines a contractive representation of a high-dimensional probability distribution, which preserves its mean-square norm. We show that unsupervised learning can be casted as an optimization of the space contraction to preserve the volume occupied by unlabeled examples, at each layer of the network. Supervised learning and classification are performed with an averaged scattering, which provides scattering estimations for multiple classes.
Extrinsic Jensen-Shannon Divergence: Applications to Variable-Length Coding
Venue: arXiv cs.IT 2015
Year: 2015

This paper considers the problem of variable-length coding over a discrete memoryless channel (DMC) with noiseless feedback. The paper provides a stochastic control view of the problem whose solution is analyzed via a newly proposed symmetrized divergence, termed extrinsic Jensen-Shannon (EJS) divergence. It is shown that strictly positive lower bounds on EJS divergence provide non-asymptotic upper bounds on the expected code length. The paper presents strictly positive lower bounds on EJS divergence, and hence non-asymptotic upper bounds on the expected code length, for the following two coding schemes: variable-length posterior matching and MaxEJS coding scheme which is based on a greedy maximization of the EJS divergence. As an asymptotic corollary of the main results, this paper also provides a rate-reliability test. Variable-length coding schemes that satisfy the condition(s) of the test for parameters $R$ and $E$, are guaranteed to achieve rate $R$ and error exponent $E$. The results are specialized for posterior matching and MaxEJS to obtain deterministic one-phase coding schemes achieving capacity and optimal error exponent. For the special case of symmetric binary-input channels, simpler deterministic schemes of optimal performance are proposed and analyzed.
Complexity of the FIFO Stack-Up Problem
Venue: arXiv cs.DS 2015
Year: 2015

We study the combinatorial FIFO stack-up problem. In delivery industry, bins have to be stacked-up from conveyor belts onto pallets with respect to customer orders. Given k sequences q_1, ..., q_k of labeled bins and a positive integer p, the aim is to stack-up the bins by iteratively removing the first bin of one of the k sequences and put it onto an initially empty pallet of unbounded capacity located at one of p stack-up places. Bins with different pallet labels have to be placed on different pallets, bins with the same pallet label have to be placed on the same pallet. After all bins for a pallet have been removed from the given sequences, the corresponding stack-up place will be cleared and becomes available for a further pallet. The FIFO stack-up problem is to find a stack-up sequence such that all pallets can be build-up with the available p stack-up places. In this paper, we introduce two digraph models for the FIFO stack-up problem, namely the processing graph and the sequence graph. We show that there is a processing of some list of sequences with at most p stack-up places if and only if the sequence graph of this list has directed pathwidth at most p-1. This connection implies that the FIFO stack-up problem is NP-complete in general, even if there are at most 6 bins for every pallet and that the problem can be solved in polynomial time, if the number p of stack-up places is assumed to be fixed. Further the processing graph allows us to show that the problem can be solved in polynomial time, if the number k of sequences is assumed to be fixed.
Approaching the Rate-Distortion Limit with Spatial Coupling, Belief propagation and Decimation
Venue: arXiv cs.IT 2015
Year: 2015

We investigate an encoding scheme for lossy compression of a binary symmetric source based on simple spatially coupled Low-Density Generator-Matrix codes. The degree of the check nodes is regular and the one of code-bits is Poisson distributed with an average depending on the compression rate. The performance of a low complexity Belief Propagation Guided Decimation algorithm is excellent. The algorithmic rate-distortion curve approaches the optimal curve of the ensemble as the width of the coupling window grows. Moreover, as the check degree grows both curves approach the ultimate Shannon rate-distortion limit. The Belief Propagation Guided Decimation encoder is based on the posterior measure of a binary symmetric test-channel. This measure can be interpreted as a random Gibbs measure at a "temperature" directly related to the "noise level of the test-channel". We investigate the links between the algorithmic performance of the Belief Propagation Guided Decimation encoder and the phase diagram of this Gibbs measure. The phase diagram is investigated thanks to the cavity method of spin glass theory which predicts a number of phase transition thresholds. In particular the dynamical and condensation "phase transition temperatures" (equivalently test-channel noise thresholds) are computed. We observe that: (i) the dynamical temperature of the spatially coupled construction saturates towards the condensation temperature; (ii) for large degrees the condensation temperature approaches the temperature (i.e. noise level) related to the information theoretic Shannon test-channel noise parameter of rate-distortion theory. This provides heuristic insight into the excellent performance of the Belief Propagation Guided Decimation algorithm. The paper contains an introduction to the cavity method.
Robust Quantitative Comparative Statics for a Multimarket Paradox
Venue: arXiv cs.GT 2015
Year: 2015

We introduce a quantitative approach to comparative statics that allows to bound the maximum effect of an exogenous parameter change on a system's equilibrium. The motivation for this approach is a well known paradox in multimarket Cournot competition, where a positive price shock on a monopoly market may actually reduce the monopolist's profit. We use our approach to quantify for the first time the worst case profit reduction for multimarket oligopolies exposed to arbitrary positive price shocks. For markets with affine price functions and firms with convex cost technologies, we show that the relative profit loss of any firm is at most 25% no matter how many firms compete in the oligopoly. We further investigate the impact of positive price shocks on total profit of all firms as well as on social welfare. We find tight bounds also for these measures showing that total profit and social welfare decreases by at most 25% and 16.6%, respectively. Finally, we show that in our model, mixed, correlated and coarse correlated equilibria are essentially unique, thus, all our bounds apply to these game solutions as well.
Reconstruction of Network Coded Sources From Incomplete Datasets
Venue: arXiv cs.IT 2015
Year: 2015

In this paper, we investigate the problem of recovering source information from an incomplete set of network coded data. We first study the theoretical performance of such systems under maximum a posteriori (MAP) decoding and derive the upper bound on the probability of decoding error as a function of the system parameters. We also establish the sufficient conditions on the number of network coded symbols required to achieve decoding error probability below a certain level. We then propose a low complexity iterative decoding algorithm based on message passing for decoding the network coded data of a particular class of statistically dependent sources that present pairwise linear correlation. The algorithm operates on a graph that captures the network coding constraints, while the knowledge about the source correlation is directly incorporated in the messages exchanged over the graph. We test the proposed method on both synthetic data and correlated image sequences and demonstrate that the prior knowledge about the source correlation can be effectively exploited at the decoder in order to provide a good reconstruction of the transmitted data in cases where the network coded data available at the decoder is not sufficient for exact decoding.
Geometric Inference on Kernel Density Estimates
Venue: arXiv cs.CG 2015
Year: 2015

We show that geometric inference of a point cloud can be calculated by examining its kernel density estimate with a Gaussian kernel. This allows one to consider kernel density estimates, which are robust to spatial noise, subsampling, and approximate computation in comparison to raw point sets. This is achieved by examining the sublevel sets of the kernel distance, which isomorphically map to superlevel sets of the kernel density estimate. We prove new properties about the kernel distance, demonstrating stability results and allowing it to inherit reconstruction results from recent advances in distance-based topological reconstruction. Moreover, we provide an algorithm to estimate its topology using weighted Vietoris-Rips complexes.
Dynamic Approximate All-Pairs Shortest Paths: Breaking the O(mn) Barrier and Derandomization
Venue: arXiv cs.DS 2015
Year: 2015

We study dynamic $(1+ε)$-approximation algorithms for the all-pairs shortest paths problem in unweighted undirected $n$-node $m$-edge graphs under edge deletions. The fastest algorithm for this problem is a randomized algorithm with a total update time of $\tilde O(mn/ε)$ and constant query time by Roditty and Zwick [FOCS 2004]. The fastest deterministic algorithm is from a 1981 paper by Even and Shiloach [JACM 1981]; it has a total update time of $O(mn^2)$ and constant query time. We improve these results as follows: (1) We present an algorithm with a total update time of $\tilde O(n^{5/2}/ε)$ and constant query time that has an additive error of $2$ in addition to the $1+ε$ multiplicative error. This beats the previous $\tilde O(mn/ε)$ time when $m=Ω(n^{3/2})$. Note that the additive error is unavoidable since, even in the static case, an $O(n^{3-δ})$-time (a so-called truly subcubic) combinatorial algorithm with $1+ε$ multiplicative error cannot have an additive error less than $2-ε$, unless we make a major breakthrough for Boolean matrix multiplication [Dor et al. FOCS 1996] and many other long-standing problems [Vassilevska Williams and Williams FOCS 2010]. The algorithm can also be turned into a $(2+ε)$-approximation algorithm (without an additive error) with the same time guarantees, improving the recent $(3+ε)$-approximation algorithm with $\tilde O(n^{5/2+O(\sqrt{\log{(1/ε)}/\log n})})$ running time of Bernstein and Roditty [SODA 2011] in terms of both approximation and time guarantees. (2) We present a deterministic algorithm with a total update time of $\tilde O(mn/ε)$ and a query time of $O(\log\log n)$. The algorithm has a multiplicative error of $1+ε$ and gives the first improved deterministic algorithm since 1981. It also answers an open question raised by Bernstein [STOC 2013].
On Barriers and the Gap between Active and Passive Replication (Full Version)
Venue: arXiv cs.DC 2015
Year: 2015

Active replication is commonly built on top of the atomic broadcast primitive. Passive replication, which has been recently used in the popular ZooKeeper coordination system, can be naturally built on top of the primary-order atomic broadcast primitive. Passive replication differs from active replication in that it requires processes to cross a barrier before they become primaries and start broadcasting messages. In this paper, we propose a barrier function tau that explains and encapsulates the differences between existing primary-order atomic broadcast algorithms, namely semi-passive replication and Zookeeper atomic broadcast (Zab), as well as the differences between Paxos and Zab. We also show that implementing primary-order atomic broadcast on top of a generic consensus primitive and tau inherently results in higher time complexity than atomic broadcast, as witnessed by existing algorithms. We overcome this problem by presenting an alternative, primary-order atomic broadcast implementation that builds on top of a generic consensus primitive and uses consensus itself to form a barrier. This algorithm is modular and matches the time complexity of existing tau-based algorithms.
Exploiting Binary Floating-Point Representations for Constraint Propagation: The Complete Unabridged Version
Venue: arXiv cs.AI 2015
Year: 2015

Floating-point computations are quickly finding their way in the design of safety- and mission-critical systems, despite the fact that designing floating-point algorithms is significantly more difficult than designing integer algorithms. For this reason, verification and validation of floating-point computations is a hot research topic. An important verification technique, especially in some industrial sectors, is testing. However, generating test data for floating-point intensive programs proved to be a challenging problem. Existing approaches usually resort to random or search-based test data generation, but without symbolic reasoning it is almost impossible to generate test inputs that execute complex paths controlled by floating-point computations. Moreover, as constraint solvers over the reals or the rationals do not natively support the handling of rounding errors, the need arises for efficient constraint solvers over floating-point domains. In this paper, we present and fully justify improved algorithms for the propagation of arithmetic IEEE 754 binary floating-point constraints. The key point of these algorithms is a generalization of an idea by B. Marre and C. Michel that exploits a property of the representation of floating-point numbers.
Using ILP/SAT to determine pathwidth, visibility representations, and other grid-based graph drawings
Venue: arXiv cs.CG 2015
Year: 2015

We present a simple and versatile formulation of grid-based graph representation problems as an integer linear program (ILP) and a corresponding SAT instance. In a grid-based representation vertices and edges correspond to axis-parallel boxes on an underlying integer grid; boxes can be further constrained in their shapes and interactions by additional problem-specific constraints. We describe a general d-dimensional model for grid representation problems. This model can be used to solve a variety of NP-hard graph problems, including pathwidth, bandwidth, optimum st-orientation, area-minimal (bar-k) visibility representation, boxicity-k graphs and others. We implemented SAT-models for all of the above problems and evaluated them on the Rome graphs collection. The experiments show that our model successfully solves NP-hard problems within few minutes on small to medium-size Rome graphs.
The placement of the head that minimizes online memory: a complex systems approach
Venue: arXiv cs.CL 2015
Year: 2015

It is well known that the length of a syntactic dependency determines its online memory cost. Thus, the problem of the placement of a head and its dependents (complements or modifiers) that minimizes online memory is equivalent to the problem of the minimum linear arrangement of a star tree. However, how that length is translated into cognitive cost is not known. This study shows that the online memory cost is minimized when the head is placed at the center, regardless of the function that transforms length into cost, provided only that this function is strictly monotonically increasing. Online memory defines a quasi-convex adaptive landscape with a single central minimum if the number of elements is odd and two central minima if that number is even. We discuss various aspects of the dynamics of word order of subject (S), verb (V) and object (O) from a complex systems perspective and suggest that word orders tend to evolve by swapping adjacent constituents from an initial or early SOV configuration that is attracted towards a central word order by online memory minimization. We also suggest that the stability of SVO is due to at least two factors, the quasi-convex shape of the adaptive landscape in the online memory dimension and online memory adaptations that avoid regression to SOV. Although OVS is also optimal for placing the verb at the center, its low frequency is explained by its long distance to the seminal SOV in the permutation space.
Refinement in the Function-Behaviour-Structure Framework
Venue: arXiv cs.SE 2015
Year: 2015

We introduce refinement in the function-behaviour-structure framework for design, as described by John Gero, in order to deal with complexity. We do this by connecting the frameworks for the design of two models, one the refinement of the other. The result is a framework for the design of an object that supports levels of abstraction in the design. This framework can easily be extended for the design of an object on more than two levels of abstraction.
The Average Sensitivity of an Intersection of Half Spaces
Venue: arXiv cs.CC 2015
Year: 2015

We prove new bounds on the average sensitivity of the indicator function of an intersection of $k$ halfspaces. In particular, we prove the optimal bound of $O(\sqrt{n\log(k)})$. This generalizes a result of Nazarov, who proved the analogous result in the Gaussian case, and improves upon a result of Harsha, Klivans and Meka. Furthermore, our result has implications for the runtime required to learn intersections of halfspaces.
Analysis of Scientific Cloud Computing requirements
Venue: arXiv cs.DC 2015
Year: 2015

While the requirements of enterprise and web applications have driven the development of Cloud computing, some of its key features, such as customized environments and rapid elasticity, could also benefit scientific applications. However, neither virtualization techniques nor Cloud-like access to resources is common in scientific computing centers due to the negative perception of the impact that virtualization techniques introduce. In this paper we discuss the feasibility of the IaaS cloud model to satisfy some of the computational science requirements and the main drawbacks that need to be addressed by cloud resource providers so that the maximum benefit can be obtained from a given cloud infrastructure.
Intersection Information based on Common Randomness
Venue: arXiv cs.IT 2015
Year: 2015

The introduction of the partial information decomposition generated a flurry of proposals for defining an intersection information that quantifies how much of "the same information" two or more random variables specify about a target random variable. As of yet, none is wholly satisfactory. A palatable measure of intersection information would provide a principled way to quantify slippery concepts, such as synergy. Here, we introduce an intersection information measure based on the Gács-Körner common random variable that is the first to satisfy the coveted target monotonicity property. Our measure is imperfect, too, and we suggest directions for improvement.