OpenRaiser/Intern-Atlas huggingface dataset:
Variable-Length Lossy Compression Allowing Positive Overflow and Excess Distortion Probabilities
Venue: arXiv cs.IT 2018
Year: 2018

This paper investigates the problem of variable-length lossy source coding allowing a positive excess distortion probability and an overflow probability of codeword lengths. Novel one-shot achievability and converse bounds of the optimal rate are established by a new quantity based on the smooth max entropy (the smooth Rényi entropy of order zero). To derive the achievability bounds, we give an explicit code construction based on a distortion ball instead of using the random coding argument. The basic idea of the code construction is similar to the optimal code construction in the variable-length lossless source coding. Our achievability bounds are slightly different, depending on whether the encoder is stochastic or deterministic. One-shot results yield a general formula of the optimal rate for blocklength $n$. In addition, our general formula is applied to asymptotic analysis for a stationary memoryless source. As a result, we derive a single-letter characterization of the optimal rate by using the rate-distortion and rate-dispersion functions.
"The Capacity of the Relay Channel": Solution to Cover's Problem in the Gaussian Case
Venue: arXiv cs.IT 2018
Year: 2018

Consider a memoryless relay channel, where the relay is connected to the destination with an isolated bit pipe of capacity $C_0$. Let $C(C_0)$ denote the capacity of this channel as a function of $C_0$. What is the critical value of $C_0$ such that $C(C_0)$ first equals $C(\infty)$? This is a long-standing open problem posed by Cover and named "The Capacity of the Relay Channel," in $Open \ Problems \ in \ Communication \ and \ Computation$, Springer-Verlag, 1987. In this paper, we answer this question in the Gaussian case and show that $C(C_0)$ can not equal to $C(\infty)$ unless $C_0=\infty$, regardless of the SNR of the Gaussian channels. This result follows as a corollary to a new upper bound we develop on the capacity of this channel. Instead of "single-letterizing" expressions involving information measures in a high-dimensional space as is typically done in converse results in information theory, our proof directly quantifies the tension between the pertinent $n$-letter forms. This is done by translating the information tension problem to a problem in high-dimensional geometry. As an intermediate result, we develop an extension of the classical isoperimetric inequality on a high-dimensional sphere, which can be of interest in its own right.
On the Upward/Downward Closures of Petri Nets
Venue: arXiv cs.FL 2018
Year: 2018

We study the size and the complexity of computing finite state automata (FSA) representing and approximating the downward and the upward closure of Petri net languages with coverability as the acceptance condition. We show how to construct an FSA recognizing the upward closure of a Petri net language in doubly-exponential time, and therefore the size is at most doubly exponential. For downward closures, we prove that the size of the minimal automata can be non-primitive recursive. In the case of BPP nets, a well-known subclass of Petri nets, we show that an FSA accepting the downward/upward closure can be constructed in exponential time. Furthermore, we consider the problem of checking whether a simple regular language is included in the downward/upward closure of a Petri net/BPP net language. We show that this problem is EXPSPACE-complete (resp. NP-complete) in the case of Petri nets (resp. BPP nets). Finally, we show that it is decidable whether a Petri net language is upward/downward closed. To this end, we prove that one can decide whether a given regular language is a subset of a Petri net coverability language.
Logics for Word Transductions with Synthesis
Venue: arXiv cs.FL 2018
Year: 2018

We introduce a logic, called LT, to express properties of transductions, i.e. binary relations from input to output (finite) words. In LT, the input/output dependencies are modelled via an origin function which associates to any position of the output word, the input position from which it originates. LT is well-suited to express relations (which are not necessarily functional), and can express all regular functional transductions, i.e. transductions definable for instance by deterministic two-way transducers. Despite its high expressive power, LT has decidable satisfiability and equivalence problems, with tight non-elementary and elementary complexities, depending on specific representation of LT-formulas. Our main contribution is a synthesis result: from any transduction R defined in LT , it is possible to synthesise a regular functional transduction f such that for all input words u in the domain of R, f is defined and (u,f(u)) belongs to R. As a consequence, we obtain that any functional transduction is regular iff it is LT-definable. We also investigate the algorithmic and expressiveness properties of several extensions of LT, and explicit a correspondence between transductions and data words. As a side-result, we obtain a new decidable logic for data words.
An Information-Theoretic Analysis of Deduplication
Venue: arXiv cs.IT 2018
Year: 2018

Deduplication finds and removes long-range data duplicates. It is commonly used in cloud and enterprise server settings and has been successfully applied to primary, backup, and archival storage. Despite its practical importance as a source-coding technique, its analysis from the point of view of information theory is missing. This paper provides such an information-theoretic analysis of data deduplication. It introduces a new source model adapted to the deduplication setting. It formalizes the two standard fixed-length and variable-length deduplication schemes, and it introduces a novel multi-chunk deduplication scheme. It then provides an analysis of these three deduplication variants, emphasizing the importance of boundary synchronization between source blocks and deduplication chunks. In particular, under fairly mild assumptions, the proposed multi-chunk deduplication scheme is shown to be order optimal.
Online Learning with Regularized Kernel for One-class Classification
Venue: arXiv cs.LG 2018
Year: 2018

This paper presents an online learning with regularized kernel based one-class extreme learning machine (ELM) classifier and is referred as online RK-OC-ELM. The baseline kernel hyperplane model considers whole data in a single chunk with regularized ELM approach for offline learning in case of one-class classification (OCC). Further, the basic hyper plane model is adapted in an online fashion from stream of training samples in this paper. Two frameworks viz., boundary and reconstruction are presented to detect the target class in online RKOC-ELM. Boundary framework based one-class classifier consists of single node output architecture and classifier endeavors to approximate all data to any real number. However, one-class classifier based on reconstruction framework is an autoencoder architecture, where output nodes are identical to input nodes and classifier endeavor to reconstruct input layer at the output layer. Both these frameworks employ regularized kernel ELM based online learning and consistency based model selection has been employed to select learning algorithm parameters. The performance of online RK-OC-ELM has been evaluated on standard benchmark datasets as well as on artificial datasets and the results are compared with existing state-of-the art one-class classifiers. The results indicate that the online learning one-class classifier is slightly better or same as batch learning based approaches. As, base classifier used for the proposed classifiers are based on the ELM, hence, proposed classifiers would also inherit the benefit of the base classifier i.e. it will perform faster computation compared to traditional autoencoder based one-class classifier.
The fast parallel algorithm for CNF SAT without algebra
Venue: arXiv cs.DS 2018
Year: 2018

A novel parallel algorithm for solving the classical Decision Boolean Satisfiability problem with clauses in conjunctive normal form is depicted. My approach for solving SAT is without using algebra or other computational search strategies such as branch and bound, back-forward, tree representation, etc. The method is based on the special class of SAT problems, Simple SAT (SSAT). The algorithm's design includes parallel execution, object oriented, and short termination as my previous versions but it keep track of the tested unsatisfactory binary values to improve the efficiency and to favor short termination. The resulting algorithm is linear with respect to the number of clauses plus a process data on the partial solutions of the subproblems SSAT of an arbitrary SAT and it is bounded by $2^{n}$ iterations where $n$ is the number of logical variables. The novelty for the solution of arbitrary SAT problems is a linear algorithm, such its complexity is less or equal than the algorithms of the state of the art for solving SAT.
Reactors: A Case for Predictable, Virtualized Actor Database Systems
Venue: arXiv cs.DB 2018
Year: 2018

The requirements for OLTP database systems are becoming ever more demanding. Domains such as finance and computer games increasingly mandate that developers be able to encode complex application logic and control transaction latencies in in-memory databases. At the same time, infrastructure engineers in these domains need to experiment with and deploy OLTP database architectures that ensure application scalability and maximize resource utilization in modern machines. In this paper, we propose a relational actor programming model for in-memory databases as a novel, holistic approach towards fulfilling these challenging requirements. Conceptually, relational actors, or reactors for short, are application-defined, isolated logical actors that encapsulate relations and process function calls asynchronously. Reactors ease reasoning about correctness by guaranteeing serializability of application-level function calls. In contrast to classic transactional models, however, reactors allow developers to take advantage of intra-transaction parallelism and state encapsulation in their applications to reduce latency and improve locality. Moreover, reactors enable a new degree of flexibility in database deployment. We present ReactDB, a system design exposing reactors that allows for flexible virtualization of database architecture between the extremes of shared-nothing and shared-everything without changes to application code. Our experiments illustrate latency control, low overhead, and asynchronicity trade-offs with ReactDB in OLTP benchmarks.
Efficient computation of higher order cumulant tensors
Venue: arXiv cs.NA 2018
Year: 2018

In this paper, we introduce a novel algorithm for calculating arbitrary order cumulants of multidimensional data. Since the $d^\text{th}$ order cumulant can be presented in the form of an $d$-dimensional tensor, the algorithm is presented using tensor operations. The algorithm provided in the paper takes advantage of super-symmetry of cumulant and moment tensors. We show that the proposed algorithm considerably reduces the computational complexity and the computational memory requirement of cumulant calculation as compared with existing algorithms. For the sizes of interest, the reduction is of the order of $d!$ compared to the naive algorithm.
Perfect phylogenies via branchings in acyclic digraphs and a generalization of Dilworth's theorem
Venue: arXiv cs.DM 2018
Year: 2018

Motivated by applications in cancer genomics and following the work of Hajirasouliha and Raphael (WABI 2014), Hujdurović et al. (IEEE TCBB, to appear) introduced the minimum conflict-free row split (MCRS) problem: split each row of a given binary matrix into a bitwise OR of a set of rows so that the resulting matrix corresponds to a perfect phylogeny and has the minimum possible number of rows among all matrices with this property. Hajirasouliha and Raphael also proposed the study of a similar problem, in which the task is to minimize the number of distinct rows of the resulting matrix. Hujdurović et al. proved that both problems are NP-hard, gave a related characterization of transitively orientable graphs, and proposed a polynomial-time heuristic algorithm for the MCRS problem based on coloring cocomparability graphs. We give new, more transparent formulations of the two problems, showing that the problems are equivalent to two optimization problems on branchings in a derived directed acyclic graph. Building on these formulations, we obtain new results on the two problems, including: (i) a strengthening of the heuristic by Hujdurović et al. via a new min-max result in digraphs generalizing Dilworth's theorem, which may be of independent interest, (ii) APX-hardness results for both problems, (iii) approximation algorithms, and (iv) exponential-time algorithms solving the two problems to optimality faster than the naïve brute-force approach. Our work relates to several well studied notions in combinatorial optimization: chain partitions in partially ordered sets, laminar hypergraphs, and (classical and weighted) colorings of graphs.
Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we present the Lipschitz regularization theory and algorithms for a novel Loss-Sensitive Generative Adversarial Network (LS-GAN). Specifically, it trains a loss function to distinguish between real and fake samples by designated margins, while learning a generator alternately to produce realistic samples by minimizing their losses. The LS-GAN further regularizes its loss function with a Lipschitz regularity condition on the density of real data, yielding a regularized model that can better generalize to produce new data from a reasonable number of training examples than the classic GAN. We will further present a Generalized LS-GAN (GLS-GAN) and show it contains a large family of regularized GAN models, including both LS-GAN and Wasserstein GAN, as its special cases. Compared with the other GAN models, we will conduct experiments to show both LS-GAN and GLS-GAN exhibit competitive ability in generating new images in terms of the Minimum Reconstruction Error (MRE) assessed on a separate test set. We further extend the LS-GAN to a conditional form for supervised and semi-supervised learning problems, and demonstrate its outstanding performance on image classification tasks.
Some impossibilities of ranking in generalized tournaments
Venue: arXiv cs.GT 2018
Year: 2018

In a generalized tournament, players may have an arbitrary number of matches against each other and the outcome of the games is measured on a cardinal scale with a lower and upper bound. An axiomatic approach is applied to the problem of ranking the competitors. Self-consistency requires assigning the same rank for players with equivalent results, while a player showing an obviously better performance than another should be ranked strictly higher. According to order preservation, if two players have the same pairwise ranking in two tournaments where the same players have played the same number of matches, then their pairwise ranking is not allowed to change in the aggregated tournament. We reveal that these two properties cannot be satisfied simultaneously on this universal domain.
A Projected Gradient Descent Method for CRF Inference allowing End-To-End Training of Arbitrary Pairwise Potentials
Venue: arXiv cs.CV 2018
Year: 2018

Are we using the right potential functions in the Conditional Random Field models that are popular in the Vision community? Semantic segmentation and other pixel-level labelling tasks have made significant progress recently due to the deep learning paradigm. However, most state-of-the-art structured prediction methods also include a random field model with a hand-crafted Gaussian potential to model spatial priors, label consistencies and feature-based image conditioning. In this paper, we challenge this view by developing a new inference and learning framework which can learn pairwise CRF potentials restricted only by their dependence on the image pixel values and the size of the support. Both standard spatial and high-dimensional bilateral kernels are considered. Our framework is based on the observation that CRF inference can be achieved via projected gradient descent and consequently, can easily be integrated in deep neural networks to allow for end-to-end training. It is empirically demonstrated that such learned potentials can improve segmentation accuracy and that certain label class interactions are indeed better modelled by a non-Gaussian potential. In addition, we compare our inference method to the commonly used mean-field algorithm. Our framework is evaluated on several public benchmarks for semantic segmentation with improved performance compared to previous state-of-the-art CNN+CRF models.
Historic Emergence of Diversity in Painting: Heterogeneity in Chromatic Distance in Images and Characterization of Massive Painting Data Set
Venue: arXiv cs.CV 2018
Year: 2018

Painting is an art form that has long functioned as a major channel for the creative expression and communication of humans, its evolution taking place under an interplay with the science, technology, and social environments of the times. Therefore, understanding the process based on comprehensive data could shed light on how humans acted and manifested creatively under changing conditions. Yet, there exist few systematic frameworks that characterize the process for painting, which would require robust statistical methods for defining painting characteristics and identifying human's creative developments, and data of high quality and sufficient quantity. Here we propose that the color contrast of a painting image signifying the heterogeneity in inter-pixel chromatic distance can be a useful representation of its style, integrating both the color and geometry. From the color contrasts of paintings from a large-scale, comprehensive archive of 179,853 high-quality images spanning several centuries we characterize the temporal evolutionary patterns of paintings, and present a deep study of an extraordinary expansion in creative diversity and individuality that came to define the modern era.
Fast Exact k-Means, k-Medians and Bregman Divergence Clustering in 1D
Venue: arXiv cs.DS 2018
Year: 2018

The $k$-Means clustering problem on $n$ points is NP-Hard for any dimension $d\ge 2$, however, for the 1D case there exists exact polynomial time algorithms. Previous literature reported an $O(kn^2)$ time dynamic programming algorithm that uses $O(kn)$ space. It turns out that the problem has been considered under a different name more than twenty years ago. We present all the existing work that had been overlooked and compare the various solutions theoretically. Moreover, we show how to reduce the space usage for some of them, as well as generalize them to data structures that can quickly report an optimal $k$-Means clustering for any $k$. Finally we also generalize all the algorithms to work for the absolute distance and to work for any Bregman Divergence. We complement our theoretical contributions by experiments that compare the practical performance of the various algorithms.
Joint Uplink-Downlink Cell Associations for Interference Networks with Local Connectivity
Venue: arXiv cs.IT 2018
Year: 2018

We study information theoretic models of interference networks that consist of K Base Station (BS) - Mobile Terminal (MT) pairs. Each BS is connected to the MT carrying the same index as well as L following MTs, where the connectivity parameter L >= 1. We fix the value of L and study large networks as K goes to infinity. We assume that each MT can be associated with Nc BSs, and these associations are determined by a cloud-based controller that has a global view of the network. An MT has to be associated with a BS, in order for the BS to transmit its message in the downlink, or decode its message in the uplink. In previous work, the cell associations that maximize the average uplink-downlink per user degrees of freedom (puDoF) were identified for the case when L=1. Further, when only the downlink is considered, the problem was settled for all values of L when we are restricted to use only zero-forcing interference cancellation schemes. In this work, we first propose puDoF inner bounds for arbitrary values of L when only the uplink is considered, and characterize the uplink puDoF value when only zero-forcing schemes are allowed. We then introduce new achievable average uplink-downlink puDoF values. We show that the new scheme is optimal for the range when Nc <= L/2 when we restrict our attention to zero forcing schemes. Additionally we conjecture that the having unity puDoF during uplink is optimal when Nc >= L.
Efficient Kinematic Planning for Mobile Manipulators with Non-holonomic Constraints Using Optimal Control
Venue: arXiv cs.RO 2018
Year: 2018

This work addresses the problem of kinematic trajectory planning for mobile manipulators with non-holonomic constraints, and holonomic operational-space tracking constraints. We obtain whole-body trajectories and time-varying kinematic feedback controllers by solving a Constrained Sequential Linear Quadratic Optimal Control problem. The employed algorithm features high efficiency through a continuous-time formulation that benefits from adaptive step-size integrators and through linear complexity in the number of integration steps. In a first application example, we solve kinematic trajectory planning problems for a 26 DoF wheeled robot. In a second example, we apply Constrained SLQ to a real-world mobile manipulator in a receding-horizon optimal control fashion, where we obtain optimal controllers and plans at rates up to 100 Hz.
Emotion Recognition From Speech With Recurrent Neural Networks
Venue: arXiv cs.CL 2018
Year: 2018

In this paper the task of emotion recognition from speech is considered. Proposed approach uses deep recurrent neural network trained on a sequence of acoustic features calculated over small speech intervals. At the same time special probabilistic-nature CTC loss function allows to consider long utterances containing both emotional and neutral parts. The effectiveness of such an approach is shown in two ways. Firstly, the comparison with recent advances in this field is carried out. Secondly, human performance on the same task is measured. Both criteria show the high quality of the proposed method.
On the Capacity of the Slotted Strongly Asynchronous Channel with a Bursty User
Venue: arXiv cs.IT 2018
Year: 2018

In this paper, the trade-off between the number of transmissions (or burstiness) $K_n=e^{nν}$ of a user, the asynchronism level $A_n=e^{nα}$ in a slotted strongly asynchronous channel, and the ability to distinguish $M_n=e^{nR}$ messages per transmission with vanishingly error probability is investigated in the asymptotic regime as blocklength $n$ goes to infinity. The receiver must locate and decode, with vanishing error probability in $n$, all of the transmitted messages. Achievability and converse bounds on the trade-off among $(R,α,ν)$ is derived. For cases where $ν=0$ and $ R=0$, achievability and converse bounds coincide. A second model for a bursty user with random access in which the user may access and transmit a message in each block with probability $e^{-nβ}$ in then considered. Achievability and converse bounds on the trade-off between $(R, α, β)$ is also characterized. For cases where $β=α$ and $R=0$, the achievability and converse bounds match.