OpenRaiser/Intern-Atlas huggingface dataset:
The Commutativity Problem of the MapReduce Framework: A Transducer-based Approach
Venue: arXiv cs.FL 2016
Year: 2016

MapReduce is a popular programming model for data parallel computation. In MapReduce, the reducer produces an output from a list of inputs. Due to the scheduling policy of the platform, the inputs may arrive at the reducers in different order. The commutativity problem of reducers asks if the output of a reducer is independent of the order of its inputs. Although the problem is undecidable in general, the MapReduce programs in practice are usually used for data analytics and thus require very simple control flow. By exploiting the simplicity, we propose a programming language for reducers where the commutativity problem is decidable. The main idea of the reducer language is to separate the control and data flow of programs and disallow arithmetic operations in the control flow. The decision procedure for the commutativity problem is obtained through a reduction to the equivalence problem of streaming numerical transducers (SNTs), a novel automata model over infinite alphabets introduced in this paper. The design of SNTs is inspired by streaming transducers (Alur and Cerny, POPL 2011). Nevertheless, the two models are intrinsically different since the outputs of SNTs are integers while those of streaming transducers are data words. The decidability of the equivalence of SNTs is achieved with an involved combinatorial analysis of the evolvement of the values of the integer variables during the runs of SNTs.
A combinatorial problem about binary necklaces and attractors of Boolean automata networks
Venue: arXiv cs.DM 2016
Year: 2016

It is known that there are no more Lyndon words of length n than there are periodic necklaces of same length. This paper considers a similar problem where, additionally, the necklaces must be without some forbidden factors. This problem relates to a different context, concerned with the behaviours of particular discrete dynamical systems, namely, Boolean automata networks. A formal argument supporting the following idea is provided: addition of cycle intersections in network structures causes exponential reduction of the networks' number of attractors.
On entropic convergence of algorithms in terms of domain partitions
Venue: arXiv cs.CC 2016
Year: 2016

The paper describes an approach to measuring convergence of an algorithm to its result in terms of an entropy-like function of partitions of its inputs of a given length. The goal is to look at the algorithmic data processing from the viewpoint of information transformation, with a hope to better understand the work of algorithm, and maybe its complexity. The entropy is a measure of uncertainty, it does not correspond to our intuitive understanding of information. However, it is what we have in this area. In order to realize this approach we introduce a measure on the inputs of a given length based on the Principle of Maximal Uncertainty: all results should be equiprobable to the algorithm at the beginning. An algorithm is viewed as a set of events, each event is an application of a command. The commands are very basic. To measure the convergence we introduce a measure that is called entropic weight of events of the algorithm. The approach is illustrated by two examples.
ODE - Augmented Training Improves Anomaly Detection in Sensor Data from Machines
Venue: arXiv cs.AI 2016
Year: 2016

Machines of all kinds from vehicles to industrial equipment are increasingly instrumented with hundreds of sensors. Using such data to detect anomalous behaviour is critical for safety and efficient maintenance. However, anomalies occur rarely and with great variety in such systems, so there is often insufficient anomalous data to build reliable detectors. A standard approach to mitigate this problem is to use one class methods relying only on data from normal behaviour. Unfortunately, even these approaches are more likely to fail in the scenario of a dynamical system with manual control input(s). Normal behaviour in response to novel control input(s) might look very different to the learned detector which may be incorrectly detected as anomalous. In this paper, we address this issue by modelling time-series via Ordinary Differential Equations (ODE) and utilising such an ODE model to simulate the behaviour of dynamical systems under varying control inputs. The available data is then augmented with data generated from the ODE, and the anomaly detector is retrained on this augmented dataset. Experiments demonstrate that ODE-augmented training data allows better coverage of possible control input(s) and results in learning more accurate distinctions between normal and anomalous behaviour in time-series.
Effect of Cultural Factors on Students of Pakistan
Venue: arXiv cs.CY 2016
Year: 2016

Pakistan as culturally diverse country possesses wide range of cultural factors. These cultural factors affect the living style, traditions, values and norms as well as the education. As e-learning is global mode of education so learners from different backgrounds enrolled in learning management system. Diversity of culture and learning styles should keep under considerations while designing e-learning environment. In this research work e learning cultural factors highlighted by Bentley ET. AL are incorporated to find the learner level. After assignment of level, learner respective Learning Management System is allocated to learner. The focus is to concentrate on cultural factor in e-learning system. Furthermore; a prototype of the proposed system is implemented for the validation of proposed architecture.
Agent-Based Model for River-Side Land-living: Portrait of Bandung Indonesian Cikapundung Park Case Study
Venue: arXiv cs.CY 2016
Year: 2016

A city park has been built from the organic urban settlement in the Cikapundung River, Bandung, Indonesia. While the aim for the development is the revitalization of the river for being unhealthy from the waste coming from the settlement. A study on how Indonesian people, in general, treating water source, like river, lake, and ocean is revisited. Throwing waste into the river has actually become paradox with the collective mental understanding about water among Indonesians. Two scenarios of agent-based simulation is presented, to see the dynamics of organic settlement and life of the city park after being opened for public. The simulation is delivered upon the imagery of landscape taken from the satellite and drone. While experience for presented problems gives insights, the computational social laboratory also awaits for further theoretical explorations and endeavors to sharpen good policymaking.
A Layered Caching Architecture for the Interference Channel
Venue: arXiv cs.IT 2016
Year: 2016

Recent work has studied the benefits of caching in the interference channel, particularly by placing caches at the transmitters. In this paper, we study the two-user Gaussian interference channel in which caches are placed at both the transmitters and the receivers. We propose a separation strategy that divides the physical and network layers. While a natural separation approach might be to abstract the physical layer into several independent bit pipes at the network layer, we argue that this is inefficient. Instead, the separation approach we propose exposes interacting bit pipes at the network layer, so that the receivers observe related (yet not identical) quantities. We find the optimal strategy within this layered architecture, and we compute the degrees-of-freedom it achieves. Finally, we show that separation is optimal in regimes where the receiver caches are large.
DCTNet and PCANet for acoustic signal feature extraction
Venue: arXiv cs.SD 2016
Year: 2016

We introduce the use of DCTNet, an efficient approximation and alternative to PCANet, for acoustic signal classification. In PCANet, the eigenfunctions of the local sample covariance matrix (PCA) are used as filterbanks for convolution and feature extraction. When the eigenfunctions are well approximated by the Discrete Cosine Transform (DCT) functions, each layer of of PCANet and DCTNet is essentially a time-frequency representation. We relate DCTNet to spectral feature representation methods, such as the the short time Fourier transform (STFT), spectrogram and linear frequency spectral coefficients (LFSC). Experimental results on whale vocalization data show that DCTNet improves classification rate, demonstrating DCTNet's applicability to signal processing problems such as underwater acoustics.
Adaptive Mesh Booleans
Venue: arXiv cs.GR 2016
Year: 2016

We present a new method for performing Boolean operations on volumes represented as triangle meshes. In contrast to existing methods which treat meshes as 3D polyhedra and try to partition the faces at their exact intersection curves, we treat meshes as adaptive surfaces which can be arbitrarily refined. Rather than depending on computing precise face intersections, our approach refines the input meshes in the intersection regions, then discards intersecting triangles and fills the resulting holes with high-quality triangles. The original intersection curves are approximated to a user-definable precision, and our method can identify and preserve creases and sharp features. Advantages of our approach include the ability to trade speed for accuracy, support for open meshes, and the ability to incorporate tolerances to handle cases where large numbers of faces are slightly inter-penetrating or near-coincident.
Two-sided Quantum Amplitude Amplification and Exact-Error Algorithms
Venue: arXiv cs.CC 2016
Year: 2016

Amplitude amplification is a central tool used in Grover's quantum search algorithm and has been used in various forms in numerous quantum algorithms since then. It has been shown to completely eliminate one-sided error of quantum search algorithms where input is accessed in the form of black-box queries. We generalize amplitude amplification for two-sided error quantum algorithm for decision problems in the familiar form where input is accessed in the form of initial states of quantum circuits and where arbitrary projective measurements may be used to ascertain success or failure. This generalization allows us to derive interesting applications of amplitude amplification for distinguishing between two given distributions based on their samples, detection of faults in quantum circuits and eliminating error of one and two-sided quantum algorithms with exact errors.
Arbitrarily Varying Networks: Capacity-achieving Computationally Efficient Codes
Venue: arXiv cs.IT 2016
Year: 2016

We consider the problem of communication over a network containing a hidden and malicious adversary that can control a subset of network resources, and aims to disrupt communications. We focus on omniscient node-based adversaries, i.e., the adversaries can control a subset of nodes, and know the message, network code and packets on all links. Characterizing information-theoretically optimal communication rates as a function of network parameters and bounds on the adversarially controlled network is in general open, even for unicast (single source, single destination) problems. In this work we characterize the information-theoretically optimal randomized capacity of such problems, i.e., under the assumption that the source node shares (an asymptotically negligible amount of) independent common randomness with each network node a priori (for instance, as part of network design). We propose a novel computationally-efficient communication scheme whose rate matches a natural information-theoretically "erasure outer bound" on the optimal rate. Our schemes require no prior knowledge of network topology, and can be implemented in a distributed manner as an overlay on top of classical distributed linear network coding.
The KB paradigm and its application to interactive configuration
Venue: arXiv cs.AI 2016
Year: 2016

The knowledge base paradigm aims to express domain knowledge in a rich formal language, and to use this domain knowledge as a knowledge base to solve various problems and tasks that arise in the domain by applying multiple forms of inference. As such, the paradigm applies a strict separation of concerns between information and problem solving. In this paper, we analyze the principles and feasibility of the knowledge base paradigm in the context of an important class of applications: interactive configuration problems. In interactive configuration problems, a configuration of interrelated objects under constraints is searched, where the system assists the user in reaching an intended configuration. It is widely recognized in industry that good software solutions for these problems are very difficult to develop. We investigate such problems from the perspective of the KB paradigm. We show that multiple functionalities in this domain can be achieved by applying different forms of logical inferences on a formal specification of the configuration domain. We report on a proof of concept of this approach in a real-life application with a banking company. To appear in Theory and Practice of Logic Programming (TPLP).
On the Performance of Mobile Visible Light Communications
Venue: arXiv cs.IT 2016
Year: 2016

We experimentally characterize the performance of mobile VLC and propose using OCT precoding to combat mobility-induced performance degradation. Results show that for approximate 300-Mb/s mobile VLC transmission, OCT precoding outperforms adaptive-loaded DMT and offers significant packet loss rate reduction.
Partial DNA Assembly: A Rate-Distortion Perspective
Venue: arXiv cs.IT 2016
Year: 2016

Earlier formulations of the DNA assembly problem were all in the context of perfect assembly; i.e., given a set of reads from a long genome sequence, is it possible to perfectly reconstruct the original sequence? In practice, however, it is very often the case that the read data is not sufficiently rich to permit unambiguous reconstruction of the original sequence. While a natural generalization of the perfect assembly formulation to these cases would be to consider a rate-distortion framework, partial assemblies are usually represented in terms of an assembly graph, making the definition of a distortion measure challenging. In this work, we introduce a distortion function for assembly graphs that can be understood as the logarithm of the number of Eulerian cycles in the assembly graph, each of which correspond to a candidate assembly that could have generated the observed reads. We also introduce an algorithm for the construction of an assembly graph and analyze its performance on real genomes.
Coded Caching for a Large Number Of Users
Venue: arXiv cs.IT 2016
Year: 2016

Information theoretic analysis of a coded caching system is considered, in which a server with a database of N equal-size files, each F bits long, serves K users. Each user is assumed to have a local cache that can store M files, i.e., capacity of MF bits. Proactive caching to user terminals is considered, in which the caches are filled by the server in advance during the placement phase, without knowing the user requests. Each user requests a single file, and all the requests are satisfied simultaneously through a shared error-free link during the delivery phase. First, centralized coded caching is studied assuming both the number and the identity of the active users in the delivery phase are known by the server during the placement phase. A novel group-based centralized coded caching (GBC) scheme is proposed for a cache capacity of M = N/K. It is shown that this scheme achieves a smaller delivery rate than all the known schemes in the literature. The improvement is then extended to a wider range of cache capacities through memory-sharing between the proposed scheme and other known schemes in the literature. Next, the proposed centralized coded caching idea is exploited in the decentralized setting, in which the identities of the users that participate in the delivery phase are assumed to be unknown during the placement phase. It is shown that the proposed decentralized caching scheme also achieves a delivery rate smaller than the state-of-the-art. Numerical simulations are also presented to corroborate our theoretical results.
Mixing Dirichlet Topic Models and Word Embeddings to Make lda2vec
Venue: arXiv cs.CL 2016
Year: 2016

Distributed dense word vectors have been shown to be effective at capturing token-level semantic and syntactic regularities in language, while topic models can form interpretable representations over documents. In this work, we describe lda2vec, a model that learns dense word vectors jointly with Dirichlet-distributed latent document-level mixtures of topic vectors. In contrast to continuous dense document representations, this formulation produces sparse, interpretable document mixtures through a non-negative simplex constraint. Our method is simple to incorporate into existing automatic differentiation frameworks and allows for unsupervised document representations geared for use by scientists while simultaneously learning word vectors and the linear relationships between them.
A Graph-based Model for GPU Caching Problems
Venue: arXiv cs.DC 2016
Year: 2016

Modeling data sharing in GPU programs is a challenging task because of the massive parallelism and complex data sharing patterns provided by GPU architectures. Better GPU caching efficiency can be achieved through careful task scheduling among different threads. Traditionally, in the field of parallel computing, graph partition models are used to model data communication and guide task scheduling. However, we discover that the previous methods are either inaccurate or expensive when applied to GPU programs. In this paper, we propose a novel task partition model that is accurate and gives rise to the development of fast and high quality task/data reorganization algorithms. We demonstrate the effectiveness of the proposed model by rigorous theoretical analysis of the algorithm bounds and extensive experimental analysis. The experimental results show that it achieves significant performance improvement across a representative set of GPU applications.
Some Simulation Results for Emphatic Temporal-Difference Learning Algorithms
Venue: arXiv cs.LG 2016
Year: 2016

This is a companion note to our recent study of the weak convergence properties of constrained emphatic temporal-difference learning (ETD) algorithms from a theoretic perspective. It supplements the latter analysis with simulation results and illustrates the behavior of some of the ETD algorithms using three example problems.
On Improving Informativity and Grammaticality for Multi-Sentence Compression
Venue: arXiv cs.CL 2016
Year: 2016

Multi Sentence Compression (MSC) is of great value to many real world applications, such as guided microblog summarization, opinion summarization and newswire summarization. Recently, word graph-based approaches have been proposed and become popular in MSC. Their key assumption is that redundancy among a set of related sentences provides a reliable way to generate informative and grammatical sentences. In this paper, we propose an effective approach to enhance the word graph-based MSC and tackle the issue that most of the state-of-the-art MSC approaches are confronted with: i.e., improving both informativity and grammaticality at the same time. Our approach consists of three main components: (1) a merging method based on Multiword Expressions (MWE); (2) a mapping strategy based on synonymy between words; (3) a re-ranking step to identify the best compression candidates generated using a POS-based language model (POS-LM). We demonstrate the effectiveness of this novel approach using a dataset made of clusters of English newswire sentences. The observed improvements on informativity and grammaticality of the generated compressions show that our approach is superior to state-of-the-art MSC methods.
The Complexity of Simulation and Matrix Multiplication
Venue: arXiv cs.CC 2016
Year: 2016

Computing the simulation preorder of a given Kripke structure (i.e., a directed graph with $n$ labeled vertices) has crucial applications in model checking of temporal logic. It amounts to solving a specific two-players reachability game, called simulation game. We offer the first conditional lower bounds for this problem, and we relate its complexity (for computation, verification, and certification) to some variants of $n\times n$ matrix multiplication. We show that any $O(n^α)$-time algorithm for simulation games, even restricting to acyclic games/structures, can be used to compute $n\times n$ boolean matrix multiplication (BMM) in $O(n^α)$ time. This is the first evidence that improving the existing $O(n^{3})$-time solutions may be difficult, without resorting to fast matrix multiplication. In the acyclic case, we match this lower bound presenting the first subcubic algorithm, based on fast BMM, and running in $n^{ω+o(1)}$ time (where $ω<2.376$ is the exponent of matrix multiplication). For both acyclic and cyclic structures, we point out the existence of natural and canonical $O(n^{2})$-size certificates, that can be verified in truly subcubic time. In the acyclic case, $O(n^{2})$ time is sufficient, employing standard matrix product verification. In the cyclic case, a $\max$-semi-boolean matrix multiplication (MSBMM) is used, i.e., a matrix multiplication on the semi-ring $(\max,\times)$ where one matrix contains only $0$'s and $1$'s. This MSBMM is computable (hence verifiable) in truly subcubic $n^{(3+ω)/2+o(1)}$ time by reduction to $(\max,\min)$-multiplication. Finally, we show a reduction from MSBMM to cyclic simulation games which implies a separation between the cyclic and the acyclic cases, unless MSBMM can be verified in $n^{ω+o(1)}$ time.
Distributed stochastic optimization for deep learning (thesis)
Venue: arXiv cs.LG 2016
Year: 2016

We study the problem of how to distribute the training of large-scale deep learning models in the parallel computing environment. We propose a new distributed stochastic optimization method called Elastic Averaging SGD (EASGD). We analyze the convergence rate of the EASGD method in the synchronous scenario and compare its stability condition with the existing ADMM method in the round-robin scheme. An asynchronous and momentum variant of the EASGD method is applied to train deep convolutional neural networks for image classification on the CIFAR and ImageNet datasets. Our approach accelerates the training and furthermore achieves better test accuracy. It also requires a much smaller amount of communication than other common baseline approaches such as the DOWNPOUR method. We then investigate the limit in speedup of the initial and the asymptotic phase of the mini-batch SGD, the momentum SGD, and the EASGD methods. We find that the spread of the input data distribution has a big impact on their initial convergence rate and stability region. We also find a surprising connection between the momentum SGD and the EASGD method with a negative moving average rate. A non-convex case is also studied to understand when EASGD can get trapped by a saddle point. Finally, we scale up the EASGD method by using a tree structured network topology. We show empirically its advantage and challenge. We also establish a connection between the EASGD and the DOWNPOUR method with the classical Jacobi and the Gauss-Seidel method, thus unifying a class of distributed stochastic optimization methods.