OpenRaiser/Intern-Atlas huggingface dataset:
Robust Cooperative Manipulation without Force/Torque Measurements: Control Design and Experiments
Venue: arXiv cs.RO 2019
Year: 2019

This paper presents two novel control methodologies for the cooperative manipulation of an object by N robotic agents. Firstly, we design an adaptive control protocol which employs quaternion feedback for the object orientation to avoid potential representation singularities. Secondly, we propose a control protocol that guarantees predefined transient and steady-state performance for the object trajectory. Both methodologies are decentralized, since the agents calculate their own signals without communicating with each other, as well as robust to external disturbances and model uncertainties. Moreover, we consider that the grasping points are rigid, and avoid the need for force/torque measurements. Load distribution is also included via a grasp matrix pseudo-inverse to account for potential differences in the agents' power capabilities. Finally, simulation and experimental results with two robotic arms verify the theoretical findings.
Multiple Instance Hybrid Estimator for Hyperspectral Target Characterization and Sub-pixel Target Detection
Venue: arXiv cs.CV 2019
Year: 2019

The Multiple Instance Hybrid Estimator for discriminative target characterization from imprecisely labeled hyperspectral data is presented. In many hyperspectral target detection problems, acquiring accurately labeled training data is difficult. Furthermore, each pixel containing target is likely to be a mixture of both target and non-target signatures (i.e., sub-pixel targets), making extracting a pure prototype signature for the target class from the data extremely difficult. The proposed approach addresses these problems by introducing a data mixing model and optimizing the response of the hybrid sub-pixel detector within a multiple instance learning framework. The proposed approach iterates between estimating a set of discriminative target and non-target signatures and solving a sparse unmixing problem. After learning target signatures, a signature based detector can then be applied on test data. Both simulated and real hyperspectral target detection experiments show the proposed algorithm is effective at learning discriminative target signatures and achieves superior performance over state-of-the-art comparison algorithms.
Credimus
Venue: arXiv cs.CC 2019
Year: 2019

We believe that economic design and computational complexity---while already important to each other---should become even more important to each other with each passing year. But for that to happen, experts in on the one hand such areas as social choice, economics, and political science and on the other hand computational complexity will have to better understand each other's worldviews. This article, written by two complexity theorists who also work in computational social choice theory, focuses on one direction of that process by presenting a brief overview of how most computational complexity theorists view the world. Although our immediate motivation is to make the lens through which complexity theorists see the world be better understood by those in the social sciences, we also feel that even within computer science it is very important for nontheoreticians to understand how theoreticians think, just as it is equally important within computer science for theoreticians to understand how nontheoreticians think.
Stochastic Variational Inference for Bayesian Sparse Gaussian Process Regression
Venue: arXiv cs.LG 2019
Year: 2019

This paper presents a novel variational inference framework for deriving a family of Bayesian sparse Gaussian process regression (SGPR) models whose approximations are variationally optimal with respect to the full-rank GPR model enriched with various corresponding correlation structures of the observation noises. Our variational Bayesian SGPR (VBSGPR) models jointly treat both the distributions of the inducing variables and hyperparameters as variational parameters, which enables the decomposability of the variational lower bound that in turn can be exploited for stochastic optimization. Such a stochastic optimization involves iteratively following the stochastic gradient of the variational lower bound to improve its estimates of the optimal variational distributions of the inducing variables and hyperparameters (and hence the predictive distribution) of our VBSGPR models and is guaranteed to achieve asymptotic convergence to them. We show that the stochastic gradient is an unbiased estimator of the exact gradient and can be computed in constant time per iteration, hence achieving scalability to big data. We empirically evaluate the performance of our proposed framework on two real-world, massive datasets.
Adversarial Learning of Structure-Aware Fully Convolutional Networks for Landmark Localization
Venue: arXiv cs.CV 2019
Year: 2019

Landmark/pose estimation in single monocular images have received much effort in computer vision due to its important applications. It remains a challenging task when input images severe occlusions caused by, e.g., adverse camera views. Under such circumstances, biologically implausible pose predictions may be produced. In contrast, human vision is able to predict poses by exploiting geometric constraints of landmark point inter-connectivity. To address the problem, by incorporating priors about the structure of pose components, we propose a novel structure-aware fully convolutional network to implicitly take such priors into account during training of the deep network. Explicit learning of such constraints is typically challenging. Instead, inspired by how human identifies implausible poses, we design discriminators to distinguish the real poses from the fake ones (such as biologically implausible ones). If the pose generator G generates results that the discriminator fails to distinguish from real ones, the network successfully learns the priors. Training of the network follows the strategy of conditional Generative Adversarial Networks (GANs). The effectiveness of the proposed network is evaluated on three pose-related tasks: 2D single human pose estimation, 2D facial landmark estimation and 3D single human pose estimation. The proposed approach significantly outperforms the state-of-the-art methods and almost always generates plausible pose predictions, demonstrating the usefulness of implicit learning of structures using GANs.
Data, Depth, and Design: Learning Reliable Models for Skin Lesion Analysis
Venue: arXiv cs.CV 2019
Year: 2019

Deep learning fostered a leap ahead in automated skin lesion analysis in the last two years. Those models are expensive to train and difficult to parameterize. Objective: We investigate methodological issues for designing and evaluating deep learning models for skin lesion analysis. We explore 10 choices faced by researchers: use of transfer learning, model architecture, train dataset, image resolution, type of data augmentation, input normalization, use of segmentation, duration of training, additional use of SVMs, and test data augmentation. Methods: We perform two full factorial experiments, for five different test datasets, resulting in 2560 exhaustive trials in our main experiment, and 1280 trials in our assessment of transfer learning. We analyze both with multi-way ANOVA. We use the exhaustive trials to simulate sequential decisions and ensembles, with and without the use of privileged information from the test set. Results -- main experiment: Amount of train data has disproportionate influence, explaining almost half the variation in performance. Of the other factors, test data augmentation and input resolution are the most influential. Deeper models, when combined, with extra data, also help. -- transfer experiment: Transfer learning is critical, its absence brings huge performance penalties. -- simulations: Ensembles of models are the best option to provide reliable results with limited resources, without using privileged information and sacrificing methodological rigor. Conclusions and Significance: Advancing research on automated skin lesion analysis requires curating larger public datasets. Indirect use of privileged information from the test set to design the models is a subtle, but frequent methodological mistake that leads to overoptimistic results. Ensembles of models are a cost-effective alternative to the expensive full-factorial and to the unstable sequential designs.
Extracting an English-Persian Parallel Corpus from Comparable Corpora
Venue: arXiv cs.CL 2019
Year: 2019

Parallel data are an important part of a reliable Statistical Machine Translation (SMT) system. The more of these data are available, the better the quality of the SMT system. However, for some language pairs such as Persian-English, parallel sources of this kind are scarce. In this paper, a bidirectional method is proposed to extract parallel sentences from English and Persian document aligned Wikipedia. Two machine translation systems are employed to translate from Persian to English and the reverse after which an IR system is used to measure the similarity of the translated sentences. Adding the extracted sentences to the training data of the existing SMT systems is shown to improve the quality of the translation. Furthermore, the proposed method slightly outperforms the one-directional approach. The extracted corpus consists of about 200,000 sentences which have been sorted by their degree of similarity calculated by the IR system and is freely available for public access on the Web.
Modeling Duct Flow for Molecular Communication
Venue: arXiv cs.ET 2019
Year: 2019

Active transport such as fluid flow is sought in molecular communication to extend coverage, improve reliability, and mitigate interference. Flow models are often over-simplified, assuming one-dimensional diffusion with constant drift. However, diffusion and flow are usually encountered in three-dimensional bounded environments where the flow is highly non-uniform such as in blood vessels or microfluidic channels. For a qualitative understanding of the relevant physical effects inherent to these channels, based on the Peclet number and the transmitter-receiver distance, we study when simplified models of uniform flow and advection-only transport are applicable. For these two regimes, analytical expressions for the channel impulse response are derived and validated by particle-based simulation. Furthermore, as advection-only transport is typically overlooked and hence not analyzed in the molecular communication literature, we evaluate the symbol error rate for exemplary on-off keying as performance metric.
Online Tool Condition Monitoring Based on Parsimonious Ensemble+
Venue: arXiv cs.LG 2019
Year: 2019

Accurate diagnosis of tool wear in metal turning process remains an open challenge for both scientists and industrial practitioners because of inhomogeneities in workpiece material, nonstationary machining settings to suit production requirements, and nonlinear relations between measured variables and tool wear. Common methodologies for tool condition monitoring still rely on batch approaches which cannot cope with a fast sampling rate of metal cutting process. Furthermore they require a retraining process to be completed from scratch when dealing with a new set of machining parameters. This paper presents an online tool condition monitoring approach based on Parsimonious Ensemble+, pENsemble+. The unique feature of pENsemble+ lies in its highly flexible principle where both ensemble structure and base-classifier structure can automatically grow and shrink on the fly based on the characteristics of data streams. Moreover, the online feature selection scenario is integrated to actively sample relevant input attributes. The paper presents advancement of a newly developed ensemble learning algorithm, pENsemble+, where online active learning scenario is incorporated to reduce operator labelling effort. The ensemble merging scenario is proposed which allows reduction of ensemble complexity while retaining its diversity. Experimental studies utilising real-world manufacturing data streams and comparisons with well known algorithms were carried out. Furthermore, the efficacy of pENsemble was examined using benchmark concept drift data streams. It has been found that pENsemble+ incurs low structural complexity and results in a significant reduction of operator labelling effort.
Towards Optimal Energy Harvesting Receiver Design in MIMO Systems
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we investigate a multiple-input multiple-output (MIMO) system with simultaneous information detection (ID) and energy harvesting (EH) receiver. This point-to-point system operates in the vicinity of active interfering nodes. The receiver performs power splitting where a portion of received signal undergoes analog energy harvesting circuitry. Further, the information content of the other portion is extracted after performing digital beamforming. In this MIMO system, information carrier eigen-modes are not necessarily the eigen-modes with the strongest energy level. Hence, it is beneficial to perform independent beamforming at the receiver of MIMO-P2P channel. Here, we utilize a hybrid analog/digital beamforming for the purpose of simultaneous ID and EH in such scenarios. This design, provides extra design degrees-of-freedom in eigen-mode selection for ID and EH purposes independently. Worst-case performance of this receiver structure is discussed. Finally, its benefits is compared to the classical receiver structure and the gains are highlighted.
Self-referential basis of undecidable dynamics: from The Liar Paradox and The Halting Problem to The Edge of Chaos
Venue: arXiv cs.LO 2019
Year: 2019

In this paper we explore several fundamental relations between formal systems, algorithms, and dynamical systems, focussing on the roles of undecidability, universality, diagonalization, and self-reference in each of these computational frameworks. Some of these interconnections are well-known, while some are clarified in this study as a result of a fine-grained comparison between recursive formal systems, Turing machines, and Cellular Automata (CAs). In particular, we elaborate on the diagonalization argument applied to distributed computation carried out by CAs, illustrating the key elements of Gödel's proof for CAs. The comparative analysis emphasizes three factors which underlie the capacity to generate undecidable dynamics within the examined computational frameworks: (i) the program-data duality; (ii) the potential to access an infinite computational medium; and (iii) the ability to implement negation. The considered adaptations of Gödel's proof distinguish between computational universality and undecidability, and show how the diagonalization argument exploits, on several levels, the self-referential basis of undecidability.
SWIPT using Hybrid ARQ over Time Varying Channels
Venue: arXiv cs.NI 2019
Year: 2019

We consider a class of wireless powered devices employing Hybrid Automatic Repeat reQuest (HARQ) to ensure reliable end-to-end communications over a two-state time-varying channel. A receiver, with no power source, relies on the energy transferred by a Simultaneous Wireless Information and Power Transfer (SWIPT) enabled transmitter to \emph{receive} and \emph{decode} information. Under the two-state channel model, information is received at two different rates while it is only possible to harvest energy in one of the states. The receiver aims to decode its messages with minimum expected number of re-transmissions. Dynamic and continuous nature of the problem motivated us to use a novel Markovian framework to bypass the complexities plaguing the conventional approaches such as MDP. Using the theory of absorbing Markov chains, we show that there exists an optimal policy utilizing the incoming RF signal solely to harvest energy or to accumulate mutual information. Hence, we convert the original problem with continuous action and state space into an equivalent one with discrete state and action space. For independent and identically distributed channels, we prove the optimality of a simple-to-implement {\em harvest-first-store-later} type policy. However, for time-correlated channels, we demonstrate that statistical knowledge of the channel may significantly improve the performance over such policies.
Clustering with feature selection using alternating minimization, Application to computational biology
Venue: arXiv cs.LG 2019
Year: 2019

This paper deals with unsupervised clustering with feature selection. The problem is to estimate both labels and a sparse projection matrix of weights. To address this combinatorial non-convex problem maintaining a strict control on the sparsity of the matrix of weights, we propose an alternating minimization of the Frobenius norm criterion. We provide a new efficient algorithm named K-sparse which alternates k-means with projection-gradient minimization. The projection-gradient step is a method of splitting type, with exact projection on the $\ell^1$ ball to promote sparsity. The convergence of the gradient-projection step is addressed, and a preliminary analysis of the alternating minimization is made. The Frobenius norm criterion converges as the number of iterates in Algorithm K-sparse goes to infinity. Experiments on Single Cell RNA sequencing datasets show that our method significantly improves the results of PCA k-means, spectral clustering, SIMLR, and Sparcl methods, and achieves a relevant selection of genes. The complexity of K-sparse is linear in the number of samples (cells), so that the method scales up to large datasets.
Syntax-Directed Attention for Neural Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

Attention mechanism, including global attention and local attention, plays a key role in neural machine translation (NMT). Global attention attends to all source words for word prediction. In comparison, local attention selectively looks at fixed-window source words. However, alignment weights for the current target word often decrease to the left and right by linear distance centering on the aligned source position and neglect syntax-directed distance constraints. In this paper, we extend local attention with syntax-distance constraint, to focus on syntactically related source words with the predicted target word, thus learning a more effective context vector for word prediction. Moreover, we further propose a double context NMT architecture, which consists of a global context vector and a syntax-directed context vector over the global attention, to provide more translation performance for NMT from source representation. The experiments on the large-scale Chinese-to-English and English-to-Germen translation tasks show that the proposed approach achieves a substantial and significant improvement over the baseline system.
A Joint Encryption-Encoding Scheme Using QC-LDPC Codes Based on Finite Geometry
Venue: arXiv cs.IT 2019
Year: 2019

Joint encryption-encoding schemes have been released to fulfill both reliability and security desires in a single step. Using Low Density Parity Check (LDPC) codes in joint encryption-encoding schemes, as an alternative to classical linear codes, would shorten the key size as well as improving error correction capability. In this article, we present a joint encryption-encoding scheme using Quasi Cyclic-Low Density Parity Check (QC-LDPC) codes based on finite geometry. We observed that our proposed scheme not only outperforms its predecessors in key size and transmission rate, but also remains secure against all known cryptanalyses of code-based secret key cryptosystems. We subsequently show that our scheme benefits from low computational complexity. In our proposed joint encryption-encoding scheme, by taking the advantage of QC-LDPC codes based on finite geometries, the key size decreases to 1/5 of that of the so far best similar system. In addition, using our proposed scheme a wide range of desirable transmission rates are achievable. This variety of codes makes our cryptosystem suitable for a number of different communication and cryptographic standards.
Motif-based Convolutional Neural Network on Graphs
Venue: arXiv cs.LG 2019
Year: 2019

This paper introduces a generalization of Convolutional Neural Networks (CNNs) to graphs with irregular linkage structures, especially heterogeneous graphs with typed nodes and schemas. We propose a novel spatial convolution operation to model the key properties of local connectivity and translation invariance, using high-order connection patterns or motifs. We develop a novel deep architecture Motif-CNN that employs an attention model to combine the features extracted from multiple patterns, thus effectively capturing high-order structural and feature information. Our experiments on semi-supervised node classification on real-world social networks and multiple representative heterogeneous graph datasets indicate significant gains of 6-21% over existing graph CNNs and other state-of-the-art techniques.
Singular value automata and approximate minimization
Venue: arXiv cs.FL 2019
Year: 2019

The present paper uses spectral theory of linear operators to construct approximately minimal realizations of weighted languages. Our new contributions are: (i) a new algorithm for the SVD decomposition of infinite Hankel matrices based on their representation in terms of weighted automata, (ii) a new canonical form for weighted automata arising from the SVD of its corresponding Hankel matrix and (iii) an algorithm to construct approximate minimizations of given weighted automata by truncating the canonical form. We give bounds on the quality of our approximation.
Hindsight policy gradients
Venue: arXiv cs.LG 2019
Year: 2019

A reinforcement learning agent that needs to pursue different goals across episodes requires a goal-conditional policy. In addition to their potential to generalize desirable behavior to unseen goals, such policies may also enable higher-level planning based on subgoals. In sparse-reward environments, the capacity to exploit information about the degree to which an arbitrary goal has been achieved while another goal was intended appears crucial to enable sample efficient learning. However, reinforcement learning agents have only recently been endowed with such capacity for hindsight. In this paper, we demonstrate how hindsight can be introduced to policy gradient methods, generalizing this idea to a broad class of successful algorithms. Our experiments on a diverse selection of sparse-reward environments show that hindsight leads to a remarkable increase in sample efficiency.
Learning to Organize Knowledge and Answer Questions with N-Gram Machines
Venue: arXiv cs.CL 2019
Year: 2019

Though deep neural networks have great success in natural language processing, they are limited at more knowledge intensive AI tasks, such as open-domain Question Answering (QA). Existing end-to-end deep QA models need to process the entire text after observing the question, and therefore their complexity in responding a question is linear in the text size. This is prohibitive for practical tasks such as QA from Wikipedia, a novel, or the Web. We propose to solve this scalability issue by using symbolic meaning representations, which can be indexed and retrieved efficiently with complexity that is independent of the text size. We apply our approach, called the N-Gram Machine (NGM), to three representative tasks. First as proof-of-concept, we demonstrate that NGM successfully solves the bAbI tasks of synthetic text. Second, we show that NGM scales to large corpus by experimenting on "life-long bAbI", a special version of bAbI that contains millions of sentences. Lastly on the WikiMovies dataset, we use NGM to induce latent structure (i.e. schema) and answer questions from natural language Wikipedia text, with only QA pairs as weak supervision.