OpenRaiser/Intern-Atlas huggingface dataset:
Temporal Grounding Graphs for Language Understanding with Accrued Visual-Linguistic Context
Venue: arXiv cs.RO 2018
Year: 2018

A robot's ability to understand or ground natural language instructions is fundamentally tied to its knowledge about the surrounding world. We present an approach to grounding natural language utterances in the context of factual information gathered through natural-language interactions and past visual observations. A probabilistic model estimates, from a natural language utterance, the objects,relations, and actions that the utterance refers to, the objectives for future robotic actions it implies, and generates a plan to execute those actions while updating a state representation to include newly acquired knowledge from the visual-linguistic context. Grounding a command necessitates a representation for past observations and interactions; however, maintaining the full context consisting of all possible observed objects, attributes, spatial relations, actions, etc., over time is intractable. Instead, our model, Temporal Grounding Graphs, maintains a learned state representation for a belief over factual groundings, those derived from natural-language interactions, and lazily infers new groundings from visual observations using the context implied by the utterance. This work significantly expands the range of language that a robot can understand by incorporating factual knowledge and observations of its workspace in its inference about the meaning and grounding of natural-language utterances.
Exploring Tradeoffs in Models for Low-latency Speech Enhancement
Venue: arXiv cs.SD 2018
Year: 2018

We explore a variety of neural networks configurations for one- and two-channel spectrogram-mask-based speech enhancement. Our best model improves on previous state-of-the-art performance on the CHiME2 speech enhancement task by 0.4 decibels in signal-to-distortion ratio (SDR). We examine trade-offs such as non-causal look-ahead, computation, and parameter count versus enhancement performance and find that zero-look-ahead models can achieve, on average, within 0.03 dB SDR of our best bidirectional model. Further, we find that 200 milliseconds of look-ahead is sufficient to achieve equivalent performance to our best bidirectional model.
Combining Fact Extraction and Verification with Neural Semantic Matching Networks
Venue: arXiv cs.CL 2018
Year: 2018

The increasing concern with misinformation has stimulated research efforts on automatic fact checking. The recently-released FEVER dataset introduced a benchmark fact-verification task in which a system is asked to verify a claim using evidential sentences from Wikipedia documents. In this paper, we present a connected system consisting of three homogeneous neural semantic matching models that conduct document retrieval, sentence selection, and claim verification jointly for fact extraction and verification. For evidence retrieval (document retrieval and sentence selection), unlike traditional vector space IR models in which queries and sources are matched in some pre-designed term vector space, we develop neural models to perform deep semantic matching from raw textual input, assuming no intermediate term representation and no access to structured external knowledge bases. We also show that Pageview frequency can also help improve the performance of evidence retrieval results, that later can be matched by using our neural semantic matching network. For claim verification, unlike previous approaches that simply feed upstream retrieved evidence and the claim to a natural language inference (NLI) model, we further enhance the NLI model by providing it with internal semantic relatedness scores (hence integrating it with the evidence retrieval modules) and ontological WordNet features. Experiments on the FEVER dataset indicate that (1) our neural semantic matching method outperforms popular TF-IDF and encoder models, by significant margins on all evidence retrieval metrics, (2) the additional relatedness score and WordNet features improve the NLI model via better semantic awareness, and (3) by formalizing all three subtasks as a similar semantic matching problem and improving on all three stages, the complete model is able to achieve the state-of-the-art results on the FEVER test set.
Polyphonic audio tagging with sequentially labelled data using CRNN with learnable gated linear units
Venue: arXiv cs.SD 2018
Year: 2018

Audio tagging aims to detect the types of sound events occurring in an audio recording. To tag the polyphonic audio recordings, we propose to use Connectionist Temporal Classification (CTC) loss function on the top of Convolutional Recurrent Neural Network (CRNN) with learnable Gated Linear Units (GLU-CTC), based on a new type of audio label data: Sequentially Labelled Data (SLD). In GLU-CTC, CTC objective function maps the frame-level probability of labels to clip-level probability of labels. To compare the mapping ability of GLU-CTC for sound events, we train a CRNN with GLU based on Global Max Pooling (GLU-GMP) and a CRNN with GLU based on Global Average Pooling (GLU-GAP). And we also compare the proposed GLU-CTC system with the baseline system, which is a CRNN trained using CTC loss function without GLU. The experiments show that the GLU-CTC achieves an Area Under Curve (AUC) score of 0.882 in audio tagging, outperforming the GLU-GMP of 0.803, GLU-GAP of 0.766 and baseline system of 0.837. That means based on the same CRNN model with GLU, the performance of CTC mapping is better than the GMP and GAP mapping. Given both based on the CTC mapping, the CRNN with GLU outperforms the CRNN without GLU.
Boosting the Robustness Verification of DNN by Identifying the Achilles's Heel
Venue: arXiv cs.LG 2018
Year: 2018

Deep Neural Network (DNN) is a widely used deep learning technique. How to ensure the safety of DNN-based system is a critical problem for the research and application of DNN. Robustness is an important safety property of DNN. However, existing work of verifying DNN's robustness is time-consuming and hard to scale to large-scale DNNs. In this paper, we propose a boosting method for DNN robustness verification, aiming to find counter-examples earlier. Our observation is DNN's different inputs have different possibilities of existing counter-examples around them, and the input with a small difference between the largest output value and the second largest output value tends to be the achilles's heel of the DNN. We have implemented our method and applied it on Reluplex, a state-of-the-art DNN verification tool, and four DNN attacking methods. The results of the extensive experiments on two benchmarks indicate the effectiveness of our boosting method.
Recursive Sparse Pseudo-input Gaussian Process SARSA
Venue: arXiv cs.LG 2018
Year: 2018

The class of Gaussian Process (GP) methods for Temporal Difference learning has shown promise for data-efficient model-free Reinforcement Learning. In this paper, we consider a recent variant of the GP-SARSA algorithm, called Sparse Pseudo-input Gaussian Process SARSA (SPGP-SARSA), and derive recursive formulas for its predictive moments. This extension promotes greater memory efficiency, since previous computations can be reused and, interestingly, it provides a technique for updating value estimates on a multiple timescales
Representation Mixing for TTS Synthesis
Venue: arXiv cs.LG 2018
Year: 2018

Recent character and phoneme-based parametric TTS systems using deep learning have shown strong performance in natural speech generation. However, the choice between character or phoneme input can create serious limitations for practical deployment, as direct control of pronunciation is crucial in certain cases. We demonstrate a simple method for combining multiple types of linguistic information in a single encoder, named representation mixing, enabling flexible choice between character, phoneme, or mixed representations during inference. Experiments and user studies on a public audiobook corpus show the efficacy of our approach.
Ethical Dimensions of Visualization Research
Venue: arXiv cs.HC 2018
Year: 2018

Visualizations have a potentially enormous influence on how data are used to make decisions across all areas of human endeavor. However, it is not clear how this power connects to ethical duties: what obligations do we have when it comes to visualizations and visual analytics systems, beyond our duties as scientists and engineers? Drawing on historical and contemporary examples, I address the moral components of the design and use of visualizations, identify some ongoing areas of visualization research with ethical dilemmas, and propose a set of additional moral obligations that we have as designers, builders, and researchers of visualizations.
Learning to infer: RL-based search for DNN primitive selection on Heterogeneous Embedded Systems
Venue: arXiv cs.CV 2018
Year: 2018

Deep Learning is increasingly being adopted by industry for computer vision applications running on embedded devices. While Convolutional Neural Networks' accuracy has achieved a mature and remarkable state, inference latency and throughput are a major concern especially when targeting low-cost and low-power embedded platforms. CNNs' inference latency may become a bottleneck for Deep Learning adoption by industry, as it is a crucial specification for many real-time processes. Furthermore, deployment of CNNs across heterogeneous platforms presents major compatibility issues due to vendor-specific technology and acceleration libraries. In this work, we present QS-DNN, a fully automatic search based on Reinforcement Learning which, combined with an inference engine optimizer, efficiently explores through the design space and empirically finds the optimal combinations of libraries and primitives to speed up the inference of CNNs on heterogeneous embedded devices. We show that, an optimized combination can achieve 45x speedup in inference latency on CPU compared to a dependency-free baseline and 2x on average on GPGPU compared to the best vendor library. Further, we demonstrate that, the quality of results and time "to-solution" is much better than with Random Search and achieves up to 15x better results for a short-time search.
Nonlinear control of a swinging pendulum on a wheeled mobile robot with nonholonomic constraints
Venue: arXiv cs.RO 2018
Year: 2018

In this paper, we propose a nonlinear control strategy for swinging up a pendulum to its upright equilibrium position by shaping its swinging energy along with regulating the cart to a desired location. While the base of a usual cart-pole system is restricted to move in a straight line, the present system is allowed to move in the x-y plane with a nonholonomic consraint that its allowable velocity is only along its orientation. A simple time invariant control law has been presented and its effectiveness has been demonstrated using numerical experiments.
Shannon meets von Neumann: A Minimax Theorem for Channel Coding in the Presence of a Jammer
Venue: arXiv cs.IT 2018
Year: 2018

We study the setting of channel coding over a family of channels whose state is controlled by an adversarial jammer by viewing it as a zero-sum game between a finite blocklength encoder-decoder team, and the jammer. The encoder-decoder team choose stochastic encoding and decoding strategies to minimize the average probability of error in transmission, while the jammer chooses a distribution on the state-space to maximize this probability. The min-max value of this game is equivalent to channel coding for a compound channel -- we call this the Shannon solution of the problem. The max-min value corresponds to finding a mixed channel with the largest value of the minimum achievable probability of error. When the min-max and max-min values are equal, the problem is said to admit a saddle-point or von Neumann solution. While a Shannon solution always exists, a von Neumann solution need not, owing to inherent nonconvexity in the communicating team's problem. Despite this, we show that the min-max and max-min values become equal asymptotically in the large blocklength limit, for all but finitely many rates. We explicitly characterize this limiting value as a function of the rate and obtain tight finite blocklength bounds on the min-max and max-min value. As a corollary we get an explicit expression for the $ε$-capacity of a compound channel under stochastic codes -- the first such result, to the best of our knowledge. Our results demonstrate a deeper relation between the compound channel and mixed channel than was previously known. They also show that the conventional information-theoretic viewpoint, articulated via the Shannon solution, coincides asymptotically with the game-theoretic one articulated via the von Neumann solution.
WISE: Lightweight Intelligent Swarm Attestation Scheme for IoT (The Verifier's Perspective)
Venue: arXiv cs.CR 2018
Year: 2018

The growing pervasiveness of Internet of Things (IoT) expands the attack surface by connecting more and more attractive attack targets, i.e. embedded devices, to the Internet. One key component in securing these devices is software integrity checking, which typically attained with Remote Attestation (RA). RA is realized as an interactive protocol, whereby a trusted party, verifier, verifies the software integrity of a potentially compromised remote device, prover. In the vast majority of IoT applications, smart devices operate in swarms, thus triggering the need for efficient swarm attestation schemes. In this paper, we present WISE, the first intelligent swarm attestation protocol that aims to minimize the communication overhead while preserving an adequate level of security. WISE depends on a resource-efficient smart broadcast authentication scheme where devices are organized in fine-grained multi-clusters, and whenever needed, the most likely compromised devices are attested. The candidate devices are selected intelligently taking into account the attestation history and the diverse characteristics (and constraints) of each device in the swarm. We show that WISE is very suitable for resource-constrained embedded devices, highly efficient and scalable in heterogenous IoT networks, and offers an adjustable level of security.
Privacy Preserving Utility Mining: A Survey
Venue: arXiv cs.DB 2018
Year: 2018

In big data era, the collected data usually contains rich information and hidden knowledge. Utility-oriented pattern mining and analytics have shown a powerful ability to explore these ubiquitous data, which may be collected from various fields and applications, such as market basket analysis, retail, click-stream analysis, medical analysis, and bioinformatics. However, analysis of these data with sensitive private information raises privacy concerns. To achieve better trade-off between utility maximizing and privacy preserving, Privacy-Preserving Utility Mining (PPUM) has become a critical issue in recent years. In this paper, we provide a comprehensive overview of PPUM. We first present the background of utility mining, privacy-preserving data mining and PPUM, then introduce the related preliminaries and problem formulation of PPUM, as well as some key evaluation criteria for PPUM. In particular, we present and discuss the current state-of-the-art PPUM algorithms, as well as their advantages and deficiencies in detail. Finally, we highlight and discuss some technical challenges and open directions for future research on PPUM.
The Preemptive Resource Allocation Problem
Venue: arXiv cs.DS 2018
Year: 2018

We revisit a classical scheduling model to incorporate modern trends in data center networks and cloud services. Addressing some key challenges in the allocation of shared resources to user requests (jobs) in such settings, we consider the following variants of the classic {\em resource allocation problem} (\textsf{RAP}). The input to our problems is a set $J$ of jobs and a set $M$ of homogeneous hosts, each has an available amount of some resource. A job is associated with a release time, a due date, a weight, and a given length, as well as its resource requirement. A \emph{feasible} schedule is an allocation of the resource to a subset of the jobs, satisfying the job release times/due dates as well as the resource constraints. A crucial distinction between classic {\textsf{RAP}} and our problems is that we allow preemption and migration of jobs, motivated by virtualization techniques. We consider two natural objectives: {\em throughput maximization} (\textsf{MaxT}), which seeks a maximum weight subset of the jobs that can be feasibly scheduled on the hosts in $M$, and {\em resource minimization} (\textsf{MinR}), that is finding the minimum number of (homogeneous) hosts needed to feasibly schedule all jobs. Both problems are known to be NP-hard. We first present a $Ω(1)$-approximation algorithm for \textsf{MaxT} instances where time-windows form a laminar family of intervals. We then extend the algorithm to handle instances with arbitrary time-windows, assuming there is sufficient slack for each job to be completed. For \textsf{MinR} we study a more general setting with $d$ resources and derive an $O(\log d)$-approximation for any fixed $d \geq 1$, under the assumption that time-windows are not too small. This assumption can be removed leading to a slightly worse ratio of $O(\log d\log^* T)$, where $T$ is the maximum due date of any job.
Generalizable Adversarial Training via Spectral Normalization
Venue: arXiv cs.LG 2018
Year: 2018

Deep neural networks (DNNs) have set benchmarks on a wide array of supervised learning tasks. Trained DNNs, however, often lack robustness to minor adversarial perturbations to the input, which undermines their true practicality. Recent works have increased the robustness of DNNs by fitting networks using adversarially-perturbed training samples, but the improved performance can still be far below the performance seen in non-adversarial settings. A significant portion of this gap can be attributed to the decrease in generalization performance due to adversarial training. In this work, we extend the notion of margin loss to adversarial settings and bound the generalization error for DNNs trained under several well-known gradient-based attack schemes, motivating an effective regularization scheme based on spectral normalization of the DNN's weight matrices. We also provide a computationally-efficient method for normalizing the spectral norm of convolutional layers with arbitrary stride and padding schemes in deep convolutional networks. We evaluate the power of spectral normalization extensively on combinations of datasets, network architectures, and adversarial training schemes. The code is available at https://github.com/jessemzhang/dl_spectral_normalization.
Multiuser Computation Offloading and Downloading for Edge Computing with Virtualization
Venue: arXiv cs.IT 2018
Year: 2018

Mobile-edge computing (MEC) is an emerging technology for enhancing the computational capabilities of mobile devices and reducing their energy consumption via offloading complex computation tasks to the nearby servers. Multiuser MEC at servers is widely realized via parallel computing based on virtualization. Due to finite shared I/O resources, interference between virtual machines (VMs), called I/O interference, degrades the computation performance. In this paper, we study the problem of joint radio-and-computation resource allocation (RCRA) in multiuser MEC systems in the presence of I/O interference. Specifically, offloading scheduling algorithms are designed targeting two system performance metrics: sum offloading throughput maximization and sum mobile energy consumption minimization. Their designs are formulated as non-convex mixed-integer programming problems, which account for latency due to offloading, result downloading and parallel computing. A set of low-complexity algorithms are designed based on a decomposition approach and leveraging classic techniques from combinatorial optimization. The resultant algorithms jointly schedule offloading users, control their offloading sizes, and divide time for communication (offloading and downloading) and computation. They are either optimal or can achieve close-to-optimality as shown by simulation. Comprehensive simulation results demonstrate considering of I/O interference can endow on an offloading controller robustness against the performance-degradation factor.
A Note on Two Constructions of Zero-Difference Balanced Functions
Venue: arXiv cs.IT 2018
Year: 2018

Notes on two constructions of zero-difference balanced (ZDB) functions are made in this letter. Then ZDB functions over $\mathbb{Z}_{e}\times \prod_{i=0}^{k}{\mathbb{F}_{q_i}}$ are obtained. And it shows that all the known ZDB functions using cyclotomic cosets over $\mathbb{Z}_{n}$ are special cases of a generic construction. Moreover, applications of these ZDB functions are presented.
CA3Net: Contextual-Attentional Attribute-Appearance Network for Person Re-Identification
Venue: arXiv cs.CV 2018
Year: 2018

Person re-identification aims to identify the same pedestrian across non-overlapping camera views. Deep learning techniques have been applied for person re-identification recently, towards learning representation of pedestrian appearance. This paper presents a novel Contextual-Attentional Attribute-Appearance Network (CA3Net) for person re-identification. The CA3Net simultaneously exploits the complementarity between semantic attributes and visual appearance, the semantic context among attributes, visual attention on attributes as well as spatial dependencies among body parts, leading to discriminative and robust pedestrian representation. Specifically, an attribute network within CA3Net is designed with an Attention-LSTM module. It concentrates the network on latent image regions related to each attribute as well as exploits the semantic context among attributes by a LSTM module. An appearance network is developed to learn appearance features from the full body, horizontal and vertical body parts of pedestrians with spatial dependencies among body parts. The CA3Net jointly learns the attribute and appearance features in a multi-task learning manner, generating comprehensive representation of pedestrians. Extensive experiments on two challenging benchmarks, i.e., Market-1501 and DukeMTMC-reID datasets, have demonstrated the effectiveness of the proposed approach.
Iso-level tool path planning for free-form surfaces
Venue: arXiv cs.GR 2018
Year: 2018

The aim of tool path planning is to maximize the efficiency against some given precision criteria. In practice, scallop height should be kept constant to avoid unnecessary cutting, while the tool path should be smooth enough to maintain a high feed rate. However, iso-scallop and smoothness often conflict with each other. Existing methods smooth iso-scallop paths one-by-one, which make the final tool path far from being globally optimal. This paper proposes a new framework for tool path optimization. It views a family of iso-level curves of a scalar function defined over the surface as tool path so that desired tool path can be generated by finding the function that minimizes certain energy functional and different objectives can be considered simultaneously. We use the framework to plan globally optimal tool path with respect to iso-scallop and smoothness. The energy functionals for planning iso-scallop, smoothness, and optimal tool path are respectively derived, and the path topology is studied too. Experimental results are given to show the effectiveness of the proposed methods.