OpenRaiser/Intern-Atlas huggingface dataset:
Leveraging Distributional Semantics for Multi-Label Learning
Venue: arXiv cs.LG 2017
Year: 2017

We present a novel and scalable label embedding framework for large-scale multi-label learning a.k.a ExMLDS (Extreme Multi-Label Learning using Distributional Semantics). Our approach draws inspiration from ideas rooted in distributional semantics, specifically the Skip Gram Negative Sampling (SGNS) approach, widely used to learn word embeddings for natural language processing tasks. Learning such embeddings can be reduced to a certain matrix factorization. Our approach is novel in that it highlights interesting connections between label embedding methods used for multi-label learning and paragraph/document embedding methods commonly used for learning representations of text data. The framework can also be easily extended to incorporate auxiliary information such as label-label correlations; this is crucial especially when there are a lot of missing labels in the training data. We demonstrate the effectiveness of our approach through an extensive set of experiments on a variety of benchmark datasets, and show that the proposed learning methods perform favorably compared to several baselines and state-of-the-art methods for large-scale multi-label learning. To facilitate end-to-end learning, we develop a joint learning algorithm that can learn the embeddings as well as a regression model that predicts these embeddings given input features, via efficient gradient-based methods.
Multi-Person Pose Estimation via Column Generation
Venue: arXiv cs.CV 2017
Year: 2017

We study the problem of multi-person pose estimation in natural images. A pose estimate describes the spatial position and identity (head, foot, knee, etc.) of every non-occluded body part of a person. Pose estimation is difficult due to issues such as deformation and variation in body configurations and occlusion of parts, while multi-person settings add complications such as an unknown number of people, with unknown appearance and possible interactions in their poses and part locations. We give a novel integer program formulation of the multi-person pose estimation problem, in which variables correspond to assignments of parts in the image to poses in a two-tier, hierarchical way. This enables us to develop an efficient custom optimization procedure based on column generation, where columns are produced by exact optimization of very small scale integer programs. We demonstrate improved accuracy and speed for our method on the MPII multi-person pose estimation benchmark.
From Logic to Biology via Physics: a survey
Venue: arXiv cs.OH 2017
Year: 2017

This short text summarizes the work in biology proposed in our book, Perspectives on Organisms, where we analyse the unity proper to organisms by looking at it from different viewpoints. We discuss the theoretical roles of biological time, complexity, theoretical symmetries, singularities and critical transitions. We explicitly borrow from the conclusions in some key chapters and introduce them by a reflection on "incompleteness", also proposed in the book. We consider that incompleteness is a fundamental notion to understand the way in which we construct knowledge. Then we will introduce an approach to biological dynamics where randomness is central to the theoretical determination: randomness does not oppose biological stability but contributes to it by variability, adaptation, and diversity. Then, evolutionary and ontogenetic trajectories are continual changes of coherence structures involving symmetry changes within an ever-changing global stability.
Guided Deep Reinforcement Learning for Swarm Systems
Venue: arXiv cs.MA 2017
Year: 2017

In this paper, we investigate how to learn to control a group of cooperative agents with limited sensing capabilities such as robot swarms. The agents have only very basic sensor capabilities, yet in a group they can accomplish sophisticated tasks, such as distributed assembly or search and rescue tasks. Learning a policy for a group of agents is difficult due to distributed partial observability of the state. Here, we follow a guided approach where a critic has central access to the global state during learning, which simplifies the policy evaluation problem from a reinforcement learning point of view. For example, we can get the positions of all robots of the swarm using a camera image of a scene. This camera image is only available to the critic and not to the control policies of the robots. We follow an actor-critic approach, where the actors base their decisions only on locally sensed information. In contrast, the critic is learned based on the true global state. Our algorithm uses deep reinforcement learning to approximate both the Q-function and the policy. The performance of the algorithm is evaluated on two tasks with simple simulated 2D agents: 1) finding and maintaining a certain distance to each others and 2) locating a target.
Deep dip teardown of tubeless insulin pump
Venue: arXiv cs.CR 2017
Year: 2017

This paper introduces a deep level teardown process of a personal medical device - the OmniPod wireless tubeless insulin pump. This starts with mechanical teardown exposing the engineering solutions used inside the device. Then the electronic part of the device is analysed followed by components identification. Finally, the firmware extraction is performed allowing further analysis of the firmware inside the device as well as real-time debugging. This paper also evaluates the security of the main controller IC of the device. It reveals some weaknesses in the device design process which lead to the possibility of the successful teardown. Should the hardware security of the controller inside the device was well thought through, the teardown process would be far more complicated. This paper demonstrates what the typical teardown process of a personal medical device involves. This knowledge could help in improving the hardware security of sensitive devices.
Coupled Ensembles of Neural Networks
Venue: arXiv cs.CV 2017
Year: 2017

We investigate in this paper the architecture of deep convolutional networks. Building on existing state of the art models, we propose a reconfiguration of the model parameters into several parallel branches at the global network level, with each branch being a standalone CNN. We show that this arrangement is an efficient way to significantly reduce the number of parameters without losing performance or to significantly improve the performance with the same level of performance. The use of branches brings an additional form of regularization. In addition to the split into parallel branches, we propose a tighter coupling of these branches by placing the "fuse (averaging) layer" before the Log-Likelihood and SoftMax layers during training. This gives another significant performance improvement, the tighter coupling favouring the learning of better representations, even at the level of the individual branches. We refer to this branched architecture as "coupled ensembles". The approach is very generic and can be applied with almost any DCNN architecture. With coupled ensembles of DenseNet-BC and parameter budget of 25M, we obtain error rates of 2.92%, 15.68% and 1.50% respectively on CIFAR-10, CIFAR-100 and SVHN tasks. For the same budget, DenseNet-BC has error rate of 3.46%, 17.18%, and 1.8% respectively. With ensembles of coupled ensembles, of DenseNet-BC networks, with 50M total parameters, we obtain error rates of 2.72%, 15.13% and 1.42% respectively on these tasks.
Cache-Aware Lock-Free Concurrent Hash Tries
Venue: arXiv cs.DS 2017
Year: 2017

This report describes an implementation of a non-blocking concurrent shared-memory hash trie based on single-word compare-and-swap instructions. Insert, lookup and remove operations modifying different parts of the hash trie can be run independent of each other and do not contend. Remove operations ensure that the unneeded memory is freed and that the trie is kept compact. A pseudocode for these operations is presented and a proof of correctness is given -- we show that the implementation is linearizable and lock-free. Finally, benchmarks are presented which compare concurrent hash trie operations against the corresponding operations on other concurrent data structures, showing their performance and scalability.
Network Deployment for Maximal Energy Efficiency in Uplink with Zero-Forcing
Venue: arXiv cs.IT 2017
Year: 2017

This work aims to design a cellular network for maximal energy efficiency (EE). In particular, we consider the uplink with multi-antenna base stations and assume that zero- forcing (ZF) combining is used for data detection with imperfect channel state information. Using stochastic geometry and a new lower bound on the average per-user spectral efficiency of the network, we optimize the pilot reuse factor, number of antennas and users per base station. Closed-form expressions are computed from which valuable insights into the interplay between the optimization variables, hardware characteristics, and propagation environment are obtained. Numerical results are used to validate the analysis and make comparisons with a network using maximum ratio (MR) combining. The results show that a Massive MIMO setup arises as the EE-optimal network configuration. In addition, ZF provides higher EE than MR while allowing a smaller pilot reuse factor and a more dense network deployment.
Deep Graph Attention Model
Venue: arXiv cs.LG 2017
Year: 2017

Graph classification is a problem with practical applications in many different domains. Most of the existing methods take the entire graph into account when calculating graph features. In a graphlet-based approach, for instance, the entire graph is processed to get the total count of different graphlets or sub-graphs. In the real-world, however, graphs can be both large and noisy with discriminative patterns confined to certain regions in the graph only. In this work, we study the problem of attentional processing for graph classification. The use of attention allows us to focus on small but informative parts of the graph, avoiding noise in the rest of the graph. We present a novel RNN model, called the Graph Attention Model (GAM), that processes only a portion of the graph by adaptively selecting a sequence of "interesting" nodes. The model is equipped with an external memory component which allows it to integrate information gathered from different parts of the graph. We demonstrate the effectiveness of the model through various experiments.
Iterative Policy Learning in End-to-End Trainable Task-Oriented Neural Dialog Models
Venue: arXiv cs.CL 2017
Year: 2017

In this paper, we present a deep reinforcement learning (RL) framework for iterative dialog policy optimization in end-to-end task-oriented dialog systems. Popular approaches in learning dialog policy with RL include letting a dialog agent to learn against a user simulator. Building a reliable user simulator, however, is not trivial, often as difficult as building a good dialog agent. We address this challenge by jointly optimizing the dialog agent and the user simulator with deep RL by simulating dialogs between the two agents. We first bootstrap a basic dialog agent and a basic user simulator by learning directly from dialog corpora with supervised training. We then improve them further by letting the two agents to conduct task-oriented dialogs and iteratively optimizing their policies with deep RL. Both the dialog agent and the user simulator are designed with neural network models that can be trained end-to-end. Our experiment results show that the proposed method leads to promising improvements on task success rate and total task reward comparing to supervised training and single-agent RL training baseline models.
A dissipativity theorem for p-dominant systems
Venue: arXiv cs.SY 2017
Year: 2017

We revisit the classical dissipativity theorem of linear-quadratic theory in a generalized framework where the quadratic storage is negative definite in a p-dimensional subspace and positive definite in a complementary subspace. The classical theory assumes p = 0 and provides an inter- connection theory for stability analysis, i.e. convergence to a zero dimensional attractor. The generalized theory is shown to provide an interconnection theory for p-dominance analysis, i.e. convergence to a p-dimensional dominant subspace. In turn, this property is the differential characterization of a generalized contraction property for nonlinear systems. The proposed generalization opens a novel avenue for the analysis of interconnected systems with low-dimensional attractors.
Paraphrasing verbal metonymy through computational methods
Venue: arXiv cs.CL 2017
Year: 2017

Verbal metonymy has received relatively scarce attention in the field of computational linguistics despite the fact that a model to accurately paraphrase metonymy has applications both in academia and the technology sector. The method described in this paper makes use of data from the British National Corpus in order to create word vectors, find instances of verbal metonymy and generate potential paraphrases. Two different ways of creating word vectors are evaluated in this study: Continuous bag of words and Skip-grams. Skip-grams are found to outperform the Continuous bag of words approach. Furthermore, the Skip-gram model is found to operate with better-than-chance accuracy and there is a strong positive relationship (phi coefficient = 0.61) between the model's classification and human judgement of the ranked paraphrases. This study lends credence to the viability of modelling verbal metonymy through computational methods based on distributional semantics.
Dynamic Oracle for Neural Machine Translation in Decoding Phase
Venue: arXiv cs.CL 2017
Year: 2017

The past several years have witnessed the rapid progress of end-to-end Neural Machine Translation (NMT). However, there exists discrepancy between training and inference in NMT when decoding, which may lead to serious problems since the model might be in a part of the state space it has never seen during training. To address the issue, Scheduled Sampling has been proposed. However, there are certain limitations in Scheduled Sampling and we propose two dynamic oracle-based methods to improve it. We manage to mitigate the discrepancy by changing the training process towards a less guided scheme and meanwhile aggregating the oracle's demonstrations. Experimental results show that the proposed approaches improve translation quality over standard NMT system.
Coded Caching in Partially Cooperative D2D Communication Networks
Venue: arXiv cs.IT 2017
Year: 2017

The backhaul traffic is becoming a major concern in wireless and cellular networks (e.g., 4G-LTE and 5G) with the increasing demands for online video streaming. Caching the popular content in the cache memory of the network users (e.g., mobile devices) is an effective technique to reduce the traffic during the networks' peak time. However, due to the dynamic nature of these networks, users privacy settings, or energy limitations, some users may not be available or intend to participate during the caching procedures. In this paper, we propose caching schemes for device-to-device communication networks where a group of users show selfish characteristics. The selfish users along with the non-selfish users will cache the popular content, but will not share their useful cache content with the other users to satisfy a user request. We show that our proposed schemes are able to satisfy any arbitrary user requests under partial cooperation of the network users.
Physical Layer Security in Heterogeneous Networks with Jammer Selection and Full-Duplex Users
Venue: arXiv cs.IT 2017
Year: 2017

In this paper, we enhance physical layer security for downlink heterogeneous networks (HetNets) by using friendly jammers and full-duplex users. The jammers are selected to transmit jamming signal if their interfering power on the scheduled users is below a threshold, meanwhile the scheduled users confound the eavesdroppers using artificial noise by full-duplexing. Using the tools of stochastic geometry, we derive the expressions of connection probability and secrecy probability. In particular, the locations of active jammers are modeled by a modified Poisson hole process (PHP). Determining the jammer selection threshold is further investigated for connection probability maximization subject to the security constraints. A greedy algorithm is proposed to efficiently solve this problem. The accuracy of the theoretical analysis and the efficiency of the proposed algorithm are evaluated by numerical simulations.
Exploring Human-like Attention Supervision in Visual Question Answering
Venue: arXiv cs.CV 2017
Year: 2017

Attention mechanisms have been widely applied in the Visual Question Answering (VQA) task, as they help to focus on the area-of-interest of both visual and textual information. To answer the questions correctly, the model needs to selectively target different areas of an image, which suggests that an attention-based model may benefit from an explicit attention supervision. In this work, we aim to address the problem of adding attention supervision to VQA models. Since there is a lack of human attention data, we first propose a Human Attention Network (HAN) to generate human-like attention maps, training on a recently released dataset called Human ATtention Dataset (VQA-HAT). Then, we apply the pre-trained HAN on the VQA v2.0 dataset to automatically produce the human-like attention maps for all image-question pairs. The generated human-like attention map dataset for the VQA v2.0 dataset is named as Human-Like ATtention (HLAT) dataset. Finally, we apply human-like attention supervision to an attention-based VQA model. The experiments show that adding human-like supervision yields a more accurate attention together with a better performance, showing a promising future for human-like attention supervision in VQA.
Aspect-Based Sentiment Analysis Using a Two-Step Neural Network Architecture
Venue: arXiv cs.CL 2017
Year: 2017

The World Wide Web holds a wealth of information in the form of unstructured texts such as customer reviews for products, events and more. By extracting and analyzing the expressed opinions in customer reviews in a fine-grained way, valuable opportunities and insights for customers and businesses can be gained. We propose a neural network based system to address the task of Aspect-Based Sentiment Analysis to compete in Task 2 of the ESWC-2016 Challenge on Semantic Sentiment Analysis. Our proposed architecture divides the task in two subtasks: aspect term extraction and aspect-specific sentiment extraction. This approach is flexible in that it allows to address each subtask independently. As a first step, a recurrent neural network is used to extract aspects from a text by framing the problem as a sequence labeling task. In a second step, a recurrent network processes each extracted aspect with respect to its context and predicts a sentiment label. The system uses pretrained semantic word embedding features which we experimentally enhance with semantic knowledge extracted from WordNet. Further features extracted from SenticNet prove to be beneficial for the extraction of sentiment labels. As the best performing system in its category, our proposed system proves to be an effective approach for the Aspect-Based Sentiment Analysis.
MetaLDA: a Topic Model that Efficiently Incorporates Meta information
Venue: arXiv cs.CL 2017
Year: 2017

Besides the text content, documents and their associated words usually come with rich sets of meta informa- tion, such as categories of documents and semantic/syntactic features of words, like those encoded in word embeddings. Incorporating such meta information directly into the generative process of topic models can improve modelling accuracy and topic quality, especially in the case where the word-occurrence information in the training data is insufficient. In this paper, we present a topic model, called MetaLDA, which is able to leverage either document or word meta information, or both of them jointly. With two data argumentation techniques, we can derive an efficient Gibbs sampling algorithm, which benefits from the fully local conjugacy of the model. Moreover, the algorithm is favoured by the sparsity of the meta information. Extensive experiments on several real world datasets demonstrate that our model achieves comparable or improved performance in terms of both perplexity and topic quality, particularly in handling sparse texts. In addition, compared with other models using meta information, our model runs significantly faster.
Selfish Jobs with Favorite Machines: Price of Anarchy vs Strong Price of Anarchy
Venue: arXiv cs.GT 2017
Year: 2017

We consider the well-studied game-theoretic version of machine scheduling in which jobs correspond to self-interested users and machines correspond to resources. Here each user chooses a machine trying to minimize her own cost, and such selfish behavior typically results in some equilibrium which is not globally optimal: An equilibrium is an allocation where no user can reduce her own cost by moving to another machine, which in general need not minimize the makespan, i.e., the maximum load over the machines. We provide tight bounds on two well-studied notions in algorithmic game theory, namely, the price of anarchy and the strong price of anarchy on machine scheduling setting which lies in between the related and the unrelated machine case. Both notions study the social cost (makespan) of the worst equilibrium compared to the optimum, with the strong price of anarchy restricting to a stronger form of equilibria. Our results extend a prior study comparing the price of anarchy to the strong price of anarchy for two related machines (Epstein, Acta Informatica 2010), thus providing further insights on the relation between these concepts. Our exact bounds give a qualitative and quantitative comparison between the two models. The bounds also show that the setting is indeed easier than the two unrelated machines: In the latter, the strong price of anarchy is $2$, while in ours it is strictly smaller.
A Deep Learning-based Framework for Conducting Stealthy Attacks in Industrial Control Systems
Venue: arXiv cs.CR 2017
Year: 2017

Industrial control systems (ICS), which in many cases are components of critical national infrastructure, are increasingly being connected to other networks and the wider internet motivated by factors such as enhanced operational functionality and improved efficiency. However, set in this context, it is easy to see that the cyber attack surface of these systems is expanding, making it more important than ever that innovative solutions for securing ICS be developed and that the limitations of these solutions are well understood. The development of anomaly based intrusion detection techniques has provided capability for protecting ICS from the serious physical damage that cyber breaches are capable of delivering to them by monitoring sensor and control signals for abnormal activity. Recently, the use of so-called stealthy attacks has been demonstrated where the injection of false sensor measurements can be used to mimic normal control system signals, thereby defeating anomaly detectors whilst still delivering attack objectives. In this paper we define a deep learning-based framework which allows an attacker to conduct stealthy attacks with minimal a-priori knowledge of the target ICS. Specifically, we show that by intercepting the sensor and/or control signals in an ICS for a period of time, a malicious program is able to automatically learn to generate high-quality stealthy attacks which can achieve specific attack goals whilst bypassing a black box anomaly detector. Furthermore, we demonstrate the effectiveness of our framework for conducting stealthy attacks using two real-world ICS case studies. We contend that our results motivate greater attention on this area by the security community as we demonstrate that currently assumed barriers for the successful execution of such attacks are relaxed.