OpenRaiser/Intern-Atlas huggingface dataset:
Blind Community Detection from Low-rank Excitations of a Graph Filter
Venue: arXiv cs.SI 2019
Year: 2019

This paper considers a new framework to detect communities in a graph from the observation of signals at its nodes. We model the observed signals as noisy outputs of an unknown network process, represented as a graph filter that is excited by a set of unknown low-rank inputs/excitations. Application scenarios of this model include diffusion dynamics, pricing experiments, and opinion dynamics. Rather than learning the precise parameters of the graph itself, we aim at retrieving the community structure directly. The paper shows that communities can be detected by applying a spectral method to the covariance matrix of graph signals. Our analysis indicates that the community detection performance depends on a `low-pass' property of the graph filter. We also show that the performance can be improved via a low-rank matrix plus sparse decomposition method when the latent parameter vectors are known. Numerical experiments demonstrate that our approach is effective.
Skip-gram word embeddings in hyperbolic space
Venue: arXiv cs.CL 2019
Year: 2019

Recent work has demonstrated that embeddings of tree-like graphs in hyperbolic space surpass their Euclidean counterparts in performance by a large margin. Inspired by these results and scale-free structure in the word co-occurrence graph, we present an algorithm for learning word embeddings in hyperbolic space from free text. An objective function based on the hyperbolic distance is derived and included in the skip-gram negative-sampling architecture of word2vec. The hyperbolic word embeddings are then evaluated on word similarity and analogy benchmarks. The results demonstrate the potential of hyperbolic word embeddings, particularly in low dimensions, though without clear superiority over their Euclidean counterparts. We further discuss subtleties in the formulation of the analogy task in curved spaces.
Telexistence and Teleoperation for Walking Humanoid Robots
Venue: arXiv cs.RO 2019
Year: 2019

This paper proposes an architecture for achieving telexistence and teleoperation of humanoid robots. The architecture combines several technological set-ups, methodologies, locomotion and manipulation algorithms in a novel manner, thus building upon and extending works available in literature. The approach allows a human operator to command and telexist with the robot. Therefore, in this work we treat aspects pertaining not only to the proposed architecture structure and implementation, but also the human operator experience in terms of ability to adapt to the robot and to the architecture. Also the proprioception aspects and embodiment of the robot are studied through specific experimental results, which are also treated in a somewhat formal, albeit high-level manner. Application of the proposed architecture and experiments incorporating user training and experience are addressed using an illustrative bipedal humanoid robot, namely the iCub robot.
Coverage-Based Designs Improve Sample Mining and Hyper-Parameter Optimization
Venue: arXiv cs.LG 2019
Year: 2019

Sampling one or more effective solutions from large search spaces is a recurring idea in machine learning, and sequential optimization has become a popular solution. Typical examples include data summarization, sample mining for predictive modeling and hyper-parameter optimization. Existing solutions attempt to adaptively trade-off between global exploration and local exploitation, wherein the initial exploratory sample is critical to their success. While discrepancy-based samples have become the de facto approach for exploration, results from computer graphics suggest that coverage-based designs, e.g. Poisson disk sampling, can be a superior alternative. In order to successfully adopt coverage-based sample designs to ML applications, which were originally developed for 2-d image analysis, we propose fundamental advances by constructing a parameterized family of designs with provably improved coverage characteristics, and by developing algorithms for effective sample synthesis. Using experiments in sample mining and hyper-parameter optimization for supervised learning, we show that our approach consistently outperforms existing exploratory sampling methods in both blind exploration, and sequential search with Bayesian optimization.
Survey on UAV Cellular Communications: Practical Aspects, Standardization Advancements, Regulation, and Security Challenges
Venue: arXiv cs.NI 2019
Year: 2019

The rapid growth of consumer Unmanned Aerial Vehicles (UAVs) is creating promising new business opportunities for cellular operators. On the one hand, UAVs can be connected to cellular networks as new types of user equipment, therefore generating significant revenues for the operators that can guarantee their stringent service requirements. On the other hand, UAVs offer the unprecedented opportunity to realize UAV-mounted flying base stations that can dynamically reposition themselves to boost coverage, spectral efficiency, and user quality of experience. Indeed, the standardization bodies are currently exploring possibilities for serving commercial UAVs with cellular networks. Industries are beginning to trial early prototypes of flying base stations or user equipments, while academia is in full swing researching mathematical and algorithmic solutions to address interesting new problems arising from flying nodes in cellular networks. In this article, we provide a comprehensive survey of all of these developments promoting smooth integration of UAVs into cellular networks. Specifically, we survey (i) the types of consumer UAVs currently available off-the-shelf, (ii) the interference issues and potential solutions addressed by standardization bodies for serving aerial users with the existing terrestrial base stations, (iii) the challenges and opportunities for assisting cellular communications with UAV-based flying relays and base stations, (iv) the ongoing prototyping and test bed activities, (v) the new regulations being developed to manage the commercial use of UAVs, and (vi) the cyber-physical security of UAV-assisted cellular communications.
Security Metrics of Networked Control Systems under Sensor Attacks (extended preprint)
Venue: arXiv cs.SY 2019
Year: 2019

As more attention is paid to security in the context of control systems and as attacks occur to real control systems throughout the world, it has become clear that some of the most nefarious attacks are those that evade detection. The term stealthy has come to encompass a variety of techniques that attackers can employ to avoid being detected. In this manuscript, for a class of perturbed linear time-invariant systems, we propose two security metrics to quantify the potential impact that stealthy attacks could have on the system dynamics by tampering with sensor measurements. We provide analysis mathematical tools (in terms of linear matrix inequalities) to quantify these metrics for given system dynamics, control structure, system monitor, and set of sensors being attacked. Then, we provide synthesis tools (in terms of semidefinite programs) to redesign controllers and monitors such that the impact of stealthy attacks is minimized and the required attack-free system performance is guaranteed.
A "poor man's" approach to topology optimization of natural convection problems
Venue: arXiv cs.CE 2019
Year: 2019

Topology optimization of natural convection problems is computationally expensive, due to the large number of degrees of freedom (DOFs) in the model and its two-way coupled nature. Herein, a method is presented to reduce the computational effort by use of a reduced-order model governed by simplified physics. The proposed method models the fluid flow using a potential flow model, which introduces an additional fluid property. This material property currently requires tuning of the model by comparison to numerical Navier-Stokes based solutions. Topology optimization based on the reduced-order model is shown to provide qualitatively similar designs, as those obtained using a full Navier-Stokes based model. The number of DOFs is reduced by 50% in two dimensions and the computational complexity is evaluated to be approximately 12.5% of the full model. We further compare to optimized designs obtained utilizing Newton's convection law.
Memory Replay GANs: learning to generate images from new categories without forgetting
Venue: arXiv cs.CV 2019
Year: 2019

Previous works on sequential learning address the problem of forgetting in discriminative models. In this paper we consider the case of generative models. In particular, we investigate generative adversarial networks (GANs) in the task of learning new categories in a sequential fashion. We first show that sequential fine tuning renders the network unable to properly generate images from previous categories (i.e. forgetting). Addressing this problem, we propose Memory Replay GANs (MeRGANs), a conditional GAN framework that integrates a memory replay generator. We study two methods to prevent forgetting by leveraging these replays, namely joint training with replay and replay alignment. Qualitative and quantitative experimental results in MNIST, SVHN and LSUN datasets show that our memory replay approach can generate competitive images while significantly mitigating the forgetting of previous categories.
A Low Complexity Detection Algorithm Based on Alternating Minimization
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we propose an algorithm based on the Alternating Minimization technique to solve the uplink massive MIMO detection problem. The proposed algorithm provides a lower complexity compared to the conventional MMSE detection technique, especially when the number of user equipment (UE) antennas is close to the number of base station (BS) antennas. This improvement is obtained without any matrix inversion. Moreover, the algorithm re-formulates the maximum likelihood (ML) detection problem as a sum of convex functions based on decomposing the received vector into multiple vectors. Each vector represents the contribution of one of the transmitted symbols in the received vector. Alternating Minimization is used to solve the new formulated problem in an iterative manner with a closed form solution update in every iteration. Simulation results demonstrate the efficacy of the proposed algorithm in the uplink massive MIMO setting for both coded and uncoded cases
Character-Aware Decoder for Translation into Morphologically Rich Languages
Venue: arXiv cs.CL 2019
Year: 2019

Neural machine translation (NMT) systems operate primarily on words (or sub-words), ignoring lower-level patterns of morphology. We present a character-aware decoder designed to capture such patterns when translating into morphologically rich languages. We achieve character-awareness by augmenting both the softmax and embedding layers of an attention-based encoder-decoder model with convolutional neural networks that operate on the spelling of a word. To investigate performance on a wide variety of morphological phenomena, we translate English into 14 typologically diverse target languages using the TED multi-target dataset. In this low-resource setting, the character-aware decoder provides consistent improvements with BLEU score gains of up to $+3.05$. In addition, we analyze the relationship between the gains obtained and properties of the target language and find evidence that our model does indeed exploit morphological patterns.
Cell-aware Stacked LSTMs for Modeling Sentences
Venue: arXiv cs.CL 2019
Year: 2019

We propose a method of stacking multiple long short-term memory (LSTM) layers for modeling sentences. In contrast to the conventional stacked LSTMs where only hidden states are fed as input to the next layer, the suggested architecture accepts both hidden and memory cell states of the preceding layer and fuses information from the left and the lower context using the soft gating mechanism of LSTMs. Thus the architecture modulates the amount of information to be delivered not only in horizontal recurrence but also in vertical connections, from which useful features extracted from lower layers are effectively conveyed to upper layers. We dub this architecture Cell-aware Stacked LSTM (CAS-LSTM) and show from experiments that our models bring significant performance gain over the standard LSTMs on benchmark datasets for natural language inference, paraphrase detection, sentiment classification, and machine translation. We also conduct extensive qualitative analysis to understand the internal behavior of the suggested approach.
Scaling Video Analytics Systems to Large Camera Deployments
Venue: arXiv cs.DC 2019
Year: 2019

Driven by advances in computer vision and the falling costs of camera hardware, organizations are deploying video cameras en masse for the spatial monitoring of their physical premises. Scaling video analytics to massive camera deployments, however, presents a new and mounting challenge, as compute cost grows proportionally to the number of camera feeds. This paper is driven by a simple question: can we scale video analytics in such a way that cost grows sublinearly, or even remains constant, as we deploy more cameras, while inference accuracy remains stable, or even improves. We believe the answer is yes. Our key observation is that video feeds from wide-area camera deployments demonstrate significant content correlations (e.g. to other geographically proximate feeds), both in space and over time. These spatio-temporal correlations can be harnessed to dramatically reduce the size of the inference search space, decreasing both workload and false positive rates in multi-camera video analytics. By discussing use-cases and technical challenges, we propose a roadmap for scaling video analytics to large camera networks, and outline a plan for its realization.
Information-Theoretic Active Learning for Content-Based Image Retrieval
Venue: arXiv cs.CV 2019
Year: 2019

We propose Information-Theoretic Active Learning (ITAL), a novel batch-mode active learning method for binary classification, and apply it for acquiring meaningful user feedback in the context of content-based image retrieval. Instead of combining different heuristics such as uncertainty, diversity, or density, our method is based on maximizing the mutual information between the predicted relevance of the images and the expected user feedback regarding the selected batch. We propose suitable approximations to this computationally demanding problem and also integrate an explicit model of user behavior that accounts for possible incorrect labels and unnameable instances. Furthermore, our approach does not only take the structure of the data but also the expected model output change caused by the user feedback into account. In contrast to other methods, ITAL turns out to be highly flexible and provides state-of-the-art performance across various datasets, such as MIRFLICKR and ImageNet.
Trick Me If You Can: Human-in-the-loop Generation of Adversarial Examples for Question Answering
Venue: arXiv cs.CL 2019
Year: 2019

Adversarial evaluation stress tests a model's understanding of natural language. While past approaches expose superficial patterns, the resulting adversarial examples are limited in complexity and diversity. We propose human-in-the-loop adversarial generation, where human authors are guided to break models. We aid the authors with interpretations of model predictions through an interactive user interface. We apply this generation framework to a question answering task called Quizbowl, where trivia enthusiasts craft adversarial questions. The resulting questions are validated via live human--computer matches: although the questions appear ordinary to humans, they systematically stump neural and information retrieval models. The adversarial questions cover diverse phenomena from multi-hop reasoning to entity type distractors, exposing open challenges in robust question answering.
On Coding over Sliced Information
Venue: arXiv cs.IT 2019
Year: 2019

The interest in channel models in which the data is sent as an unordered set of binary strings has increased lately, due to emerging applications in DNA storage, among others. In this paper we analyze the minimal redundancy of binary codes for this channel under substitution errors, and provide several constructions, some of which are shown to be asymptotically optimal up to constants. The surprising result in this paper is that while the information vector is sliced into a set of unordered strings, the amount of redundant bits that are required to correct errors is order-wise equivalent to the amount required in the classical error correcting paradigm.
Faithful Multimodal Explanation for Visual Question Answering
Venue: arXiv cs.CL 2019
Year: 2019

AI systems' ability to explain their reasoning is critical to their utility and trustworthiness. Deep neural networks have enabled significant progress on many challenging problems such as visual question answering (VQA). However, most of them are opaque black boxes with limited explanatory capability. This paper presents a novel approach to developing a high-performing VQA system that can elucidate its answers with integrated textual and visual explanations that faithfully reflect important aspects of its underlying reasoning while capturing the style of comprehensible human explanations. Extensive experimental evaluation demonstrates the advantages of this approach compared to competing methods with both automatic evaluation metrics and human evaluation metrics.
SOTorrent: Studying the Origin, Evolution, and Usage of Stack Overflow Code Snippets
Venue: arXiv cs.SE 2019
Year: 2019

Stack Overflow (SO) is the most popular question-and-answer website for software developers, providing a large amount of copyable code snippets. Like other software artifacts, code on SO evolves over time, for example when bugs are fixed or APIs are updated to the most recent version. To be able to analyze how code and the surrounding text on SO evolves, we built SOTorrent, an open dataset based on the official SO data dump. SOTorrent provides access to the version history of SO content at the level of whole posts and individual text and code blocks. It connects code snippets from SO posts to other platforms by aggregating URLs from surrounding text blocks and comments, and by collecting references from GitHub files to SO posts. Our vision is that researchers will use SOTorrent to investigate and understand the evolution and maintenance of code on SO and its relation to other platforms such as GitHub.
Efficient and Robust Parallel DNN Training through Model Parallelism on Multi-GPU Platform
Venue: arXiv cs.DC 2019
Year: 2019

The training process of Deep Neural Network (DNN) is compute-intensive, often taking days to weeks to train a DNN model. Therefore, parallel execution of DNN training on GPUs is a widely adopted approach to speed up the process nowadays. Due to the implementation simplicity, data parallelism is currently the most commonly used parallelization method. Nonetheless, data parallelism suffers from excessive inter-GPU communication overhead due to frequent weight synchronization among GPUs. Another approach is pipelined model parallelism, which partitions a DNN model among GPUs, and processes multiple mini-batches concurrently. This approach can significantly reduce inter-GPU communication cost compared to data parallelism. However, pipelined model parallelism faces the weight staleness issue; that is, gradients are computed with stale weights, leading to training instability and accuracy loss. In this paper, we present a pipelined model parallel execution method that enables high GPU utilization while maintaining robust training accuracy via a novel weight prediction technique, SpecTrain. Experimental results show that our proposal achieves up to 8.91x speedup compared to data parallelism on a 4-GPU platform while maintaining comparable model accuracy.
Evidence-based lean logic profiles for conceptual data modelling languages
Venue: arXiv cs.AI 2019
Year: 2019

Multiple logic-based reconstructions of conceptual data modelling languages such as EER, UML Class Diagrams, and ORM exist. They mainly cover various fragments of the languages and none are formalised such that the logic applies simultaneously for all three modelling language families as unifying mechanism. This hampers interchangeability, interoperability, and tooling support. In addition, due to the lack of a systematic design process of the logic used for the formalisation, hidden choices permeate the formalisations that have rendered them incompatible. We aim to address these problems, first, by structuring the logic design process in a methodological way. We generalise and extend the DSL design process to apply to logic language design more generally and, in particular, by incorporating an ontological analysis of language features in the process. Second, we specify minimal logic profiles availing of this extended process, including the ontological commitments embedded in the languages, of evidence gathered of language feature usage, and of computational complexity insights from Description Logics (DL). The profiles characterise the essential logic structure needed to handle the semantics of conceptual models, therewith enabling the development of interoperability tools. There is no known DL language that matches exactly the features of those profiles and the common core is small (in the tractable DL $\mathcal{ALNI}$). Although hardly any inconsistencies can be derived with the profiles, it is promising for scalable runtime use of conceptual data models.
Online Learning for Position-Aided Millimeter Wave Beam Training
Venue: arXiv cs.IT 2019
Year: 2019

Accurate beam alignment is essential for beam-based millimeter wave communications. Conventional beam sweeping solutions often have large overhead, which is unacceptable for mobile applications like vehicle-to-everything. Learning-based solutions that leverage sensor data like position to identify good beam directions are one approach to reduce the overhead. Most existing solutions, though, are supervised-learning where the training data is collected beforehand. In this paper, we use a multi-armed bandit framework to develop online learning algorithms for beam pair selection and refinement. The beam pair selection algorithm learns coarse beam directions in some predefined beam codebook, e.g., in discrete angles separated by the 3dB beamwidths. The beam refinement fine-tunes the identified directions to match the peak of the power angular spectrum at that position. The beam pair selection uses the upper confidence bound (UCB) with a newly proposed risk-aware feature, while the beam refinement uses a modified optimistic optimization algorithm. The proposed algorithms learn to recommend good beam pairs quickly. When using 16x16 arrays at both the transmitter and receiver, it can achieve on average 1dB gain over the exhaustive search (over 271x271 beam pairs) on the unrefined codebook within 100 time-steps with a training budget of only 30 beam pairs.
Fast and Efficient Information Transmission with Burst Spikes in Deep Spiking Neural Networks
Venue: arXiv cs.NE 2019
Year: 2019

The spiking neural networks (SNNs) are considered as one of the most promising artificial neural networks due to their energy efficient computing capability. Recently, conversion of a trained deep neural network to an SNN has improved the accuracy of deep SNNs. However, most of the previous studies have not achieved satisfactory results in terms of inference speed and energy efficiency. In this paper, we propose a fast and energy-efficient information transmission method with burst spikes and hybrid neural coding scheme in deep SNNs. Our experimental results showed the proposed methods can improve inference energy efficiency and shorten the latency.