OpenRaiser/Intern-Atlas huggingface dataset:
Adversarial Examples in Modern Machine Learning: A Review
Venue: arXiv cs.LG 2019
Year: 2019

Recent research has found that many families of machine learning models are vulnerable to adversarial examples: inputs that are specifically designed to cause the target model to produce erroneous outputs. In this survey, we focus on machine learning models in the visual domain, where methods for generating and detecting such examples have been most extensively studied. We explore a variety of adversarial attack methods that apply to image-space content, real world adversarial attacks, adversarial defenses, and the transferability property of adversarial examples. We also discuss strengths and weaknesses of various methods of adversarial attack and defense. Our aim is to provide an extensive coverage of the field, furnishing the reader with an intuitive understanding of the mechanics of adversarial attack and defense mechanisms and enlarging the community of researchers studying this fundamental set of problems.
Buffer-aware Wireless Scheduling based on Deep Reinforcement Learning
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, the downlink packet scheduling problem for cellular networks is modeled, which jointly optimizes throughput, fairness and packet drop rate. Two genie-aided heuristic search methods are employed to explore the solution space. A deep reinforcement learning (DRL) framework with A2C algorithm is proposed for the optimization problem. Several methods have been utilized in the framework to improve the sampling and training efficiency and to adapt the algorithm to a specific scheduling problem. Numerical results show that DRL outperforms the baseline algorithm and achieves similar performance as genie-aided methods without using the future information.
Topological Stability: a New Algorithm for Selecting The Nearest Neighbors in Non-Linear Dimensionality Reduction Techniques
Venue: arXiv cs.LG 2019
Year: 2019

In the machine learning field, dimensionality reduction is an important task. It mitigates the undesired properties of high-dimensional spaces to facilitate classification, compression, and visualization of high-dimensional data. During the last decade, researchers proposed many new (non-linear) techniques for dimensionality reduction. Most of these techniques are based on the intuition that data lies on or near a complex low-dimensional manifold that is embedded in the high-dimensional space. New techniques for dimensionality reduction aim at identifying and extracting the manifold from the high-dimensional space. Isomap is one of widely-used low-dimensional embedding methods, where geodesic distances on a weighted graph are incorporated with the classical scaling (metric multidimensional scaling). The Isomap chooses the nearest neighbours based on the distance only which causes bridges and topological instability. In this paper, we propose a new algorithm to choose the nearest neighbours to reduce the number of short-circuit errors and hence improves the topological stability. Because at any point on the manifold, that point and its nearest neighbours form a vector subspace and the orthogonal to that subspace is orthogonal to all vectors spans the vector subspace. The prposed algorithmuses the point itself and its two nearest neighbours to find the bases of the subspace and the orthogonal to that subspace which belongs to the orthogonal complementary subspace. The proposed algorithm then adds new points to the two nearest neighbours based on the distance and the angle between each new point and the orthogonal to the subspace. The superior performance of the new algorithm in choosing the nearest neighbours is confirmed through experimental work with several datasets.
Reliable THz Communications for Outdoor based Applications- Use Cases and Methods
Venue: arXiv cs.NI 2019
Year: 2019

Future (beyond 5G) wireless networks will demand high throughput and low latency and would benefit from greenfield, contiguous, and wider bandwidth, all of which THz spectrum can provide. Although THz has been envisioned to be deployed in an indoor setting, with proper enforcement and planning, we can draw a limited number of use cases for outdoor THz communication. THz can provide high capacity and ultra-high throughput but at the cost of high path loss and sensitivity to device orientation/mobility.. We identify scenarios where the use of the THz spectrum for an outdoor setting is justified and their critical operating parameters. We further categorize the applications based on the relative mobility between the access point (AP) and user equipment (UE). We present an approach for deploying THz on an outdoor framework by presenting preliminary technical parameter analysis for scenarios, like wireless backhaul, high-speed kiosks, and the aerial base station (ABS). Our preliminary analysis shows that the application for each of these scenarios is limited based on multiple parameters, such as distance, device mobility, device orientation, user geometry, antenna gain, and environment settings, which requires separate consideration and optimization.
Finding Social Media Trolls: Dynamic Keyword Selection Methods for Rapidly-Evolving Online Debates
Venue: arXiv cs.LG 2019
Year: 2019

Online harassment is a significant social problem. Prevention of online harassment requires rapid detection of harassing, offensive, and negative social media posts. In this paper, we propose the use of word embedding models to identify offensive and harassing social media messages in two aspects: detecting fast-changing topics for more effective data collection and representing word semantics in different domains. We demonstrate with preliminary results that using the GloVe (Global Vectors for Word Representation) model facilitates the discovery of new and relevant keywords to use for data collection and trolling detection. Our paper concludes with a discussion of a research agenda to further develop and test word embedding models for identification of social media harassment and trolling.
A Stable Variational Autoencoder for Text Modelling
Venue: arXiv cs.CL 2019
Year: 2019

Variational Autoencoder (VAE) is a powerful method for learning representations of high-dimensional data. However, VAEs can suffer from an issue known as latent variable collapse (or KL loss vanishing), where the posterior collapses to the prior and the model will ignore the latent codes in generative tasks. Such an issue is particularly prevalent when employing VAE-RNN architectures for text modelling (Bowman et al., 2016). In this paper, we present a simple architecture called holistic regularisation VAE (HR-VAE), which can effectively avoid latent variable collapse. Compared to existing VAE-RNN architectures, we show that our model can achieve much more stable training process and can generate text with significantly better quality.
SAVEHR: Self Attention Vector Representations for EHR based Personalized Chronic Disease Onset Prediction and Interpretability
Venue: arXiv cs.LG 2019
Year: 2019

Chronic disease progression is emerging as an important area of investment for healthcare providers. As the quantity and richness of available clinical data continue to increase along with advances in machine learning, there is great potential to advance our approaches to caring for patients. An ideal approach to this problem should generate good performance on at least three axes namely, a) perform across many clinical conditions without requiring deep clinical expertise or extensive data scientist effort, b) generalization across populations, and c) be explainable (model interpretability). We present SAVEHR, a self-attention based architecture on heterogeneous structured EHR data that achieves $>$ 0.51 AUC-PR and $>$ 0.87 AUC-ROC gains on predicting the onset of four clinical conditions (CHF, Kidney Failure, Diabetes and COPD) 15-months in advance, and transfers with high performance onto a new population. We demonstrate that SAVEHR model performs superior to ten baselines on all three axes stated formerly.
Learning to Communicate in Multi-Agent Reinforcement Learning : A Review
Venue: arXiv cs.LG 2019
Year: 2019

We consider the issue of multiple agents learning to communicate through reinforcement learning within partially observable environments, with a focus on information asymmetry in the second part of our work. We provide a review of the recent algorithms developed to improve the agents' policy by allowing the sharing of information between agents and the learning of communication strategies, with a focus on Deep Recurrent Q-Network-based models. We also describe recent efforts to interpret the languages generated by these agents and study their properties in an attempt to generate human-language-like sentences. We discuss the metrics used to evaluate the generated communication strategies and propose a novel entropy-based evaluation metric. Finally, we address the issue of the cost of communication and introduce the idea of an experimental setup to expose this cost in cooperative-competitive game.
Statistical Deformation Reconstruction Using Multi-organ Shape Features for Pancreatic Cancer Localization
Venue: arXiv cs.CV 2019
Year: 2019

Respiratory motion and the associated deformations of abdominal organs and tumors are essential information in clinical applications. However, inter- and intra-patient multi-organ deformations are complex and have not been statistically formulated, whereas single organ deformations have been widely studied. In this paper, we introduce a multi-organ deformation library and its application to deformation reconstruction based on the shape features of multiple abdominal organs. Statistical multi-organ motion/deformation models of the stomach, liver, left and right kidneys, and duodenum were generated by shape matching their region labels defined on four-dimensional computed tomography images. A total of 250 volumes were measured from 25 pancreatic cancer patients. This paper also proposes a per-region-based deformation learning using the reproducing kernel to predict the displacement of pancreatic cancer for adaptive radiotherapy. The experimental results show that the proposed concept estimates deformations better than general per-patient-based learning models and achieves a clinically acceptable estimation error with a mean distance of 1.2 $\pm$ 0.7 mm and a Hausdorff distance of 4.2 $\pm$ 2.3 mm throughout the respiratory motion.
Going Negative Online? -- A Study of Negative Advertising on Social Media
Venue: arXiv cs.SI 2019
Year: 2019

A growing number of empirical studies suggest that negative advertising is effective in campaigning, while the mechanisms are rarely mentioned. With the scandal of Cambridge Analytica and Russian intervention behind the Brexit and the 2016 presidential election, people have become aware of the political ads on social media and have pressured congress to restrict political advertising on social media. Following the related legislation, social media companies began disclosing their political ads archive for transparency during the summer of 2018 when the midterm election campaign was just beginning. This research collects the data of the related political ads in the context of the U.S. midterm elections since August to study the overall pattern of political ads on social media and uses sets of machine learning methods to conduct sentiment analysis on these ads to classify the negative ads. A novel approach is applied that uses AI image recognition to study the image data. Through data visualization, this research shows that negative advertising is still the minority, Republican advertisers and third party organizations are more likely to engage in negative advertising than their counterparts. Based on ordinal regressions, this study finds that anger evoked information-seeking is one of the main mechanisms causing negative ads to be more engaging and effective rather than the negative bias theory. Overall, this study provides a unique understanding of political advertising on social media by applying innovative data science methods. Further studies can extend the findings, methods, and datasets in this study, and several suggestions are given for future research.
HDDL -- A Language to Describe Hierarchical Planning Problems
Venue: arXiv cs.AI 2019
Year: 2019

The research in hierarchical planning has made considerable progress in the last few years. Many recent systems do not rely on hand-tailored advice anymore to find solutions, but are supposed to be domain-independent systems that come with sophisticated solving techniques. In principle, this development would make the comparison between systems easier (because the domains are not tailored to a single system anymore) and -- much more important -- also the integration into other systems, because the modeling process is less tedious (due to the lack of advice) and there is no (or less) commitment to a certain planning system the model is created for. However, these advantages are destroyed by the lack of a common input language and feature set supported by the different systems. In this paper, we propose an extension to PDDL, the description language used in non-hierarchical planning, to the needs of hierarchical planning systems. We restrict our language to a basic feature set shared by many recent systems, give an extension of PDDL's EBNF syntax definition, and discuss our extensions with respect to several planner-specific input languages from related work.
Compressive Transformers for Long-Range Sequence Modelling
Venue: arXiv cs.LG 2019
Year: 2019

We present the Compressive Transformer, an attentive sequence model which compresses past memories for long-range sequence learning. We find the Compressive Transformer obtains state-of-the-art language modelling results in the WikiText-103 and Enwik8 benchmarks, achieving 17.1 ppl and 0.97 bpc respectively. We also find it can model high-frequency speech effectively and can be used as a memory mechanism for RL, demonstrated on an object matching task. To promote the domain of long-range sequence learning, we propose a new open-vocabulary language modelling benchmark derived from books, PG-19.
An E2E Simulator for 5G NR Networks
Venue: arXiv cs.NI 2019
Year: 2019

As the specification of the new 5G NR standard proceeds inside 3GPP, the availability of a versatile, full-stack, End-to-End (E2E), and open source simulator becomes a necessity to extract insights from the recently approved 3GPP specifications. This paper presents an extension to ns-3, a well-known discrete-event network simulator, to support the NR Radio Access Network. The present work describes the design and implementation choices at the MAC and PHY layers, and it discusses a technical solution for managing different bandwidth parts. Finally, we present calibration results, according to 3GPP procedures, and we show how to get E2E performance indicators in a realistic deployment scenario, with special emphasis on the E2E latency.
Energy-Efficient UAV Backscatter Communication with Joint Trajectory Design and Resource Optimization
Venue: arXiv cs.IT 2019
Year: 2019

Backscatter communication which enables wireless-powered backscatter devices (BDs) to transmit information by reflecting incident signals, is an energy- and cost-efficient communication technology for Internet-of-Things. This paper considers an unmanned aerial vehicle (UAV)-assisted backscatter communication network (UBCN) consisting of multiple BDs and carrier emitters (CEs) on the ground as well as a UAV. A communicate-while-fly scheme is first designed, in which the BDs illuminated by their associated CEs transmit information to the flying UAV in a time-division-multiple-access manner. Considering the critical issue of the UAV's limited on-board energy and the CEs' transmission energy, we maximize the energy efficiency (EE) of the UBCN by jointly optimizing the UAV's trajectory, the BDs' scheduling, and the CEs' transmission power, subject to the BDs' throughput constraints and harvested energy constraints, as well as other practical constraints. Furthermore, we propose an iterative algorithm based on the block coordinated decent method to solve the formulated mixed-integer non-convex problem, in each iteration of which the variables are alternatively optimized by leveraging the cutting-plane technique, the Dinkelbach's method and the successive convex approximation technique. Also, the convergence and complexity of the proposed algorithm are analyzed. Finally, simulation results show that the proposed communicate-while-fly scheme achieves significant EE gains compared with the benchmark hover-and-fly scheme. Useful insights on the optimal trajectory design and resource allocation are also obtained.
Structured Sparsification of Gated Recurrent Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Recently, a lot of techniques were developed to sparsify the weights of neural networks and to remove networks' structure units, e.g. neurons. We adjust the existing sparsification approaches to the gated recurrent architectures. Specifically, in addition to the sparsification of weights and neurons, we propose sparsifying the preactivations of gates. This makes some gates constant and simplifies LSTM structure. We test our approach on the text classification and language modeling tasks. We observe that the resulting structure of gate sparsity depends on the task and connect the learned structure to the specifics of the particular tasks. Our method also improves neuron-wise compression of the model in most of the tasks.
An Effective Hit-or-Miss Layer Favoring Feature Interpretation as Learned Prototypes Deformations
Venue: arXiv cs.LG 2019
Year: 2019

Neural networks designed for the task of classification have become a commodity in recent years. Many works target the development of more effective networks, which results in a complexification of their architectures with more layers, multiple sub-networks, or even the combination of multiple classifiers, but this often comes at the expense of producing uninterpretable black boxes. In this paper, we redesign a simple capsule network to enable it to synthesize class-representative samples, called prototypes, by replacing the last layer with a novel Hit-or-Miss layer. This layer contains activated vectors, called capsules, that we train to hit or miss a fixed target capsule by tailoring a specific centripetal loss function. This possibility allows to develop a data augmentation step combining information from the data space and the feature space, resulting in a hybrid data augmentation process. We show that our network, named HitNet, is able to reach better performances than those reproduced with the initial CapsNet on several datasets, while allowing to visualize the nature of the features extracted as deformations of the prototypes, which provides a direct insight into the feature representation learned by the network .
Affective Computing for Large-Scale Heterogeneous Multimedia Data: A Survey
Venue: arXiv cs.MM 2019
Year: 2019

The wide popularity of digital photography and social networks has generated a rapidly growing volume of multimedia data (i.e., image, music, and video), resulting in a great demand for managing, retrieving, and understanding these data. Affective computing (AC) of these data can help to understand human behaviors and enable wide applications. In this article, we survey the state-of-the-art AC technologies comprehensively for large-scale heterogeneous multimedia data. We begin this survey by introducing the typical emotion representation models from psychology that are widely employed in AC. We briefly describe the available datasets for evaluating AC algorithms. We then summarize and compare the representative methods on AC of different multimedia types, i.e., images, music, videos, and multimodal data, with the focus on both handcrafted features-based methods and deep learning methods. Finally, we discuss some challenges and future directions for multimedia affective computing.
Multi-Purpose Aerial Drones for Network Coverage and Package Delivery
Venue: arXiv cs.IT 2019
Year: 2019

Unmanned aerial vehicles (UAVs) have become important in many applications including last-mile deliveries, surveillance and monitoring, and wireless networks. This paper aims to design UAV trajectories that simultaneously perform multiple tasks. We aim to design UAV trajectories that minimize package delivery time, and at the same time provide uniform coverage over a neighborhood area which is needed for applications such as network coverage or surveillance. We first consider multi-task UAVs for a simplified scenario where the neighborhood area is a circular region with the post office located at its center and the houses are assumed to be uniformly distributed on the circle boundary. We propose a trajectory process such that if according to which the drones move, a uniform coverage can be achieved while the delivery efficiency is still preserved. We then consider a more practical scenario in which the delivery destinations are arbitrarily distributed in an arbitrarily-shaped region. We also do not assume any restrictions on the package arrivals. We show that simultaneous uniform coverage and efficient package delivery is possible for such realistic scenarios. This is shown using both rigorous analyses as well as simulations.
TPM-FAIL: TPM meets Timing and Lattice Attacks
Venue: arXiv cs.CR 2019
Year: 2019

Trusted Platform Module (TPM) serves as a hardware-based root of trust that protects cryptographic keys from privileged system and physical adversaries. In this work, we perform a black-box timing analysis of TPM 2.0 devices deployed on commodity computers. Our analysis reveals that some of these devices feature secret-dependent execution times during signature generation based on elliptic curves. In particular, we discovered timing leakage on an Intel firmware-based TPM as well as a hardware TPM. We show how this information allows an attacker to apply lattice techniques to recover 256-bit private keys for ECDSA and ECSchnorr signatures. On Intel fTPM, our key recovery succeeds after about 1,300 observations and in less than two minutes. Similarly, we extract the private ECDSA key from a hardware TPM manufactured by STMicroelectronics, which is certified at Common Criteria (CC) EAL 4+, after fewer than 40,000 observations. We further highlight the impact of these vulnerabilities by demonstrating a remote attack against a StrongSwan IPsec VPN that uses a TPM to generate the digital signatures for authentication. In this attack, the remote client recovers the server's private authentication key by timing only 45,000 authentication handshakes via a network connection. The vulnerabilities we have uncovered emphasize the difficulty of correctly implementing known constant-time techniques, and show the importance of evolutionary testing and transparent evaluation of cryptographic implementations. Even certified devices that claim resistance against attacks require additional scrutiny by the community and industry, as we learn more about these attacks.