OpenRaiser/Intern-Atlas huggingface dataset:
Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

Non-Autoregressive Neural Machine Translation (NAT) achieves significant decoding speedup through generating target words independently and simultaneously. However, in the context of non-autoregressive translation, the word-level cross-entropy loss cannot model the target-side sequential dependency properly, leading to its weak correlation with the translation quality. As a result, NAT tends to generate influent translations with over-translation and under-translation errors. In this paper, we propose to train NAT to minimize the Bag-of-Ngrams (BoN) difference between the model output and the reference sentence. The bag-of-ngrams training objective is differentiable and can be efficiently calculated, which encourages NAT to capture the target-side sequential dependency and correlates well with the translation quality. We validate our approach on three translation tasks and show that our approach largely outperforms the NAT baseline by about 5.0 BLEU scores on WMT14 En$\leftrightarrow$De and about 2.5 BLEU scores on WMT16 En$\leftrightarrow$Ro.
Zero Knowledge Proof based authentication protocol using graph isomorphism
Venue: arXiv cs.CR 2019
Year: 2019

We live in an era of information and it is very important to handle the exchange of information. While sending data to an authorized source, we need to protect it from unauthorized sources, changes, and authentication. ZKP technique can be used in designing secure authentication systems that dont involve any direct exchange of information between the claimant and the verifier thus preventing any possible leak of personal information. We propose a Zero-Knowledge Proof (ZKP) algorithm based on isomorphic graphs. We suggest most of the computations should be carried out on the users' web browser without revealing the password to the server at any point in time. Instead, it will generate random graphs and their permutations based on the login ID and password.
Generating Diverse Translation by Manipulating Multi-Head Attention
Venue: arXiv cs.CL 2019
Year: 2019

Transformer model has been widely used on machine translation tasks and obtained state-of-the-art results. In this paper, we report an interesting phenomenon in its encoder-decoder multi-head attention: different attention heads of the final decoder layer align to different word translation candidates. We empirically verify this discovery and propose a method to generate diverse translations by manipulating heads. Furthermore, we make use of these diverse translations with the back-translation technique for better data augmentation. Experiment results show that our method generates diverse translations without severe drop in translation quality. Experiments also show that back-translation with these diverse translations could bring significant improvement on performance on translation tasks. An auxiliary experiment of conversation response generation task proves the effect of diversity as well.
Incorporating Textual Evidence in Visual Storytelling
Venue: arXiv cs.CL 2019
Year: 2019

Previous work on visual storytelling mainly focused on exploring image sequence as evidence for storytelling and neglected textual evidence for guiding story generation. Motivated by human storytelling process which recalls stories for familiar images, we exploit textual evidence from similar images to help generate coherent and meaningful stories. To pick the images which may provide textual experience, we propose a two-step ranking method based on image object recognition techniques. To utilize textual information, we design an extended Seq2Seq model with two-channel encoder and attention. Experiments on the VIST dataset show that our method outperforms state-of-the-art baseline models without heavy engineering.
Empirical Autopsy of Deep Video Captioning Frameworks
Venue: arXiv cs.CV 2019
Year: 2019

Contemporary deep learning based video captioning follows encoder-decoder framework. In encoder, visual features are extracted with 2D/3D Convolutional Neural Networks (CNNs) and a transformed version of those features is passed to the decoder. The decoder uses word embeddings and a language model to map visual features to natural language captions. Due to its composite nature, the encoder-decoder pipeline provides the freedom of multiple choices for each of its components, e.g the choices of CNNs models, feature transformations, word embeddings, and language models etc. Component selection can have drastic effects on the overall video captioning performance. However, current literature is void of any systematic investigation in this regard. This article fills this gap by providing the first thorough empirical analysis of the role that each major component plays in a contemporary video captioning pipeline. We perform extensive experiments by varying the constituent components of the video captioning framework, and quantify the performance gains that are possible by mere component selection. We use the popular MSVD dataset as the test-bed, and demonstrate that substantial performance gains are possible by careful selection of the constituent components without major changes to the pipeline itself. These results are expected to provide guiding principles for future research in the fast growing direction of video captioning.
Assessing Cyber-Physical Security in Industrial Control Systems
Venue: arXiv cs.CR 2019
Year: 2019

Over the last years, Industrial Control Systems (ICS) have become increasingly exposed to a wide range of cyber-physical threats. Efficient models and techniques able to capture their complex structure and identify critical cyber-physical components are therefore essential. AND/OR graphs have proven very useful in this context as they are able to semantically grasp intricate logical interdependencies among ICS components. However, identifying critical nodes in AND/OR graphs is an NP-complete problem. In addition, ICS settings normally involve various cyber and physical security measures that simultaneously protect multiple ICS components in overlapping manners, which makes this problem even harder. In this paper, we present an extended security metric based on AND/OR hypergraphs which efficiently identifies the set of critical ICS components and security measures that should be compromised, with minimum cost (effort) for an attacker, in order to disrupt the operation of vital ICS assets. Our approach relies on MAX-SAT techniques, which we have incorporated in META4ICS, a Java-based security metric analyser for ICS. We also provide a thorough performance evaluation that shows the feasibility of our method. Finally, we illustrate our methodology through a case study in which we analyse the security posture of a realistic Water Transport Network (WTN).
Accurate Hydrologic Modeling Using Less Information
Venue: arXiv cs.LG 2019
Year: 2019

Joint models are a common and important tool in the intersection of machine learning and the physical sciences, particularly in contexts where real-world measurements are scarce. Recent developments in rainfall-runoff modeling, one of the prime challenges in hydrology, show the value of a joint model with shared representation in this important context. However, current state-of-the-art models depend on detailed and reliable attributes characterizing each site to help the model differentiate correctly between the behavior of different sites. This dependency can present a challenge in data-poor regions. In this paper, we show that we can replace the need for such location-specific attributes with a completely data-driven learned embedding, and match previous state-of-the-art results with less information.
Time Varying Channel Tracking for Multi-UAV Wideband Communications with Beam Squint
Venue: arXiv cs.IT 2019
Year: 2019

Unmanned aerial vehicle (UAV) has become an appealing solution for a wide range of commercial and civilian applications because of its high mobility and flexible deployment. Due to the continuous UAV navigation, the channel between UAV and base station (BS) is subject to the Doppler effect. Meanwhile, when the BS is equipped with massive number of antennas, the non-negligible propagation delay across the array aperture would cause beam squint effect. In this paper, we first investigate the channel of UAV communications under both Doppler shift effect and beam squint effect. Then, we design a gridless compressed sensing (GCS) based channel tracking method, where the high dimension uplink channel can be derived by estimating a few physical parameters such as the direction of arrival (DOA), Doppler shift, and the complex gain information. Besides, with the Doppler shift reciprocity and angular reciprocity, the downlink channel can be derived by only one pilot symbol, which greatly decreases the downlink channel training overhead. Various simulation results are provided to verify the effectiveness of the proposed methods.
Beyond Max-SNR: Joint Encoding for Reconfigurable Intelligent Surfaces
Venue: arXiv cs.IT 2019
Year: 2019

A communication link aided by a Reconfigurable Intelligent Surface (RIS) is studied, in which the transmitter can control the state of the RIS via a finite-rate control link. Prior work mostly assumed a fixed RIS configuration irrespective of the transmitted information. In contrast, this work derives information-theoretic limits, and demonstrates that the capacity is achieved by a scheme that jointly encodes information in the transmitted signal as well as in the RIS configuration. In addition, a novel signaling strategy based on layered encoding is proposed that enables practical successive cancellation-type decoding at the receiver. Numerical experiments demonstrate that the standard max-SNR scheme that fixes the configuration of the RIS as to maximize the Signal-to-Noise Ratio (SNR) at the receiver is strictly suboptimal, and is outperformed by the proposed strategies at all practical SNR levels.
Insider threats in Cyber Security: The enemy within the gates
Venue: arXiv cs.CR 2019
Year: 2019

Insider threats have become reality for civilian firms such as Tesla, which experienced sabotage and intellectual property theft, and Capital One, which suffered from fraud. Even greater social impact was caused by the data breach at the US Department of Defense, perpetrated by well-known attackers Chelsea Manning and Edward Snowden, whose espionage and hacktivist activities are widely known. The dramatic increase of such incidents in recent years and the incalculable damage committed by insiders must serve as a warning for all members of the cyber security community. It is no longer acceptable to continue to underestimate the problem of insider threats. Firms, organizations, institutions and governments need to lead and embrace a cultural change in their security posture. Through the adoption of an Insider Threat Program that engages all the strategic branches (including HR, Legal, Information Assurance, Cyber Security and Intelligence), coordinated by the chief information security officer and supported by c-level executive, it is possible to implement a framework that can prevent, detect, and respond to disloyal and/or unintentional insider threats. Hence, defending your enterprise from insider threats is a vital part of information security best practices. It is essential that your company highly valuable classified data and assets are protected from its greatest threat: the enemy within the gates.
AdaFilter: Adaptive Filter Fine-tuning for Deep Transfer Learning
Venue: arXiv cs.CV 2019
Year: 2019

There is an increasing number of pre-trained deep neural network models. However, it is still unclear how to effectively use these models for a new task. Transfer learning, which aims to transfer knowledge from source tasks to a target task, is an effective solution to this problem. Fine-tuning is a popular transfer learning technique for deep neural networks where a few rounds of training are applied to the parameters of a pre-trained model to adapt them to a new task. Despite its popularity, in this paper, we show that fine-tuning suffers from several drawbacks. We propose an adaptive fine-tuning approach, called AdaFilter, which selects only a part of the convolutional filters in the pre-trained model to optimize on a per-example basis. We use a recurrent gated network to selectively fine-tune convolutional filters based on the activations of the previous layer. We experiment with 7 public image classification datasets and the results show that AdaFilter can reduce the average classification error of the standard fine-tuning by 2.54%.
Regularizing Neural Networks by Stochastically Training Layer Ensembles
Venue: arXiv cs.LG 2019
Year: 2019

Dropout and similar stochastic neural network regularization methods are often interpreted as implicitly averaging over a large ensemble of models. We propose STE (stochastically trained ensemble) layers, which enhance the averaging properties of such methods by training an ensemble of weight matrices with stochastic regularization while explicitly averaging outputs. This provides stronger regularization with no additional computational cost at test time. We show consistent improvement on various image classification tasks using standard network topologies.
Automatically Neutralizing Subjective Bias in Text
Venue: arXiv cs.CL 2019
Year: 2019

Texts like news, encyclopedias, and some social media strive for objectivity. Yet bias in the form of inappropriate subjectivity - introducing attitudes via framing, presupposing truth, and casting doubt - remains ubiquitous. This kind of bias erodes our collective trust and fuels social conflict. To address this issue, we introduce a novel testbed for natural language generation: automatically bringing inappropriately subjective text into a neutral point of view ("neutralizing" biased text). We also offer the first parallel corpus of biased language. The corpus contains 180,000 sentence pairs and originates from Wikipedia edits that removed various framings, presuppositions, and attitudes from biased sentences. Last, we propose two strong encoder-decoder baselines for the task. A straightforward yet opaque CONCURRENT system uses a BERT encoder to identify subjective words as part of the generation process. An interpretable and controllable MODULAR algorithm separates these steps, using (1) a BERT-based classifier to identify problematic words and (2) a novel join embedding through which the classifier can edit the hidden states of the encoder. Large-scale human evaluation across four domains (encyclopedias, news headlines, books, and political speeches) suggests that these algorithms are a first step towards the automatic identification and reduction of bias.
Leveraging Sensing at the Infrastructure for mmWave Communication
Venue: arXiv cs.IT 2019
Year: 2019

Vehicle-to-everything (V2X) communication in the mmWave band is one way to achieve high data-rates for applications like infotainment, cooperative perception, and augmented reality assisted driving etc. MmWave communication relies on large antennas arrays, and configuring these arrays poses high training overhead. In this article, we motivate the use of infrastructure mounted sensors (which will be part of future smart cities) for mmWave communication. We provide numerical and measurement results to demonstrate that information from these infrastructure sensors reduces the mmWave array configuration overhead. Finally, we outline future research directions to help materialize the use of infrastructure sensors for mmWave communication.
A Comparative Evaluation of SGM Variants (including a New Variant, tMGM) for Dense Stereo Matching
Venue: arXiv cs.CV 2019
Year: 2019

Our goal here is threefold: [1] To present a new dense-stereo matching algorithm, tMGM, that by combining the hierarchical logic of tSGM with the support structure of MGM achieves 6-8\% performance improvement over the baseline SGM (these performance numbers are posted under tMGM-16 in the Middlebury Benchmark V3 ); and [2] Through an exhaustive quantitative and qualitative comparative study, to compare how the major variants of the SGM approach to dense stereo matching, including the new tMGM, perform in the presence of: (a) illumination variations and shadows, (b) untextured or weakly textured regions, (c) repetitive patterns in the scene in the presence of large stereo rectification errors. [3] To present a novel DEM-Sculpting approach for estimating initial disparity search bounds for multi-date satellite stereo pairs. Based on our study, we have found that tMGM generally performs best with respect to all these data conditions. Both tSGM and MGM improve the density of stereo disparity maps and combining the two in tMGM makes it possible to accurately estimate the disparities at a significant number of pixels that would otherwise be declared invalid by SGM. The datasets we have used in our comparative evaluation include the Middlebury2014, KITTI2015, and ETH3D datasets and the satellite images over the San Fernando area from the MVS Challenge dataset.
Zero-Resource Cross-Lingual Named Entity Recognition
Venue: arXiv cs.CL 2019
Year: 2019

Recently, neural methods have achieved state-of-the-art (SOTA) results in Named Entity Recognition (NER) tasks for many languages without the need for manually crafted features. However, these models still require manually annotated training data, which is not available for many languages. In this paper, we propose an unsupervised cross-lingual NER model that can transfer NER knowledge from one language to another in a completely unsupervised way without relying on any bilingual dictionary or parallel data. Our model achieves this through word-level adversarial learning and augmented fine-tuning with parameter sharing and feature augmentation. Experiments on five different languages demonstrate the effectiveness of our approach, outperforming existing models by a good margin and setting a new SOTA for each language pair.
Differentiable Algorithm for Marginalising Changepoints
Venue: arXiv cs.LG 2019
Year: 2019

We present an algorithm for marginalising changepoints in time-series models that assume a fixed number of unknown changepoints. Our algorithm is differentiable with respect to its inputs, which are the values of latent random variables other than changepoints. Also, it runs in time O(mn) where n is the number of time steps and m the number of changepoints, an improvement over a naive marginalisation method with O(n^m) time complexity. We derive the algorithm by identifying quantities related to this marginalisation problem, showing that these quantities satisfy recursive relationships, and transforming the relationships to an algorithm via dynamic programming. Since our algorithm is differentiable, it can be applied to convert a model non-differentiable due to changepoints to a differentiable one, so that the resulting models can be analysed using gradient-based inference or learning techniques. We empirically show the effectiveness of our algorithm in this application by tackling the posterior inference problem on synthetic and real-world data.
Archipelago: A Scalable Low-Latency Serverless Platform
Venue: arXiv cs.DC 2019
Year: 2019

The increased use of micro-services to build web applications has spurred the rapid growth of Function-as-a-Service (FaaS) or serverless computing platforms. While FaaS simplifies provisioning and scaling for application developers, it introduces new challenges in resource management that need to be handled by the cloud provider. Our analysis of popular serverless workloads indicates that schedulers need to handle functions that are very short-lived, have unpredictable arrival patterns, and require expensive setup of sandboxes. The challenge of running a large number of such functions in a multi-tenant cluster makes existing scheduling frameworks unsuitable. We present Archipelago, a platform that enables low latency request execution in a multi-tenant serverless setting. Archipelago views each application as a DAG of functions, and every DAG in associated with a latency deadline. Archipelago achieves its per-DAG request latency goals by: (1) partitioning a given cluster into a number of smaller worker pools, and associating each pool with a semi-global scheduler (SGS), (2) using a latency-aware scheduler within each SGS along with proactive sandbox allocation to reduce overheads, and (3) using a load balancing layer to route requests for different DAGs to the appropriate SGS, and automatically scale the number of SGSs per DAG. Our testbed results show that Archipelago meets the latency deadline for more than 99% of realistic application request workloads, and reduces tail latencies by up to 36X compared to state-of-the-art serverless platforms.
Data Programming using Continuous and Quality-Guided Labeling Functions
Venue: arXiv cs.LG 2019
Year: 2019

Scarcity of labeled data is a bottleneck for supervised learning models. A paradigm that has evolved for dealing with this problem is data programming. An existing data programming paradigm allows human supervision to be provided as a set of discrete labeling functions (LF) that output possibly noisy labels to input instances and a generative modelfor consolidating the weak labels. We enhance and generalize this paradigm by supporting functions that output a continuous score (instead of a hard label) that noisily correlates with labels. We show across five applications that continuous LFs are more natural to program and lead to improved recall. We also show that accuracy of existing generative models is unstable with respect to initialization, training epochs, and learning rates. We give control to the data programmer to guide the training process by providing intuitive quality guides with each LF. We propose an elegant method of incorporating these guides into the generative model. Our overall method, called CAGE, makes the data programming paradigm more reliable than other tricks based on initialization, sign-penalties, or soft-accuracy constraints.
On the Robustness of Signal Characteristic-Based Sender Identification
Venue: arXiv cs.CR 2019
Year: 2019

Vehicles become more vulnerable to remote attackers in modern days due to their increasing connectivity and range of functionality. Such increased attack vectors enable adversaries to access a vehicle Electronic Control Unit (ECU). As of today in-vehicle access can cause drastic consequences, because the most commonly used in-vehicle bus technology, the Controller Area Network (CAN), lacks sender identification. With low limits on bandwidth and payload, as well as resource constrains on hardware, usage of cryptographic measures is limited. As an alternative, sender identification methods were presented, identifying the sending ECU on the basis of its analog message signal. While prior works showed promising results on the security and feasibility for those approaches, the potential changes in signals over a vehicle's lifetime have only been partly addressed. This paper closes this gap. We conduct a 4~months measurement campaign containing more than 80,000 frames from a real vehicle. The data reflects different driving situations, different seasons and weather conditions, a 19-week break, and a car repair altering the physical CAN properties. We demonstrate the impact of temperature dependencies, analyze the signal changes and define strategies for their handling. In the evaluation, the identification rate can be increased from 91.23% to 99.98% by a targeted updating of the system parameters. At the same time, the detection of intrusions can be improved from 76.83% to 99.74%, while no false positives occured during evaluation. Lastly, we show how to increase the overall performance of such systems by double monitoring the bus at different positions.