OpenRaiser/Intern-Atlas huggingface dataset:
Question Generation from Paragraphs: A Tale of Two Hierarchical Models
Venue: arXiv cs.CL 2019
Year: 2019

Automatic question generation from paragraphs is an important and challenging problem, particularly due to the long context from paragraphs. In this paper, we propose and study two hierarchical models for the task of question generation from paragraphs. Specifically, we propose (a) a novel hierarchical BiLSTM model with selective attention and (b) a novel hierarchical Transformer architecture, both of which learn hierarchical representations of paragraphs. We model a paragraph in terms of its constituent sentences, and a sentence in terms of its constituent words. While the introduction of the attention mechanism benefits the hierarchical BiLSTM model, the hierarchical Transformer, with its inherent attention and positional encoding mechanisms also performs better than flat transformer model. We conducted empirical evaluation on the widely used SQuAD and MS MARCO datasets using standard metrics. The results demonstrate the overall effectiveness of the hierarchical models over their flat counterparts. Qualitatively, our hierarchical models are able to generate fluent and relevant questions
Wreath Products of Distributive Forest Algebras
Venue: arXiv cs.FL 2019
Year: 2019

It is an open problem whether definability in Propositional Dynamic Logic (PDL) on forests is decidable. Based on an algebraic characterization by Bojańczyk, et. al.,(2012) in terms of forest algebras, Straubing (2013) described an approach to PDL based on a k-fold iterated distributive law. A proof that all languages satisfying such a k-fold iterated distributive law are in PDL would settle decidability of PDL. We solve this problem in the case k=2: All languages recognized by forest algebras satisfying a 2-fold iterated distributive law are in PDL. Furthermore, we show that this class is decidable. This provides a novel nontrivial decidable subclass of PDL, and demonstrates the viability of the proposed approach to deciding PDL in general.
An Annotation Scheme of A Large-scale Multi-party Dialogues Dataset for Discourse Parsing and Machine Comprehension
Venue: arXiv cs.CL 2019
Year: 2019

In this paper, we propose the scheme for annotating large-scale multi-party chat dialogues for discourse parsing and machine comprehension. The main goal of this project is to help understand multi-party dialogues. Our dataset is based on the Ubuntu Chat Corpus. For each multi-party dialogue, we annotate the discourse structure and question-answer pairs for dialogues. As we know, this is the first large scale corpus for multi-party dialogues discourse parsing, and we firstly propose the task for multi-party dialogues machine reading comprehension.
Transmit-Receive Generalized Spatial Modulation Based on Dual-layered MIMO Transmission
Venue: arXiv cs.IT 2019
Year: 2019

We propose a novel scheme for downlink multiuser multiple-input multiple-output (MIMO) systems, called dual-layered transmit-receive generalized spatial modulation (DL-TR-GSM). The proposed scheme is based on the concept of dual-layered transmission (DLT) which uses two receive antenna power levels instead of receive antenna activation/inactivation to transmit data in the receive spatial domain. Hence, in order to minimize the bit error rate (BER) for DL-TR-GSM, the optimal ratio between the two power levels is determined. To further characterize DL-TR-GSM, we fully derive the computational complexity and show a significant computational complexity reduction as well as a required hardware complexity reduction of DL-TR-GSM, compared to a state-of-the-art benchmark scheme. Simulation results confirm the performance advantages of DL-TR-GSM.
Attack Trees for Security and Privacy in Social Virtual Reality Learning Environments
Venue: arXiv cs.CR 2019
Year: 2019

Social Virtual Reality Learning Environment (VRLE) is a novel edge computing platform for collaboration amongst distributed users. Given that VRLEs are used for critical applications (e.g., special education, public safety training), it is important to ensure security and privacy issues. In this paper, we present a novel framework to obtain quantitative assessments of threats and vulnerabilities for VRLEs. Based on the use cases from an actual social VRLE viz., vSocial, we first model the security and privacy using the attack trees. Subsequently, these attack trees are converted into stochastic timed automata representations that allow for rigorous statistical model checking. Such an analysis helps us adopt pertinent design principles such as hardening, diversity and principle of least privilege to enhance the resilience of social VRLEs. Through experiments in a vSocial case study, we demonstrate the effectiveness of our attack tree modeling with a reduction of 26% in probability of loss of integrity (security) and 80% in privacy leakage (privacy) in before and after scenarios pertaining to the adoption of the design principles.
Vision-Based Lane-Changing Behavior Detection Using Deep Residual Neural Network
Venue: arXiv cs.CV 2019
Year: 2019

Accurate lane localization and lane change detection are crucial in advanced driver assistance systems and autonomous driving systems for safer and more efficient trajectory planning. Conventional localization devices such as Global Positioning System only provide road-level resolution for car navigation, which is incompetent to assist in lane-level decision making. The state of art technique for lane localization is to use Light Detection and Ranging sensors to correct the global localization error and achieve centimeter-level accuracy, but the real-time implementation and popularization for LiDAR is still limited by its computational burden and current cost. As a cost-effective alternative, vision-based lane change detection has been highly regarded for affordable autonomous vehicles to support lane-level localization. A deep learning-based computer vision system is developed to detect the lane change behavior using the images captured by a front-view camera mounted on the vehicle and data from the inertial measurement unit for highway driving. Testing results on real-world driving data have shown that the proposed method is robust with real-time working ability and could achieve around 87% lane change detection accuracy. Compared to the average human reaction to visual stimuli, the proposed computer vision system works 9 times faster, which makes it capable of helping make life-saving decisions in time.
Zero-Shot Paraphrase Generation with Multilingual Language Models
Venue: arXiv cs.CL 2019
Year: 2019

Leveraging multilingual parallel texts to automatically generate paraphrases has drawn much attention as size of high-quality paraphrase corpus is limited. Round-trip translation, also known as the pivoting method, is a typical approach to this end. However, we notice that the pivoting process involves multiple machine translation models and is likely to incur semantic drift during the two-step translations. In this paper, inspired by the Transformer-based language models, we propose a simple and unified paraphrasing model, which is purely trained on multilingual parallel data and can conduct zero-shot paraphrase generation in one step. Compared with the pivoting approach, paraphrases generated by our model is more semantically similar to the input sentence. Moreover, since our model shares the same architecture as GPT (Radford et al., 2018), we are able to pre-train the model on large-scale unparallel corpus, which further improves the fluency of the output sentences. In addition, we introduce the mechanism of denoising auto-encoder (DAE) to improve diversity and robustness of the model. Experimental results show that our model surpasses the pivoting method in terms of relevance, diversity, fluency and efficiency.
Analyzing Bias in Sensitive Personal Information Used to Train Financial Models
Venue: arXiv cs.CR 2019
Year: 2019

Bias in data can have unintended consequences that propagate to the design, development, and deployment of machine learning models. In the financial services sector, this can result in discrimination from certain financial instruments and services. At the same time, data privacy is of paramount importance, and recent data breaches have seen reputational damage for large institutions. Presented in this paper is a trusted model-lifecycle management platform that attempts to ensure consumer data protection, anonymization, and fairness. Specifically, we examine how datasets can be reproduced using deep learning techniques to effectively retain important statistical features in datasets whilst simultaneously protecting data privacy and enabling safe and secure sharing of sensitive personal information beyond the current state-of-practice.
Missing Features Reconstruction and Its Impact on Classification Accuracy
Venue: arXiv cs.LG 2019
Year: 2019

In real-world applications, we can encounter situations when a well-trained model has to be used to predict from a damaged dataset. The damage caused by missing or corrupted values can be either on the level of individual instances or on the level of entire features. Both situations have a negative impact on the usability of the model on such a dataset. This paper focuses on the scenario where entire features are missing which can be understood as a specific case of transfer learning. Our aim is to experimentally research the influence of various imputation methods on the performance of several classification models. The imputation impact is researched on a combination of traditional methods such as k-NN, linear regression, and MICE compared to modern imputation methods such as multi-layer perceptron (MLP) and gradient boosted trees (XGBT). For linear regression, MLP, and XGBT we also propose two approaches to using them for multiple features imputation. The experiments were performed on both real world and artificial datasets with continuous features where different numbers of features, varying from one feature to 50%, were missing. The results show that MICE and linear regression are generally good imputers regardless of the conditions. On the other hand, the performance of MLP and XGBT is strongly dataset dependent. Their performance is the best in some cases, but more often they perform worse than MICE or linear regression.
Bootstrapping Disjoint Datasets for Multilingual Multimodal Representation Learning
Venue: arXiv cs.CL 2019
Year: 2019

Recent work has highlighted the advantage of jointly learning grounded sentence representations from multiple languages. However, the data used in these studies has been limited to an aligned scenario: the same images annotated with sentences in multiple languages. We focus on the more realistic disjoint scenario in which there is no overlap between the images in multilingual image--caption datasets. We confirm that training with aligned data results in better grounded sentence representations than training with disjoint data, as measured by image--sentence retrieval performance. In order to close this gap in performance, we propose a pseudopairing method to generate synthetically aligned English--German--image triplets from the disjoint sets. The method works by first training a model on the disjoint data, and then creating new triples across datasets using sentence similarity under the learned model. Experiments show that pseudopairs improve image--sentence retrieval performance compared to disjoint training, despite requiring no external data or models. However, we do find that using an external machine translation model to generate the synthetic data sets results in better performance.
A Machine to Machine framework for the charging of Electric Autonomous Vehicles
Venue: arXiv cs.CY 2019
Year: 2019

Electric Autonomous Vehicles (EAVs) have gained increasing attention of industry, governments and scientific communities concerned about issues related to classic transportation including accidents and casualties, gas emissions and air pollution, intensive traffic and city viability. One of the aspects, however, that prevent a broader adoption of this technology is the need for human interference to charge EAVs, which is still mostly manual and time-consuming. This study approaches such a problem by introducing the Inno-EAV, an open-source charging framework for EAVs that employs machine-to-machine (M2M) distributed communication. The idea behind M2M is to have networked devices that can interact, exchange information and perform actions without any manual assistance of humans. The advantages of the Inno-EAV include the automation of charging processes and the collection of relevant data that can support better decision making in the spheres of energy distribution. In this paper, we present the software design of the framework, the development process, the emphasis on the distributed architecture and the networked communication, and we discuss the back-end database that is used to store information about car owners, cars, and charging stations.
Code-Mixed to Monolingual Translation Framework
Venue: arXiv cs.CL 2019
Year: 2019

The use of multilingualism in the new generation is widespread in the form of code-mixed data on social media, and therefore a robust translation system is required for catering to the monolingual users, as well as for easier comprehension by language processing models. In this work, we present a translation framework that uses a translation-transliteration strategy for translating code-mixed data into their equivalent monolingual instances. For converting the output to a more fluent form, it is reordered using a target language model. The most important advantage of the proposed framework is that it does not require a code-mixed to monolingual parallel corpus at any point. On testing the framework, it achieved BLEU and TER scores of 16.47 and 55.45, respectively. Since the proposed framework deals with various sub-modules, we dive deeper into the importance of each of them, analyze the errors and finally, discuss some improvement strategies.
Enforcing Encoder-Decoder Modularity in Sequence-to-Sequence Models
Venue: arXiv cs.CL 2019
Year: 2019

Inspired by modular software design principles of independence, interchangeability, and clarity of interface, we introduce a method for enforcing encoder-decoder modularity in seq2seq models without sacrificing the overall model quality or its full differentiability. We discretize the encoder output units into a predefined interpretable vocabulary space using the Connectionist Temporal Classification (CTC) loss. Our modular systems achieve near SOTA performance on the 300h Switchboard benchmark, with WER of 8.3% and 17.6% on the SWB and CH subsets, using seq2seq models with encoder and decoder modules which are independent and interchangeable.
A Robust Blind 3-D Mesh Watermarking based on Wavelet Transform for Copyright Protection
Venue: arXiv cs.MM 2019
Year: 2019

Nowadays, three-dimensional meshes have been extensively used in several applications such as, industrial, medical, computer-aided design (CAD) and entertainment due to the processing capability improvement of computers and the development of the network infrastructure. Unfortunately, like digital images and videos, 3-D meshes can be easily modified, duplicated and redistributed by unauthorized users. Digital watermarking came up while trying to solve this problem. In this paper, we propose a blind robust watermarking scheme for three-dimensional semiregular meshes for Copyright protection. The watermark is embedded by modifying the norm of the wavelet coefficient vectors associated with the lowest resolution level using the edge normal norms as synchronizing primitives. The experimental results show that in comparison with alternative 3-D mesh watermarking approaches, the proposed method can resist to a wide range of common attacks, such as similarity transformations including translation, rotation, uniform scaling and their combination, noise addition, Laplacian smoothing, quantization, while preserving high imperceptibility.
Synthesis of coordination programs from linear temporal logic
Venue: arXiv cs.PL 2019
Year: 2019

This paper presents a method for synthesizing a reactive program which coordinates the actions of a group of other reactive programs, so that the combined system satisfies a temporal specification of its desired long-term behavior. Traditionally, reactive synthesis has been applied to the construction of a stateful hardware circuit. This work is motivated by applications to other domains, such as the IoT (the Internet of Things) and robotics, where it is necessary to coordinate the actions of multiple sensors, devices, and robots. The mathematical model represents such entities as individual processes in Hoare's CSP model. Given a network of interacting entities, called an \emph{environment}, and a temporal specification of long-term behavior, the synthesis method constructs a \emph{coordinator} process (if one exists) that guides the actions of the environment entities so that the combined system is deadlock-free and satisfies the given specification. The main technical challenge is that a coordinator may have only \emph{partial knowledge} of the environment state, due to non-determinism within the environment, and environment actions that are hidden from the coordinator. This is the first method to handle both sources of partial knowledge, and to do so for arbitrary linear temporal logic specifications. It is shown that the coordination synthesis problem is \PSPACE-hard in the size of the environment. A prototype implementation is able to synthesize compact solutions for a number of coordination problems.
Modelling Bahdanau Attention using Election methods aided by Q-Learning
Venue: arXiv cs.LG 2019
Year: 2019

Neural Machine Translation has lately gained a lot of "attention" with the advent of more and more sophisticated but drastically improved models. Attention mechanism has proved to be a boon in this direction by providing weights to the input words, making it easy for the decoder to identify words representing the present context. But by and by, as newer attention models with more complexity came into development, they involved large computation, making inference slow. In this paper, we have modelled the attention network using techniques resonating with social choice theory. Along with that, the attention mechanism, being a Markov Decision Process, has been represented by reinforcement learning techniques. Thus, we propose to use an election method ($k$-Borda), fine-tuned using Q-learning, as a replacement for attention networks. The inference time for this network is less than a standard Bahdanau translator, and the results of the translation are comparable. This not only experimentally verifies the claims stated above but also helped provide a faster inference.
Semantic Noise Matters for Neural Natural Language Generation
Venue: arXiv cs.CL 2019
Year: 2019

Neural natural language generation (NNLG) systems are known for their pathological outputs, i.e. generating text which is unrelated to the input specification. In this paper, we show the impact of semantic noise on state-of-the-art NNLG models which implement different semantic control mechanisms. We find that cleaned data can improve semantic correctness by up to 97%, while maintaining fluency. We also find that the most common error is omitting information, rather than hallucination.
Language Model-Driven Unsupervised Neural Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

Unsupervised neural machine translation(NMT) is associated with noise and errors in synthetic data when executing vanilla back-translations. Here, we explicitly exploits language model(LM) to drive construction of an unsupervised NMT system. This features two steps. First, we initialize NMT models using synthetic data generated via temporary statistical machine translation(SMT). Second, unlike vanilla back-translation, we formulate a weight function, that scores synthetic data at each step of subsequent iterative training; this allows unsupervised training to an improved outcome. We present the detailed mathematical construction of our method. Experimental WMT2014 English-French, and WMT2016 English-German and English-Russian translation tasks revealed that our method outperforms the best prior systems by more than 3 BLEU points.
TSK-Streams: Learning TSK Fuzzy Systems on Data Streams
Venue: arXiv cs.LG 2019
Year: 2019

The problem of adaptive learning from evolving and possibly non-stationary data streams has attracted a lot of interest in machine learning in the recent past, and also stimulated research in related fields, such as computational intelligence and fuzzy systems. In particular, several rule-based methods for the incremental induction of regression models have been proposed. In this paper, we develop a method that combines the strengths of two existing approaches rooted in different learning paradigms. More concretely, our method adopts basic principles of the state-of-the-art learning algorithm AMRules and enriches them by the representational advantages of fuzzy rules. In a comprehensive experimental study, TSK-Streams is shown to be highly competitive in terms of performance.
Implementing geometric complexity theory: On the separation of orbit closures via symmetries
Venue: arXiv cs.CC 2019
Year: 2019

Understanding the difference between group orbits and their closures is a key difficulty in geometric complexity theory (GCT): While the GCT program is set up to separate certain orbit closures, many beautiful mathematical properties are only known for the group orbits, in particular close relations with symmetry groups and invariant spaces, while the orbit closures seem much more difficult to understand. However, in order to prove lower bounds in algebraic complexity theory, considering group orbits is not enough. In this paper we tighten the relationship between the orbit of the power sum polynomial and its closure, so that we can separate this orbit closure from the orbit closure of the product of variables by just considering the symmetry groups of both polynomials and their representation theoretic decomposition coefficients. In a natural way our construction yields a multiplicity obstruction that is neither an occurrence obstruction, nor a so-called vanishing ideal occurrence obstruction. All multiplicity obstructions so far have been of one of these two types. Our paper is the first implementation of the ambitious approach that was originally suggested in the first papers on geometric complexity theory by Mulmuley and Sohoni (SIAM J Comput 2001, 2008): Before our paper, all existence proofs of obstructions only took into account the symmetry group of one of the two polynomials (or tensors) that were to be separated. In our paper the multiplicity obstruction is obtained by comparing the representation theoretic decomposition coefficients of both symmetry groups. Our proof uses a semi-explicit description of the coordinate ring of the orbit closure of the power sum polynomial in terms of Young tableaux, which enables its comparison to the coordinate ring of the orbit.
Target tracking in the framework of possibility theory: The possibilistic Bernoulli filter
Venue: arXiv cs.IT 2019
Year: 2019

The Bernoulli filter is a Bayes filter for joint detection and tracking of a target in the presence of false and miss detections. This paper presents a mathematical formulation of the Bernoulli filter in the framework of possibility theory, where uncertainty is represented using {\em possibility} functions, rather than {\em probability} distributions. Possibility functions model the uncertainty in a non-additive manner, and have the capacity to deal with partial (incomplete) problem specification. Thus, the main advantage of the possibilistic Bernoulli filter, derived in this paper, is that it can operate even in the absence of precise measurement and/or dynamic model parameters. This feature of the proposed filter is demonstrated in the context of target tracking using multi-static Doppler shifts as measurements.