OpenRaiser/Intern-Atlas huggingface dataset:
Machine Generation and Detection of Arabic Manipulated and Fake News
Venue: arXiv cs.CL 2020
Year: 2020

Fake news and deceptive machine-generated text are serious problems threatening modern societies, including in the Arab world. This motivates work on detecting false and manipulated stories online. However, a bottleneck for this research is lack of sufficient data to train detection models. We present a novel method for automatically generating Arabic manipulated (and potentially fake) news stories. Our method is simple and only depends on availability of true stories, which are abundant online, and a part of speech tagger (POS). To facilitate future work, we dispense with both of these requirements altogether by providing AraNews, a novel and large POS-tagged news dataset that can be used off-the-shelf. Using stories generated based on AraNews, we carry out a human annotation study that casts light on the effects of machine manipulation on text veracity. The study also measures human ability to detect Arabic machine manipulated text generated by our method. Finally, we develop the first models for detecting manipulated Arabic news and achieve state-of-the-art results on Arabic fake news detection (macro F1=70.06). Our models and data are publicly available.
Semi-supervised URL Segmentation with Recurrent Neural Networks Pre-trained on Knowledge Graph Entities
Venue: arXiv cs.CL 2020
Year: 2020

Breaking domain names such as openresearch into component words open and research is important for applications like Text-to-Speech synthesis and web search. We link this problem to the classic problem of Chinese word segmentation and show the effectiveness of a tagging model based on Recurrent Neural Networks (RNNs) using characters as input. To compensate for the lack of training data, we propose a pre-training method on concatenated entity names in a large knowledge database. Pre-training improves the model by 33% and brings the sequence accuracy to 85%.
GHFP: Gradually Hard Filter Pruning
Venue: arXiv cs.CV 2020
Year: 2020

Filter pruning is widely used to reduce the computation of deep learning, enabling the deployment of Deep Neural Networks (DNNs) in resource-limited devices. Conventional Hard Filter Pruning (HFP) method zeroizes pruned filters and stops updating them, thus reducing the search space of the model. On the contrary, Soft Filter Pruning (SFP) simply zeroizes pruned filters, keeping updating them in the following training epochs, thus maintaining the capacity of the network. However, SFP, together with its variants, converges much slower than HFP due to its larger search space. Our question is whether SFP-based methods and HFP can be combined to achieve better performance and speed up convergence. Firstly, we generalize SFP-based methods and HFP to analyze their characteristics. Then we propose a Gradually Hard Filter Pruning (GHFP) method to smoothly switch from SFP-based methods to HFP during training and pruning, thus maintaining a large search space at first, gradually reducing the capacity of the model to ensure a moderate convergence speed. Experimental results on CIFAR-10/100 show that our method achieves the state-of-the-art performance.
On Systematic Polarization-Adjusted Convolutional (PAC) Codes
Venue: arXiv cs.IT 2020
Year: 2020

Polarization-adjusted convolutional (PAC) codes were recently proposed and arouse the interest of the channel coding community because they were shown to approach theoretical bounds for the (128,64) code size. In this letter, we propose systematic PAC codes. Thanks to the systematic property, improvement in the bit-error rate of up to 0.2 dB is observed, while preserving the frame-error rate performance. Moreover, a genetic-algorithm based construction method targeted to approach the theoretical bound is provided. It is then shown that using the proposed construction method systematic and non-systematic PAC codes can approach the theoretical bound even for higher code sizes such as (256,128).
Web Application Attack Detection using Deep Learning
Venue: arXiv cs.CR 2020
Year: 2020

Modern web applications are dominated by HTTP/HTTPS messages that consist of one or more headers, where most of the exploits and payloads can be injected by attackers. According to the OWASP, the 80 percent of the web attacks are done through HTTP/HTTPS requests queries. In this paper, we present a deep learning based web application attacks detection model. The model uses auto-encoder that can learn from the sequences of word and weight each word or character according to them. The classification engine is trained on ECML-KDD dataset for classification of anomaly queries with respect to specific attack type. The proposed web application detection engine is trained with anomaly and benign web queries to achieve the accuracy of receiver operating characteristic curve of 1. The experimental results show that the proposed model can detect web applications attack successfully with low false positive rate.
Task-Graph Scheduling Extensions for Efficient Synchronization and Communication
Venue: arXiv cs.DC 2020
Year: 2020

Task graphs have been studied for decades as a foundation for scheduling irregular parallel applications and incorporated in programming models such as OpenMP. While many high-performance parallel libraries are based on task graphs, they also have additional scheduling requirements, such as synchronization from inner levels of data parallelism and internal blocking communications. In this paper, we extend task-graph scheduling to support efficient synchronization and communication within tasks. Our scheduler avoids deadlock and oversubscription of worker threads, and refines victim selection to increase the overlap of sibling tasks. To the best of our knowledge, our approach is the first to combine gang-scheduling and work-stealing in a single runtime. Our approach has been evaluated on the SLATE highperformance linear algebra library. Relative to the LLVM OMP runtime, our runtime demonstrates performance improvements of up to 13.82%, 15.2%, and 36.94% for LU, QR, and Cholesky, respectively, evaluated across different configurations.
Secure Performance Analysis and Optimization for FD-NOMA Vehicular Communications
Venue: arXiv cs.IT 2020
Year: 2020

Vehicle-to-vehicle (V2V) communication appeals to increasing research interest as a result of its applications to provide safety information as well as infotainment services. The increasing demand of transmit rates and various requirements of quality of services (QoS) in vehicular communication scenarios call for the integration of V2V communication systems and potential techniques in the future wireless communications, such as full duplex (FD) and non-orthogonal multiple access (NOMA) which enhance spectral efficiency and provide massive connectivity. However, the large amount of data transmission and user connectivity give rise to the concern of security issues and personal privacy. In order to analyze the security performance of V2V communications, we introduce a cooperative NOMA V2V system model with an FD relay. This paper focuses on the security performance of the FD-NOMA based V2V system on the physical layer perspective. We first derive several analytical results of the ergodic secrecy capacity. Then, we propose a secrecy sum rate optimization scheme utilizing the instantaneous channel state information (CSI), which is formulated as a non-convex optimization problem. Based on the differential structure of the non-convex constraints, the original problem is approximated and solved by a series of convex optimization problems. Simulation results validate the analytical results and the effectiveness of the secrecy sum rate optimization algorithm.
Resource-Constrained Federated Learning with Heterogeneous Labels and Models
Venue: arXiv cs.LG 2020
Year: 2020

Various IoT applications demand resource-constrained machine learning mechanisms for different applications such as pervasive healthcare, activity monitoring, speech recognition, real-time computer vision, etc. This necessitates us to leverage information from multiple devices with few communication overheads. Federated Learning proves to be an extremely viable option for distributed and collaborative machine learning. Particularly, on-device federated learning is an active area of research, however, there are a variety of challenges in addressing statistical (non-IID data) and model heterogeneities. In addition, in this paper we explore a new challenge of interest -- to handle label heterogeneities in federated learning. To this end, we propose a framework with simple $α$-weighted federated aggregation of scores which leverages overlapping information gain across labels, while saving bandwidth costs in the process. Empirical evaluation on Animals-10 dataset (with 4 labels for effective elucidation of results) indicates an average deterministic accuracy increase of at least ~16.7%. We also demonstrate the on-device capabilities of our proposed framework by experimenting with federated learning and inference across different iterations on a Raspberry Pi 2, a single-board computing platform.
Alquist 2.0: Alexa Prize Socialbot Based on Sub-Dialogue Models
Venue: arXiv cs.CL 2020
Year: 2020

This paper presents the second version of the dialogue system named Alquist competing in Amazon Alexa Prize 2018. We introduce a system leveraging ontology-based topic structure called topic nodes. Each of the nodes consists of several sub-dialogues, and each sub-dialogue has its own LSTM-based model for dialogue management. The sub-dialogues can be triggered according to the topic hierarchy or a user intent which allows the bot to create a unique experience during each session.
Semi-Supervised Low-Resource Style Transfer of Indonesian Informal to Formal Language with Iterative Forward-Translation
Venue: arXiv cs.CL 2020
Year: 2020

In its daily use, the Indonesian language is riddled with informality, that is, deviations from the standard in terms of vocabulary, spelling, and word order. On the other hand, current available Indonesian NLP models are typically developed with the standard Indonesian in mind. In this work, we address a style-transfer from informal to formal Indonesian as a low-resource machine translation problem. We build a new dataset of parallel sentences of informal Indonesian and its formal counterpart. We benchmark several strategies to perform style transfer from informal to formal Indonesian. We also explore augmenting the training set with artificial forward-translated data. Since we are dealing with an extremely low-resource setting, we find that a phrase-based machine translation approach outperforms the Transformer-based approach. Alternatively, a pre-trained GPT-2 fined-tuned to this task performed equally well but costs more computational resource. Our findings show a promising step towards leveraging machine translation models for style transfer. Our code and data are available in https://github.com/haryoa/stif-indonesia
Scalable Unsupervised Multi-Criteria Trajectory Segmentation and Driving Preference Mining
Venue: arXiv cs.RO 2020
Year: 2020

We present analysis techniques for large trajectory data sets that aim to provide a semantic understanding of trajectories reaching beyond them being point sequences in time and space. The presented techniques use a driving preference model w.r.t. road segment traversal costs, e.g., travel time and distance, to analyze and explain trajectories. In particular, we present trajectory mining techniques that can (a) find interesting points within a trajectory indicating, e.g., a via-point, and (b) recover the driving preferences of a driver based on their chosen trajectory. We evaluate our techniques on the tasks of via-point identification and personalized routing using a data set of more than 1 million vehicle trajectories collected throughout Denmark during a 3-year period. Our techniques can be implemented efficiently and are highly parallelizable, allowing them to scale to millions or billions of trajectories.
Deep learning architectures for inference of AC-OPF solutions
Venue: arXiv cs.LG 2020
Year: 2020

We present a systematic comparison between neural network (NN) architectures for inference of AC-OPF solutions. Using fully connected NNs as a baseline we demonstrate the efficacy of leveraging network topology in the models by constructing abstract representations of electrical grids in the graph domain, for both convolutional and graph NNs. The performance of the NN architectures is compared for regression (predicting optimal generator set-points) and classification (predicting the active set of constraints) settings. Computational gains for obtaining optimal solutions are also presented.
Explaining Differences in Classes of Discrete Sequences
Venue: arXiv cs.HC 2020
Year: 2020

While there are many machine learning methods to classify and cluster sequences, they fail to explain what are the differences in groups of sequences that make them distinguishable. Although in some cases having a black box model is sufficient, there is a need for increased explainability in research areas focused on human behaviors. For example, psychologists are less interested in having a model that predicts human behavior with high accuracy and more concerned with identifying differences between actions that lead to divergent human behavior. This paper presents techniques for understanding differences between classes of discrete sequences. Approaches introduced in this paper can be utilized to interpret black box machine learning models on sequences. The first approach compares k-gram representations of sequences using the silhouette score. The second method characterizes differences by analyzing the distance matrix of subsequences. As a case study, we trained black box supervised learning methods to classify sequences of GitHub teams and then utilized our sequence analysis techniques to measure and characterize differences between event sequences of teams with bots and teams without bots. In our second case study, we classified Minecraft event sequences to infer their high-level actions and analyzed differences between low-level event sequences of actions.
FDNAS: Improving Data Privacy and Model Diversity in AutoML
Venue: arXiv cs.LG 2020
Year: 2020

To prevent the leakage of private information while enabling automated machine intelligence, there is an emerging trend to integrate federated learning and Neural Architecture Search (NAS). Although promising as it may seem, the coupling of difficulties from both two tenets makes the algorithm development quite challenging. In particular, how to efficiently search the optimal neural architecture directly from massive non-iid data of clients in a federated manner remains to be a hard nut to crack. To tackle this challenge, in this paper, by leveraging the advances in proxy-less NAS, we propose a Federated Direct Neural Architecture Search (FDNAS) framework that allows hardware-aware NAS from decentralized non-iid data of clients. To further adapt for various data distributions of clients, inspired by meta-learning, a cluster Federated Direct Neural Architecture Search (CFDNAS) framework is proposed to achieve client-aware NAS, in the sense that each client can learn a tailored deep learning model for its particular data distribution. Extensive experiments on real-world non-iid datasets show state-of-the-art accuracy-efficiency trade-offs for various hardware and data distributions of clients. Our codes will be released publicly upon paper acceptance.
Joint Sensing and Communication over Memoryless Broadcast Channels
Venue: arXiv cs.IT 2020
Year: 2020

A memoryless state-dependent broadcast channel (BC) is considered, where the transmitter wishes to convey two private messages to two receivers while simultaneously estimating the respective states via generalized feedback. The model at hand is motivated by a joint radar and communication system where radar and data applications share the same frequency band. For physically degraded BCs with i.i.d. state sequences, we characterize the capacity-distortion tradeoff region. For general BCs, we provide inner and outer bounds on the capacitydistortion region, as well as a sufficient condition when it is equal to the product of the capacity region and the set of achievable distortion. Interestingly, the proposed synergetic design significantly outperforms a conventional approach that splits the resource either for sensing or communication.
C-ITS bundling for integrated traffic management
Venue: arXiv cs.OH 2020
Year: 2020

Cooperative Intelligent Transportation Systems (C-ITS) enable vehicles communication with each other (Vehicle-to-Vehicle, V2V) and with roadside infrastructure (Vehicle-to-Infrastructure, V2I). In the context of traffic efficiency, C-ITS technologies could assist in road network status visualization and monitoring, through data exchange, improving this way traffic control organization and traffic management implementation. Bundling is the provision of several C-ITS services as one combined service. The purpose of bundling is to harvest the usability of C-ITS services by developing a strategy for the operation and exploitation of services in real-time and within varying geographical areas. Two different dimensions of bundling have been recognized covering: 1) end-users, and 2) operators-managers. The objective of the operators-managers dimension is the integration of C-ITS services in operational traffic management. This work spotlights the operators-managers bundling dimension, presenting a framework based on a step-by-step approach for integrating C-ITS services in traditional traffic management.
Threats and Opportunities: Blockchain Meets Quantum Computation
Venue: arXiv cs.CR 2020
Year: 2020

This article considered deficiencies of the flourishing blockchain technology manifested by the development of quantum computation. We show that the future blockchain technology would under constant threats from the following aspects: 1) Speed up the generation of nonces; 2) Faster searching for hash collisions; 3) Break the security of the classical encryption. We also demonstrate that incorporating some quantum properties into blockchain makes it more robust and more efficient. For example people can establish a quantum-security blockchain system that utilizes quantum key distribution (QKD), and quantum synchronization and detectable Byzantine agreement (DBA) can help the blockchain systems achieve faster consensus even if there exist a number of malicious nodes.
An Unsupervised method for OCR Post-Correction and Spelling Normalisation for Finnish
Venue: arXiv cs.CL 2020
Year: 2020

Historical corpora are known to contain errors introduced by OCR (optical character recognition) methods used in the digitization process, often said to be degrading the performance of NLP systems. Correcting these errors manually is a time-consuming process and a great part of the automatic approaches have been relying on rules or supervised machine learning. We build on previous work on fully automatic unsupervised extraction of parallel data to train a character-based sequence-to-sequence NMT (neural machine translation) model to conduct OCR error correction designed for English, and adapt it to Finnish by proposing solutions that take the rich morphology of the language into account. Our new method shows increased performance while remaining fully unsupervised, with the added benefit of spelling normalisation. The source code and models are available on GitHub and Zenodo.
Large-scale multilingual audio visual dubbing
Venue: arXiv cs.CV 2020
Year: 2020

We describe a system for large-scale audiovisual translation and dubbing, which translates videos from one language to another. The source language's speech content is transcribed to text, translated, and automatically synthesized into target language speech using the original speaker's voice. The visual content is translated by synthesizing lip movements for the speaker to match the translated audio, creating a seamless audiovisual experience in the target language. The audio and visual translation subsystems each contain a large-scale generic synthesis model trained on thousands of hours of data in the corresponding domain. These generic models are fine-tuned to a specific speaker before translation, either using an auxiliary corpus of data from the target speaker, or using the video to be translated itself as the input to the fine-tuning process. This report gives an architectural overview of the full system, as well as an in-depth discussion of the video dubbing component. The role of the audio and text components in relation to the full system is outlined, but their design is not discussed in detail. Translated and dubbed demo videos generated using our system can be viewed at https://www.youtube.com/playlist?list=PLSi232j2ZA6_1Exhof5vndzyfbxAhhEs5
Contrastive Topographic Models: Energy-based density models applied to the understanding of sensory coding and cortical topography
Venue: arXiv cs.NE 2020
Year: 2020

We address the problem of building theoretical models that help elucidate the function of the visual brain at computational/algorithmic and structural/mechanistic levels. We seek to understand how the receptive fields and topographic maps found in visual cortical areas relate to underlying computational desiderata. We view the development of sensory systems from the popular perspective of probability density estimation; this is motivated by the notion that an effective internal representational scheme is likely to reflect the statistical structure of the environment in which an organism lives. We apply biologically based constraints on elements of the model. The thesis begins by surveying the relevant literature from the fields of neurobiology, theoretical neuroscience, and machine learning. After this review we present our main theoretical and algorithmic developments: we propose a class of probabilistic models, which we refer to as "energy-based models", and show equivalences between this framework and various other types of probabilistic model such as Markov random fields and factor graphs; we also develop and discuss approximate algorithms for performing maximum likelihood learning and inference in our energy based models. The rest of the thesis is then concerned with exploring specific instantiations of such models. By performing constrained optimisation of model parameters to maximise the likelihood of appropriate, naturalistic datasets we are able to qualitatively reproduce many of the receptive field and map properties found in vivo, whilst simultaneously learning about statistical regularities in the data.
Infer XPath
Venue: arXiv cs.PL 2020
Year: 2020

We propose reformulation of discovery of data structure within a web page as relations between sets of document nodes. We start by reformulating web page analysis as finding expressions in extension of XPath. Then we propose to automatically discover these XPath expressions with InferXPath meta-language. Our goal is to automate laborious process of conversion of manually created web pages that serve as software documentations, wikis, and reference documents, and speed up their conversion into tabular data that can be directly fed into data pipeline.
Highly Available Data Parallel ML training on Mesh Networks
Venue: arXiv cs.LG 2020
Year: 2020

Data parallel ML models can take several days or weeks to train on several accelerators. The long duration of training relies on the cluster of resources to be available for the job to keep running for the entire duration. On a mesh network this is challenging because failures will create holes in the mesh. Packets must be routed around the failed chips for full connectivity. In this paper, we present techniques to route gradient summation allreduce traffic around failed chips on 2-D meshes. We evaluate performance of our fault tolerant allreduce techniques via the MLPerf-v0.7 ResNet-50 and BERT benchmarks. Performance results show minimal impact to training throughput on 512 and 1024 TPU-v3 chips.