OpenRaiser/Intern-Atlas huggingface dataset:
Pilot Decontamination for Massive MIMO Network with UAVs
Venue: arXiv cs.IT 2020
Year: 2020

This letter studies the pilot contamination (PC) problem for massive multiple-input multiple-output (MIMO) networks with coexisting terrestrial users and unmanned aerial vehicles (UAVs). Due to the strong line-of-sight (LoS) air-to-ground channels between UAVs and base stations (BSs), UAVs usually cause a more severe PC issue as compared to the traditional terrestrial users. To mitigate the PC caused by UAVs, we propose a low-complexity distributed scheme by exploiting the full-dimensional beamforming of massive MIMO BSs and the angle-dependent LoS channels between them and high-altitude UAVs. Numerical results show the effectiveness of the proposed pilot decontamination scheme and the significant signal-to-interference-plus-noise ratio (SINR) gains in both the uplink and downlink after pilot decontamination.
UAV Secure Downlink NOMA Transmissions: A Secure Users Oriented Perspective
Venue: arXiv cs.IT 2020
Year: 2020

This paper proposes a secure downlink multi-user transmission scheme enabled by a flexible unmanned aerial vehicle base station (UAV-BS) and non-orthogonal multiple access (NOMA). According to their heterogeneous service requirements, multiple legitimate users are categorized as security-required users (SUs) and quality of service (QoS)-required users (QUs), while these QUs can potentially act as internal eavesdroppers which are curious about the secrecy transmissions of SUs. In such a context, our goal is to maximize the achievable minimum secrecy rate among SUs through the joint optimization of user scheduling, power allocation, and trajectory design, subject to the QoS requirements of QUs and the mobility constraint of UAV-BS. Due to the non-convexity of the problem, an efficient iterative algorithm is firstly proposed, based on the alternative optimization (AO) and successive convex approximation (SCA) methods and along with a penalty-based algorithm to deal with the introduced binary integer variables, to obtain a sub-optimal solution. Then, we propose an SUs-oriented low-complexity algorithm by taking advantage of the inherent characteristics of the optimization problem, which can efficiently reduce the computational complexity and can act as a reasonable initial solution for the previous iterative algorithm to achieve better performance. Finally, the superiority of our proposed scheme compared with the conventional orthogonal multiple access (OMA) one is validated by numerical simulation results.
Neural Network Activation Quantization with Bitwise Information Bottlenecks
Venue: arXiv cs.CV 2020
Year: 2020

Recent researches on information bottleneck shed new light on the continuous attempts to open the black box of neural signal encoding. Inspired by the problem of lossy signal compression for wireless communication, this paper presents a Bitwise Information Bottleneck approach for quantizing and encoding neural network activations. Based on the rate-distortion theory, the Bitwise Information Bottleneck attempts to determine the most significant bits in activation representation by assigning and approximating the sparse coefficient associated with each bit. Given the constraint of a limited average code rate, the information bottleneck minimizes the rate-distortion for optimal activation quantization in a flexible layer-by-layer manner. Experiments over ImageNet and other datasets show that, by minimizing the quantization rate-distortion of each layer, the neural network with information bottlenecks achieves the state-of-the-art accuracy with low-precision activation. Meanwhile, by reducing the code rate, the proposed method can improve the memory and computational efficiency by over six times compared with the deep neural network with standard single-precision representation. Codes will be available on GitHub when the paper is accepted \url{https://github.com/BitBottleneck/PublicCode}.
On the complexity of the universality and inclusion problems for unambiguous context-free grammars (technical report)
Venue: arXiv cs.FL 2020
Year: 2020

We study the computational complexity of universality and inclusion problems for unambiguous finite automata and context-free grammars. We observe that several such problems can be reduced to the universality problem for unambiguous context-free grammars. The latter problem has long been known to be decidable and we propose a PSPACE algorithm that works by reduction to the zeroness problem of recurrence equations with convolution. We are not aware of any non-trivial complexity lower bounds. However, we show that computing the coin-flip measure of an unambiguous context-free language, a quantitative generalisation of universality, is hard for the long-standing open problem SQRTSUM.
On Data Augmentation for GAN Training
Venue: arXiv cs.CV 2020
Year: 2020

Recent successes in Generative Adversarial Networks (GAN) have affirmed the importance of using more data in GAN training. Yet it is expensive to collect data in many domains such as medical applications. Data Augmentation (DA) has been applied in these applications. In this work, we first argue that the classical DA approach could mislead the generator to learn the distribution of the augmented data, which could be different from that of the original data. We then propose a principled framework, termed Data Augmentation Optimized for GAN (DAG), to enable the use of augmented data in GAN training to improve the learning of the original distribution. We provide theoretical analysis to show that using our proposed DAG aligns with the original GAN in minimizing the Jensen-Shannon (JS) divergence between the original distribution and model distribution. Importantly, the proposed DAG effectively leverages the augmented data to improve the learning of discriminator and generator. We conduct experiments to apply DAG to different GAN models: unconditional GAN, conditional GAN, self-supervised GAN and CycleGAN using datasets of natural images and medical images. The results show that DAG achieves consistent and considerable improvements across these models. Furthermore, when DAG is used in some GAN models, the system establishes state-of-the-art Frechet Inception Distance (FID) scores. Our code is available.
MeshWalker: Deep Mesh Understanding by Random Walks
Venue: arXiv cs.CV 2020
Year: 2020

Most attempts to represent 3D shapes for deep learning have focused on volumetric grids, multi-view images and point clouds. In this paper we look at the most popular representation of 3D shapes in computer graphics - a triangular mesh - and ask how it can be utilized within deep learning. The few attempts to answer this question propose to adapt convolutions & pooling to suit Convolutional Neural Networks (CNNs). This paper proposes a very different approach, termed MeshWalker, to learn the shape directly from a given mesh. The key idea is to represent the mesh by random walks along the surface, which "explore" the mesh's geometry and topology. Each walk is organized as a list of vertices, which in some manner imposes regularity on the mesh. The walk is fed into a Recurrent Neural Network (RNN) that "remembers" the history of the walk. We show that our approach achieves state-of-the-art results for two fundamental shape analysis tasks: shape classification and semantic segmentation. Furthermore, even a very small number of examples suffices for learning. This is highly important, since large datasets of meshes are difficult to acquire.
Combination of abstractive and extractive approaches for summarization of long scientific texts
Venue: arXiv cs.CL 2020
Year: 2020

In this research work, we present a method to generate summaries of long scientific documents that uses the advantages of both extractive and abstractive approaches. Before producing a summary in an abstractive manner, we perform the extractive step, which then is used for conditioning the abstractor module. We used pre-trained transformer-based language models, for both extractor and abstractor. Our experiments showed that using extractive and abstractive models jointly significantly improves summarization results and ROUGE scores.
Comparison of Citations Trends between the COVID-19 Pandemic and SARS-CoV, MERS-CoV, Ebola, Zika, Avian and Swine Influenza Epidemics
Venue: arXiv cs.DL 2020
Year: 2020

Objective: The novel coronavirus COVID-19 outbreak rapidly evolved into pandemic. Global research efforts focus on this topic and with the collaboration of the scientific journals publication industry produced more than 16,000 related published articles in PubMed within five months from the onset of the outbreak. Herein, a comparison of the COVID-19 citations in PubMed and Web of Science was performed with SARS-CoV, MERS-CoV, Ebola, Zika, avian and swine influenza epidemics. Methods: The citations were searched and collected using the disease terms and the date of publication restriction. The total number of PubMed citations and the HIV associated papers during the same chronological periods were examined in parallel. The journal category and country information of the publications were gathered from Web of Science. The collected data were statistically analyzed and compared. Results: Significant correlations were found between COVID-19 and MERS (CC=0.988; p=0.003; q=0.006), Ebola (CC=0.987; p=0.003; q=0.011), and SARS (CC=0.964; p=0.015; q=0.028) epidemics five-month pick of novel citations in PubMed. However, COVID-19 publications were accumulated earlier and in larger numbers than any other 21st century major communicable disease outbreak. Conclusion: The acceleration and the total number of COVID-19 publications represent an unprecedented landmark event in the medical library history. The immediate adoption of the fast-track peer-reviewing and publishing as well as the open access publication policies by the journal publishers are significant contributors to this bibliographic phenomenon.
A Machine Learning Early Warning System: Multicenter Validation in Brazilian Hospitals
Venue: arXiv cs.LG 2020
Year: 2020

Early recognition of clinical deterioration is one of the main steps for reducing inpatient morbidity and mortality. The challenging task of clinical deterioration identification in hospitals lies in the intense daily routines of healthcare practitioners, in the unconnected patient data stored in the Electronic Health Records (EHRs) and in the usage of low accuracy scores. Since hospital wards are given less attention compared to the Intensive Care Unit, ICU, we hypothesized that when a platform is connected to a stream of EHR, there would be a drastic improvement in dangerous situations awareness and could thus assist the healthcare team. With the application of machine learning, the system is capable to consider all patient's history and through the use of high-performing predictive models, an intelligent early warning system is enabled. In this work we used 121,089 medical encounters from six different hospitals and 7,540,389 data points, and we compared popular ward protocols with six different scalable machine learning methods (three are classic machine learning models, logistic and probabilistic-based models, and three gradient boosted models). The results showed an advantage in AUC (Area Under the Receiver Operating Characteristic Curve) of 25 percentage points in the best Machine Learning model result compared to the current state-of-the-art protocols. This is shown by the generalization of the algorithm with leave-one-group-out (AUC of 0.949) and the robustness through cross-validation (AUC of 0.961). We also perform experiments to compare several window sizes to justify the use of five patient timestamps. A sample dataset, experiments, and code are available for replicability purposes.
Guidelines for the Search Strategy to Update Systematic Literature Reviews in Software Engineering
Venue: arXiv cs.SE 2020
Year: 2020

Context: Systematic Literature Reviews (SLRs) have been adopted within Software Engineering (SE) for more than a decade to provide meaningful summaries of evidence on several topics. Many of these SLRs are now potentially not fully up-to-date, and there are no standard proposals on how to update SLRs in SE. Objective: The objective of this paper is to propose guidelines on how to best search for evidence when updating SLRs in SE, and to evaluate these guidelines using an SLR that was not employed during the formulation of the guidelines. Method: To propose our guidelines, we compare and discuss outcomes from applying different search strategies to identify primary studies in a published SLR, an SLR update, and two replications in the area of effort estimation. These guidelines are then evaluated using an SLR in the area of software ecosystems, its update and a replication. Results: The use of a single iteration forward snowballing with Google Scholar, and employing as a seed set the original SLR and its primary studies is the most cost-effective way to search for new evidence when updating SLRs. Furthermore, the importance of having more than one researcher involved in the selection of papers when applying the inclusion and exclusion criteria is highlighted through the results. Conclusions: Our proposed guidelines formulated based upon an effort estimation SLR, its update and two replications, were supported when using an SLR in the area of software ecosystems, its update and a replication. Therefore, we put forward that our guidelines ought to be adopted for updating SLRs in SE.
Resolution-Enhanced MRI-Guided Navigation of Spinal Cellular Injection Robot
Venue: arXiv cs.RO 2020
Year: 2020

This paper presents a method of navigating a surgical robot beyond the resolution of magnetic resonance imaging (MRI) by using a resolution enhancement technique enabled by high-precision piezoelectric actuation. The surgical robot was specifically designed for injecting stem cells into the spinal cord. This particular therapy can be performed in a shorter time by using a MRI-compatible robotic platform than by using a manual needle positioning platform. Imaging resolution of fiducial markers attached to the needle guide tubing was enhanced by reconstructing a high-resolution image from multiple images with sub-pixel movements of the robot. The parallel-plane direct-drive needle positioning mechanism positioned the needle guide with a high spatial precision that is two orders of magnitude higher than typical MRI resolution up to 1 mm. Reconstructed resolution enhanced images were used to navigate the robot precisely that would not have been possible by using standard MRI. Experiments were conducted to verify the effectiveness of the proposed enhanced-resolution image-guided intervention.
Causal Discovery from Incomplete Data using An Encoder and Reinforcement Learning
Venue: arXiv cs.LG 2020
Year: 2020

Discovering causal structure among a set of variables is a fundamental problem in many domains. However, state-of-the-art methods seldom consider the possibility that the observational data has missing values (incomplete data), which is ubiquitous in many real-world situations. The missing value will significantly impair the performance and even make the causal discovery algorithms fail. In this paper, we propose an approach to discover causal structures from incomplete data by using a novel encoder and reinforcement learning (RL). The encoder is designed for missing data imputation as well as feature extraction. In particular, it learns to encode the currently available information (with missing values) into a robust feature representation which is then used to determine where to search the best graph. The encoder is integrated into a RL framework that can be optimized using the actor-critic algorithm. Our method takes the incomplete observational data as input and generates a causal structure graph. Experimental results on synthetic and real data demonstrate that our method can robustly generate causal structures from incomplete data. Compared with the direct combination of data imputation and causal discovery methods, our method performs generally better and can even obtain a performance gain as much as 43.2%.
Machine Learning and Control Theory
Venue: arXiv cs.LG 2020
Year: 2020

We survey in this article the connections between Machine Learning and Control Theory. Control Theory provide useful concepts and tools for Machine Learning. Conversely Machine Learning can be used to solve large control problems. In the first part of the paper, we develop the connections between reinforcement learning and Markov Decision Processes, which are discrete time control problems. In the second part, we review the concept of supervised learning and the relation with static optimization. Deep learning which extends supervised learning, can be viewed as a control problem. In the third part, we present the links between stochastic gradient descent and mean-field theory. Conversely, in the fourth and fifth parts, we review machine learning approaches to stochastic control problems, and focus on the deterministic case, to explain, more easily, the numerical algorithms.
Deep Learning for Change Detection in Remote Sensing Images: Comprehensive Review and Meta-Analysis
Venue: arXiv cs.CV 2020
Year: 2020

Deep learning (DL) algorithms are considered as a methodology of choice for remote-sensing image analysis over the past few years. Due to its effective applications, deep learning has also been introduced for automatic change detection and achieved great success. The present study attempts to provide a comprehensive review and a meta-analysis of the recent progress in this subfield. Specifically, we first introduce the fundamentals of deep learning methods which arefrequently adopted for change detection. Secondly, we present the details of the meta-analysis conducted to examine the status of change detection DL studies. Then, we focus on deep learning-based change detection methodologies for remote sensing images by giving a general overview of the existing methods. Specifically, these deep learning-based methods were classified into three groups; fully supervised learning-based methods, fully unsupervised learning-based methods and transfer learning-based techniques. As a result of these investigations, promising new directions were identified for future research. This study will contribute in several ways to our understanding of deep learning for change detection and will provide a basis for further research.
Towards Jacamo-rest: A Resource-Oriented Abstraction for Managing Multi-Agent Systems
Venue: arXiv cs.MA 2020
Year: 2020

The Multi-Agent Oriented Programming (MAOP) paradigm provides abstractions to model and implements entities of agents, as well as of their organisations and environments. In recent years, researchers have started to explore the integration of MAOP and the resource-oriented web architecture (REST). This paper further advances this line of research by presenting an ongoing work on jacamo-rest, a resource-oriented web-based abstraction for the multi-agent programming platform JaCaMo. Jacamo-rest takes Multi-Agent System (MAS) interoperability to a new level, enabling MAS to not only interact with services or applications of the World Wide Web but also to be managed and updated in their specifications by other applications. To add a developer interface to JaCaMo that is suitable for the Web, we provide a novel conceptual perspective on the management of MAOP specification entities as web resources. We tested jacamo-rest using it as a middleware of a programming interface application that provides modern software engineering facilities such as continuous deployments and iterative software development for MAS.
Understanding Points of Correspondence between Sentences for Abstractive Summarization
Venue: arXiv cs.CL 2020
Year: 2020

Fusing sentences containing disparate content is a remarkable human ability that helps create informative and succinct summaries. Such a simple task for humans has remained challenging for modern abstractive summarizers, substantially restricting their applicability in real-world scenarios. In this paper, we present an investigation into fusing sentences drawn from a document by introducing the notion of points of correspondence, which are cohesive devices that tie any two sentences together into a coherent text. The types of points of correspondence are delineated by text cohesion theory, covering pronominal and nominal referencing, repetition and beyond. We create a dataset containing the documents, source and fusion sentences, and human annotations of points of correspondence between sentences. Our dataset bridges the gap between coreference resolution and summarization. It is publicly shared to serve as a basis for future work to measure the success of sentence fusion systems. (https://github.com/ucfnlp/points-of-correspondence)
Data Augmentation for Training Dialog Models Robust to Speech Recognition Errors
Venue: arXiv cs.CL 2020
Year: 2020

Speech-based virtual assistants, such as Amazon Alexa, Google assistant, and Apple Siri, typically convert users' audio signals to text data through automatic speech recognition (ASR) and feed the text to downstream dialog models for natural language understanding and response generation. The ASR output is error-prone; however, the downstream dialog models are often trained on error-free text data, making them sensitive to ASR errors during inference time. To bridge the gap and make dialog models more robust to ASR errors, we leverage an ASR error simulator to inject noise into the error-free text data, and subsequently train the dialog models with the augmented data. Compared to other approaches for handling ASR errors, such as using ASR lattice or end-to-end methods, our data augmentation approach does not require any modification to the ASR or downstream dialog models; our approach also does not introduce any additional latency during inference time. We perform extensive experiments on benchmark data and show that our approach improves the performance of downstream dialog models in the presence of ASR errors, and it is particularly effective in the low-resource situations where there are constraints on model size or the training data is scarce.
OpEvo: An Evolutionary Method for Tensor Operator Optimization
Venue: arXiv cs.LG 2020
Year: 2020

Training and inference efficiency of deep neural networks highly rely on the performance of tensor operators on hardware platforms. Manually optimizing tensor operators has limitations in terms of supporting new operators or hardware platforms. Therefore, automatically optimizing device code configurations of tensor operators is getting increasingly attractive. However, current methods for tensor operator optimization usually suffer from poor sample-efficiency due to the combinatorial search space. In this work, we propose a novel evolutionary method, OpEvo, which efficiently explores the search spaces of tensor operators by introducing a topology-aware mutation operation based on q-random walk to leverage the topological structures over the search spaces. Our comprehensive experiment results show that compared with state-of-the-art (SOTA) methods OpEvo can find the best configuration with the lowest variance and least efforts in the number of trials and wall-clock time. All code of this work is available online.
Position Masking for Language Models
Venue: arXiv cs.CL 2020
Year: 2020

Masked language modeling (MLM) pre-training models such as BERT corrupt the input by replacing some tokens with [MASK] and then train a model to reconstruct the original tokens. This is an effective technique which has led to good results on all NLP benchmarks. We propose to expand upon this idea by masking the positions of some tokens along with the masked input token ids. We follow the same standard approach as BERT masking a percentage of the tokens positions and then predicting their original values using an additional fully connected classifier stage. This approach has shown good performance gains (.3\% improvement) for the SQUAD additional improvement in convergence times. For the Graphcore IPU the convergence of BERT Base with position masking requires only 50\% of the tokens from the original BERT paper.