OpenRaiser/Intern-Atlas huggingface dataset:
Exploring and Predicting Transferability across NLP Tasks
Venue: arXiv cs.CL 2020
Year: 2020

Recent advances in NLP demonstrate the effectiveness of training large-scale language models and transferring them to downstream tasks. Can fine-tuning these models on tasks other than language modeling further improve performance? In this paper, we conduct an extensive study of the transferability between 33 NLP tasks across three broad classes of problems (text classification, question answering, and sequence labeling). Our results show that transfer learning is more beneficial than previously thought, especially when target task data is scarce, and can improve performance even when the source task is small or differs substantially from the target task (e.g., part-of-speech tagging transfers well to the DROP QA dataset). We also develop task embeddings that can be used to predict the most transferable source tasks for a given target task, and we validate their effectiveness in experiments controlled for source and target data size. Overall, our experiments reveal that factors such as source data size, task and domain similarity, and task complexity all play a role in determining transferability.
KinGDOM: Knowledge-Guided DOMain adaptation for sentiment analysis
Venue: arXiv cs.CL 2020
Year: 2020

Cross-domain sentiment analysis has received significant attention in recent years, prompted by the need to combat the domain gap between different applications that make use of sentiment analysis. In this paper, we take a novel perspective on this task by exploring the role of external commonsense knowledge. We introduce a new framework, KinGDOM, which utilizes the ConceptNet knowledge graph to enrich the semantics of a document by providing both domain-specific and domain-general background concepts. These concepts are learned by training a graph convolutional autoencoder that leverages inter-domain concepts in a domain-invariant manner. Conditioning a popular domain-adversarial baseline method with these learned concepts helps improve its performance over state-of-the-art approaches, demonstrating the efficacy of our proposed framework.
Teaching Machine Comprehension with Compositional Explanations
Venue: arXiv cs.CL 2020
Year: 2020

Advances in machine reading comprehension (MRC) rely heavily on the collection of large scale human-annotated examples in the form of (question, paragraph, answer) triples. In contrast, humans are typically able to generalize with only a few examples, relying on deeper underlying world knowledge, linguistic sophistication, and/or simply superior deductive powers. In this paper, we focus on "teaching" machines reading comprehension, using a small number of semi-structured explanations that explicitly inform machines why answer spans are correct. We extract structured variables and rules from explanations and compose neural module teachers that annotate instances for training downstream MRC models. We use learnable neural modules and soft logic to handle linguistic variation and overcome sparse coverage; the modules are jointly optimized with the MRC model to improve final performance. On the SQuAD dataset, our proposed method achieves 70.14% F1 score with supervision from 26 explanations, comparable to plain supervised learning using 1,100 labeled instances, yielding a 12x speed up.
DroTrack: High-speed Drone-based Object Tracking Under Uncertainty
Venue: arXiv cs.CV 2020
Year: 2020

We present DroTrack, a high-speed visual single-object tracking framework for drone-captured video sequences. Most of the existing object tracking methods are designed to tackle well-known challenges, such as occlusion and cluttered backgrounds. The complex motion of drones, i.e., multiple degrees of freedom in three-dimensional space, causes high uncertainty. The uncertainty problem leads to inaccurate location predictions and fuzziness in scale estimations. DroTrack solves such issues by discovering the dependency between object representation and motion geometry. We implement an effective object segmentation based on Fuzzy C Means (FCM). We incorporate the spatial information into the membership function to cluster the most discriminative segments. We then enhance the object segmentation by using a pre-trained Convolution Neural Network (CNN) model. DroTrack also leverages the geometrical angular motion to estimate a reliable object scale. We discuss the experimental results and performance evaluation using two datasets of 51,462 drone-captured frames. The combination of the FCM segmentation and the angular scaling increased DroTrack precision by up to $9\%$ and decreased the centre location error by $162$ pixels on average. DroTrack outperforms all the high-speed trackers and achieves comparable results in comparison to deep learning trackers. DroTrack offers high frame rates up to 1000 frame per second (fps) with the best location precision, more than a set of state-of-the-art real-time trackers.
Transfer Learning-Based Received Power Prediction with Ray-tracing Simulation and Small Amount of Measurement Data
Venue: arXiv cs.NI 2020
Year: 2020

This paper proposes a method to predict received power in urban area deterministically, which can learn a prediction model from small amount of measurement data by a simulation-aided transfer learning and data augmentation. Recent development in machine learning such as artificial neural network (ANN) enables us to predict radio propagation and path loss accurately. However, training a high-performance ANN model requires a significant number of data, which are difficult to obtain in real environments. The main motivation for this work was to facilitate accurate prediction using small amount of measurement data. To this end, we propose a transfer learning-based prediction method with data augmentation. The proposed method pre-trains a prediction model using data generated from ray-tracing simulations, increases the number of data using simulation-assisted data augmentation, and then fine-tunes a model using the augmented data to fit the target environment. Experiments using Wi-Fi devices were conducted, and the results demonstrate that the proposed method predicts received power with 50% (or less) of the RMS error of conventional methods.
Towards Deep Learning Methods for Quality Assessment of Computer-Generated Imagery
Venue: arXiv cs.MM 2020
Year: 2020

Video gaming streaming services are growing rapidly due to new services such as passive video streaming, e.g. Twitch.tv, and cloud gaming, e.g. Nvidia Geforce Now. In contrast to traditional video content, gaming content has special characteristics such as extremely high motion for some games, special motion patterns, synthetic content and repetitive content, which makes the state-of-the-art video and image quality metrics perform weaker for this special computer generated content. In this paper, we outline our plan to build a deep learningbased quality metric for video gaming quality assessment. In addition, we present initial results by training the network based on VMAF values as a ground truth to give some insights on how to build a metric in future. The paper describes the method that is used to choose an appropriate Convolutional Neural Network architecture. Furthermore, we estimate the size of the required subjective quality dataset which achieves a sufficiently high performance. The results show that by taking around 5k images for training of the last six modules of Xception, we can obtain a relatively high performance metric to assess the quality of distorted video games.
Visualization of Diseases at Risk in the COVID-19 Literature
Venue: arXiv cs.IR 2020
Year: 2020

This paper presents a project, named VIDAR-19, able to extract automatically diseases from the CORD-19 dataset, and also diseases which might be considered as risk factors. The project relies on the ICD-11 classification of diseases maintained by the WHO. This nomenclature is used as a data source of the extraction mechanism, and also as the repository for the results. Developed for the COVID-19, the project has the ability to extract diseases at risk and to calculate relevant indicators. The outcome of the project is presented in a dashboard which enables the user to explore graphically diseases at risk which are put back in the classification hierarchy. Beyond the COVID-19, VIDAR has much broader applications and might be directly used for any corpus dealing with other pathologies.
Minerva: A Portable Machine Learning Microservice Framework for Traditional Enterprise SaaS Applications
Venue: arXiv cs.SE 2020
Year: 2020

In traditional SaaS enterprise applications, microservices are an essential ingredient to deploy machine learning (ML) models successfully. In general, microservices result in efficiencies in software service design, development, and delivery. As they become ubiquitous in the redesign of monolithic software, with the addition of machine learning, the traditional applications are also becoming increasingly intelligent. Here, we propose a portable ML microservice framework Minerva (microservices container for applied ML) as an efficient way to modularize and deploy intelligent microservices in traditional legacy SaaS applications suite, especially in the enterprise domain. We identify and discuss the needs, challenges and architecture to incorporate ML microservices in such applications. Minervas design for optimal integration with legacy applications using microservices architecture leveraging lightweight infrastructure accelerates deploying ML models in such applications.
Energy Efficient Neural Network Embedding in IoT over Passive Optical Networks
Venue: arXiv cs.NI 2020
Year: 2020

In the near future, IoT based application services are anticipated to collect massive amounts of data on which complex and diverse tasks are expected to be performed. Machine learning algorithms such as Artificial Neural Networks (ANN) are increasingly used in smart environments to predict the output for a given problem based on a set of tuning parameters as the input. To this end, we present an energy efficient neural network (EE-NN) service embedding framework for IoT based smart homes. The developed framework considers the idea of Service Oriented Architecture (SOA) to provide service abstraction for multiple complex modules of a NN which can be used by a higher application layer. We utilize Mixed Integer Linear Programming (MILP) to formulate the embedding problem to minimize the total power consumption of networking and processing simultaneously. The results of the MILP model show that our optimized NN can save up to 86% by embedding processing modules in IoT devices and up to 72% in fog nodes due to the limited capacity of IoT devices.
Electrically-Tunable Stochasticity for Spin-based Neuromorphic Circuits: Self-Adjusting to Variation
Venue: arXiv cs.ET 2020
Year: 2020

Energy-efficient methods are addressed for leveraging low energy barrier nanomagnetic devices within neuromorphic architectures. Using a Magnetoresistive Random Access Memory (MRAM) probabilistic device (p-bit) as the basis of neuronal structures in Deep Belief Networks (DBNs), the impact of reducing the Magnetic Tunnel Junction's (MTJ's) energy barrier is assessed and optimized for the resulting stochasticity present in the learning system. This can mitigate the process variation sensitivity of stochastic DBNs which encounter a sharp drop-off when energy barriers exceed near-zero kT. As evaluated for the MNIST dataset for energy barriers at near-zero kT to 2.0 kT in increments of 0.5 kT, it is shown that the stability factor changes by 5 orders of magnitude. The self-compensating circuit developed herein provides a compact, and low complexity approach to mitigating process variation impacts towards practical implementation and fabrication.
Millimeter-Wave Beam Search with Iterative Deactivation and Beam Shifting
Venue: arXiv cs.IT 2020
Year: 2020

Millimeter Wave (mmWave) communications rely on highly directional beams to combat severe propagation loss. In this paper, an adaptive beam search algorithm based on spatial scanning, called Iterative Deactivation and Beam Shifting (IDBS), is proposed for mmWave beam alignment. IDBS does not require advance information such as the Signal-to-Noise Ratio (SNR) and channel statistics, and matches the training overhead to the unknown SNR to achieve satisfactory performance. The algorithm works by gradually deactivating beams using a Bayesian probability criterion based on a uniform improper prior, where beam deactivation can be implemented with low-complexity operations that require computing a low-degree polynomial or a search through a look-up table. Numerical results confirm that IDBS adapts to different propagation scenarios such as line-of-sight and non-line-of-sight and to different SNRs. It can achieve better tradeoffs between training overhead and beam alignment accuracy than existing non-adaptive algorithms that have fixed training overheads.
Investigating the Effects of Robot Engagement Communication on Learning from Demonstration
Venue: arXiv cs.HC 2020
Year: 2020

Robot Learning from Demonstration (RLfD) is a technique for robots to derive policies from instructors' examples. Although the reciprocal effects of student engagement on teacher behavior are widely recognized in the educational community, it is unclear whether the same phenomenon holds true for RLfD. To fill this gap, we first design three types of robot engagement behavior (attention, imitation, and a hybrid of the two) based on the learning literature. We then conduct, in a simulation environment, a within-subject user study to investigate the impact of different robot engagement cues on humans compared to a "without-engagement" condition. Results suggest that engagement communication significantly changes the human's estimation of the robots' capability and significantly raises their expectation towards the learning outcomes, even though we do not run actual learning algorithms in the experiments. Moreover, imitation behavior affects humans more than attention does in all metrics, while their combination has the most profound influences on humans. We also find that communicating engagement via imitation or the combined behavior significantly improve humans' perception towards the quality of demonstrations, even if all demonstrations are of the same quality.
Autoencoders for strategic decision support
Venue: arXiv cs.LG 2020
Year: 2020

In the majority of executive domains, a notion of normality is involved in most strategic decisions. However, few data-driven tools that support strategic decision-making are available. We introduce and extend the use of autoencoders to provide strategically relevant granular feedback. A first experiment indicates that experts are inconsistent in their decision making, highlighting the need for strategic decision support. Furthermore, using two large industry-provided human resources datasets, the proposed solution is evaluated in terms of ranking accuracy, synergy with human experts, and dimension-level feedback. This three-point scheme is validated using (a) synthetic data, (b) the perspective of data quality, (c) blind expert validation, and (d) transparent expert evaluation. Our study confirms several principal weaknesses of human decision-making and stresses the importance of synergy between a model and humans. Moreover, unsupervised learning and in particular the autoencoder are shown to be valuable tools for strategic decision-making.
A Decoupled Learning Strategy for Massive Access Optimization in Cellular IoT Networks
Venue: arXiv cs.NI 2020
Year: 2020

Cellular-based networks are expected to offer connectivity for massive Internet of Things (mIoT) systems. However, their Random Access CHannel (RACH) procedure suffers from unreliability, due to the collision from the simultaneous massive access. Despite that this collision problem has been treated in existing RACH schemes, these schemes usually organize IoT devices' transmission and re-transmission along with fixed parameters, thus can hardly adapt to time-varying traffic patterns. Without adaptation, the RACH procedure easily suffers from high access delay, high energy consumption, or even access unavailability. With the goal of improving the RACH procedure, this paper targets to optimize the RACH procedure in real-time by maximizing a long-term hybrid multi-objective function, which consists of the number of access success devices, the average energy consumption, and the average access delay. To do so, we first optimize the long-term objective in the number of access success devices by using Deep Reinforcement Learning (DRL) algorithms for different RACH schemes, including Access Class Barring (ACB), Back-Off (BO), and Distributed Queuing (DQ). The converging capability and efficiency of different DRL algorithms including Policy Gradient (PG), Actor-Critic (AC), Deep Q-Network (DQN), and Deep Deterministic Policy Gradients (DDPG) are compared. Inspired by the results from this comparison, a decoupled learning strategy is developed to jointly and dynamically adapt the access control factors of those three access schemes. This decoupled strategy first leverage a Recurrent Neural Network (RNN) model to predict the real-time traffic values of the network environment, and then uses multiple DRL agents to cooperatively configure parameters of each RACH scheme.
Correcting the Autocorrect: Context-Aware Typographical Error Correction via Training Data Augmentation
Venue: arXiv cs.CL 2020
Year: 2020

In this paper, we explore the artificial generation of typographical errors based on real-world statistics. We first draw on a small set of annotated data to compute spelling error statistics. These are then invoked to introduce errors into substantially larger corpora. The generation methodology allows us to generate particularly challenging errors that require context-aware error detection. We use it to create a set of English language error detection and correction datasets. Finally, we examine the effectiveness of machine learning models for detecting and correcting errors based on this data. The datasets are available at http://typo.nlproc.org
Multi-cell Edge Coverage Enhancement Using Mobile UAV-Relay
Venue: arXiv cs.NI 2020
Year: 2020

Unmanned aerial vehicle (UAV)-assisted communication is a promising technology in future wireless communication networks. UAVs can not only help offload data traffic from ground base stations (GBSs), but also improve the quality of service of cell-edge users (CEUs). In this paper, we consider the enhancement of cell-edge communications through a mobile relay, i.e., UAV, in multi-cell networks. During each transmission period, GBSs first send data to the UAV, and then the UAV forwards its received data to CEUs according to a certain association strategy. In order to maximize the sum rate of all CEUs, we jointly optimize the UAV mobility management, including trajectory, velocity, and acceleration, and association strategy of CEUs to the UAV, subject to minimum rate requirements of CEUs, mobility constraints of the UAV and causal buffer constraints in practice. To address the mixed-integer nonconvex problem, we transform it into two convex subproblems by applying tight bounds and relaxations. An iterative algorithm was proposed to solve the two subproblems in an alternating manner. Numerical results show that the proposed algorithm achieves higher rates of CEUs as compared with existing benchmark schemes.
An empirical comparison of deep-neural-network architectures for next activity prediction using context-enriched process event logs
Venue: arXiv cs.LG 2020
Year: 2020

Researchers have proposed a variety of predictive business process monitoring (PBPM) techniques aiming to predict future process behaviour during the process execution. Especially, techniques for the next activity prediction anticipate great potential in improving operational business processes. To gain more accurate predictions, a plethora of these techniques rely on deep neural networks (DNNs) and consider information about the context, in which the process is running. However, an in-depth comparison of such techniques is missing in the PBPM literature, which prevents researchers and practitioners from selecting the best solution for a given event log. To remedy this problem, we empirically evaluate the predictive quality of three promising DNN architectures, combined with five proven encoding techniques and based on five context-enriched real-life event logs. We provide four findings that can support researchers and practitioners in designing novel PBPM techniques for predicting the next activities.
TIMELY: Pushing Data Movements and Interfaces in PIM Accelerators Towards Local and in Time Domain
Venue: arXiv cs.LG 2020
Year: 2020

Resistive-random-access-memory (ReRAM) based processing-in-memory (R$^2$PIM) accelerators show promise in bridging the gap between Internet of Thing devices' constrained resources and Convolutional/Deep Neural Networks' (CNNs/DNNs') prohibitive energy cost. Specifically, R$^2$PIM accelerators enhance energy efficiency by eliminating the cost of weight movements and improving the computational density through ReRAM's high density. However, the energy efficiency is still limited by the dominant energy cost of input and partial sum (Psum) movements and the cost of digital-to-analog (D/A) and analog-to-digital (A/D) interfaces. In this work, we identify three energy-saving opportunities in R$^2$PIM accelerators: analog data locality, time-domain interfacing, and input access reduction, and propose an innovative R$^2$PIM accelerator called TIMELY, with three key contributions: (1) TIMELY adopts analog local buffers (ALBs) within ReRAM crossbars to greatly enhance the data locality, minimizing the energy overheads of both input and Psum movements; (2) TIMELY largely reduces the energy of each single D/A (and A/D) conversion and the total number of conversions by using time-domain interfaces (TDIs) and the employed ALBs, respectively; (3) we develop an only-once input read (O$^2$IR) mapping method to further decrease the energy of input accesses and the number of D/A conversions. The evaluation with more than 10 CNN/DNN models and various chip configurations shows that, TIMELY outperforms the baseline R$^2$PIM accelerator, PRIME, by one order of magnitude in energy efficiency while maintaining better computational density (up to 31.2$\times$) and throughput (up to 736.6$\times$). Furthermore, comprehensive studies are performed to evaluate the effectiveness of the proposed ALB, TDI, and O$^2$IR innovations in terms of energy savings and area reduction.
Formal Verification of Solidity contracts in Event-B
Venue: arXiv cs.SE 2020
Year: 2020

Smart contracts are the artifact of the blockchain that provide immutable and verifiable specifications of physical transactions. Solidity is a domain-specific programming language with the purpose of defining smart contracts. It aims at reducing the transaction costs occasioned by the execution of contracts on the distributed ledgers such as the Ethereum. However, Solidity contracts need to adhere safety and security requirements that require formal verification and certification. This paper proposes a method to meet such requirements by translating Solidity contracts to Event-B models, supporting certification. To that purpose, we define a restrained Solidity subset and a transfer function which translates Solidity contracts to Event-B models. Then we take advantage of Event-B method capabilities to refine models at different levels of abstraction to verify Solidity contracts' properties. And we can verify the generated proof obligations of the Event-B model with the help of the Rodin platform.