OpenRaiser/Intern-Atlas huggingface dataset:
Multitask Learning For Different Subword Segmentations In Neural Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

In Neural Machine Translation (NMT) the usage of subwords and characters as source and target units offers a simple and flexible solution for translation of rare and unseen words. However, selecting the optimal subword segmentation involves a trade-off between expressiveness and flexibility, and is language and dataset-dependent. We present Block Multitask Learning (BMTL), a novel NMT architecture that predicts multiple targets of different granularities simultaneously, removing the need to search for the optimal segmentation strategy. Our multi-task model exhibits improvements of up to 1.7 BLEU points on each decoder over single-task baseline models with the same number of parameters on datasets from two language pairs of IWSLT15 and one from IWSLT19. The multiple hypotheses generated at different granularities can be combined as a post-processing step to give better translations, which improves over hypothesis combination from baseline models while using substantially fewer parameters.
Landmark Ordinal Embedding
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we aim to learn a low-dimensional Euclidean representation from a set of constraints of the form "item j is closer to item i than item k". Existing approaches for this "ordinal embedding" problem require expensive optimization procedures, which cannot scale to handle increasingly larger datasets. To address this issue, we propose a landmark-based strategy, which we call Landmark Ordinal Embedding (LOE). Our approach trades off statistical efficiency for computational efficiency by exploiting the low-dimensionality of the latent embedding. We derive bounds establishing the statistical consistency of LOE under the popular Bradley-Terry-Luce noise model. Through a rigorous analysis of the computational complexity, we show that LOE is significantly more efficient than conventional ordinal embedding approaches as the number of items grows. We validate these characterizations empirically on both synthetic and real datasets. We also present a practical approach that achieves the "best of both worlds", by using LOE to warm-start existing methods that are more statistically efficient but computationally expensive.
Computational design of organic solar cell active layer through genetic algorithm
Venue: arXiv cs.NE 2019
Year: 2019

The active layer microstructure of organic solar cells is critical to efficiency. By studying the photovoltaic properties of organic solar cell's microstructure, it is possible to increase the efficiency of the solar cell. A graph-based microstructure model was employed to approximate the efficiency, measured as short circuit current, of a solar cell given a microstructure. Through probabilistic graph-based optimization, a class of microstructures were found with an efficiency surpassing that of more conventional morphologies. These optimized solar cells surpass the efficiency of more conventional photovoltaic devices as they better facilitate charge transport, generation, and dissociation. A device was designed with a 40.29% increase in short circuit current from the solar cells with the currently believed optimal morphology. The designed morphologies feature two dendritic clusters of the donor material poly(3-hexylthiophene-2,5-diyl) (P3HT) and the acceptor material phenyl-C61-Butyric-Acid-Methyl Ester (PCBM). The designed microstructure's increase in performance contrasts with more conventional structures featuring interdigitated or bilayer strands of P3HT and PCBM. The change of microstructure morphology through graph-based evolution obtains an organic solar cell with an efficiency significantly greater than conventional organic solar cells, proves the validity of graph-based microstructure models for simulation in materials science, and advances the vision of an inexpensive, efficient form of renewable energy.
Attention-Gated Graph Convolutions for Extracting Drug Interaction Information from Drug Labels
Venue: arXiv cs.CL 2019
Year: 2019

Preventable adverse events as a result of medical errors present a growing concern in the healthcare system. As drug-drug interactions (DDIs) may lead to preventable adverse events, being able to extract DDIs from drug labels into a machine-processable form is an important step toward effective dissemination of drug safety information. In this study, we tackle the problem of jointly extracting drugs and their interactions, including interaction outcome, from drug labels. Our deep learning approach entails composing various intermediate representations including sequence and graph based context, where the latter is derived using graph convolutions (GC) with a novel attention-based gating mechanism (holistically called GCA). These representations are then composed in meaningful ways to handle all subtasks jointly. To overcome scarcity in training data, we additionally propose transfer learning by pre-training on related DDI data. Our model is trained and evaluated on the 2018 TAC DDI corpus. Our GCA model in conjunction with transfer learning performs at 39.20% F1 and 26.09% F1 on entity recognition (ER) and relation extraction (RE) respectively on the first official test set and at 45.30% F1 and 27.87% F1 on ER and RE respectively on the second official test set corresponding to an improvement over our prior best results by up to 6 absolute F1 points. After controlling for available training data, our model exhibits state-of-the-art performance by improving over the next comparable best outcome by roughly three F1 points in ER and 1.5 F1 points in RE evaluation across two official test sets.
Human-AI Co-Learning for Data-Driven AI
Venue: arXiv cs.HC 2019
Year: 2019

Human and AI are increasingly interacting and collaborating to accomplish various complex tasks in the context of diverse application domains (e.g., healthcare, transportation, and creative design). Two dynamic, learning entities (AI and human) have distinct mental model, expertise, and ability; such fundamental difference/mismatch offers opportunities for bringing new perspectives to achieve better results. However, this mismatch can cause unexpected failure and result in serious consequences. While recent research has paid much attention to enhancing interpretability or explainability to allow machine to explain how it makes a decision for supporting humans, this research argues that there is urging the need for both human and AI should develop specific, corresponding ability to interact and collaborate with each other to form a human-AI team to accomplish superior results. This research introduces a conceptual framework called "Co-Learning," in which people can learn with/from and grow with AI partners over time. We characterize three key concepts of co-learning: "mutual understanding," "mutual benefits," and "mutual growth" for facilitating human-AI collaboration on complex problem solving. We will present proof-of-concepts to investigate whether and how our approach can help human-AI team to understand and benefit each other, and ultimately improve productivity and creativity on creative problem domains. The insights will contribute to the design of Human-AI collaboration.
A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media
Venue: arXiv cs.SI 2019
Year: 2019

Generated hateful and toxic content by a portion of users in social media is a rising phenomenon that motivated researchers to dedicate substantial efforts to the challenging direction of hateful content identification. We not only need an efficient automatic hate speech detection model based on advanced machine learning and natural language processing, but also a sufficiently large amount of annotated data to train a model. The lack of a sufficient amount of labelled hate speech data, along with the existing biases, has been the main issue in this domain of research. To address these needs, in this study we introduce a novel transfer learning approach based on an existing pre-trained language model called BERT (Bidirectional Encoder Representations from Transformers). More specifically, we investigate the ability of BERT at capturing hateful context within social media content by using new fine-tuning methods based on transfer learning. To evaluate our proposed approach, we use two publicly available datasets that have been annotated for racism, sexism, hate, or offensive content on Twitter. The results show that our solution obtains considerable performance on these datasets in terms of precision and recall in comparison to existing approaches. Consequently, our model can capture some biases in data annotation and collection process and can potentially lead us to a more accurate model.
FD-Net with Auxiliary Time Steps: Fast Prediction of PDEs using Hessian-Free Trust-Region Methods
Venue: arXiv cs.LG 2019
Year: 2019

Discovering the underlying physical behavior of complex systems is a crucial, but less well-understood topic in many engineering disciplines. This study proposes a finite-difference inspired convolutional neural network framework to learn hidden partial differential equations from given data and iteratively estimate future dynamical behavior. The methodology designs the filter sizes such that they mimic the finite difference between the neighboring points. By learning the governing equation, the network predicts the future evolution of the solution by using only a few trainable parameters. In this paper, we provide numerical results to compare the efficiency of the second-order Trust-Region Conjugate Gradient (TRCG) method with the first-order ADAM optimizer.
Large-Scale Characterization and Segmentation of Internet Path Delays with Infinite HMMs
Venue: arXiv cs.NI 2019
Year: 2019

Round-Trip Times are one of the most commonly collected performance metrics in computer networks. Measurement platforms such as RIPE Atlas provide researchers and network operators with an unprecedented amount of historical Internet delay measurements. It would be very useful to automate the processing of these measurements (statistical characterization of paths performance, change detection, recognition of recurring patterns, etc.). Humans are pretty good at finding patterns in network measurements but it can be difficult to automate this to enable many time series being processed at the same time. In this article we introduce a new model, the HDP-HMM or infinite hidden Markov model, whose performance in trace segmentation is very close to human cognition. This is obtained at the cost of a greater complexity and the ambition of this article is to make the theory accessible to network monitoring and management researchers. We demonstrate that this model provides very accurate results on a labeled dataset and on RIPE Atlas and CAIDA MANIC data. This method has been implemented in Atlas and we introduce the publicly accessible Web API.
Human Interface for Teleoperated Object Manipulation with a Soft Growing Robot
Venue: arXiv cs.RO 2019
Year: 2019

Soft growing robots are proposed for use in applications such as complex manipulation tasks or navigation in disaster scenarios. Safe interaction and ease of production promote the usage of this technology, but soft robots can be challenging to teleoperate due to their unique degrees of freedom. In this paper, we propose a human-centered interface that allows users to teleoperate a soft growing robot for manipulation tasks using arm movements. A study was conducted to assess the intuitiveness of the interface and the performance of our soft robot, involving a pick-and-place manipulation task. The results show that users completed the task with a success rate of 97%, achieving placement errors below 2 cm on average. These results demonstrate that our body-movement-based interface is an effective method for control of a soft growing robot manipulator.
De-homogenization of optimal multi-scale 3D topologies
Venue: arXiv cs.CE 2019
Year: 2019

This paper presents a highly efficient method to obtain high-resolution, near-optimal 3D topologies optimized for minimum compliance on a standard PC. Using an implicit geometry description we derive a single-scale interpretation of optimal multi-scale designs on a very fine mesh (de-homogenization). By performing homogenization-based topology optimization, optimal multi-scale designs are obtained on a relatively coarse mesh resulting in a low computational cost. As microstructure parameterization we use orthogonal rank-3 microstructures, which are known to be optimal for a single loading case. Furthermore, a method to get explicit control of the minimum feature size and complexity of the final shapes will be discussed. Numerical examples show excellent performance of these fine-scale designs resulting in objective values similar to the homogenization-based designs. Comparisons with well-established density-based topology optimization methods show a reduction in computational cost of 3 orders of magnitude, paving the way for giga-scale designs on a standard PC.
Neural Similarity Learning
Venue: arXiv cs.LG 2019
Year: 2019

Inner product-based convolution has been the founding stone of convolutional neural networks (CNNs), enabling end-to-end learning of visual representation. By generalizing inner product with a bilinear matrix, we propose the neural similarity which serves as a learnable parametric similarity measure for CNNs. Neural similarity naturally generalizes the convolution and enhances flexibility. Further, we consider the neural similarity learning (NSL) in order to learn the neural similarity adaptively from training data. Specifically, we propose two different ways of learning the neural similarity: static NSL and dynamic NSL. Interestingly, dynamic neural similarity makes the CNN become a dynamic inference network. By regularizing the bilinear matrix, NSL can be viewed as learning the shape of kernel and the similarity measure simultaneously. We further justify the effectiveness of NSL with a theoretical viewpoint. Most importantly, NSL shows promising performance in visual recognition and few-shot learning, validating the superiority of NSL over the inner product-based convolution counterparts.
Efficient Approximation Algorithms for Adaptive Target Profit Maximization
Venue: arXiv cs.SI 2019
Year: 2019

Given a social network $G$, the profit maximization (PM) problem asks for a set of seed nodes to maximize the profit, i.e., revenue of influence spread less the cost of seed selection. The target profit maximization (TPM) problem, which generalizes the PM problem, aims to select a subset of seed nodes from a target user set $T$ to maximize the profit. Existing algorithms for PM mostly consider the nonadaptive setting, where all seed nodes are selected in one batch without any knowledge on how they may influence other users. In this paper, we study TPM in adaptive setting, where the seed users are selected through multiple batches, such that the selection of a batch exploits the knowledge of actual influence in the previous batches. To acquire an overall understanding, we study the adaptive TPM problem under both the oracle model and the noise model, and propose ADG and AddATP algorithms to address them with strong theoretical guarantees, respectively. In addition, to better handle the sampling errors under the noise model, we propose the idea of hybrid error based on which we design a novel algorithm HATP that boosts the efficiency of AddATP significantly. We conduct extensive experiments on real social networks to evaluate the performance, and the experimental results strongly confirm the superiorities and effectiveness of our solutions.
Resource Allocation Using Gradient Boosting Aided Deep Q-Network for IoT in C-RANs
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we investigate dynamic resource allocation (DRA) problems for Internet of Things (IoT) in real-time cloud radio access networks (C-RANs), by combining gradient boosting approximation and deep reinforcement learning to solve the following two major problems. Firstly, in C-RANs, the decision making process of resource allocation is time-consuming and computational-expensive, motivating us to use an approximation method, i.e. the gradient boosting decision tree (GBDT) to approximate the solutions of second order cone programming (SOCP) problem. Moreover, considering the innumerable states in real-time C-RAN systems, we employ a deep reinforcement learning framework, i.e., deep Q-network (DQN) to generate a robust policy that controls the status of remote radio heads (RRHs). We propose a GBDT-based DQN framework for the DRA problem, where the heavy computation to solve SOCP problems is cut down and great power consumption is saved in the whole C-RAN system. We demonstrate that the generated policy is error-tolerant even the gradient boosting regression may not be strictly subject to the constraints of the original problem. Comparisons between the proposed method and existing baseline methods confirm the advantages of our method.
An Efficient Model for Sentiment Analysis of Electronic Product Reviews in Vietnamese
Venue: arXiv cs.CL 2019
Year: 2019

In the past few years, the growth of e-commerce and digital marketing in Vietnam has generated a huge volume of opinionated data. Analyzing those data would provide enterprises with insight for better business decisions. In this work, as part of the Advosights project, we study sentiment analysis of product reviews in Vietnamese. The final solution is based on Self-attention neural networks, a flexible architecture for text classification task with about 90.16% of accuracy in 0.0124 second, a very fast inference time.
LabPipe: an extensible informatics platform to streamline management of metabolomics data and metadata
Venue: arXiv cs.CE 2019
Year: 2019

Summary: Data management in clinical metabolomics studies is often inadequate. To improve this situation we created LabPipe to provide a guided, customisable approach to study-specific sample collection. It is driven through a local client which manages the process and pushes local data to a remote server through an access controlled web API. The platform is able to support data management for different sampling approaches across multiple sites / studies and is now an essential study management component for supporting clinical metabolomics locally at the EPSRC/MRC funded East Midlands Breathomics Pathology Node. Availability and Implementation: LabPipe is freely available to download under a non-commercial open-source license (NPOSL 3.0) along with documentation and installation instructions at http://labpipe.org. Contact: rob.free@le.ac.uk
Sentence Embeddings for Russian NLU
Venue: arXiv cs.CL 2019
Year: 2019

We investigate the performance of sentence embeddings models on several tasks for the Russian language. In our comparison, we include such tasks as multiple choice question answering, next sentence prediction, and paraphrase identification. We employ FastText embeddings as a baseline and compare it to ELMo and BERT embeddings. We conduct two series of experiments, using both unsupervised (i.e., based on similarity measure only) and supervised approaches for the tasks. Finally, we present datasets for multiple choice question answering and next sentence prediction in Russian.
DeepWiFi: Cognitive WiFi with Deep Learning
Venue: arXiv cs.NI 2019
Year: 2019

We present the DeepWiFi protocol, which hardens the baseline WiFi (IEEE 802.11ac) with deep learning and sustains high throughput by mitigating out-of-network interference. DeepWiFi is interoperable with baseline WiFi and builds upon the existing WiFi's PHY transceiver chain without changing the MAC frame format. Users run DeepWiFi for i) RF front end processing; ii) spectrum sensing and signal classification; iii) signal authentication; iv) channel selection and access; v) power control; vi) modulation and coding scheme (MCS) adaptation; and vii) routing. DeepWiFi mitigates the effects of probabilistic, sensing-based, and adaptive jammers. RF front end processing applies a deep learning-based autoencoder to extract spectrum-representative features. Then a deep neural network is trained to classify waveforms reliably as idle, WiFi, or jammer. Utilizing channel labels, users effectively access idle or jammed channels, while avoiding interference with legitimate WiFi transmissions (authenticated by machine learning-based RF fingerprinting) resulting in higher throughput. Users optimize their transmit power for low probability of intercept/detection and their MCS to maximize link rates used by backpressure algorithm for routing. Supported by embedded platform implementation, DeepWiFi provides major throughput gains compared to baseline WiFi and another jamming-resistant protocol, especially when channels are likely to be jammed and the signal-to-interference-plus-noise-ratio is low.
Zephyr: Hiding Metadata in a Messaging System
Venue: arXiv cs.CR 2019
Year: 2019

Private messaging over internet related services is difficult to implement. Regular end-to-end encryption messaging systems are prone to man in the middle attacks and only hide messages but not the identity of its users. For example, WhatsApp offers a strong privacy guarantee but does not hide much Metadata because it uses end-to-end encryption. Other messaging systems such as Skype can be monitored by government agencies and have backdoors implemented into its software. Zephyr is an anonymous messaging system that protects the privacy of message contents and message metadata. Users that use Zephyr do not reveal who they are talking to or the contents of their messages. The goal of Zephyr is to decrease the amount of information being sent by the user and hide as much metadata as possible.
Weakly-Supervised Deep Learning for Domain Invariant Sentiment Classification
Venue: arXiv cs.LG 2019
Year: 2019

The task of learning a sentiment classification model that adapts well to any target domain, different from the source domain, is a challenging problem. Majority of the existing approaches focus on learning a common representation by leveraging both source and target data during training. In this paper, we introduce a two-stage training procedure that leverages weakly supervised datasets for developing simple lift-and-shift-based predictive models without being exposed to the target domain during the training phase. Experimental results show that transfer with weak supervision from a source domain to various target domains provides performance very close to that obtained via supervised training on the target domain itself.
An Ultimate Approach of Mitigating Attacks in RPL Based Low Power Lossy Networks
Venue: arXiv cs.CR 2019
Year: 2019

The Routing Protocol for Low-Power and Lossy Networks (RPL) is the existing routing protocol for Internet of Things (IoT). RPL is a proactive,lightweight, Distance Vector protocol which offers security against various forms of routing attacks. Still, there are various attacks(as rank, version attacks and many more ) which is possible in this network due to problem of unauthenticated or unencrypted control frames, centralized root controller, compromised or unauthenticated devices and many more ways. There are various solutions present in the literature but every solution has its pros and cons. There is no appropriate system framework till now which completely solves these all issues. So, we present an ultimate approach to mitigate these RPL attacks more efficiently and effectively. We use IDS based system for internal attacks and a mini-firewall for removing the external attacks. In IDS based approach, we use intrusion detection system at multiple locations for analyzing the behaviour of nodes. The final decision whether the node is attacker or not depends on mainly three things as: trust between the neighbouring nodes, local decision by multiple sink nodes and global decision by root node. We also use some blockchain features in this framework for better internal security. We use some threshold values and rules in mini-firewall for removing external attacks. In this paper, we provide the proposed approach and theoretical analysis of this approach which provide better protection from these attacks than any other method.