pwc-archive/papers-with-abstracts huggingface dataset:
A supervised active learning method for identifying critical nodes in Wireless Sensor Network
Authors: Behnam Ojaghi, Mohammad Mahdi Dehshibi
Date: 2020-04-19 00:00:00
Tasks: Active Learning, Clustering

Energy Efficiency of a wireless sensor network (WSN) relies on its main characteristics, including hop-number, user's location, allocated power, and relay. Identifying nodes, which have more impact on these characteristics, is, however, subject to a substantial computational overhead and energy consumption. In this paper, we proposed an active learning approach to address the computational overhead of identifying critical nodes in a WSN. The proposed approach can overcome biasing in identifying non-critical nodes and needs much less effort in fine-tuning to adapt to the dynamic nature of WSN. This method benefits from the cooperation of clustering and classification modules to iteratively decrease the required number of data in a typical supervised learning scenario and to increase the accuracy in the presence of uninformative examples, i.e., non-critical nodes. Experiments show that the proposed method has more flexibility, compared to the state-of-the-art, to be employed in large scale WSN environments, the fifth-generation mobile networks (5G), and massively distributed IoT (i.e., sensor networks), where it can prolong the network lifetime.
Variational Policy Propagation for Multi-agent Reinforcement Learning
Authors: Chao Qu, Hui Li, Chang Liu, Junwu Xiong, James Zhang, Wei Chu, Weiqiang Wang, Yuan Qi, Le Song
Date: 2020-04-19 00:00:00
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Variational Inference

We propose a \emph{collaborative} multi-agent reinforcement learning algorithm named variational policy propagation (VPP) to learn a \emph{joint} policy through the interactions over agents. We prove that the joint policy is a Markov Random Field under some mild conditions, which in turn reduces the policy space effectively. We integrate the variational inference as special differentiable layers in policy such that the actions can be efficiently sampled from the Markov Random Field and the overall policy is differentiable. We evaluate our algorithm on several large scale challenging tasks and demonstrate that it outperforms previous state-of-the-arts.
Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation
Authors: Qianyu Zhou, Zhengyang Feng, Qiqi Gu, Guangliang Cheng, Xuequan Lu, Jianping Shi, Lizhuang Ma
Date: 2020-04-19 00:00:00
Tasks: Domain Adaptation, Semantic Segmentation, Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) aims to adapt existing models of the source domain to a new target domain with only unlabeled data. Most existing methods suffer from noticeable negative transfer resulting from either the error-prone discriminator network or the unreasonable teacher model. Besides, the local regional consistency in UDA has been largely neglected, and only extracting the global-level pattern information is not powerful enough for feature alignment due to the abuse use of contexts. To this end, we propose an uncertainty-aware consistency regularization method for cross-domain semantic segmentation. Firstly, we introduce an uncertainty-guided consistency loss with a dynamic weighting scheme by exploiting the latent uncertainty information of the target samples. As such, more meaningful and reliable knowledge from the teacher model can be transferred to the student model. We further reveal the reason why the current consistency regularization is often unstable in minimizing the domain discrepancy. Besides, we design a ClassDrop mask generation algorithm to produce strong class-wise perturbations. Guided by this mask, we propose a ClassOut strategy to realize effective regional consistency in a fine-grained manner. Experiments demonstrate that our method outperforms the state-of-the-art methods on four domain adaptation benchmarks, i.e., GTAV $\rightarrow $ Cityscapes and SYNTHIA $\rightarrow $ Cityscapes, Virtual KITTI $\rightarrow$ KITTI and Cityscapes $\rightarrow$ KITTI.
Superkernel Neural Architecture Search for Image Denoising
Authors: Marcin Możejko, Tomasz Latkowski, Łukasz Treszczotko, Michał Szafraniuk, Krzysztof Trojanowski
Date: 2020-04-19 00:00:00
Tasks: Denoising, GPU, image-classification, Image Classification, Image Denoising, Image Enhancement, Neural Architecture Search, object-detection, Object Detection, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Semantic Segmentation

Recent advancements in Neural Architecture Search(NAS) resulted in finding new state-of-the-art Artificial Neural Network (ANN) solutions for tasks like image classification, object detection, or semantic segmentation without substantial human supervision. In this paper, we focus on exploring NAS for a dense prediction task that is image denoising. Due to a costly training procedure, most NAS solutions for image enhancement rely on reinforcement learning or evolutionary algorithm exploration, which usually take weeks (or even months) to train. Therefore, we introduce a new efficient implementation of various superkernel techniques that enable fast (6-8 RTX2080 GPU hours) single-shot training of models for dense predictions. We demonstrate the effectiveness of our method on the SIDD+ benchmark for image denoising.
A Universal Approximation Theorem of Deep Neural Networks for Expressing Probability Distributions
Authors: Yulong Lu, Jianfeng Lu
Date: 2020-04-19 00:00:00
Proceeding: NeurIPS 2020 12

This paper studies the universal approximation property of deep neural networks for representing probability distributions. Given a target distribution $\pi$ and a source distribution $p_z$ both defined on $\mathbb{R}^d$, we prove under some assumptions that there exists a deep neural network $g:\mathbb{R}^d\rightarrow \mathbb{R}$ with ReLU activation such that the push-forward measure $(\nabla g)_\# p_z$ of $p_z$ under the map $\nabla g$ is arbitrarily close to the target measure $\pi$. The closeness are measured by three classes of integral probability metrics between probability distributions: $1$-Wasserstein distance, maximum mean distance (MMD) and kernelized Stein discrepancy (KSD). We prove upper bounds for the size (width and depth) of the deep neural network in terms of the dimension $d$ and the approximation error $\varepsilon$ with respect to the three discrepancies. In particular, the size of neural network can grow exponentially in $d$ when $1$-Wasserstein distance is used as the discrepancy, whereas for both MMD and KSD the size of neural network only depends on $d$ at most polynomially. Our proof relies on convergence estimates of empirical measures under aforementioned discrepancies and semi-discrete optimal transport.
Space Debris Ontology for ADR Capture Methods Selection
Authors: Marko Jankovic, Mehmed Yüksel, Mohammad Mohammadzadeh Babr, Francesca Letizia, Vitali Braun
Date: 2020-04-19 00:00:00
Tasks: Management

Studies have concluded that active debris removal (ADR) of the existing in-orbit mass is necessary. However, the quest for an optimal solution does not have a unique answer and the available data often lacks coherence. To improve this situation, modern knowledge representation techniques, that have been shaping the World Wide Web, medicine and pharmacy, should be employed. Prior efforts in the domain of space debris have only focused onto space situational awareness, neglecting ADR. To bridge this gap we present a domain-ontology of intact derelict objects, i.e. payloads and rocket bodies, for ADR capture methods selection. The ontology is defined on a minimal set of physical, dynamical and statistical parameters of a target object. The practicality and validity of the ontology are demonstrated by applying it onto a database of 30 representative objects, built by combining structured and unstructured data from publicly available sources. The analysis of results proves the ontology capable of inferring the most suited ADR capture methods for considered objects. Furthermore, it confirms its ability to handle the input data from different sources transparently, minimizing user input. The developed ontology provides an initial step towards a more comprehensive knowledge representation framework meant to improve data management and knowledge discovery in the domain of space debris. Furthermore, it provides a tool that should make the initial planning of future ADR missions simpler yet more systematic.
Role-Wise Data Augmentation for Knowledge Distillation
Authors: Jie Fu, Xue Geng, Zhijian Duan, Bohan Zhuang, Xingdi Yuan, Adam Trischler, Jie Lin, Chris Pal, Hao Dong
Date: 2020-04-19 00:00:00
Proceeding: ICLR 2020 1
Tasks: Data Augmentation, Knowledge Distillation

Knowledge Distillation (KD) is a common method for transferring the ``knowledge'' learned by one machine learning model (the \textit{teacher}) into another model (the \textit{student}), where typically, the teacher has a greater capacity (e.g., more parameters or higher bit-widths). To our knowledge, existing methods overlook the fact that although the student absorbs extra knowledge from the teacher, both models share the same input data -- and this data is the only medium by which the teacher's knowledge can be demonstrated. Due to the difference in model capacities, the student may not benefit fully from the same data points on which the teacher is trained. On the other hand, a human teacher may demonstrate a piece of knowledge with individualized examples adapted to a particular student, for instance, in terms of her cultural background and interests. Inspired by this behavior, we design data augmentation agents with distinct roles to facilitate knowledge distillation. Our data augmentation agents generate distinct training data for the teacher and student, respectively. We find empirically that specially tailored data points enable the teacher's knowledge to be demonstrated more effectively to the student. We compare our approach with existing KD methods on training popular neural architectures and demonstrate that role-wise data augmentation improves the effectiveness of KD over strong prior approaches. The code for reproducing our results can be found at https://github.com/bigaidream-projects/role-kd
Make E Smart Again
Authors: Zarathustra Amadeus Goertzel
Date: 2020-04-19 00:00:00

In this work in progress, we demonstrate a new use-case for the ENIGMA system. The ENIGMA system using the XGBoost implementation of gradient boosted decision trees has demonstrated high capability to learn to guide the E theorem prover's inferences in real-time. Here, we strip E to the bare bones: we replace the KBO term ordering with an identity relation as the minimal possible ordering, disable literal selection, and replace evolved strategies with a simple combination of the clause weight and FIFO (first in first out) clause evaluation functions. We experimentally demonstrate that ENIGMA can learn to guide E as well as the smart, evolved strategies even without these standard automated theorem prover functionalities. To this end, we experiment with XGBoost's meta-parameters over a dozen loops.
Local Differential Privacy based Federated Learning for Internet of Things
Authors: Yang Zhao, Jun Zhao, Mengmeng Yang, Teng Wang, Ning Wang, Lingjuan Lyu, Dusit Niyato, Kwok-Yan Lam
Date: 2020-04-19 00:00:00
Tasks: BIG-bench Machine Learning, Federated Learning, Management

Internet of Vehicles (IoV) is a promising branch of the Internet of Things. IoV simulates a large variety of crowdsourcing applications such as Waze, Uber, and Amazon Mechanical Turk, etc. Users of these applications report the real-time traffic information to the cloud server which trains a machine learning model based on traffic information reported by users for intelligent traffic management. However, crowdsourcing application owners can easily infer users' location information, which raises severe location privacy concerns of the users. In addition, as the number of vehicles increases, the frequent communication between vehicles and the cloud server incurs unexpected amount of communication cost. To avoid the privacy threat and reduce the communication cost, in this paper, we propose to integrate federated learning and local differential privacy (LDP) to facilitate the crowdsourcing applications to achieve the machine learning model. Specifically, we propose four LDP mechanisms to perturb gradients generated by vehicles. The Three-Outputs mechanism is proposed which introduces three different output possibilities to deliver a high accuracy when the privacy budget is small. The output possibilities of Three-Outputs can be encoded with two bits to reduce the communication cost. Besides, to maximize the performance when the privacy budget is large, an optimal piecewise mechanism (PM-OPT) is proposed. We further propose a suboptimal mechanism (PM-SUB) with a simple formula and comparable utility to PM-OPT. Then, we build a novel hybrid mechanism by combining Three-Outputs and PM-SUB.
Approximate Nearest Neighbour Search on Privacy-aware Encoding of User Locations to Identify Susceptible Infections in Simulated Epidemics
Authors: Chandan Biswas, Debasis Ganguly, Ujjwal Bhattacharya
Date: 2020-04-19 00:00:00
Tasks: Information Retrieval, Retrieval

Amidst an increasing number of infected cases during the Covid-19 pandemic, it is essential to trace, as early as possible, the susceptible people who might have been infected by the disease due to their close proximity with people who were tested positive for the virus. This early contact tracing is likely to limit the rate of spread of the infection within a locality. In this paper, we investigate how effectively and efficiently can such a list of susceptible people be found given a list of infected persons and their locations. To address this problem from an information retrieval (search) perspective, we represent the location of each person at each time instant as a point in a vector space. By using the locations of the given list of infected persons as queries, we investigate the feasibility of applying approximate nearest neighbour (ANN) based indexing and retrieval approaches to obtain a list of top-k suspected users in real-time. Since leveraging information from true user location data can lead to security and privacy concerns, we also investigate what effects does distance-preserving encoding methods have on the effectiveness of the ANN methods. Experiments conducted on real and synthetic datasets demonstrate that the top-k retrieved lists of susceptible users retrieved with existing ANN approaches (KD-tree and HNSW) yield satisfactory precision and recall values, thus indicating that ANN approaches can potentially be applied in practice to facilitate real-time contact tracing even under the presence of imposed privacy constraints.
Dynamic Knowledge Graph-based Dialogue Generation with Improved Adversarial Meta-Learning
Authors: Hongcai Xu, Junpeng Bao, Gaojie Zhang
Date: 2020-04-19 00:00:00
Tasks: Adversarial Attack, Dialogue Generation, Meta-Learning

Knowledge graph-based dialogue systems are capable of generating more informative responses and can implement sophisticated reasoning mechanisms. However, these models do not take into account the sparseness and incompleteness of knowledge graph (KG)and current dialogue models cannot be applied to dynamic KG. This paper proposes a dynamic Knowledge graph-based dialogue generation method with improved adversarial Meta-Learning (KDAD). KDAD formulates dynamic knowledge triples as a problem of adversarial attack and incorporates the objective of quickly adapting to dynamic knowledge-aware dialogue generation. We train a knowledge graph-based dialog model with improved ADML using minimal training samples. The model can initialize the parameters and adapt to previous unseen knowledge so that training can be quickly completed based on only a few knowledge triples. We show that our model significantly outperforms other baselines. We evaluate and demonstrate that our method adapts extremely fast and well to dynamic knowledge graph-based dialogue generation.
Improving Robot Dual-System Motor Learning with Intrinsically Motivated Meta-Control and Latent-Space Experience Imagination
Authors: Muhammad Burhan Hafez, Cornelius Weber, Matthias Kerzel, Stefan Wermter
Date: 2020-04-19 00:00:00
Tasks: Robotic Grasping

Combining model-based and model-free learning systems has been shown to improve the sample efficiency of learning to perform complex robotic tasks. However, dual-system approaches fail to consider the reliability of the learned model when it is applied to make multiple-step predictions, resulting in a compounding of prediction errors and performance degradation. In this paper, we present a novel dual-system motor learning approach where a meta-controller arbitrates online between model-based and model-free decisions based on an estimate of the local reliability of the learned model. The reliability estimate is used in computing an intrinsic feedback signal, encouraging actions that lead to data that improves the model. Our approach also integrates arbitration with imagination where a learned latent-space model generates imagined experiences, based on its local reliability, to be used as additional training data. We evaluate our approach against baseline and state-of-the-art methods on learning vision-based robotic grasping in simulation and real world. The results show that our approach outperforms the compared methods and learns near-optimal grasping policies in dense- and sparse-reward environments.
DeepCFD: Efficient Steady-State Laminar Flow Approximation with Deep Convolutional Neural Networks
Authors: Mateus Dias Ribeiro, Abdul Rehman, Sheraz Ahmed, Andreas Dengel
Date: 2020-04-19 00:00:00

Computational Fluid Dynamics (CFD) simulation by the numerical solution of the Navier-Stokes equations is an essential tool in a wide range of applications from engineering design to climate modeling. However, the computational cost and memory demand required by CFD codes may become very high for flows of practical interest, such as in aerodynamic shape optimization. This expense is associated with the complexity of the fluid flow governing equations, which include non-linear partial derivative terms that are of difficult solution, leading to long computational times and limiting the number of hypotheses that can be tested during the process of iterative design. Therefore, we propose DeepCFD: a convolutional neural network (CNN) based model that efficiently approximates solutions for the problem of non-uniform steady laminar flows. The proposed model is able to learn complete solutions of the Navier-Stokes equations, for both velocity and pressure fields, directly from ground-truth data generated using a state-of-the-art CFD code. Using DeepCFD, we found a speedup of up to 3 orders of magnitude compared to the standard CFD approach at a cost of low error rates.
A Chinese Corpus for Fine-grained Entity Typing
Authors: Chin Lee, Hongliang Dai, Yangqiu Song, Xin Li
Date: 2020-04-19 00:00:00
Proceeding: LREC 2020 5
Tasks: Cross-Lingual Transfer, Entity Typing, Transfer Learning

Fine-grained entity typing is a challenging task with wide applications. However, most existing datasets for this task are in English. In this paper, we introduce a corpus for Chinese fine-grained entity typing that contains 4,800 mentions manually labeled through crowdsourcing. Each mention is annotated with free-form entity types. To make our dataset useful in more possible scenarios, we also categorize all the fine-grained types into 10 general types. Finally, we conduct experiments with some neural models whose structures are typical in fine-grained entity typing and show how well they perform on our dataset. We also show the possibility of improving Chinese fine-grained entity typing through cross-lingual transfer learning.
Human Activity Recognition using Inertial, Physiological and Environmental Sensors: a Comprehensive Survey
Authors: Florenc Demrozi, Graziano Pravadelli, Azra Bihorac, Parisa Rashidi
Date: 2020-04-19 00:00:00
Tasks: Activity Recognition, BIG-bench Machine Learning, Human Activity Recognition

In the last decade, Human Activity Recognition (HAR) has become a vibrant research area, especially due to the spread of electronic devices such as smartphones, smartwatches and video cameras present in our daily lives. In addition, the advance of deep learning and other machine learning algorithms has allowed researchers to use HAR in various domains including sports, health and well-being applications. For example, HAR is considered as one of the most promising assistive technology tools to support elderly's daily life by monitoring their cognitive and physical function through daily activities. This survey focuses on critical role of machine learning in developing HAR applications based on inertial sensors in conjunction with physiological and environmental sensors.
Graph-Structured Referring Expression Reasoning in The Wild
Authors: Sibei Yang, Guanbin Li, Yizhou Yu
Date: 2020-04-19 00:00:00
Proceeding: CVPR 2020 6
Tasks: Referring Expression

Grounding referring expressions aims to locate in an image an object referred to by a natural language expression. The linguistic structure of a referring expression provides a layout of reasoning over the visual contents, and it is often crucial to align and jointly understand the image and the referring expression. In this paper, we propose a scene graph guided modular network (SGMN), which performs reasoning over a semantic graph and a scene graph with neural modules under the guidance of the linguistic structure of the expression. In particular, we model the image as a structured semantic graph, and parse the expression into a language scene graph. The language scene graph not only decodes the linguistic structure of the expression, but also has a consistent representation with the image semantic graph. In addition to exploring structured solutions to grounding referring expressions, we also propose Ref-Reasoning, a large-scale real-world dataset for structured referring expression reasoning. We automatically generate referring expressions over the scene graphs of images using diverse expression templates and functional programs. This dataset is equipped with real-world visual contents as well as semantically rich expressions with different reasoning layouts. Experimental results show that our SGMN not only significantly outperforms existing state-of-the-art algorithms on the new Ref-Reasoning dataset, but also surpasses state-of-the-art structured methods on commonly used benchmark datasets. It can also provide interpretable visual evidences of reasoning. Data and code are available at https://github.com/sibeiyang/sgmn
Knowledge-graph based Proactive Dialogue Generation with Improved Meta-Learning
Authors: Hongcai Xu, Junpeng Bao, Junqing Wang
Date: 2020-04-19 00:00:00
Tasks: Dialogue Generation, Knowledge Graphs, Meta-Learning, Sentence, Sentence Embedding, Sentence-Embedding

Knowledge graph-based dialogue systems can narrow down knowledge candidates for generating informative and diverse responses with the use of prior information, e.g., triple attributes or graph paths. However, most current knowledge graph (KG) cover incomplete domain-specific knowledge. To overcome this drawback, we propose a knowledge graph based proactive dialogue generation model (KgDg) with three components, improved model-agnostic meta-learning algorithm (MAML), knowledge selection in knowledge triplets embedding, and knowledge aware proactive response generator. For knowledge triplets embedding and selection, we formulate it as a problem of sentence embedding to better capture semantic information. Our improved MAML algorithm is capable of learning general features from a limited number of knowledge graphs, which can also quickly adapt to dialogue generation with unseen knowledge triplets. Extensive experiments are conducted on a knowledge aware dialogue dataset (DuConv). The results show that KgDg adapts both fast and well to knowledge graph-based dialogue generation and outperforms state-of-the-art baseline.