pwc-archive/papers-with-abstracts huggingface dataset:
Deep Uncertainty Quantification: A Machine Learning Approach for Weather Forecasting
Authors: Bin Wang, Jie Lu, Zheng Yan, Huaishao Luo, Tianrui Li, Yu Zheng, Guangquan Zhang
Date: 2018-12-22 00:00:00
Tasks: BIG-bench Machine Learning, Uncertainty Quantification, Weather Forecasting

Weather forecasting is usually solved through numerical weather prediction
(NWP), which can sometimes lead to unsatisfactory performance due to
inappropriate setting of the initial states. In this paper, we design a
data-driven method augmented by an effective information fusion mechanism to
learn from historical data that incorporates prior knowledge from NWP. We cast
the weather forecasting problem as an end-to-end deep learning problem and
solve it by proposing a novel negative log-likelihood error (NLE) loss
function. A notable advantage of our proposed method is that it simultaneously
implements single-value forecasting and uncertainty quantification, which we
refer to as deep uncertainty quantification (DUQ). Efficient deep ensemble
strategies are also explored to further improve performance. This new approach
was evaluated on a public dataset collected from weather stations in Beijing,
China. Experimental results demonstrate that the proposed NLE loss
significantly improves generalization compared to mean squared error (MSE) loss
and mean absolute error (MAE) loss. Compared with NWP, this approach
significantly improves accuracy by 47.76%, which is a state-of-the-art result
on this benchmark dataset. The preliminary version of the proposed method won
2nd place in an online competition for daily weather forecasting.
Fault Location in Power Distribution Systems via Deep Graph Convolutional Networks
Authors: Kunjin Chen, Jun Hu, Yu Zhang, Zhanqing Yu, Jinliang He
Date: 2018-12-22 00:00:00
Tasks: Data Augmentation, Data Visualization

This paper develops a novel graph convolutional network (GCN) framework for fault location in power distribution networks. The proposed approach integrates multiple measurements at different buses while taking system topology into account. The effectiveness of the GCN model is corroborated by the IEEE 123 bus benchmark system. Simulation results show that the GCN model significantly outperforms other widely-used machine learning schemes with very high fault location accuracy. In addition, the proposed approach is robust to measurement noise and data loss errors. Data visualization results of two competing neural networks are presented to explore the mechanism of GCN's superior performance. A data augmentation procedure is proposed to increase the robustness of the model under various levels of noise and data loss errors. Further experiments show that the model can adapt to topology changes of distribution networks and perform well with a limited number of measured buses.
A Survey on Deep Learning for Named Entity Recognition
Authors: Jing Li, Aixin Sun, Jianglei Han, Chenliang Li
Date: 2018-12-22 00:00:00
Tasks: Decoder, Deep Learning, Machine Translation, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Question Answering, Semantic Composition, Survey, TAG, Text Summarization, Translation

Named entity recognition (NER) is the task to identify mentions of rigid designators from text belonging to predefined semantic types such as person, location, organization etc. NER always serves as the foundation for many natural language applications such as question answering, text summarization, and machine translation. Early NER systems got a huge success in achieving good performance with the cost of human engineering in designing domain-specific features and rules. In recent years, deep learning, empowered by continuous real-valued vector representations and semantic composition through nonlinear processing, has been employed in NER systems, yielding stat-of-the-art performance. In this paper, we provide a comprehensive review on existing deep learning techniques for NER. We first introduce NER resources, including tagged NER corpora and off-the-shelf NER tools. Then, we systematically categorize existing works based on a taxonomy along three axes: distributed representations for input, context encoder, and tag decoder. Next, we survey the most representative methods for recent applied techniques of deep learning in new NER problem settings and applications. Finally, we present readers with the challenges faced by NER systems and outline future directions in this area.
Deep autoregressive neural networks for high-dimensional inverse problems in groundwater contaminant source identification
Authors: Shaoxing Mo, Nicholas Zabaras, Xiaoqing Shi, Jichun Wu
Date: 2018-12-22 00:00:00
Tasks: Computational Efficiency

Identification of a groundwater contaminant source simultaneously with the
hydraulic conductivity in highly-heterogeneous media often results in a
high-dimensional inverse problem. In this study, a deep autoregressive neural
network-based surrogate method is developed for the forward model to allow us
to solve efficiently such high-dimensional inverse problems. The surrogate is
trained using limited evaluations of the forward model. Since the relationship
between the time-varying inputs and outputs of the forward transport model is
complex, we propose an autoregressive strategy, which treats the output at the
previous time step as input to the network for predicting the output at the
current time step. We employ a dense convolutional encoder-decoder network
architecture in which the high-dimensional input and output fields of the model
are treated as images to leverage the robust capability of convolutional
networks in image-like data processing. An iterative local updating ensemble
smoother (ILUES) algorithm is used as the inversion framework. The proposed
method is evaluated using a synthetic contaminant source identification problem
with 686 uncertain input parameters. Results indicate that, with relatively
limited training data, the deep autoregressive neural network consisting of 27
convolutional layers is capable of providing an accurate approximation for the
high-dimensional model input-output relationship. The autoregressive strategy
substantially improves the network's accuracy and computational efficiency. The
application of the surrogate-based ILUES in solving the inverse problem shows
that it can achieve accurate inversion results and predictive uncertainty
estimates.
Graph Transformation Policy Network for Chemical Reaction Prediction
Authors: Kien Do, Truyen Tran, Svetha Venkatesh
Date: 2018-12-22 00:00:00
Tasks: Chemical Reaction Prediction, Prediction, Reinforcement Learning

We address a fundamental problem in chemistry known as chemical reaction
product prediction. Our main insight is that the input reactant and reagent
molecules can be jointly represented as a graph, and the process of generating
product molecules from reactant molecules can be formulated as a sequence of
graph transformations. To this end, we propose Graph Transformation Policy
Network (GTPN) -- a novel generic method that combines the strengths of graph
neural networks and reinforcement learning to learn the reactions directly from
data with minimal chemical knowledge. Compared to previous methods, GTPN has
some appealing properties such as: end-to-end learning, and making no
assumption about the length or the order of graph transformations. In order to
guide model search through the complex discrete space of sets of bond changes
effectively, we extend the standard policy gradient loss by adding useful
constraints. Evaluation results show that GTPN improves the top-1 accuracy over
the current state-of-the-art method by about 3% on the large USPTO dataset. Our
model's performances and prediction errors are also analyzed carefully in the
paper.
Dissociable neural representations of adversarially perturbed images in convolutional neural networks and the human brain
Authors: Chi Zhang, Xiaohan Duan, Linyuan Wang, Yongli Li, Bin Yan, Guoen Hu, Ruyuan Zhang, Li Tong
Date: 2018-12-22 00:00:00

Despite the remarkable similarities between convolutional neural networks (CNN) and the human brain, CNNs still fall behind humans in many visual tasks, indicating that there still exist considerable differences between the two systems. Here, we leverage adversarial noise (AN) and adversarial interference (AI) images to quantify the consistency between neural representations and perceptual outcomes in the two systems. Humans can successfully recognize AI images as corresponding categories but perceive AN images as meaningless noise. In contrast, CNNs can correctly recognize AN images but mistakenly classify AI images into wrong categories with surprisingly high confidence. We use functional magnetic resonance imaging to measure brain activity evoked by regular and adversarial images in the human brain, and compare it to the activity of artificial neurons in a prototypical CNN-AlexNet. In the human brain, we find that the representational similarity between regular and adversarial images largely echoes their perceptual similarity in all early visual areas. In AlexNet, however, the neural representations of adversarial images are inconsistent with network outputs in all intermediate processing layers, providing no neural foundations for perceptual similarity. Furthermore, we show that voxel-encoding models trained on regular images can successfully generalize to the neural responses to AI images but not AN images. These remarkable differences between the human brain and AlexNet in the representation-perception relation suggest that future CNNs should emulate both behavior and the internal neural presentations of the human brain.
Dynamic Graph Representation Learning via Self-Attention Networks
Authors: Aravind Sankar, Yanhong Wu, Liang Gou, Wei zhang, Hao Yang
Date: 2018-12-22 00:00:00
Tasks: General Classification, Graph Embedding, Graph Representation Learning, Link Prediction, Node Classification, Representation Learning

Learning latent representations of nodes in graphs is an important and ubiquitous task with widespread applications such as link prediction, node classification, and graph visualization. Previous methods on graph representation learning mainly focus on static graphs, however, many real-world graphs are dynamic and evolve over time. In this paper, we present Dynamic Self-Attention Network (DySAT), a novel neural architecture that operates on dynamic graphs and learns node representations that capture both structural properties and temporal evolutionary patterns. Specifically, DySAT computes node representations by jointly employing self-attention layers along two dimensions: structural neighborhood and temporal dynamics. We conduct link prediction experiments on two classes of graphs: communication networks and bipartite rating networks. Our experimental results show that DySAT has a significant performance gain over several different state-of-the-art graph embedding baselines.
Image Embedding of PMU Data for Deep Learning towards Transient Disturbance Classification
Authors: Yongli Zhu, Chengxi Liu, Kai Sun
Date: 2018-12-22 00:00:00
Tasks: Classification, General Classification, Time Series, Time Series Analysis

This paper presents a study on power grid disturbance classification by Deep
Learning (DL). A real synchrophasor set composing of three different types of
disturbance events from the Frequency Monitoring Network (FNET) is used. An
image embedding technique called Gramian Angular Field is applied to transform
each time series of event data to a two-dimensional image for learning. Two
main DL algorithms, i.e. CNN (Convolutional Neural Network) and RNN (Recurrent
Neural Network) are tested and compared with two widely used data mining tools,
the Support Vector Machine and Decision Tree. The test results demonstrate the
superiority of the both DL algorithms over other methods in the application of
power system transient disturbance classification.
Distributed sequential method for analyzing massive data
Authors: Zhanfeng Wang, Yuan-Chin Ivan Chang
Date: 2018-12-22 00:00:00

To analyse a very large data set containing lengthy variables, we adopt a
sequential estimation idea and propose a parallel divide-and-conquer method. We
conduct several conventional sequential estimation procedures separately, and
properly integrate their results while maintaining the desired statistical
properties. Additionally, using a criterion from the statistical experiment
design, we adopt an adaptive sample selection, together with an adaptive
shrinkage estimation method, to simultaneously accelerate the estimation
procedure and identify the effective variables. We confirm the cogency of our
methods through theoretical justifications and numerical results derived from
synthesized data sets. We then apply the proposed method to three real data
sets, including those pertaining to appliance energy use and particulate matter
concentration.
Exploiting Problem Structure in Combinatorial Landscapes: A Case Study on Pure Mathematics Application
Authors: Xiao-Feng Xie, Zun-Jing Wang
Date: 2018-12-22 00:00:00
Tasks: Combinatorial Optimization

In this paper, we present a method using AI techniques to solve a case of
pure mathematics applications for finding narrow admissible tuples. The
original problem is formulated into a combinatorial optimization problem. In
particular, we show how to exploit the local search structure to formulate the
problem landscape for dramatic reductions in search space and for non-trivial
elimination in search barriers, and then to realize intelligent search
strategies for effectively escaping from local minima. Experimental results
demonstrate that the proposed method is able to efficiently find best known
solutions. This research sheds light on exploiting the local problem structure
for an efficient search in combinatorial landscapes as an application of AI to
a new problem domain.
Towards resilient machine learning for ransomware detection
Authors: Li Chen, Chih-Yuan Yang, Anindya Paul, Ravi Sahita
Date: 2018-12-21 00:00:00
Tasks: BIG-bench Machine Learning, Generative Adversarial Network, Malware Analysis

There has been a surge of interest in using machine learning (ML) to automatically detect malware through their dynamic behaviors. These approaches have achieved significant improvement in detection rates and lower false positive rates at large scale compared with traditional malware analysis methods. ML in threat detection has demonstrated to be a good cop to guard platform security. However it is imperative to evaluate - is ML-powered security resilient enough? In this paper, we juxtapose the resiliency and trustworthiness of ML algorithms for security, via a case study of evaluating the resiliency of ransomware detection via the generative adversarial network (GAN). In this case study, we propose to use GAN to automatically produce dynamic features that exhibit generalized malicious behaviors that can reduce the efficacy of black-box ransomware classifiers. We examine the quality of the GAN-generated samples by comparing the statistical similarity of these samples to real ransomware and benign software. Further we investigate the latent subspace where the GAN-generated samples lie and explore reasons why such samples cause a certain class of ransomware classifiers to degrade in performance. Our focus is to emphasize necessary defense improvement in ML-based approaches for ransomware detection before deployment in the wild. Our results and discoveries should pose relevant questions for defenders such as how ML models can be made more resilient for robust enforcement of security objectives.
Multi-Step Prediction of Occupancy Grid Maps with Recurrent Neural Networks
Authors: Nima Mohajerin, Mohsen Rohani
Date: 2018-12-21 00:00:00
Proceeding: CVPR 2019 6
Tasks: Autonomous Driving, Autonomous Vehicles, Prediction, Prediction Of Occupancy Grid Maps

We investigate the multi-step prediction of the drivable space, represented
by Occupancy Grid Maps (OGMs), for autonomous vehicles. Our motivation is that
accurate multi-step prediction of the drivable space can efficiently improve
path planning and navigation resulting in safe, comfortable and optimum paths
in autonomous driving. We train a variety of Recurrent Neural Network (RNN)
based architectures on the OGM sequences from the KITTI dataset. The results
demonstrate significant improvement of the prediction accuracy using our
proposed difference learning method, incorporating motion related features,
over the state of the art. We remove the egomotion from the OGM sequences by
transforming them into a common frame. Although in the transformed sequences
the KITTI dataset is heavily biased toward static objects, by learning the
difference between subsequent OGMs, our proposed method provides accurate
prediction over both the static and moving objects.
Human-AI Learning Performance in Multi-Armed Bandits
Authors: Ravi Pandya, Sandy H. Huang, Dylan Hadfield-Menell, Anca D. Dragan
Date: 2018-12-21 00:00:00
Tasks: Decision Making, Multi-Armed Bandits

People frequently face challenging decision-making problems in which outcomes
are uncertain or unknown. Artificial intelligence (AI) algorithms exist that
can outperform humans at learning such tasks. Thus, there is an opportunity for
AI agents to assist people in learning these tasks more effectively. In this
work, we use a multi-armed bandit as a controlled setting in which to explore
this direction. We pair humans with a selection of agents and observe how well
each human-agent team performs. We find that team performance can beat both
human and agent performance in isolation. Interestingly, we also find that an
agent's performance in isolation does not necessarily correlate with the
human-agent team's performance. A drop in agent performance can lead to a
disproportionately large drop in team performance, or in some settings can even
improve team performance. Pairing a human with an agent that performs slightly
better than them can make them perform much better, while pairing them with an
agent that performs the same can make them them perform much worse. Further,
our results suggest that people have different exploration strategies and might
perform better with agents that match their strategy. Overall, optimizing
human-agent team performance requires going beyond optimizing agent
performance, to understanding how the agent's suggestions will influence human
decision-making.
Multi-Frame Super-Resolution Reconstruction with Applications to Medical Imaging
Authors: Thomas Köhler
Date: 2018-12-21 00:00:00
Tasks: Multi-Frame Super-Resolution, Super-Resolution

The optical resolution of a digital camera is one of its most crucial
parameters with broad relevance for consumer electronics, surveillance systems,
remote sensing, or medical imaging. However, resolution is physically limited
by the optics and sensor characteristics. In addition, practical and economic
reasons often stipulate the use of out-dated or low-cost hardware.
Super-resolution is a class of retrospective techniques that aims at
high-resolution imagery by means of software. Multi-frame algorithms approach
this task by fusing multiple low-resolution frames to reconstruct
high-resolution images. This work covers novel super-resolution methods along
with new applications in medical imaging.
Wireless Software Synchronization of Multiple Distributed Cameras
Authors: Sameer Ansari, Neal Wadhwa, Rahul Garg, Jiawen Chen
Date: 2018-12-21 00:00:00
Tasks: Depth Estimation, Stereo Depth Estimation

We present a method for precisely time-synchronizing the capture of image sequences from a collection of smartphone cameras connected over WiFi. Our method is entirely software-based, has only modest hardware requirements, and achieves an accuracy of less than 250 microseconds on unmodified commodity hardware. It does not use image content and synchronizes cameras prior to capture. The algorithm operates in two stages. In the first stage, we designate one device as the leader and synchronize each client device's clock to it by estimating network delay. Once clocks are synchronized, the second stage initiates continuous image streaming, estimates the relative phase of image timestamps between each client and the leader, and shifts the streams into alignment. We quantitatively validate our results on a multi-camera rig imaging a high-precision LED array and qualitatively demonstrate significant improvements to multi-view stereo depth estimation and stitching of dynamic scenes. We release as open source 'libsoftwaresync', an Android implementation of our system, to inspire new types of collective capture applications.
NeuroX: A Toolkit for Analyzing Individual Neurons in Neural Networks
Authors: Fahim Dalvi, Avery Nortonsmith, D. Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, James Glass
Date: 2018-12-21 00:00:00

We present a toolkit to facilitate the interpretation and understanding of
neural network models. The toolkit provides several methods to identify salient
neurons with respect to the model itself or an external task. A user can
visualize selected neurons, ablate them to measure their effect on the model
accuracy, and manipulate them to control the behavior of the model at the test
time. Such an analysis has a potential to serve as a springboard in various
research directions, such as understanding the model, better architectural
choices, model distillation and controlling data biases.
What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models
Authors: Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Yonatan Belinkov, Anthony Bau, James Glass
Date: 2018-12-21 00:00:00
Tasks: Language Modeling, Language Modelling, Machine Translation, NMT, Sand

Despite the remarkable evolution of deep neural networks in natural language
processing (NLP), their interpretability remains a challenge. Previous work
largely focused on what these models learn at the representation level. We
break this analysis down further and study individual dimensions (neurons) in
the vector representation learned by end-to-end neural models in NLP tasks. We
propose two methods: Linguistic Correlation Analysis, based on a supervised
method to extract the most relevant neurons with respect to an extrinsic task,
and Cross-model Correlation Analysis, an unsupervised method to extract salient
neurons w.r.t. the model itself. We evaluate the effectiveness of our
techniques by ablating the identified neurons and reevaluating the network's
performance for two tasks: neural machine translation (NMT) and neural language
modeling (NLM). We further present a comprehensive analysis of neurons with the
aim to address the following questions: i) how localized or distributed are
different linguistic properties in the models? ii) are certain neurons
exclusive to some properties and not others? iii) is the information more or
less distributed in NMT vs. NLM? and iv) how important are the neurons
identified through the linguistic correlation method to the overall task? Our
code is publicly available as part of the NeuroX toolkit (Dalvi et al. 2019).