pwc-archive/papers-with-abstracts huggingface dataset:
Autoencoding Binary Classifiers for Supervised Anomaly Detection
Authors: Yuki Yamanaka, Tomoharu Iwata, Hiroshi Takahashi, Masanori Yamada, Sekitoshi Kanai
Date: 2019-03-26 00:00:00
Tasks: Anomaly Detection, Supervised Anomaly Detection

We propose the Autoencoding Binary Classifiers (ABC), a novel supervised
anomaly detector based on the Autoencoder (AE). There are two main approaches
in anomaly detection: supervised and unsupervised. The supervised approach
accurately detects the known anomalies included in training data, but it cannot
detect the unknown anomalies. Meanwhile, the unsupervised approach can detect
both known and unknown anomalies that are located away from normal data points.
However, it does not detect known anomalies as accurately as the supervised
approach. Furthermore, even if we have labeled normal data points and
anomalies, the unsupervised approach cannot utilize these labels. The ABC is a
probabilistic binary classifier that effectively exploits the label
information, where normal data points are modeled using the AE as a component.
By maximizing the likelihood, the AE in the proposed ABC is trained to minimize
the reconstruction error for normal data points, and to maximize it for known
anomalies. Since our approach becomes able to reconstruct the normal data
points accurately and fails to reconstruct the known and unknown anomalies, it
can accurately discriminate both known and unknown anomalies from normal data
points. Experimental results show that the ABC achieves higher detection
performance than existing supervised and unsupervised methods.
Conditioning a Recurrent Neural Network to synthesize musical instrument transients
Authors: Lonce Wyse, Muhammad Huzaifah
Date: 2019-03-26 00:00:00

A recurrent Neural Network (RNN) is trained to predict sound samples based on
audio input augmented by control parameter information for pitch, volume, and
instrument identification. During the generative phase following training,
audio input is taken from the output of the previous time step, and the
parameters are externally controlled allowing the network to be played as a
musical instrument. Building on an architecture developed in previous work, we
focus on the learning and synthesis of transients - the temporal response of
the network during the short time (tens of milliseconds) following the onset
and offset of a control signal. We find that the network learns the particular
transient characteristics of two different synthetic instruments, and
furthermore shows some ability to interpolate between the characteristics of
the instruments used in training in response to novel parameter settings. We
also study the behaviour of the units in hidden layers of the RNN using various
visualisation techniques and find a variety of volume-specific response
characteristics.
On the Theory of Dynamic Graph Regression Problem
Authors: Mostafa Haghir Chehreghani
Date: 2019-03-26 00:00:00
Tasks: Graph Regression, regression

Most of real-world graphs are dynamic, i.e., they change over time by a sequence of update operations. While the regression problem has been studied for static graphs and temporal graphs, it is not investigated for general dynamic graphs. In this paper, we study regression over dynamic graphs. First, we present the notion of update-efficient matrix embedding, that defines conditions sufficient for a matrix embedding to be effectively used for dynamic graph regression (under l2 norm). Then, we show that given a n*m update-efficient matrix embedding (e.g., the adjacency matrix) and after an update operation in the graph, the exact optimal solution of linear regression can be updated in O(nm) time for the revised graph. Moreover, we show that this also holds when the matrix embedding is the Laplacian matrix and the update operations are restricted to edge insertion/deletion. In the end, by conducting experiments over synthetic and real-world graphs, we show the high efficiency of updating the solution of graph regression.
Probabilistic Load Forecasting via Point Forecast Feature Integration
Authors: Qicheng Chang, Yishen Wang, Xiao Lu, Di Shi, Haifeng Li, Jiajun Duan, Zhiwei Wang
Date: 2019-03-26 00:00:00
Tasks: energy management, Feature Importance, Load Forecasting, Management, quantile regression, regression

Short-term load forecasting is a critical element of power systems energy
management systems. In recent years, probabilistic load forecasting (PLF) has
gained increased attention for its ability to provide uncertainty information
that helps to improve the reliability and economics of system operation
performances. This paper proposes a two-stage probabilistic load forecasting
framework by integrating point forecast as a key probabilistic forecasting
feature into PLF. In the first stage, all related features are utilized to
train a point forecast model and also obtain the feature importance. In the
second stage the forecasting model is trained, taking into consideration point
forecast features, as well as selected feature subsets. During the testing
period of the forecast model, the final probabilistic load forecast results are
leveraged to obtain both point forecasting and probabilistic forecasting.
Numerical results obtained from ISO New England demand data demonstrate the
effectiveness of the proposed approach in the hour-ahead load forecasting,
which uses the gradient boosting regression for the point forecasting and
quantile regression neural networks for the probabilistic forecasting.
Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Data
Authors: Xiaowei Hu, Yitong Jiang, Chi-Wing Fu, Pheng-Ann Heng
Date: 2019-03-26 00:00:00
Proceeding: ICCV 2019 10
Tasks: Shadow Removal

This paper presents a new method for shadow removal using unpaired data, enabling us to avoid tedious annotations and obtain more diverse training samples. However, directly employing adversarial learning and cycle-consistency constraints is insufficient to learn the underlying relationship between the shadow and shadow-free domains, since the mapping between shadow and shadow-free images is not simply one-to-one. To address the problem, we formulate Mask-ShadowGAN, a new deep framework that automatically learns to produce a shadow mask from the input shadow image and then takes the mask to guide the shadow generation via re-formulated cycle-consistency constraints. Particularly, the framework simultaneously learns to produce shadow masks and learns to remove shadows, to maximize the overall performance. Also, we prepared an unpaired dataset for shadow removal and demonstrated the effectiveness of Mask-ShadowGAN on various experiments, even it was trained on unpaired data.
Short-term Load Forecasting at Different Aggregation Levels with Predictability Analysis
Authors: Yayu Peng, Yishen Wang, Xiao Lu, Haifeng Li, Di Shi, Zhiwei Wang, Jie Li
Date: 2019-03-26 00:00:00
Tasks: Load Forecasting

Short-term load forecasting (STLF) is essential for the reliable and economic
operation of power systems. Though many STLF methods were proposed over the
past decades, most of them focused on loads at high aggregation levels only.
Thus, low-aggregation load forecast still requires further research and
development. Compared with the substation or city level loads, individual loads
are typically more volatile and much more challenging to forecast. To further
address this issue, this paper first discusses the characteristics of
small-and-medium enterprise (SME) and residential loads at different
aggregation levels and quantifies their predictability with approximate
entropy. Various STLF techniques, from the conventional linear regression to
state-of-the-art deep learning, are implemented for a detailed comparative
analysis to verify the forecasting performances as well as the predictability
using an Irish smart meter dataset. In addition, the paper also investigates
how using data processing improves individual-level residential load
forecasting with low predictability. Effectiveness of the discussed method is
validated with numerical results.
SciBERT: A Pretrained Language Model for Scientific Text
Authors: Iz Beltagy, Kyle Lo, Arman Cohan
Date: 2019-03-26 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Citation Intent Classification, Dependency Parsing, General Classification, Language Modeling, Language Modelling, Medical Named Entity Recognition, model, Named Entity Recognition (NER), Participant Intervention Comparison Outcome Extraction, Relation Extraction, Sentence, Sentence Classification

Obtaining large-scale annotated data for NLP tasks in the scientific domain is challenging and expensive. We release SciBERT, a pretrained language model based on BERT (Devlin et al., 2018) to address the lack of high-quality, large-scale labeled scientific data. SciBERT leverages unsupervised pretraining on a large multi-domain corpus of scientific publications to improve performance on downstream scientific NLP tasks. We evaluate on a suite of tasks including sequence tagging, sentence classification and dependency parsing, with datasets from a variety of scientific domains. We demonstrate statistically significant improvements over BERT and achieve new state-of-the-art results on several of these tasks. The code and pretrained models are available at https://github.com/allenai/scibert/.
Document Similarity for Texts of Varying Lengths via Hidden Topics
Authors: Hongyu Gong, Tarek Sakakini, Suma Bhat, JinJun Xiong
Date: 2019-03-26 00:00:00
Proceeding: ACL 2018 7
Tasks: Text Matching

Measuring similarity between texts is an important task for several
applications. Available approaches to measure document similarity are
inadequate for document pairs that have non-comparable lengths, such as a long
document and its summary. This is because of the lexical, contextual and the
abstraction gaps between a long document of rich details and its concise
summary of abstract information. In this paper, we present a document matching
approach to bridge this gap, by comparing the texts in a common space of hidden
topics. We evaluate the matching algorithm on two matching tasks and find that
it consistently and widely outperforms strong baselines. We also highlight the
benefits of incorporating domain knowledge to text matching.
Robustness of Neural Networks to Parameter Quantization
Authors: Abhishek Murthy, Himel Das, Md Ariful Islam
Date: 2019-03-26 00:00:00
Tasks: Binary Classification, Edge-computing, Quantization

Quantization, a commonly used technique to reduce the memory footprint of a
neural network for edge computing, entails reducing the precision of the
floating-point representation used for the parameters of the network. The
impact of such rounding-off errors on the overall performance of the neural
network is estimated using testing, which is not exhaustive and thus cannot be
used to guarantee the safety of the model. We present a framework based on
Satisfiability Modulo Theory (SMT) solvers to quantify the robustness of neural
networks to parameter perturbation. To this end, we introduce notions of local
and global robustness that capture the deviation in the confidence of class
assignments due to parameter quantization. The robustness notions are then cast
as instances of SMT problems and solved automatically using solvers, such as
dReal. We demonstrate our framework on two simple Multi-Layer Perceptrons (MLP)
that perform binary classification on a two-dimensional input. In addition to
quantifying the robustness, we also show that Rectified Linear Unit activation
results in higher robustness than linear activations for our MLPs.
Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus
Authors: Hongyu Gong, Suma Bhat, Lingfei Wu, JinJun Xiong, Wen-mei Hwu
Date: 2019-03-26 00:00:00
Proceeding: NAACL 2019 6
Tasks: Decoder, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sentence, Style Transfer, Text Style Transfer

Text style transfer rephrases a text from a source style (e.g., informal) to
a target style (e.g., formal) while keeping its original meaning. Despite the
success existing works have achieved using a parallel corpus for the two
styles, transferring text style has proven significantly more challenging when
there is no parallel training corpus. In this paper, we address this challenge
by using a reinforcement-learning-based generator-evaluator architecture. Our
generator employs an attention-based encoder-decoder to transfer a sentence
from the source style to the target style. Our evaluator is an adversarially
trained style discriminator with semantic and syntactic constraints that score
the generated sentence for style, meaning preservation, and fluency.
Experimental results on two different style transfer tasks (sentiment transfer
and formality transfer) show that our model outperforms state-of-the-art
approaches. Furthermore, we perform a manual evaluation that demonstrates the
effectiveness of the proposed method using subjective metrics of generated text
quality.
Blur Removal via Blurred-Noisy Image Pair
Authors: Chunzhi Gu, Xuequan Lu, Ying He, Chao Zhang
Date: 2019-03-26 00:00:00
Tasks: Deblurring, Image Deblurring, Optical Flow Estimation

Complex blur such as the mixup of space-variant and space-invariant blur, which is hard to model mathematically, widely exists in real images. In this paper, we propose a novel image deblurring method that does not need to estimate blur kernels. We utilize a pair of images that can be easily acquired in low-light situations: (1) a blurred image taken with low shutter speed and low ISO noise; and (2) a noisy image captured with high shutter speed and high ISO noise. Slicing the blurred image into patches, we extend the Gaussian mixture model (GMM) to model the underlying intensity distribution of each patch using the corresponding patches in the noisy image. We compute patch correspondences by analyzing the optical flow between the two images. The Expectation Maximization (EM) algorithm is utilized to estimate the parameters of GMM. To preserve sharp features, we add an additional bilateral term to the objective function in the M-step. We eventually add a detail layer to the deblurred image for refinement. Extensive experiments on both synthetic and real-world data demonstrate that our method outperforms state-of-the-art techniques, in terms of robustness, visual quality, and quantitative metrics.
Combination of Multiple Global Descriptors for Image Retrieval
Authors: HeeJae Jun, Byungsoo Ko, Youngjoon Kim, Insik Kim, Jongtack Kim
Date: 2019-03-26 00:00:00
Proceeding: arXiv 2019 3
Tasks: Image Retrieval, Retrieval

Recent studies in image retrieval task have shown that ensembling different models and combining multiple global descriptors lead to performance improvement. However, training different models for the ensemble is not only difficult but also inefficient with respect to time and memory. In this paper, we propose a novel framework that exploits multiple global descriptors to get an ensemble effect while it can be trained in an end-to-end manner. The proposed framework is flexible and expandable by the global descriptor, CNN backbone, loss, and dataset. Moreover, we investigate the effectiveness of combining multiple global descriptors with quantitative and qualitative analysis. Our extensive experiments show that the combined descriptor outperforms a single global descriptor, as it can utilize different types of feature properties. In the benchmark evaluation, the proposed framework achieves the state-of-the-art performance on the CARS196, CUB200-2011, In-shop Clothes, and Stanford Online Products on image retrieval tasks. Our model implementations and pretrained models are publicly available.
Semantic Alignment: Finding Semantically Consistent Ground-truth for Facial Landmark Detection
Authors: Zhiwei Liu, Xiangyu Zhu, Guosheng Hu, Haiyun Guo, Ming Tang, Zhen Lei, Neil M. Robertson, Jinqiao Wang
Date: 2019-03-26 00:00:00
Proceeding: CVPR 2019 6
Tasks: Face Alignment, Facial Landmark Detection

Recently, deep learning based facial landmark detection has achieved great
success. Despite this, we notice that the semantic ambiguity greatly degrades
the detection performance. Specifically, the semantic ambiguity means that some
landmarks (e.g. those evenly distributed along the face contour) do not have
clear and accurate definition, causing inconsistent annotations by annotators.
Accordingly, these inconsistent annotations, which are usually provided by
public databases, commonly work as the ground-truth to supervise network
training, leading to the degraded accuracy. To our knowledge, little research
has investigated this problem. In this paper, we propose a novel probabilistic
model which introduces a latent variable, i.e. the 'real' ground-truth which is
semantically consistent, to optimize. This framework couples two parts (1)
training landmark detection CNN and (2) searching the 'real' ground-truth.
These two parts are alternatively optimized: the searched 'real' ground-truth
supervises the CNN training; and the trained CNN assists the searching of
'real' ground-truth. In addition, to recover the unconfidently predicted
landmarks due to occlusion and low quality, we propose a global heatmap
correction unit (GHCU) to correct outliers by considering the global face shape
as a constraint. Extensive experiments on both image-based (300W and AFLW) and
video-based (300-VW) databases demonstrate that our method effectively improves
the landmark detection accuracy and achieves the state of the art performance.
Unpaired Image Captioning via Scene Graph Alignments
Authors: Jiuxiang Gu, Shafiq Joty, Jianfei Cai, Handong Zhao, Xu Yang, Gang Wang
Date: 2019-03-26 00:00:00
Proceeding: ICCV 2019 10
Tasks: Decoder, Image Captioning, Sentence

Most of current image captioning models heavily rely on paired image-caption datasets. However, getting large scale image-caption paired data is labor-intensive and time-consuming. In this paper, we present a scene graph-based approach for unpaired image captioning. Our framework comprises an image scene graph generator, a sentence scene graph generator, a scene graph encoder, and a sentence decoder. Specifically, we first train the scene graph encoder and the sentence decoder on the text modality. To align the scene graphs between images and sentences, we propose an unsupervised feature alignment method that maps the scene graph features from the image to the sentence modality. Experimental results show that our proposed model can generate quite promising results without using any image-caption training pairs, outperforming existing methods by a wide margin.
A Probabilistic Bitwise Genetic Algorithm for B-Spline based Image Deformation Estimation
Authors: Takumi Nakane, Takuya Akashi, Xuequan Lu, Chao Zhang
Date: 2019-03-26 00:00:00
Tasks: Diversity

We propose a novel genetic algorithm to solve the image deformation
estimation problem by preserving the genetic diversity. As a classical problem,
there is always a trade-off between the complexity of deformation models and
the difficulty of parameters search in image deformation. 2D cubic B-spline
surface is a highly free-form deformation model and is able to handle complex
deformations such as fluid image distortions. However, it is challenging to
estimate an apposite global solution. To tackle this problem, we develop a
genetic operation named probabilistic bitwise operation (PBO) to replace the
crossover and mutation operations, which can preserve the diversity during
generation iteration and achieve better coverage ratio of the solution space.
Furthermore, a selection strategy named annealing selection is proposed to
control the convergence. Qualitative and quantitative results on synthetic data
show the effectiveness of our method.
Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Driving
Authors: Akifumi Wachi
Date: 2019-03-26 00:00:00
Tasks: Autonomous Driving, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We examine the problem of adversarial reinforcement learning for multi-agent domains including a rule-based agent. Rule-based algorithms are required in safety-critical applications for them to work properly in a wide range of situations. Hence, every effort is made to find failure scenarios during the development phase. However, as the software becomes complicated, finding failure cases becomes difficult. Especially in multi-agent domains, such as autonomous driving environments, it is much harder to find useful failure scenarios that help us improve the algorithm. We propose a method for efficiently finding failure scenarios; this method trains the adversarial agents using multi-agent reinforcement learning such that the tested rule-based agent fails. We demonstrate the effectiveness of our proposed method using a simple environment and autonomous driving simulator.
Increasing Iterate Averaging for Solving Saddle-Point Problems
Authors: Yuan Gao, Christian Kroer, Donald Goldfarb
Date: 2019-03-26 00:00:00
Tasks: Denoising, Image Denoising

Many problems in machine learning and game theory can be formulated as saddle-point problems, for which various first-order methods have been developed and proven efficient in practice. Under the general convex-concave assumption, most first-order methods only guarantee an ergodic convergence rate, that is, the uniform averages of the iterates converge at a $O(1/T)$ rate in terms of the saddle-point residual. However, numerically, the iterates themselves can often converge much faster than the uniform averages. This observation motivates increasing averaging schemes that put more weight on later iterates, in contrast to the usual uniform averaging. We show that such increasing averaging schemes, applied to various first-order methods, are able to preserve the $O(1/T)$ convergence rate with no additional assumptions or computational overhead. Extensive numerical experiments on zero-sum game solving, market equilibrium computation and image denoising demonstrate the effectiveness of the proposed schemes. In particular, the increasing averages consistently outperform the uniform averages in all test problems by orders of magnitude. When solving matrix and extensive-form games, increasing averages consistently outperform the last iterates as well. For matrix games, a first-order method equipped with increasing averaging outperforms the highly competitive CFR$^+$ algorithm.