pwc-archive/papers-with-abstracts huggingface dataset:
On Catastrophic Forgetting and Mode Collapse in Generative Adversarial Networks
Authors: Hoang Thanh-Tung, Truyen Tran
Date: 2018-07-11 00:00:00
Tasks: Continual Learning

In this paper, we show that Generative Adversarial Networks (GANs) suffer from catastrophic forgetting even when they are trained to approximate a single target distribution. We show that GAN training is a continual learning problem in which the sequence of changing model distributions is the sequence of tasks to the discriminator. The level of mismatch between tasks in the sequence determines the level of forgetting. Catastrophic forgetting is interrelated to mode collapse and can make the training of GANs non-convergent. We investigate the landscape of the discriminator's output in different variants of GANs and find that when a GAN converges to a good equilibrium, real training datapoints are wide local maxima of the discriminator. We empirically show the relationship between the sharpness of local maxima and mode collapse and generalization in GANs. We show how catastrophic forgetting prevents the discriminator from making real datapoints local maxima, and thus causes non-convergence. Finally, we study methods for preventing catastrophic forgetting in GANs.
Medusa: A Scalable Interconnect for Many-Port DNN Accelerators and Wide DRAM Controller Interfaces
Authors: Yongming Shen, Tianchu Ji, Michael Ferdman, Peter Milder
Date: 2018-07-11 00:00:00

To cope with the increasing demand and computational intensity of deep neural
networks (DNNs), industry and academia have turned to accelerator technologies.
In particular, FPGAs have been shown to provide a good balance between
performance and energy efficiency for accelerating DNNs. While significant
research has focused on how to build efficient layer processors, the
computational building blocks of DNN accelerators, relatively little attention
has been paid to the on-chip interconnects that sit between the layer
processors and the FPGA's DRAM controller.
  We observe a disparity between DNN accelerator interfaces, which tend to
comprise many narrow ports, and FPGA DRAM controller interfaces, which tend to
be wide buses. This mismatch causes traditional interconnects to consume
significant FPGA resources. To address this problem, we designed Medusa: an
optimized FPGA memory interconnect which transposes data in the interconnect
fabric, tailoring the interconnect to the needs of DNN layer processors.
Compared to a traditional FPGA interconnect, our design can reduce LUT and FF
use by 4.7x and 6.0x, and improves frequency by 1.8x.
Causal Discovery in the Presence of Missing Data
Authors: Ruibo Tu, Kun Zhang, Paul Ackermann, Bo Christer Bertilson, Clark Glymour, Hedvig Kjellström, Cheng Zhang
Date: 2018-07-11 00:00:00
Tasks: Causal Discovery, Diagnostic

Missing data are ubiquitous in many domains including healthcare. When these data entries are not missing completely at random, the (conditional) independence relations in the observed data may be different from those in the complete data generated by the underlying causal process. Consequently, simply applying existing causal discovery methods to the observed data may lead to wrong conclusions. In this paper, we aim at developing a causal discovery method to recover the underlying causal structure from observed data that follow different missingness mechanisms, including missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR). With missingness mechanisms represented by missingness graphs, we analyse conditions under which additional correction is needed to derive conditional independence/dependence relations in the complete data. Based on our analysis, we propose the Missing Value PC (MVPC) algorithm for both continuous and binary variables, which extends the PC algorithm to incorporate additional corrections. Our proposed MVPC is shown in theory to give asymptotically correct results even on data that are MAR or MNAR. Experimental results on synthetic data show that the proposed algorithm is able to find correct causal relations even in the general case of MNAR. Moreover, we create a neuropathic pain diagnostic simulator for evaluating causal discovery methods. Evaluated on such simulated neuropathic pain diagnosis records and the other two real world applications, MVPC outperforms the other benchmark methods.
Learning Neural Models for End-to-End Clustering
Authors: Benjamin Bruno Meier, Ismail Elezi, Mohammadreza Amirian, Oliver Durr, Thilo Stadelmann
Date: 2018-07-11 00:00:00
Tasks: Clustering, Metric Learning

We propose a novel end-to-end neural network architecture that, once trained,
directly outputs a probabilistic clustering of a batch of input examples in one
pass. It estimates a distribution over the number of clusters $k$, and for each
$1 \leq k \leq k_\mathrm{max}$, a distribution over the individual cluster
assignment for each data point. The network is trained in advance in a
supervised fashion on separate data to learn grouping by any perceptual
similarity criterion based on pairwise labels (same/different group). It can
then be applied to different data containing different groups. We demonstrate
promising performance on high-dimensional data like images (COIL-100) and
speech (TIMIT). We call this ``learning to cluster'' and show its conceptual
difference to deep metric learning, semi-supervise clustering and other related
approaches while having the advantage of performing learnable clustering fully
end-to-end.
Testing Global Constraints
Authors: Aurélie Massart, Valentin Rombouts, Pierre Schaus
Date: 2018-07-11 00:00:00
Tasks: ARC

Every Constraint Programming (CP) solver exposes a library of constraints for
solving combinatorial problems. In order to be useful, CP solvers need to be
bug-free. Therefore the testing of the solver is crucial to make developers and
users confident. We present a Java library allowing any JVM based solver to
test that the implementations of the individual constraints are correct. The
library can be used in a test suite executed in a continuous integration tool
or it can also be used to discover minimalist instances violating some
properties (arc-consistency, etc) in order to help the developer to identify
the origin of the problem using standard debuggers.
Deep attention-based classification network for robust depth prediction
Authors: Ruibo Li, Ke Xian, Chunhua Shen, Zhiguo Cao, Hao Lu, Lingxiao Hang
Date: 2018-07-11 00:00:00
Tasks: Classification, Deep Attention, Depth Estimation, Depth Prediction, General Classification, Multi-class Classification, Prediction, Quantization

In this paper, we present our deep attention-based classification (DABC)
network for robust single image depth prediction, in the context of the Robust
Vision Challenge 2018 (ROB 2018). Unlike conventional depth prediction, our
goal is to design a model that can perform well in both indoor and outdoor
scenes with a single parameter set. However, robust depth prediction suffers
from two challenging problems: a) How to extract more discriminative features
for different scenes (compared to a single scene)? b) How to handle the large
differences of depth ranges between indoor and outdoor datasets? To address
these two problems, we first formulate depth prediction as a multi-class
classification task and apply a softmax classifier to classify the depth label
of each pixel. We then introduce a global pooling layer and a channel-wise
attention mechanism to adaptively select the discriminative channels of
features and to update the original features by assigning important channels
with higher weights. Further, to reduce the influence of quantization errors,
we employ a soft-weighted sum inference strategy for the final prediction.
Experimental results on both indoor and outdoor datasets demonstrate the
effectiveness of our method. It is worth mentioning that we won the 2-nd place
in single image depth prediction entry of ROB 2018, in conjunction with IEEE
Conference on Computer Vision and Pattern Recognition (CVPR) 2018.
An improved neural network model for joint POS tagging and dependency parsing
Authors: Dat Quoc Nguyen, Karin Verspoor
Date: 2018-07-11 00:00:00
Proceeding: CONLL 2018 10
Tasks: Dependency Parsing, Event Extraction, Part-Of-Speech Tagging, POS, POS Tagging

We propose a novel neural network model for joint part-of-speech (POS)
tagging and dependency parsing. Our model extends the well-known BIST
graph-based dependency parser (Kiperwasser and Goldberg, 2016) by incorporating
a BiLSTM-based tagging component to produce automatically predicted POS tags
for the parser. On the benchmark English Penn treebank, our model obtains
strong UAS and LAS scores at 94.51% and 92.87%, respectively, producing 1.5+%
absolute improvements to the BIST graph-based parser, and also obtaining a
state-of-the-art POS tagging accuracy at 97.97%. Furthermore, experimental
results on parsing 61 "big" Universal Dependencies treebanks from raw texts
show that our model outperforms the baseline UDPipe (Straka and Strakov\'a,
2017) with 0.8% higher average POS tagging score and 3.6% higher average LAS
score. In addition, with our model, we also obtain state-of-the-art downstream
task scores for biomedical event extraction and opinion analysis applications.
Our code is available together with all pre-trained models at:
https://github.com/datquocnguyen/jPTDP
Knowledge Extracted from Recurrent Deep Belief Network for Real Time Deterministic Control
Authors: Shin Kamada, Takumi Ichimura
Date: 2018-07-11 00:00:00
Tasks: Deep Learning, General Classification, GPU, Time Series, Time Series Analysis

Recently, the market on deep learning including not only software but also
hardware is developing rapidly. Big data is collected through IoT devices and
the industry world will analyze them to improve their manufacturing process.
Deep Learning has the hierarchical network architecture to represent the
complicated features of input patterns. Although deep learning can show the
high capability of classification, prediction, and so on, the implementation on
GPU devices are required. We may meet the trade-off between the higher
precision by deep learning and the higher cost with GPU devices. We can success
the knowledge extraction from the trained deep learning with high
classification capability. The knowledge that can realize faster inference of
pre-trained deep network is extracted as IF-THEN rules from the network signal
flow given input data. Some experiment results with benchmark tests for time
series data sets showed the effectiveness of our proposed method related to the
computational speed.
Adaptive Learning Method of Recurrent Temporal Deep Belief Network to Analyze Time Series Data
Authors: Takumi Ichimura, Shin Kamada
Date: 2018-07-11 00:00:00
Tasks: Time Series, Time Series Analysis

Deep Learning has the hierarchical network architecture to represent the
complicated features of input patterns. Such architecture is well known to
represent higher learning capability compared with some conventional models if
the best set of parameters in the optimal network structure is found. We have
been developing the adaptive learning method that can discover the optimal
network structure in Deep Belief Network (DBN). The learning method can
construct the network structure with the optimal number of hidden neurons in
each Restricted Boltzmann Machine and with the optimal number of layers in the
DBN during learning phase. The network structure of the learning method can be
self-organized according to given input patterns of big data set. In this
paper, we embed the adaptive learning method into the recurrent temporal RBM
and the self-generated layer into DBN. In order to verify the effectiveness of
our proposed method, the experimental results are higher classification
capability than the conventional methods in this paper.
Shortening Time Required for Adaptive Structural Learning Method of Deep Belief Network with Multi-Modal Data Arrangement
Authors: Shin Kamada, Takumi Ichimura
Date: 2018-07-11 00:00:00
Tasks: Deep Learning, Time Series, Time Series Analysis

Recently, Deep Learning has been applied in the techniques of artificial
intelligence. Especially, Deep Learning performed good results in the field of
image recognition. Most new Deep Learning architectures are naturally developed
in image recognition. For this reason, not only the numerical data and text
data but also the time-series data are transformed to the image data format.
Multi-modal data consists of two or more kinds of data such as picture and
text. The arrangement in a general method is formed in the squared array with
no specific aim. In this paper, the data arrangement are modified according to
the similarity of input-output pattern in Adaptive Structural Learning method
of Deep Belief Network. The similarity of output signals of hidden neurons is
made by the order rearrangement of hidden neurons. The experimental results for
the data rearrangement in squared array showed the shortening time required for
DBN learning.
Towards Understanding End-of-trip Instructions in a Taxi Ride Scenario
Authors: Deepthi Karkada, Ramesh Manuvinakurike, Kallirroi Georgila
Date: 2018-07-11 00:00:00
Proceeding: COLING 2018 8

We introduce a dataset containing human-authored descriptions of target
locations in an "end-of-trip in a taxi ride" scenario. We describe our data
collection method and a novel annotation scheme that supports understanding of
such descriptions of target locations. Our dataset contains target location
descriptions for both synthetic and real-world images as well as visual
annotations (ground truth labels, dimensions of vehicles and objects,
coordinates of the target location,distance and direction of the target
location from vehicles and objects) that can be used in various visual and
language tasks. We also perform a pilot experiment on how the corpus could be
applied to visual reference resolution in this domain.
A Dialogue Annotation Scheme for Weight Management Chat using the Trans-Theoretical Model of Health Behavior Change
Authors: Ramesh Manuvinakurike, Sumanth Bharadwaj, Kallirroi Georgila
Date: 2018-07-11 00:00:00
Proceeding: COLING 2018 8
Tasks: General Classification, Management

In this study we collect and annotate human-human role-play dialogues in the
domain of weight management. There are two roles in the conversation: the
"seeker" who is looking for ways to lose weight and the "helper" who provides
suggestions to help the "seeker" in their weight loss journey. The chat
dialogues collected are then annotated with a novel annotation scheme inspired
by a popular health behavior change theory called "trans-theoretical model of
health behavior change". We also build classifiers to automatically predict the
annotation labels used in our corpus. We find that classification accuracy
improves when oracle segmentations of the interlocutors' sentences are provided
compared to directly classifying unsegmented sentences.
Instance-based entropy fuzzy support vector machine for imbalanced data
Authors: Poongjin Cho, Minhyuk Lee, Woojin Chang
Date: 2018-07-11 00:00:00
Tasks: BIG-bench Machine Learning, Diversity, imbalanced classification

Imbalanced classification has been a major challenge for machine learning
because many standard classifiers mainly focus on balanced datasets and tend to
have biased results towards the majority class. We modify entropy fuzzy support
vector machine (EFSVM) and introduce instance-based entropy fuzzy support
vector machine (IEFSVM). Both EFSVM and IEFSVM use the entropy information of
k-nearest neighbors to determine the fuzzy membership value for each sample
which prioritizes the importance of each sample. IEFSVM considers the diversity
of entropy patterns for each sample when increasing the size of neighbors, k,
while EFSVM uses single entropy information of the fixed size of neighbors for
all samples. By varying k, we can reflect the component change of sample's
neighbors from near to far distance in the determination of fuzzy value
membership. Numerical experiments on 35 public and 12 real-world imbalanced
datasets are performed to validate IEFSVM and area under the receiver operating
characteristic curve (AUC) is used to compare its performance with other SVMs
and machine learning methods. IEFSVM shows a much higher AUC value for datasets
with high imbalance ratio, implying that IEFSVM is effective in dealing with
the class imbalance problem.
A Hierarchical Bayesian Linear Regression Model with Local Features for Stochastic Dynamics Approximation
Authors: Behnoosh Parsa, Keshav Rajasekaran, Franziska Meier, Ashis G. Banerjee
Date: 2018-07-11 00:00:00
Tasks: Model-based Reinforcement Learning, regression, Reinforcement Learning

One of the challenges in model-based control of stochastic dynamical systems
is that the state transition dynamics are involved, and it is not easy or
efficient to make good-quality predictions of the states. Moreover, there are
not many representational models for the majority of autonomous systems, as it
is not easy to build a compact model that captures the entire dynamical
subtleties and uncertainties. In this work, we present a hierarchical Bayesian
linear regression model with local features to learn the dynamics of a
micro-robotic system as well as two simpler examples, consisting of a
stochastic mass-spring damper and a stochastic double inverted pendulum on a
cart. The model is hierarchical since we assume non-stationary priors for the
model parameters. These non-stationary priors make the model more flexible by
imposing priors on the priors of the model. To solve the maximum likelihood
(ML) problem for this hierarchical model, we use the variational expectation
maximization (EM) algorithm, and enhance the procedure by introducing hidden
target variables. The algorithm yields parsimonious model structures, and
consistently provides fast and accurate predictions for all our examples
involving large training and test sets. This demonstrates the effectiveness of
the method in learning stochastic dynamics, which makes it suitable for future
use in a paradigm, such as model-based reinforcement learning, to compute
optimal control policies in real time.
Quantification under prior probability shift: the ratio estimator and its extensions
Authors: Afonso Fernandes Vaz, Rafael Izbicki, Rafael Bassi Stern
Date: 2018-07-11 00:00:00

The quantification problem consists of determining the prevalence of a given
label in a target population. However, one often has access to the labels in a
sample from the training population but not in the target population. A common
assumption in this situation is that of prior probability shift, that is, once
the labels are known, the distribution of the features is the same in the
training and target populations. In this paper, we derive a new lower bound for
the risk of the quantification problem under the prior shift assumption.
Complementing this lower bound, we present a new approximately minimax class of
estimators, ratio estimators, which generalize several previous proposals in
the literature. Using a weaker version of the prior shift assumption, which can
be tested, we show that ratio estimators can be used to build confidence
intervals for the quantification problem. We also extend the ratio estimator so
that it can: (i) incorporate labels from the target population, when they are
available and (ii) estimate how the prevalence of positive labels varies
according to a function of certain covariates.
Generative Adversarial Networks with Decoder-Encoder Output Noise
Authors: Guoqiang Zhong, Wei Gao, Yongbin Liu, Youzhao Yang
Date: 2018-07-11 00:00:00
Tasks: Decoder, Image Generation, Variational Inference

In recent years, research on image generation methods has been developing
fast. The auto-encoding variational Bayes method (VAEs) was proposed in 2013,
which uses variational inference to learn a latent space from the image
database and then generates images using the decoder. The generative
adversarial networks (GANs) came out as a promising framework, which uses
adversarial training to improve the generative ability of the generator.
However, the generated pictures by GANs are generally blurry. The deep
convolutional generative adversarial networks (DCGANs) were then proposed to
leverage the quality of generated images. Since the input noise vectors are
randomly sampled from a Gaussian distribution, the generator has to map from a
whole normal distribution to the images. This makes DCGANs unable to reflect
the inherent structure of the training data. In this paper, we propose a novel
deep model, called generative adversarial networks with decoder-encoder output
noise (DE-GANs), which takes advantage of both the adversarial training and the
variational Bayesain inference to improve the performance of image generation.
DE-GANs use a pre-trained decoder-encoder architecture to map the random
Gaussian noise vectors to informative ones and pass them to the generator of
the adversarial networks. Since the decoder-encoder architecture is trained by
the same images as the generators, the output vectors could carry the intrinsic
distribution information of the original images. Moreover, the loss function of
DE-GANs is different from GANs and DCGANs. A hidden-space loss function is
added to the adversarial loss function to enhance the robustness of the model.
Extensive empirical results show that DE-GANs can accelerate the convergence of
the adversarial training process and improve the quality of the generated
images.
Discovering Interesting Plots in Production Yield Data Analytics
Authors: Matthew Nero, Chuanhe Shan, Li-C. Wang, Nik Sumikawa
Date: 2018-07-11 00:00:00

An analytic process is iterative between two agents, an analyst and an
analytic toolbox. Each iteration comprises three main steps: preparing a
dataset, running an analytic tool, and evaluating the result, where dataset
preparation and result evaluation, conducted by the analyst, are largely
domain-knowledge driven. In this work, the focus is on automating the result
evaluation step. The underlying problem is to identify plots that are deemed
interesting by an analyst. We propose a methodology to learn such analyst's
intent based on Generative Adversarial Networks (GANs) and demonstrate its
applications in the context of production yield optimization using data
collected from several product lines.
Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis
Authors: Hai Pham, Thomas Manzini, Paul Pu Liang, Barnabas Poczos
Date: 2018-07-11 00:00:00
Proceeding: WS 2018 7
Tasks: Multimodal Sentiment Analysis, Sentiment Analysis, Translation

Multimodal machine learning is a core research area spanning the language,
visual and acoustic modalities. The central challenge in multimodal learning
involves learning representations that can process and relate information from
multiple modalities. In this paper, we propose two methods for unsupervised
learning of joint multimodal representations using sequence to sequence
(Seq2Seq) methods: a \textit{Seq2Seq Modality Translation Model} and a
\textit{Hierarchical Seq2Seq Modality Translation Model}. We also explore
multiple different variations on the multimodal inputs and outputs of these
seq2seq models. Our experiments on multimodal sentiment analysis using the
CMU-MOSI dataset indicate that our methods learn informative multimodal
representations that outperform the baselines and achieve improved performance
on multimodal sentiment analysis, specifically in the Bimodal case where our
model is able to improve F1 Score by twelve points. We also discuss future
directions for multimodal Seq2Seq methods.