pwc-archive/papers-with-abstracts huggingface dataset:
Cold-Start Aware User and Product Attention for Sentiment Classification
Authors: Reinald Kim Amplayo, Jihyeok Kim, Sua Sung, Seung-won Hwang
Date: 2018-06-14 00:00:00
Proceeding: ACL 2018 7
Tasks: Classification, General Classification, Sentiment Analysis, Sentiment Classification

The use of user/product information in sentiment analysis is important,
especially for cold-start users/products, whose number of reviews are very
limited. However, current models do not deal with the cold-start problem which
is typical in review websites. In this paper, we present Hybrid Contextualized
Sentiment Classifier (HCSC), which contains two modules: (1) a fast word
encoder that returns word vectors embedded with short and long range dependency
features; and (2) Cold-Start Aware Attention (CSAA), an attention mechanism
that considers the existence of cold-start problem when attentively pooling the
encoded word vectors. HCSC introduces shared vectors that are constructed from
similar users/products, and are used when the original distinct vectors do not
have sufficient information (i.e. cold-start). This is decided by a
frequency-guided selective gate vector. Our experiments show that in terms of
RMSE, HCSC performs significantly better when compared with on famous datasets,
despite having less complexity, and thus can be trained much faster. More
importantly, our model performs significantly better than previous models when
the training data is sparse and has cold-start problems.
Dense Light Field Reconstruction From Sparse Sampling Using Residual Network
Authors: Mantang Guo, Hao Zhu, Guoqing Zhou, Qing Wang
Date: 2018-06-14 00:00:00

A light field records numerous light rays from a real-world scene. However,
capturing a dense light field by existing devices is a time-consuming process.
Besides, reconstructing a large amount of light rays equivalent to multiple
light fields using sparse sampling arises a severe challenge for existing
methods. In this paper, we present a learning based method to reconstruct
multiple novel light fields between two mutually independent light fields. We
indicate that light rays distributed in different light fields have the same
consistent constraints under a certain condition. The most significant
constraint is a depth related correlation between angular and spatial
dimensions. Our method avoids working out the error-sensitive constraint by
employing a deep neural network. We solve residual values of pixels on epipolar
plane image (EPI) to reconstruct novel light fields. Our method is able to
reconstruct 2 to 4 novel light fields between two mutually independent input
light fields. We also compare our results with those yielded by a number of
alternatives elsewhere in the literature, which shows our reconstructed light
fields have better structure similarity and occlusion relationship.
Entity Commonsense Representation for Neural Abstractive Summarization
Authors: Reinald Kim Amplayo, Seonjae Lim, Seung-won Hwang
Date: 2018-06-14 00:00:00
Proceeding: NAACL 2018 6
Tasks: Abstractive Text Summarization, Decoder, Entity Linking, Sentence

A major proportion of a text summary includes important entities found in the
original text. These entities build up the topic of the summary. Moreover, they
hold commonsense information once they are linked to a knowledge base. Based on
these observations, this paper investigates the usage of linked entities to
guide the decoder of a neural text summarizer to generate concise and better
summaries. To this end, we leverage on an off-the-shelf entity linking system
(ELS) to extract linked entities and propose Entity2Topic (E2T), a module
easily attachable to a sequence-to-sequence model that transforms a list of
entities into a vector representation of the topic of the summary. Current
available ELS's are still not sufficiently effective, possibly introducing
unresolved ambiguities and irrelevant entities. We resolve the imperfections of
the ELS by (a) encoding entities with selective disambiguation, and (b) pooling
entity vectors using firm attention. By applying E2T to a simple
sequence-to-sequence model with attention mechanism as base model, we see
significant improvements of the performance in the Gigaword (sentence to title)
and CNN (long document to multi-sentence highlights) summarization datasets by
at least 2 ROUGE points.
Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace
Authors: Yoonho Lee, Seungjin Choi
Date: 2018-01-17 00:00:00
Proceeding: ICML 2018 7
Tasks: Few-Shot Image Classification, Meta-Learning

Gradient-based meta-learning methods leverage gradient descent to learn the
commonalities among various tasks. While previous such methods have been
successful in meta-learning tasks, they resort to simple gradient descent
during meta-testing. Our primary contribution is the {\em MT-net}, which
enables the meta-learner to learn on each layer's activation space a subspace
that the task-specific learner performs gradient descent on. Additionally, a
task-specific learner of an {\em MT-net} performs gradient descent with respect
to a meta-learned distance metric, which warps the activation space to be more
sensitive to task identity. We demonstrate that the dimension of this learned
subspace reflects the complexity of the task-specific learner's adaptation
task, and also that our model is less sensitive to the choice of initial
learning rates than previous gradient-based meta-learning methods. Our method
achieves state-of-the-art or comparable performance on few-shot classification
and regression tasks.
Aspect Sentiment Model for Micro Reviews
Authors: Reinald Kim Amplayo, Seung-won Hwang
Date: 2018-06-14 00:00:00
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), model, Sentiment Analysis, Sentiment Classification, Term Extraction, Topic Models

This paper aims at an aspect sentiment model for aspect-based sentiment
analysis (ABSA) focused on micro reviews. This task is important in order to
understand short reviews majority of the users write, while existing topic
models are targeted for expert-level long reviews with sufficient co-occurrence
patterns to observe. Current methods on aggregating micro reviews using
metadata information may not be effective as well due to metadata absence,
topical heterogeneity, and cold start problems. To this end, we propose a model
called Micro Aspect Sentiment Model (MicroASM). MicroASM is based on the
observation that short reviews 1) are viewed with sentiment-aspect word pairs
as building blocks of information, and 2) can be clustered into larger reviews.
When compared to the current state-of-the-art aspect sentiment models,
experiments show that our model provides better performance on aspect-level
tasks such as aspect term extraction and document-level tasks such as sentiment
classification.
On Accurate Evaluation of GANs for Language Generation
Authors: Stanislau Semeniuta, Aliaksei Severyn, Sylvain Gelly
Date: 2018-06-13 00:00:00
Tasks: Diversity, Language Modeling, Language Modelling, Text Generation

Generative Adversarial Networks (GANs) are a promising approach to language generation. The latest works introducing novel GAN models for language generation use n-gram based metrics for evaluation and only report single scores of the best run. In this paper, we argue that this often misrepresents the true picture and does not tell the full story, as GAN models can be extremely sensitive to the random initialization and small deviations from the best hyperparameter choice. In particular, we demonstrate that the previously used BLEU score is not sensitive to semantic deterioration of generated texts and propose alternative metrics that better capture the quality and diversity of the generated samples. We also conduct a set of experiments comparing a number of GAN models for text with a conventional Language Model (LM) and find that neither of the considered models performs convincingly better than the LM.
Dynamic Video Segmentation Network
Authors: Yu-Syuan Xu, Tsu-Jui Fu, Hsuan-Kung Yang, Chun-Yi Lee
Date: 2018-04-03 00:00:00
Proceeding: CVPR 2018 6
Tasks: Segmentation, Video Segmentation, Video Semantic Segmentation

In this paper, we present a detailed design of dynamic video segmentation
network (DVSNet) for fast and efficient semantic video segmentation. DVSNet
consists of two convolutional neural networks: a segmentation network and a
flow network. The former generates highly accurate semantic segmentations, but
is deeper and slower. The latter is much faster than the former, but its output
requires further processing to generate less accurate semantic segmentations.
We explore the use of a decision network to adaptively assign different frame
regions to different networks based on a metric called expected confidence
score. Frame regions with a higher expected confidence score traverse the flow
network. Frame regions with a lower expected confidence score have to pass
through the segmentation network. We have extensively performed experiments on
various configurations of DVSNet, and investigated a number of variants for the
proposed decision network. The experimental results show that our DVSNet is
able to achieve up to 70.4% mIoU at 19.8 fps on the Cityscape dataset. A high
speed version of DVSNet is able to deliver an fps of 30.4 with 63.2% mIoU on
the same dataset. DVSNet is also able to reduce up to 95% of the computational
workloads.
Learning a Tree-Structured Ising Model in Order to Make Predictions
Authors: Guy Bresler, Mina Karzand
Date: 2016-04-22 00:00:00

We study the problem of learning a tree Ising model from samples such that
subsequent predictions made using the model are accurate. The prediction task
considered in this paper is that of predicting the values of a subset of
variables given values of some other subset of variables. Virtually all
previous work on graphical model learning has focused on recovering the true
underlying graph. We define a distance ("small set TV" or ssTV) between
distributions $P$ and $Q$ by taking the maximum, over all subsets $\mathcal{S}$
of a given size, of the total variation between the marginals of $P$ and $Q$ on
$\mathcal{S}$; this distance captures the accuracy of the prediction task of
interest. We derive non-asymptotic bounds on the number of samples needed to
get a distribution (from the same class) with small ssTV relative to the one
generating the samples. One of the main messages of this paper is that far
fewer samples are needed than for recovering the underlying tree, which means
that accurate predictions are possible using the wrong tree.
Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo
Authors: Marton Havasi, José Miguel Hernández-Lobato, Juan José Murillo-Fuentes
Date: 2018-06-14 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Gaussian Processes, Variational Inference

Deep Gaussian Processes (DGPs) are hierarchical generalizations of Gaussian
Processes that combine well calibrated uncertainty estimates with the high
flexibility of multilayer models. One of the biggest challenges with these
models is that exact inference is intractable. The current state-of-the-art
inference method, Variational Inference (VI), employs a Gaussian approximation
to the posterior distribution. This can be a potentially poor unimodal
approximation of the generally multimodal posterior. In this work, we provide
evidence for the non-Gaussian nature of the posterior and we apply the
Stochastic Gradient Hamiltonian Monte Carlo method to generate samples. To
efficiently optimize the hyperparameters, we introduce the Moving Window MCEM
algorithm. This results in significantly better predictions at a lower
computational cost than its VI counterpart. Thus our method establishes a new
state-of-the-art for inference in DGPs.
Nearly Zero-Shot Learning for Semantic Decoding in Spoken Dialogue Systems
Authors: Lina M. Rojas-Barahona, Stefan Ultes, Pawel Budzianowski, Iñigo Casanueva, Milica Gasic, Bo-Hsiang Tseng, Steve Young
Date: 2018-06-14 00:00:00
Tasks: speech-recognition, Speech Recognition, Spoken Dialogue Systems, Zero-Shot Learning

This paper presents two ways of dealing with scarce data in semantic decoding
using N-Best speech recognition hypotheses. First, we learn features by using a
deep learning architecture in which the weights for the unknown and known
categories are jointly optimised. Second, an unsupervised method is used for
further tuning the weights. Sharing weights injects prior knowledge to unknown
categories. The unsupervised tuning (i.e. the risk minimisation) improves the
F-Measure when recognising nearly zero-shot data on the DSTC3 corpus. This
unsupervised method can be applied subject to two assumptions: the rank of the
class marginal is assumed to be known and the class-conditional scores of the
classifier are assumed to follow a Gaussian distribution.
Morphological and Language-Agnostic Word Segmentation for NMT
Authors: Dominik Macháček, Jonáš Vidra, Ondřej Bojar
Date: 2018-06-14 00:00:00
Tasks: GPU, Machine Translation, NMT, Translation

The state of the art of handling rich morphology in neural machine
translation (NMT) is to break word forms into subword units, so that the
overall vocabulary size of these units fits the practical limits given by the
NMT model and GPU memory capacity. In this paper, we compare two common but
linguistically uninformed methods of subword construction (BPE and STE, the
method implemented in Tensor2Tensor toolkit) and two linguistically-motivated
methods: Morfessor and one novel method, based on a derivational dictionary.
Our experiments with German-to-Czech translation, both morphologically rich,
document that so far, the non-motivated methods perform better. Furthermore, we
iden- tify a critical difference between BPE and STE and show a simple pre-
processing step for BPE that considerably increases translation quality as
evaluated by automatic measures.
Automatic Language Identification for Romance Languages using Stop Words and Diacritics
Authors: Ciprian-Octavian Truică, Julien Velcin, Alexandru Boicea
Date: 2018-06-14 00:00:00
Tasks: Language Identification

Automatic language identification is a natural language processing problem
that tries to determine the natural language of a given content. In this paper
we present a statistical method for automatic language identification of
written text using dictionaries containing stop words and diacritics. We
propose different approaches that combine the two dictionaries to accurately
determine the language of textual corpora. This method was chosen because stop
words and diacritics are very specific to a language, although some languages
have some similar words and special characters they are not all common. The
languages taken into account were romance languages because they are very
similar and usually it is hard to distinguish between them from a computational
point of view. We have tested our method using a Twitter corpus and a news
article corpus. Both corpora consists of UTF-8 encoded text, so the diacritics
could be taken into account, in the case that the text has no diacritics only
the stop words are used to determine the language of the text. The experimental
results show that the proposed method has an accuracy of over 90% for small
texts and over 99.8% for
Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Data
Authors: Jacson Rodrigues Correia-Silva, Rodrigo F. Berriel, Claudine Badue, Alberto F. de Souza, Thiago Oliveira-Santos
Date: 2018-06-14 00:00:00

In the past few years, Convolutional Neural Networks (CNNs) have been
achieving state-of-the-art performance on a variety of problems. Many companies
employ resources and money to generate these models and provide them as an API,
therefore it is in their best interest to protect them, i.e., to avoid that
someone else copies them. Recent studies revealed that state-of-the-art CNNs
are vulnerable to adversarial examples attacks, and this weakness indicates
that CNNs do not need to operate in the problem domain (PD). Therefore, we
hypothesize that they also do not need to be trained with examples of the PD in
order to operate in it.
  Given these facts, in this paper, we investigate if a target black-box CNN
can be copied by persuading it to confess its knowledge through random
non-labeled data. The copy is two-fold: i) the target network is queried with
random data and its predictions are used to create a fake dataset with the
knowledge of the network; and ii) a copycat network is trained with the fake
dataset and should be able to achieve similar performance as the target
network.
  This hypothesis was evaluated locally in three problems (facial expression,
object, and crosswalk classification) and against a cloud-based API. In the
copy attacks, images from both non-problem domain and PD were used. All copycat
networks achieved at least 93.7% of the performance of the original models with
non-problem domain data, and at least 98.6% using additional data from the PD.
Additionally, the copycat CNN successfully copied at least 97.3% of the
performance of the Microsoft Azure Emotion API. Our results show that it is
possible to create a copycat CNN by simply querying a target network as
black-box with random non-labeled data.
Efficient Active Learning for Image Classification and Segmentation using a Sample Selection and Conditional Generative Adversarial Network
Authors: Dwarikanath Mahapatra, Behzad Bozorgtabar, Jean-Philippe Thiran, Mauricio Reyes
Date: 2018-06-14 00:00:00
Tasks: Active Learning, General Classification, Generative Adversarial Network, image-classification, Image Classification, Medical Image Classification

Training robust deep learning (DL) systems for medical image classification or segmentation is challenging due to limited images covering different disease types and severity. We propose an active learning (AL) framework to select most informative samples and add to the training data. We use conditional generative adversarial networks (cGANs) to generate realistic chest xray images with different disease characteristics by conditioning its generation on a real image sample. Informative samples to add to the training set are identified using a Bayesian neural network. Experiments show our proposed AL framework is able to achieve state of the art performance by using about 35% of the full dataset, thus saving significant time and effort over conventional methods.
Relation Networks for Object Detection
Authors: Han Hu, Jiayuan Gu, Zheng Zhang, Jifeng Dai, Yichen Wei
Date: 2017-11-30 00:00:00
Proceeding: CVPR 2018 6
Tasks: Object, object-detection, Object Detection, Object Recognition, Relation

Although it is well believed for years that modeling relations between
objects would help object recognition, there has not been evidence that the
idea is working in the deep learning era. All state-of-the-art object detection
systems still rely on recognizing object instances individually, without
exploiting their relations during learning.
  This work proposes an object relation module. It processes a set of objects
simultaneously through interaction between their appearance feature and
geometry, thus allowing modeling of their relations. It is lightweight and
in-place. It does not require additional supervision and is easy to embed in
existing networks. It is shown effective on improving object recognition and
duplicate removal steps in the modern object detection pipeline. It verifies
the efficacy of modeling object relations in CNN based detection. It gives rise
to the first fully end-to-end object detector.
DiCE: The Infinitely Differentiable Monte-Carlo Estimator
Authors: Jakob Foerster, Gregory Farquhar, Maruan Al-Shedivat, Tim Rocktäschel, Eric P. Xing, Shimon Whiteson
Date: 2018-02-14 00:00:00
Tasks: Meta-Learning, Reinforcement Learning

The score function estimator is widely used for estimating gradients of
stochastic objectives in stochastic computation graphs (SCG), eg, in
reinforcement learning and meta-learning. While deriving the first-order
gradient estimators by differentiating a surrogate loss (SL) objective is
computationally and conceptually simple, using the same approach for
higher-order derivatives is more challenging. Firstly, analytically deriving
and implementing such estimators is laborious and not compliant with automatic
differentiation. Secondly, repeatedly applying SL to construct new objectives
for each order derivative involves increasingly cumbersome graph manipulations.
Lastly, to match the first-order gradient under differentiation, SL treats part
of the cost as a fixed sample, which we show leads to missing and wrong terms
for estimators of higher-order derivatives. To address all these shortcomings
in a unified way, we introduce DiCE, which provides a single objective that can
be differentiated repeatedly, generating correct estimators of derivatives of
any order in SCGs. Unlike SL, DiCE relies on automatic differentiation for
performing the requisite graph manipulations. We verify the correctness of DiCE
both through a proof and numerical evaluation of the DiCE derivative estimates.
We also use DiCE to propose and evaluate a novel approach for multi-agent
learning. Our code is available at https://www.github.com/alshedivat/lola.
Learning Cross-lingual Distributed Logical Representations for Semantic Parsing
Authors: Yanyan Zou, Wei Lu
Date: 2018-06-14 00:00:00
Proceeding: ACL 2018 7
Tasks: Semantic Parsing

With the development of several multilingual datasets used for semantic
parsing, recent research efforts have looked into the problem of learning
semantic parsers in a multilingual setup. However, how to improve the
performance of a monolingual semantic parser for a specific language by
leveraging data annotated in different languages remains a research question
that is under-explored. In this work, we present a study to show how learning
distributed representations of the logical forms from data annotated in
different languages can be used for improving the performance of a monolingual
semantic parser. We extend two existing monolingual semantic parsers to
incorporate such cross-lingual distributed logical representations as features.
Experiments show that our proposed approach is able to yield improved semantic
parsing results on the standard multilingual GeoQuery dataset.
Gradient Layer: Enhancing the Convergence of Adversarial Training for Generative Models
Authors: Atsushi Nitanda, Taiji Suzuki
Date: 2018-01-07 00:00:00

We propose a new technique that boosts the convergence of training generative
adversarial networks. Generally, the rate of training deep models reduces
severely after multiple iterations. A key reason for this phenomenon is that a
deep network is expressed using a highly non-convex finite-dimensional model,
and thus the parameter gets stuck in a local optimum. Because of this, methods
often suffer not only from degeneration of the convergence speed but also from
limitations in the representational power of the trained network. To overcome
this issue, we propose an additional layer called the gradient layer to seek a
descent direction in an infinite-dimensional space. Because the layer is
constructed in the infinite-dimensional space, we are not restricted by the
specific model structure of finite-dimensional models. As a result, we can get
out of the local optima in finite-dimensional models and move towards the
global optimal function more directly. In this paper, this phenomenon is
explained from the functional gradient method perspective of the gradient
layer. Interestingly, the optimization procedure using the gradient layer
naturally constructs the deep structure of the network. Moreover, we
demonstrate that this procedure can be regarded as a discretization method of
the gradient flow that naturally reduces the objective function. Finally, the
method is tested using several numerical experiments, which show its fast
convergence.