pwc-archive/papers-with-abstracts huggingface dataset:
The Shallow End: Empowering Shallower Deep-Convolutional Networks through Auxiliary Outputs
Authors: Yong Guo, Jian Chen, Qing Du, Anton Van Den Hengel, Qinfeng Shi, Mingkui Tan
Date: 2016-11-06 00:00:00
Tasks: Model Compression, Model Selection

Depth is one of the key factors behind the success of convolutional neural networks (CNNs). Since ResNet, we are able to train very deep CNNs as the gradient vanishing issue has been largely addressed by the introduction of skip connections. However, we observe that, when the depth is very large, the intermediate layers (especially shallow layers) may fail to receive sufficient supervision from the loss due to the severe transformation through a long backpropagation path. As a result, the representation power of intermediate layers can be very weak and the model becomes very redundant with limited performance. In this paper, we first investigate the supervision vanishing issue in existing backpropagation (BP) methods. And then, we propose to address it via an effective method, called Multi-way BP (MW-BP), which relies on multiple auxiliary losses added to the intermediate layers of the network. The proposed MW-BP method can be applied to most deep architectures with slight modifications, such as ResNet and MobileNet. Our method often gives rise to much more compact models (denoted by "Mw+Architecture") than existing methods. For example, MwResNet-44 with 44 layers performs better than ResNet-110 with 110 layers on CIFAR-10 and CIFAR-100. More critically, the resultant models even outperform the light models obtained by state-of-the-art model compression methods. Last, our method inherently produces multiple compact models with different depths at the same time, which is helpful for model selection.
Residual Attention Network for Image Classification
Authors: Fei Wang, Mengqing Jiang, Chen Qian, Shuo Yang, Cheng Li, Honggang Zhang, Xiaogang Wang, Xiaoou Tang
Date: 2017-04-23 00:00:00
Proceeding: CVPR 2017 7
Tasks: General Classification, image-classification, Image Classification, Object Recognition

In this work, we propose "Residual Attention Network", a convolutional neural
network using attention mechanism which can incorporate with state-of-art feed
forward network architecture in an end-to-end training fashion. Our Residual
Attention Network is built by stacking Attention Modules which generate
attention-aware features. The attention-aware features from different modules
change adaptively as layers going deeper. Inside each Attention Module,
bottom-up top-down feedforward structure is used to unfold the feedforward and
feedback attention process into a single feedforward process. Importantly, we
propose attention residual learning to train very deep Residual Attention
Networks which can be easily scaled up to hundreds of layers. Extensive
analyses are conducted on CIFAR-10 and CIFAR-100 datasets to verify the
effectiveness of every module mentioned above. Our Residual Attention Network
achieves state-of-the-art object recognition performance on three benchmark
datasets including CIFAR-10 (3.90% error), CIFAR-100 (20.45% error) and
ImageNet (4.8% single model and single crop, top-5 error). Note that, our
method achieves 0.6% top-1 accuracy improvement with 46% trunk depth and 69%
forward FLOPs comparing to ResNet-200. The experiment also demonstrates that
our network is robust against noisy labels.
Identifying Consistent Statements about Numerical Data with Dispersion-Corrected Subgroup Discovery
Authors: Mario Boley, Bryan R. Goldsmith, Luca M. Ghiringhelli, Jilles Vreeken
Date: 2017-01-26 00:00:00
Tasks: Subgroup Discovery

Existing algorithms for subgroup discovery with numerical targets do not
optimize the error or target variable dispersion of the groups they find. This
often leads to unreliable or inconsistent statements about the data, rendering
practical applications, especially in scientific domains, futile. Therefore, we
here extend the optimistic estimator framework for optimal subgroup discovery
to a new class of objective functions: we show how tight estimators can be
computed efficiently for all functions that are determined by subgroup size
(non-decreasing dependence), the subgroup median value, and a dispersion
measure around the median (non-increasing dependence). In the important special
case when dispersion is measured using the average absolute deviation from the
median, this novel approach yields a linear time algorithm. Empirical
evaluation on a wide range of datasets shows that, when used within
branch-and-bound search, this approach is highly efficient and indeed discovers
subgroups with much smaller errors.
Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning
Authors: Yacine Jernite, Samuel R. Bowman, David Sontag
Date: 2017-04-23 00:00:00
Tasks: Representation Learning, Sentence

This work presents a novel objective function for the unsupervised training
of neural network sentence encoders. It exploits signals from paragraph-level
discourse coherence to train these models to understand text. Our objective is
purely discriminative, allowing us to train models many times faster than was
possible under prior methods, and it yields models which perform well in
extrinsic evaluations.
Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision
Authors: Chen Liang, Jonathan Berant, Quoc Le, Kenneth D. Forbus, Ni Lao
Date: 2016-10-31 00:00:00
Proceeding: ACL 2017 7
Tasks: Feature Engineering, Structured Prediction

Harnessing the statistical power of neural networks to perform language
understanding and symbolic reasoning is difficult, when it requires executing
efficient discrete operations against a large knowledge-base. In this work, we
introduce a Neural Symbolic Machine, which contains (a) a neural "programmer",
i.e., a sequence-to-sequence model that maps language utterances to programs
and utilizes a key-variable memory to handle compositionality (b) a symbolic
"computer", i.e., a Lisp interpreter that performs program execution, and helps
find good programs by pruning the search space. We apply REINFORCE to directly
optimize the task reward of this structured prediction problem. To train with
weak supervision and improve the stability of REINFORCE, we augment it with an
iterative maximum-likelihood training process. NSM outperforms the
state-of-the-art on the WebQuestionsSP dataset when trained from
question-answer pairs only, without requiring any feature engineering or
domain-specific knowledge.
A General Theory for Training Learning Machine
Authors: Hong Zhao
Date: 2017-04-23 00:00:00
Tasks: BIG-bench Machine Learning

Though the deep learning is pushing the machine learning to a new stage,
basic theories of machine learning are still limited. The principle of
learning, the role of the a prior knowledge, the role of neuron bias, and the
basis for choosing neural transfer function and cost function, etc., are still
far from clear. In this paper, we present a general theoretical framework for
machine learning. We classify the prior knowledge into common and
problem-dependent parts, and consider that the aim of learning is to maximally
incorporate them. The principle we suggested for maximizing the former is the
design risk minimization principle, while the neural transfer function, the
cost function, as well as pretreatment of samples, are endowed with the role
for maximizing the latter. The role of the neuron bias is explained from a
different angle. We develop a Monte Carlo algorithm to establish the
input-output responses, and we control the input-output sensitivity of a
learning machine by controlling that of individual neurons. Applications of
function approaching and smoothing, pattern recognition and classification, are
provided to illustrate how to train general learning machines based on our
theory and algorithm. Our method may in addition induce new applications, such
as the transductive inference.
Misspecified Linear Bandits
Authors: Avishek Ghosh, Sayak Ray Chowdhury, Aditya Gopalan
Date: 2017-04-23 00:00:00
Tasks: Learning-To-Rank

We consider the problem of online learning in misspecified linear stochastic
multi-armed bandit problems. Regret guarantees for state-of-the-art linear
bandit algorithms such as Optimism in the Face of Uncertainty Linear bandit
(OFUL) hold under the assumption that the arms expected rewards are perfectly
linear in their features. It is, however, of interest to investigate the impact
of potential misspecification in linear bandit models, where the expected
rewards are perturbed away from the linear subspace determined by the arms
features. Although OFUL has recently been shown to be robust to relatively
small deviations from linearity, we show that any linear bandit algorithm that
enjoys optimal regret performance in the perfectly linear setting (e.g., OFUL)
must suffer linear regret under a sparse additive perturbation of the linear
model. In an attempt to overcome this negative result, we define a natural
class of bandit models characterized by a non-sparse deviation from linearity.
We argue that the OFUL algorithm can fail to achieve sublinear regret even
under models that have non-sparse deviation.We finally develop a novel bandit
algorithm, comprising a hypothesis test for linearity followed by a decision to
use either the OFUL or Upper Confidence Bound (UCB) algorithm. For perfectly
linear bandit models, the algorithm provably exhibits OFULs favorable regret
performance, while for misspecified models satisfying the non-sparse deviation
property, the algorithm avoids the linear regret phenomenon and falls back on
UCBs sublinear regret scaling. Numerical experiments on synthetic data, and on
recommendation data from the public Yahoo! Learning to Rank Challenge dataset,
empirically support our findings.
Deep Keyphrase Generation
Authors: Rui Meng, Sanqiang Zhao, Shuguang Han, Daqing He, Peter Brusilovsky, Yu Chi
Date: 2017-04-23 00:00:00
Proceeding: ACL 2017 7
Tasks: Decoder, Keyphrase Extraction, Keyphrase Generation

Keyphrase provides highly-condensed information that can be effectively used for understanding, organizing and retrieving text content. Though previous studies have provided many workable solutions for automated keyphrase extraction, they commonly divided the to-be-summarized content into multiple text chunks, then ranked and selected the most meaningful ones. These approaches could neither identify keyphrases that do not appear in the text, nor capture the real semantic meaning behind the text. We propose a generative model for keyphrase prediction with an encoder-decoder framework, which can effectively overcome the above drawbacks. We name it as deep keyphrase generation since it attempts to capture the deep semantic meaning of the content with a deep learning method. Empirical analysis on six datasets demonstrates that our proposed model not only achieves a significant performance boost on extracting keyphrases that appear in the source text, but also can generate absent keyphrases based on the semantic meaning of the text. Code and dataset are available at https://github.com/memray/OpenNMT-kpg-release.
Cognitive Mapping and Planning for Visual Navigation
Authors: Saurabh Gupta, Varun Tolani, James Davidson, Sergey Levine, Rahul Sukthankar, Jitendra Malik
Date: 2017-02-13 00:00:00
Proceeding: CVPR 2017 7
Tasks: Visual Navigation

We introduce a neural architecture for navigation in novel environments. Our
proposed architecture learns to map from first-person views and plans a
sequence of actions towards goals in the environment. The Cognitive Mapper and
Planner (CMP) is based on two key ideas: a) a unified joint architecture for
mapping and planning, such that the mapping is driven by the needs of the task,
and b) a spatial memory with the ability to plan given an incomplete set of
observations about the world. CMP constructs a top-down belief map of the world
and applies a differentiable neural net planner to produce the next action at
each time step. The accumulated belief of the world enables the agent to track
visited regions of the environment. We train and test CMP on navigation
problems in simulation environments derived from scans of real world buildings.
Our experiments demonstrate that CMP outperforms alternate learning-based
architectures, as well as, classical mapping and path planning approaches in
many cases. Furthermore, it naturally extends to semantically specified goals,
such as 'going to a chair'. We also deploy CMP on physical robots in indoor
environments, where it achieves reasonable performance, even though it is
trained entirely in simulation.
Argument Mining with Structured SVMs and RNNs
Authors: Vlad Niculae, Joonsuk Park, Claire Cardie
Date: 2017-04-23 00:00:00
Proceeding: ACL 2017 7
Tasks: Argument Mining, General Classification, Relation Prediction

We propose a novel factor graph model for argument mining, designed for
settings in which the argumentative relations in a document do not necessarily
form a tree structure. (This is the case in over 20% of the web comments
dataset we release.) Our model jointly learns elementary unit type
classification and argumentative relation prediction. Moreover, our model
supports SVM and RNN parametrizations, can enforce structure constraints (e.g.,
transitivity), and can express dependencies between adjacent relations and
propositions. Our approaches outperform unstructured baselines in both web
comments and argumentative essay datasets.
A Review on Deep Learning Techniques Applied to Semantic Segmentation
Authors: Alberto Garcia-Garcia, Sergio Orts-Escolano, Sergiu Oprea, Victor Villena-Martinez, Jose Garcia-Rodriguez
Date: 2017-04-22 00:00:00
Tasks: Autonomous Driving, Deep Learning, Scene Understanding, Segmentation, Semantic Segmentation

Image semantic segmentation is more and more being of interest for computer
vision and machine learning researchers. Many applications on the rise need
accurate and efficient segmentation mechanisms: autonomous driving, indoor
navigation, and even virtual or augmented reality systems to name a few. This
demand coincides with the rise of deep learning approaches in almost every
field or application target related to computer vision, including semantic
segmentation or scene understanding. This paper provides a review on deep
learning methods for semantic segmentation applied to various application
areas. Firstly, we describe the terminology of this field as well as mandatory
background concepts. Next, the main datasets and challenges are exposed to help
researchers decide which are the ones that best suit their needs and their
targets. Then, existing methods are reviewed, highlighting their contributions
and their significance in the field. Finally, quantitative results are given
for the described methods and the datasets in which they were evaluated,
following up with a discussion of the results. At last, we point out a set of
promising future works and draw our own conclusions about the state of the art
of semantic segmentation using deep learning techniques.
Affect-LM: A Neural Language Model for Customizable Affective Text Generation
Authors: Sayan Ghosh, Mathieu Chollet, Eugene Laksana, Louis-Philippe Morency, Stefan Scherer
Date: 2017-04-22 00:00:00
Proceeding: ACL 2017 7
Tasks: Language Modeling, Language Modelling, Text Generation

Human verbal communication includes affective messages which are conveyed
through use of emotionally colored words. There has been a lot of research in
this direction but the problem of integrating state-of-the-art neural language
models with affective information remains an area ripe for exploration. In this
paper, we propose an extension to an LSTM (Long Short-Term Memory) language
model for generating conversational text, conditioned on affect categories. Our
proposed model, Affect-LM enables us to customize the degree of emotional
content in generated sentences through an additional design parameter.
Perception studies conducted using Amazon Mechanical Turk show that Affect-LM
generates naturally looking emotional sentences without sacrificing grammatical
correctness. Affect-LM also learns affect-discriminative word representations,
and perplexity experiments show that additional affective information in
conversational text can improve language model prediction.
A hybrid primal heuristic for Robust Multiperiod Network Design
Authors: Fabio D'Andreagiovanni, Jonatan Krolikowski, Jonad Pulaj
Date: 2017-04-22 00:00:00

We investigate the Robust Multiperiod Network Design Problem, a
generalization of the classical Capacitated Network Design Problem that
additionally considers multiple design periods and provides solutions protected
against traffic uncertainty. Given the intrinsic difficulty of the problem,
which proves challenging even for state-of-the art commercial solvers, we
propose a hybrid primal heuristic based on the combination of ant colony
optimization and an exact large neighborhood search. Computational experiments
on a set of realistic instances from the SNDlib show that our heuristic can
find solutions of extremely good quality with low optimality gap.
Semi-Supervised QA with Generative Domain-Adaptive Nets
Authors: Zhilin Yang, Junjie Hu, Ruslan Salakhutdinov, William W. Cohen
Date: 2017-02-07 00:00:00
Proceeding: ACL 2017 7
Tasks: Domain Adaptation, Question Answering, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We study the problem of semi-supervised question answering----utilizing
unlabeled text to boost the performance of question answering models. We
propose a novel training framework, the Generative Domain-Adaptive Nets. In
this framework, we train a generative model to generate questions based on the
unlabeled text, and combine model-generated questions with human-generated
questions for training question answering models. We develop novel domain
adaptation algorithms, based on reinforcement learning, to alleviate the
discrepancy between the model-generated data distribution and the
human-generated data distribution. Experiments show that our proposed framework
obtains substantial improvement from unlabeled text.
Video Acceleration Magnification
Authors: Yichao Zhang, Silvia L. Pintea, Jan C. van Gemert
Date: 2017-04-13 00:00:00
Proceeding: CVPR 2017 7
Tasks: Motion Magnification, Optical Flow Estimation, Video Editing

The ability to amplify or reduce subtle image changes over time is useful in
contexts such as video editing, medical video analysis, product quality control
and sports. In these contexts there is often large motion present which
severely distorts current video amplification methods that magnify change
linearly. In this work we propose a method to cope with large motions while
still magnifying small changes. We make the following two observations: i)
large motions are linear on the temporal scale of the small changes; ii) small
changes deviate from this linearity. We ignore linear motion and propose to
magnify acceleration. Our method is pure Eulerian and does not require any
optical flow, temporal alignment or region annotations. We link temporal
second-order derivative filtering to spatial acceleration magnification. We
apply our method to moving objects where we show motion magnification and color
magnification. We provide quantitative as well as qualitative evidence for our
method while comparing to the state-of-the-art.
On the Two-View Geometry of Unsynchronized Cameras
Authors: Cenek Albl, Zuzana Kukelova, Andrew Fitzgibbon, Jan Heller, Matej Smid, Tomas Pajdla
Date: 2017-04-22 00:00:00
Proceeding: CVPR 2017 7
Tasks: Vocal Bursts Valence Prediction

We present new methods for simultaneously estimating camera geometry and time
shift from video sequences from multiple unsynchronized cameras. Algorithms for
simultaneous computation of a fundamental matrix or a homography with unknown
time shift between images are developed. Our methods use minimal correspondence
sets (eight for fundamental matrix and four and a half for homography) and
therefore are suitable for robust estimation using RANSAC. Furthermore, we
present an iterative algorithm that extends the applicability on sequences
which are significantly unsynchronized, finding the correct time shift up to
several seconds. We evaluated the methods on synthetic and wide range of real
world datasets and the results show a broad applicability to the problem of
camera synchronization.
Medical Text Classification using Convolutional Neural Networks
Authors: Mark Hughes, Irene Li, Spyros Kotoulas, Toyotaro Suzumura
Date: 2017-04-22 00:00:00
Tasks: Classification, General Classification, Sentence, text-classification, Text Classification

We present an approach to automatically classify clinical text at a sentence
level. We are using deep convolutional neural networks to represent complex
features. We train the network on a dataset providing a broad categorization of
health information. Through a detailed evaluation, we demonstrate that our
method outperforms several approaches widely used in natural language
processing tasks by about 15%.
Sarcasm SIGN: Interpreting Sarcasm with Sentiment Based Monolingual Machine Translation
Authors: Lotem Peled, Roi Reichart
Date: 2017-04-22 00:00:00
Proceeding: ACL 2017 7
Tasks: Machine Translation, Translation

Sarcasm is a form of speech in which speakers say the opposite of what they
truly mean in order to convey a strong sentiment. In other words, "Sarcasm is
the giant chasm between what I say, and the person who doesn't get it.". In
this paper we present the novel task of sarcasm interpretation, defined as the
generation of a non-sarcastic utterance conveying the same message as the
original sarcastic one. We introduce a novel dataset of 3000 sarcastic tweets,
each interpreted by five human judges. Addressing the task as monolingual
machine translation (MT), we experiment with MT algorithms and evaluation
measures. We then present SIGN: an MT based sarcasm interpretation algorithm
that targets sentiment words, a defining element of textual sarcasm. We show
that while the scores of n-gram based automatic measures are similar for all
interpretation models, SIGN's interpretations are scored higher by humans for
adequacy and sentiment polarity. We conclude with a discussion on future
research directions for our new task.
Learning Representations of Emotional Speech with Deep Convolutional Generative Adversarial Networks
Authors: Jonathan Chang, Stefan Scherer
Date: 2017-04-22 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, Generative Adversarial Network, Representation Learning

Automatically assessing emotional valence in human speech has historically
been a difficult task for machine learning algorithms. The subtle changes in
the voice of the speaker that are indicative of positive or negative emotional
states are often "overshadowed" by voice characteristics relating to emotional
intensity or emotional activation. In this work we explore a representation
learning approach that automatically derives discriminative representations of
emotional speech. In particular, we investigate two machine learning strategies
to improve classifier performance: (1) utilization of unlabeled data using a
deep convolutional generative adversarial network (DCGAN), and (2) multitask
learning. Within our extensive experiments we leverage a multitask annotated
emotional corpus as well as a large unlabeled meeting corpus (around 100
hours). Our speaker-independent classification experiments show that in
particular the use of unlabeled data in our investigations improves performance
of the classifiers and both fully supervised baseline approaches are
outperformed considerably. We improve the classification of emotional valence
on a discrete 5-point scale to 43.88% and on a 3-point scale to 49.80%, which
is competitive to state-of-the-art performance.