pwc-archive/papers-with-abstracts huggingface dataset:
Architecture Search of Dynamic Cells for Semantic Video Segmentation
Authors: Vladimir Nekrasov, Hao Chen, Chunhua Shen, Ian Reid
Date: 2019-04-04 00:00:00
Tasks: GPU, Neural Architecture Search, Optical Flow Estimation, Semantic Segmentation, Video Segmentation, Video Semantic Segmentation

In semantic video segmentation the goal is to acquire consistent dense
semantic labelling across image frames. To this end, recent approaches have
been reliant on manually arranged operations applied on top of static semantic
segmentation networks - with the most prominent building block being the
optical flow able to provide information about scene dynamics. Related to that
is the line of research concerned with speeding up static networks by
approximating expensive parts of them with cheaper alternatives, while
propagating information from previous frames. In this work we attempt to come
up with generalisation of those methods, and instead of manually designing
contextual blocks that connect per-frame outputs, we propose a neural
architecture search solution, where the choice of operations together with
their sequential arrangement are being predicted by a separate neural network.
We showcase that such generalisation leads to stable and accurate results
across common benchmarks, such as CityScapes and CamVid datasets. Importantly,
the proposed methodology takes only 2 GPU-days, finds high-performing cells and
does not rely on the expensive optical flow computation.
Template-Based Automatic Search of Compact Semantic Segmentation Architectures
Authors: Vladimir Nekrasov, Chunhua Shen, Ian Reid
Date: 2019-04-04 00:00:00
Tasks: General Classification, Holdout Set, Real-Time Semantic Segmentation, Reinforcement Learning, Semantic Segmentation

Automatic search of neural architectures for various vision and natural language tasks is becoming a prominent tool as it allows to discover high-performing structures on any dataset of interest. Nevertheless, on more difficult domains, such as dense per-pixel classification, current automatic approaches are limited in their scope - due to their strong reliance on existing image classifiers they tend to search only for a handful of additional layers with discovered architectures still containing a large number of parameters. In contrast, in this work we propose a novel solution able to find light-weight and accurate segmentation architectures starting from only few blocks of a pre-trained classification network. To this end, we progressively build up a methodology that relies on templates of sets of operations, predicts which template and how many times should be applied at each step, while also generating the connectivity structure and downsampling factors. All these decisions are being made by a recurrent neural network that is rewarded based on the score of the emitted architecture on the holdout set and trained using reinforcement learning. One discovered architecture achieves 63.2% mean IoU on CamVid and 67.8% on CityScapes having only 270K parameters. Pre-trained models and the search code are available at https://github.com/DrSleep/nas-segm-pytorch.
Spatiotemporal CNN for Video Object Segmentation
Authors: Kai Xu, Longyin Wen, Guorong Li, Liefeng Bo, Qingming Huang
Date: 2019-04-04 00:00:00
Proceeding: CVPR 2019 6
Tasks: Object, Segmentation, Semantic Segmentation, Semi-Supervised Video Object Segmentation, Video Object Segmentation, Video Segmentation, Video Semantic Segmentation, Visual Object Tracking

In this paper, we present a unified, end-to-end trainable spatiotemporal CNN
model for VOS, which consists of two branches, i.e., the temporal coherence
branch and the spatial segmentation branch. Specifically, the temporal
coherence branch pretrained in an adversarial fashion from unlabeled video
data, is designed to capture the dynamic appearance and motion cues of video
sequences to guide object segmentation. The spatial segmentation branch focuses
on segmenting objects accurately based on the learned appearance and motion
cues. To obtain accurate segmentation results, we design a coarse-to-fine
process to sequentially apply a designed attention module on multi-scale
feature maps, and concatenate them to produce the final prediction. In this
way, the spatial segmentation branch is enforced to gradually concentrate on
object regions. These two branches are jointly fine-tuned on video segmentation
sequences in an end-to-end manner. Several experiments are carried out on three
challenging datasets (i.e., DAVIS-2016, DAVIS-2017 and Youtube-Object) to show
that our method achieves favorable performance against the state-of-the-arts.
Code is available at https://github.com/longyin880815/STCNN.
A Robust Learning Approach to Domain Adaptive Object Detection
Authors: Mehran Khodabandeh, Arash Vahdat, Mani Ranjbar, William G. Macready
Date: 2019-04-04 00:00:00
Proceeding: ICCV 2019 10
Tasks: Domain Adaptation, Object, object-detection, Object Detection, Robust Object Detection, Self-Driving Cars

Domain shift is unavoidable in real-world applications of object detection. For example, in self-driving cars, the target domain consists of unconstrained road environments which cannot all possibly be observed in training data. Similarly, in surveillance applications sufficiently representative training data may be lacking due to privacy regulations. In this paper, we address the domain adaptation problem from the perspective of robust learning and show that the problem may be formulated as training with noisy labels. We propose a robust object detection framework that is resilient to noise in bounding box class labels, locations and size annotations. To adapt to the domain shift, the model is trained on the target domain using a set of noisy object bounding boxes that are obtained by a detection model trained only in the source domain. We evaluate the accuracy of our approach in various source/target domain pairs and demonstrate that the model significantly improves the state-of-the-art on multiple domain adaptation scenarios on the SIM10K, Cityscapes and KITTI datasets.
Lightweight Image Super-Resolution with Adaptive Weighted Learning Network
Authors: Chaofeng Wang, Zheng Li, Jun Shi
Date: 2019-04-04 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

PyTorch code for our paper "Lightweight Image Super-Resolution with Adaptive Weighted Learning Network"
Plan, Write, and Revise: an Interactive System for Open-Domain Story Generation
Authors: Seraphina Goldfarb-Tarrant, Haining Feng, Nanyun Peng
Date: 2019-04-04 00:00:00
Proceeding: NAACL 2019 6
Tasks: Diversity, Story Generation

Story composition is a challenging problem for machines and even for humans. We present a neural narrative generation system that interacts with humans to generate stories. Our system has different levels of human interaction, which enables us to understand at what stage of story-writing human collaboration is most productive, both to improving story quality and human engagement in the writing process. We compare different varieties of interaction in story-writing, story-planning, and diversity controls under time constraints, and show that increased types of human collaboration at both planning and writing stages results in a 10-50% improvement in story quality as compared to less interactive baselines. We also show an accompanying increase in user engagement and satisfaction with stories as compared to our own less interactive systems and to previous turn-taking approaches to interaction. Finally, we find that humans tasked with collaboratively improving a particular characteristic of a story are in fact able to do so, which has implications for future uses of human-in-the-loop systems.
MMED: A Multi-domain and Multi-modality Event Dataset
Authors: Zhenguo Yang, Zehang Lin, Min Cheng, Qing Li, Wenyin Liu
Date: 2019-04-04 00:00:00
Tasks: Articles, Question Answering, Retrieval, Visual Question Answering, Visual Question Answering (VQA)

In this work, we construct and release a multi-domain and multi-modality
event dataset (MMED), containing 25,165 textual news articles collected from
hundreds of news media sites (e.g., Yahoo News, Google News, CNN News.) and
76,516 image posts shared on Flickr social media, which are annotated according
to 412 real-world events. The dataset is collected to explore the problem of
organizing heterogeneous data contributed by professionals and amateurs in
different data domains, and the problem of transferring event knowledge
obtained from one data domain to heterogeneous data domain, thus summarizing
the data with different contributors. We hope that the release of the MMED
dataset can stimulate innovate research on related challenging problems, such
as event discovery, cross-modal (event) retrieval, and visual question
answering, etc.
Document-Level $N$-ary Relation Extraction with Multiscale Representation Learning
Authors: Robin Jia, Cliff Wong, Hoifung Poon
Date: 2019-04-04 00:00:00
Tasks: Reading Comprehension, Relation, Relation Extraction, Representation Learning, Sentence

Most information extraction methods focus on binary relations expressed within single sentences. In high-value domains, however, $n$-ary relations are of great demand (e.g., drug-gene-mutation interactions in precision oncology). Such relations often involve entity mentions that are far apart in the document, yet existing work on cross-sentence relation extraction is generally confined to small text spans (e.g., three consecutive sentences), which severely limits recall. In this paper, we propose a novel multiscale neural architecture for document-level $n$-ary relation extraction. Our system combines representations learned over various text spans throughout the document and across the subrelation hierarchy. Widening the system's purview to the entire document maximizes potential recall. Moreover, by integrating weak signals across the document, multiscale modeling increases precision, even in the presence of noisy labels from distant supervision. Experiments on biomedical machine reading show that our approach substantially outperforms previous $n$-ary relation extraction methods.
Preference Neural Network
Authors: Ayman Elgharabawy, Mukesh Prasad, Chin-Teng Lin
Date: 2019-04-04 00:00:00
Tasks: Computational Efficiency

This paper proposes a preference neural network (PNN) to address the problem of indifference preferences orders with new activation function. PNN also solves the Multi-label ranking problem, where labels may have indifference preference orders or subgroups are equally ranked. PNN follows a multi-layer feedforward architecture with fully connected neurons. Each neuron contains a novel smooth stairstep activation function based on the number of preference orders. PNN inputs represent data features and output neurons represent label indexes. The proposed PNN is evaluated using new preference mining dataset that contains repeated label values which have not experimented before. PNN outperforms five previously proposed methods for strict label ranking in terms of accurate results with high computational efficiency.
Density Matching for Bilingual Word Embedding
Authors: Chunting Zhou, Xuezhe Ma, Di Wang, Graham Neubig
Date: 2019-04-04 00:00:00
Proceeding: NAACL 2019 6
Tasks: Bilingual Lexicon Induction, Word Embeddings, Word Similarity

Recent approaches to cross-lingual word embedding have generally been based
on linear transformations between the sets of embedding vectors in the two
languages. In this paper, we propose an approach that instead expresses the two
monolingual embedding spaces as probability densities defined by a Gaussian
mixture model, and matches the two densities using a method called normalizing
flow. The method requires no explicit supervision, and can be learned with only
a seed dictionary of words that have identical strings. We argue that this
formulation has several intuitively attractive properties, particularly with
the respect to improving robustness and generalization to mappings between
difficult language pairs or word pairs. On a benchmark data set of bilingual
lexicon induction and cross-lingual word similarity, our approach can achieve
competitive or superior performance compared to state-of-the-art published
results, with particularly strong results being found on etymologically distant
and/or morphologically rich languages.
Text Generation from Knowledge Graphs with Graph Transformers
Authors: Rik Koncel-Kedziorski, Dhanush Bekal, Yi Luan, Mirella Lapata, Hannaneh Hajishirzi
Date: 2019-04-04 00:00:00
Proceeding: NAACL 2019 6
Tasks: Decoder, Dialogue Generation, KG-to-Text Generation, Knowledge Graphs, Sentence, Text Generation

Generating texts which express complex ideas spanning multiple sentences requires a structured representation of their content (document plan), but these representations are prohibitively expensive to manually produce. In this work, we address the problem of generating coherent multi-sentence texts from the output of an information extraction system, and in particular a knowledge graph. Graphical knowledge representations are ubiquitous in computing, but pose a significant challenge for text generation techniques due to their non-hierarchical nature, collapsing of long-distance dependencies, and structural variety. We introduce a novel graph transforming encoder which can leverage the relational structure of such knowledge graphs without imposing linearization or hierarchical constraints. Incorporated into an encoder-decoder setup, we provide an end-to-end trainable system for graph-to-text generation that we apply to the domain of scientific text. Automatic and human evaluations show that our technique produces more informative texts which exhibit better document structure than competitive encoder-decoder methods.
Online Risk-Bounded Motion Planning for Autonomous Vehicles in Dynamic Environments
Authors: Xin Huang, Sungkweon Hong, Andreas Hofmann, Brian C. Williams
Date: 2019-04-04 00:00:00
Tasks: Autonomous Vehicles, Heuristic Search, Intent Recognition, Motion Planning

A crucial challenge to efficient and robust motion planning for autonomous
vehicles is understanding the intentions of the surrounding agents. Ignoring
the intentions of the other agents in dynamic environments can lead to risky or
over-conservative plans. In this work, we model the motion planning problem as
a partially observable Markov decision process (POMDP) and propose an online
system that combines an intent recognition algorithm and a POMDP solver to
generate risk-bounded plans for the ego vehicle navigating with a number of
dynamic agent vehicles. The intent recognition algorithm predicts the
probabilistic hybrid motion states of each agent vehicle over a finite horizon
using Bayesian filtering and a library of pre-learned maneuver motion models.
We update the POMDP model with the intent recognition results in real time and
solve it using a heuristic search algorithm which produces policies with
upper-bound guarantees on the probability of near colliding with other dynamic
agents. We demonstrate that our system is able to generate better motion plans
in terms of efficiency and safety in a number of challenging environments
including unprotected intersection left turns and lane changes as compared to
the baseline methods.
Multi-View Intact Space Learning
Authors: Chang Xu, DaCheng Tao, Chao Xu
Date: 2019-04-04 00:00:00
Tasks: MULTI-VIEW LEARNING

It is practical to assume that an individual view is unlikely to be
sufficient for effective multi-view learning. Therefore, integration of
multi-view information is both valuable and necessary. In this paper, we
propose the Multi-view Intact Space Learning (MISL) algorithm, which integrates
the encoded complementary information in multiple views to discover a latent
intact representation of the data. Even though each view on its own is
insufficient, we show theoretically that by combing multiple views we can
obtain abundant information for latent intact space learning. Employing the
Cauchy loss (a technique used in statistical learning) as the error measurement
strengthens robustness to outliers. We propose a new definition of multi-view
stability and then derive the generalization error bound based on multi-view
stability and Rademacher complexity, and show that the complementarity between
multiple views is beneficial for the stability and generalization. MISL is
efficiently optimized using a novel Iteratively Reweight Residuals (IRR)
technique, whose convergence is theoretically analyzed. Experiments on
synthetic data and real-world datasets demonstrate that MISL is an effective
and promising algorithm for practical applications.
Consistency by Agreement in Zero-shot Neural Machine Translation
Authors: Maruan Al-Shedivat, Ankur P. Parikh
Date: 2019-04-04 00:00:00
Proceeding: NAACL 2019 6
Tasks: Machine Translation, NMT, Translation, Zero-shot Generalization, Zero-Shot Machine Translation

Generalization and reliability of multilingual translation often highly
depend on the amount of available parallel data for each language pair of
interest. In this paper, we focus on zero-shot generalization---a challenging
setup that tests models on translation directions they have not been optimized
for at training time. To solve the problem, we (i) reformulate multilingual
translation as probabilistic inference, (ii) define the notion of zero-shot
consistency and show why standard training often results in models unsuitable
for zero-shot tasks, and (iii) introduce a consistent agreement-based training
method that encourages the model to produce equivalent translations of parallel
sentences in auxiliary languages. We test our multilingual NMT models on
multiple public zero-shot translation benchmarks (IWSLT17, UN corpus, Europarl)
and show that agreement-based learning often results in 2-3 BLEU zero-shot
improvement over strong baselines without any loss in performance on supervised
translation directions.
Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation
Authors: Jiawei Wu, Xin Wang, William Yang Wang
Date: 2019-04-04 00:00:00
Proceeding: NAACL 2019 6
Tasks: Machine Translation, Sentence, Translation, Unsupervised Machine Translation

The overreliance on large parallel corpora significantly limits the
applicability of machine translation systems to the majority of language pairs.
Back-translation has been dominantly used in previous approaches for
unsupervised neural machine translation, where pseudo sentence pairs are
generated to train the models with a reconstruction loss. However, the pseudo
sentences are usually of low quality as translation errors accumulate during
training. To avoid this fundamental issue, we propose an alternative but more
effective approach, extract-edit, to extract and then edit real sentences from
the target monolingual corpora. Furthermore, we introduce a comparative
translation loss to evaluate the translated target sentences and thus train the
unsupervised translation systems. Experiments show that the proposed approach
consistently outperforms the previous state-of-the-art unsupervised machine
translation systems across two benchmarks (English-French and English-German)
and two low-resource language pairs (English-Romanian and English-Russian) by
more than 2 (up to 3.63) BLEU points.
Feature Pyramid Hashing
Authors: Yifan Yang, Libing Geng, Hanjiang Lai, Yan Pan, Jian Yin
Date: 2019-04-04 00:00:00
Tasks: Deep Hashing, Image Retrieval, Retrieval

In recent years, deep-networks-based hashing has become a leading approach
for large-scale image retrieval. Most deep hashing approaches use the high
layer to extract the powerful semantic representations. However, these methods
have limited ability for fine-grained image retrieval because the semantic
features extracted from the high layer are difficult in capturing the subtle
differences. To this end, we propose a novel two-pyramid hashing architecture
to learn both the semantic information and the subtle appearance details for
fine-grained image search. Inspired by the feature pyramids of convolutional
neural network, a vertical pyramid is proposed to capture the high-layer
features and a horizontal pyramid combines multiple low-layer features with
structural information to capture the subtle differences. To fuse the low-level
features, a novel combination strategy, called consensus fusion, is proposed to
capture all subtle information from several low-layers for finer retrieval.
Extensive evaluation on two fine-grained datasets CUB-200-2011 and Stanford
Dogs demonstrate that the proposed method achieves significant performance
compared with the state-of-art baselines.
Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations
Authors: Fred Hohman, Haekyu Park, Caleb Robinson, Duen Horng Chau
Date: 2019-04-04 00:00:00
Tasks: Decision Making

Deep learning is increasingly used in decision-making tasks. However, understanding how neural networks produce final predictions remains a fundamental challenge. Existing work on interpreting neural network predictions for images often focuses on explaining predictions for single images or neurons. As predictions are often computed from millions of weights that are optimized over millions of images, such explanations can easily miss a bigger picture. We present Summit, an interactive system that scalably and systematically summarizes and visualizes what features a deep learning model has learned and how those features interact to make predictions. Summit introduces two new scalable summarization techniques: (1) activation aggregation discovers important neurons, and (2) neuron-influence aggregation identifies relationships among such neurons. Summit combines these techniques to create the novel attribution graph that reveals and summarizes crucial neuron associations and substructures that contribute to a model's outcomes. Summit scales to large data, such as the ImageNet dataset with 1.2M images, and leverages neural network feature visualization and dataset examples to help users distill large, complex neural network models into compact, interactive visualizations. We present neural network exploration scenarios where Summit helps us discover multiple surprising insights into a prevalent, large-scale image classifier's learned representations and informs future neural network architecture design. The Summit visualization runs in modern web browsers and is open-sourced.
Guiding Extractive Summarization with Question-Answering Rewards
Authors: Kristjan Arumae, Fei Liu
Date: 2019-04-04 00:00:00
Proceeding: NAACL 2019 6
Tasks: Extractive Summarization, Question Answering

Highlighting while reading is a natural behavior for people to track salient
content of a document. It would be desirable to teach an extractive summarizer
to do the same. However, a major obstacle to the development of a supervised
summarizer is the lack of ground-truth. Manual annotation of extraction units
is cost-prohibitive, whereas acquiring labels by automatically aligning human
abstracts and source documents can yield inferior results. In this paper we
describe a novel framework to guide a supervised, extractive summarization
system with question-answering rewards. We argue that quality summaries should
serve as a document surrogate to answer important questions, and such
question-answer pairs can be conveniently obtained from human abstracts. The
system learns to promote summaries that are informative, fluent, and perform
competitively on question-answering. Our results compare favorably with those
reported by strong summarization baselines as evaluated by automatic metrics
and human assessors.