pwc-archive/papers-with-abstracts huggingface dataset:
Decoupled Novel Object Captioner
Authors: Yu Wu, Linchao Zhu, Lu Jiang, Yi Yang
Date: 2018-04-11 00:00:00
Tasks: Image Captioning, Novel Concepts, Object, Sentence

Image captioning is a challenging task where the machine automatically
describes an image by sentences or phrases. It often requires a large number of
paired image-sentence annotations for training. However, a pre-trained
captioning model can hardly be applied to a new domain in which some novel
object categories exist, i.e., the objects and their description words are
unseen during model training. To correctly caption the novel object, it
requires professional human workers to annotate the images by sentences with
the novel words. It is labor expensive and thus limits its usage in real-world
applications.
  In this paper, we introduce the zero-shot novel object captioning task where
the machine generates descriptions without extra sentences about the novel
object. To tackle the challenging problem, we propose a Decoupled Novel Object
Captioner (DNOC) framework that can fully decouple the language sequence model
from the object descriptions. DNOC has two components. 1) A Sequence Model with
the Placeholder (SM-P) generates a sentence containing placeholders. The
placeholder represents an unseen novel object. Thus, the sequence model can be
decoupled from the novel object descriptions. 2) A key-value object memory
built upon the freely available detection model, contains the visual
information and the corresponding word for each object. The SM-P will generate
a query to retrieve the words from the object memory. The placeholder will then
be filled with the correct word, resulting in a caption with novel object
descriptions. The experimental results on the held-out MSCOCO dataset
demonstrate the ability of DNOC in describing novel concepts in the zero-shot
novel object captioning task.
CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes
Authors: Yuhong Li, Xiaofan Zhang, Deming Chen
Date: 2018-02-27 00:00:00
Proceeding: CVPR 2018 6
Tasks: Crowd Counting, Scene Recognition

We propose a network for Congested Scene Recognition called CSRNet to provide
a data-driven and deep learning method that can understand highly congested
scenes and perform accurate count estimation as well as present high-quality
density maps. The proposed CSRNet is composed of two major components: a
convolutional neural network (CNN) as the front-end for 2D feature extraction
and a dilated CNN for the back-end, which uses dilated kernels to deliver
larger reception fields and to replace pooling operations. CSRNet is an
easy-trained model because of its pure convolutional structure. We demonstrate
CSRNet on four datasets (ShanghaiTech dataset, the UCF_CC_50 dataset, the
WorldEXPO'10 dataset, and the UCSD dataset) and we deliver the state-of-the-art
performance. In the ShanghaiTech Part_B dataset, CSRNet achieves 47.3% lower
Mean Absolute Error (MAE) than the previous state-of-the-art method. We extend
the targeted applications for counting other objects, such as the vehicle in
TRANCOS dataset. Results show that CSRNet significantly improves the output
quality with 15.4% lower MAE than the previous state-of-the-art approach.
Achieving Fluency and Coherency in Task-oriented Dialog
Authors: Rashmi Gangadharaiah, Balakrishnan Narayanaswamy, Charles Elkan
Date: 2018-04-11 00:00:00

We consider real world task-oriented dialog settings, where agents need to
generate both fluent natural language responses and correct external actions
like database queries and updates. We demonstrate that, when applied to
customer support chat transcripts, Sequence to Sequence (Seq2Seq) models often
generate short, incoherent and ungrammatical natural language responses that
are dominated by words that occur with high frequency in the training data.
These phenomena do not arise in synthetic datasets such as bAbI, where we show
Seq2Seq models are nearly perfect. We develop techniques to learn embeddings
that succinctly capture relevant information from the dialog history, and
demonstrate that nearest neighbor based approaches in this learned neural
embedding space generate more fluent responses. However, we see that these
methods are not able to accurately predict when to execute an external action.
We show how to combine nearest neighbor and Seq2Seq methods in a hybrid model,
where nearest neighbor is used to generate fluent responses and Seq2Seq type
models ensure dialog coherency and generate accurate external actions. We show
that this approach is well suited for customer support scenarios, where agents'
responses are typically script-driven, and correct external actions are
critically important. The hybrid model on the customer support data achieves a
78% relative improvement in fluency scores, and a 130% improvement in accuracy
of external calls.
Dynamic Multivariate Functional Data Modeling via Sparse Subspace Learning
Authors: Chen Zhang, Hao Yan, Seungho Lee, Jianjun Shi
Date: 2018-04-11 00:00:00
Tasks: regression

Multivariate functional data from a complex system are naturally
high-dimensional and have complex cross-correlation structure. The complexity
of data structure can be observed as that (1) some functions are strongly
correlated with similar features, while some others may have almost no
cross-correlations with quite diverse features; and (2) the cross-correlation
structure may also change over time due to the system evolution. With this
regard, this paper presents a dynamic subspace learning method for multivariate
functional data modeling. In particular, we consider different functions come
from different subspaces, and only functions of the same subspace have
cross-correlations with each other. The subspaces can be automatically
formulated and learned by reformatting the problem as a sparse regression. By
allowing but regularizing the regression change over time, we can describe the
cross-correlation dynamics. The model can be efficiently estimated by the fast
iterative shrinkage-thresholding algorithm (FISTA), and the features of every
subspace can be extracted using the smooth multi-channel functional PCA.
Numerical studies together with case studies demonstrate the efficiency and
applicability of the proposed methodology.
Emergent Translation in Multi-Agent Communication
Authors: Jason Lee, Kyunghyun Cho, Jason Weston, Douwe Kiela
Date: 2017-10-12 00:00:00
Proceeding: ICLR 2018 1
Tasks: Machine Translation, Sentence, Translation

While most machine translation systems to date are trained on large parallel
corpora, humans learn language in a different way: by being grounded in an
environment and interacting with other humans. In this work, we propose a
communication game where two agents, native speakers of their own respective
languages, jointly learn to solve a visual referential task. We find that the
ability to understand and translate a foreign language emerges as a means to
achieve shared goals. The emergent translation is interactive and multimodal,
and crucially does not require parallel corpora, but only monolingual,
independent text and corresponding images. Our proposed translation model
achieves this by grounding the source and target languages into a shared visual
modality, and outperforms several baselines on both word-level and
sentence-level translation tasks. Furthermore, we show that agents in a
multilingual community learn to translate better and faster than in a bilingual
communication setting.
Differentially Private Confidence Intervals for Empirical Risk Minimization
Authors: Yue Wang, Daniel Kifer, Jaewoo Lee
Date: 2018-04-11 00:00:00
Tasks: BIG-bench Machine Learning

The process of data mining with differential privacy produces results that
are affected by two types of noise: sampling noise due to data collection and
privacy noise that is designed to prevent the reconstruction of sensitive
information. In this paper, we consider the problem of designing confidence
intervals for the parameters of a variety of differentially private machine
learning models. The algorithms can provide confidence intervals that satisfy
differential privacy (as well as the more recently proposed concentrated
differential privacy) and can be used with existing differentially private
mechanisms that train models using objective perturbation and output
perturbation.
DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference
Authors: Reza Ghaeini, Sadid A. Hasan, Vivek Datla, Joey Liu, Kathy Lee, Ashequl Qadir, Yuan Ling, Aaditya Prakash, Xiaoli Z. Fern, Oladimeji Farri
Date: 2018-02-15 00:00:00
Proceeding: NAACL 2018 6
Tasks: Natural Language Inference

We present a novel deep learning architecture to address the natural language
inference (NLI) task. Existing approaches mostly rely on simple reading
mechanisms for independent encoding of the premise and hypothesis. Instead, we
propose a novel dependent reading bidirectional LSTM network (DR-BiLSTM) to
efficiently model the relationship between a premise and a hypothesis during
encoding and inference. We also introduce a sophisticated ensemble strategy to
combine our proposed models, which noticeably improves final predictions.
Finally, we demonstrate how the results can be improved further with an
additional preprocessing step. Our evaluation shows that DR-BiLSTM obtains the
best single model and ensemble model results achieving the new state-of-the-art
scores on the Stanford NLI dataset.
Geometric Consistency for Self-Supervised End-to-End Visual Odometry
Authors: Ganesh Iyer, J. Krishna Murthy, Gunshi Gupta, K. Madhava Krishna, Liam Paull
Date: 2018-04-11 00:00:00
Tasks: Visual Odometry

With the success of deep learning based approaches in tackling challenging
problems in computer vision, a wide range of deep architectures have recently
been proposed for the task of visual odometry (VO) estimation. Most of these
proposed solutions rely on supervision, which requires the acquisition of
precise ground-truth camera pose information, collected using expensive motion
capture systems or high-precision IMU/GPS sensor rigs. In this work, we propose
an unsupervised paradigm for deep visual odometry learning. We show that using
a noisy teacher, which could be a standard VO pipeline, and by designing a loss
term that enforces geometric consistency of the trajectory, we can train
accurate deep models for VO that do not require ground-truth labels. We
leverage geometry as a self-supervisory signal and propose "Composite
Transformation Constraints (CTCs)", that automatically generate supervisory
signals for training and enforce geometric consistency in the VO estimate. We
also present a method of characterizing the uncertainty in VO estimates thus
obtained. To evaluate our VO pipeline, we present exhaustive ablation studies
that demonstrate the efficacy of end-to-end, self-supervised methodologies to
train deep models for monocular VO. We show that leveraging concepts from
geometry and incorporating them into the training of a recurrent neural network
results in performance competitive to supervised deep VO methods.
Multi-Scale Generalized Plane Match for Optical Flow
Authors: Inchul Choi, Arunava Banerjee
Date: 2018-04-11 00:00:00
Tasks: object-detection, Object Detection, Optical Flow Estimation

Despite recent advances, estimating optical flow remains a challenging
problem in the presence of illumination change, large occlusions or fast
movement. In this paper, we propose a novel optical flow estimation framework
which can provide accurate dense correspondence and occlusion localization
through a multi-scale generalized plane matching approach. In our method, we
regard the scene as a collection of planes at multiple scales, and for each
such plane, compensate motion in consensus to improve match quality. We
estimate the square patch plane distortion using a robust plane model detection
method and iteratively apply a plane matching scheme within a multi-scale
framework. During the flow estimation process, our enhanced plane matching
method also clearly localizes the occluded regions. In experiments on
MPI-Sintel datasets, our method robustly estimated optical flow from given
noisy correspondences, and also revealed the occluded regions accurately.
Compared to other state-of-the-art optical flow methods, our method shows
accurate occlusion localization, comparable optical flow quality, and better
thin object detection.
Nonlinear 3D Face Morphable Model
Authors: Luan Tran, Xiaoming Liu
Date: 2018-04-11 00:00:00
Proceeding: CVPR 2018 6
Tasks: 3D Reconstruction, Face Alignment, Image Generation, model

As a classic statistical model of 3D facial shape and texture, 3D Morphable
Model (3DMM) is widely used in facial analysis, e.g., model fitting, image
synthesis. Conventional 3DMM is learned from a set of well-controlled 2D face
images with associated 3D face scans, and represented by two sets of PCA basis
functions. Due to the type and amount of training data, as well as the linear
bases, the representation power of 3DMM can be limited. To address these
problems, this paper proposes an innovative framework to learn a nonlinear 3DMM
model from a large set of unconstrained face images, without collecting 3D face
scans. Specifically, given a face image as input, a network encoder estimates
the projection, shape and texture parameters. Two decoders serve as the
nonlinear 3DMM to map from the shape and texture parameters to the 3D shape and
texture, respectively. With the projection parameter, 3D shape, and texture, a
novel analytically-differentiable rendering layer is designed to reconstruct
the original input face. The entire network is end-to-end trainable with only
weak supervision. We demonstrate the superior representation power of our
nonlinear 3DMM over its linear counterpart, and its contribution to face
alignment and 3D reconstruction.
CoT: Cooperative Training for Generative Modeling of Discrete Data
Authors: Sidi Lu, Lantao Yu, Siyuan Feng, Yaoming Zhu, Wei-Nan Zhang, Yong Yu
Date: 2018-04-11 00:00:00
Proceeding: ICLR 2019 5
Tasks: Diversity

In this paper, we study the generative models of sequential discrete data. To tackle the exposure bias problem inherent in maximum likelihood estimation (MLE), generative adversarial networks (GANs) are introduced to penalize the unrealistic generated samples. To exploit the supervision signal from the discriminator, most previous models leverage REINFORCE to address the non-differentiable problem of sequential discrete data. However, because of the unstable property of the training signal during the dynamic process of adversarial training, the effectiveness of REINFORCE, in this case, is hardly guaranteed. To deal with such a problem, we propose a novel approach called Cooperative Training (CoT) to improve the training of sequence generative models. CoT transforms the min-max game of GANs into a joint maximization framework and manages to explicitly estimate and optimize Jensen-Shannon divergence. Moreover, CoT works without the necessity of pre-training via MLE, which is crucial to the success of previous methods. In the experiments, compared to existing state-of-the-art methods, CoT shows superior or at least competitive performance on sample quality, diversity, as well as training stability.
Deep Air Learning: Interpolation, Prediction, and Feature Analysis of Fine-grained Air Quality
Authors: Zhongang Qi, Tianchun Wang, Guojie Song, Weisong Hu, Xi Li, Zhongfei, Zhang
Date: 2017-11-02 00:00:00
Tasks: feature selection, Prediction

The interpolation, prediction, and feature analysis of fine-gained air
quality are three important topics in the area of urban air computing. The
solutions to these topics can provide extremely useful information to support
air pollution control, and consequently generate great societal and technical
impacts. Most of the existing work solves the three problems separately by
different models. In this paper, we propose a general and effective approach to
solve the three problems in one model called the Deep Air Learning (DAL). The
main idea of DAL lies in embedding feature selection and semi-supervised
learning in different layers of the deep learning network. The proposed
approach utilizes the information pertaining to the unlabeled spatio-temporal
data to improve the performance of the interpolation and the prediction, and
performs feature selection and association analysis to reveal the main relevant
features to the variation of the air quality. We evaluate our approach with
extensive experiments based on real data sources obtained in Beijing, China.
Experiments show that DAL is superior to the peer models from the recent
literature when solving the topics of interpolation, prediction, and feature
analysis of fine-gained air quality.
Design of a PCIe Interface Card Control Software Based on WDF
Authors: Meng Shengwei, Lu Jianjie
Date: 2018-03-24 00:00:00

Based on a clear analysis of the latest Windows driver framework WDF, this
paper has implemented a driver of the PCIe-SpaceWire interface card device and
put forward a discussion about ensuring the stability of PCIe driver. At the
same time, Qt and OpenGL are used to design the upper application. Finally, a
functional verification of the control software is provided.
Embedding Deep Networks into Visual Explanations
Authors: Zhongang Qi, Saeed Khorram, Fuxin Li
Date: 2017-09-15 00:00:00
Tasks: image-classification, Image Classification

In this paper, we propose a novel Explanation Neural Network (XNN) to explain the predictions made by a deep network. The XNN works by learning a nonlinear embedding of a high-dimensional activation vector of a deep network layer into a low-dimensional explanation space while retaining faithfulness i.e., the original deep learning predictions can be constructed from the few concepts extracted by our explanation network. We then visualize such concepts for human to learn about the high-level concepts that the deep network is using to make decisions. We propose an algorithm called Sparse Reconstruction Autoencoder (SRAE) for learning the embedding to the explanation space. SRAE aims to reconstruct part of the original feature space while retaining faithfulness. A pull-away term is applied to SRAE to make the bases of the explanation space more orthogonal to each other. A visualization system is then introduced for human understanding of the features in the explanation space. The proposed method is applied to explain CNN models in image classification tasks. We conducted a human study, which shows that the proposed approach outperforms single saliency map baselines, and improves human performance on a difficult classification tasks. Also, several novel metrics are introduced to evaluate the performance of explanations quantitatively without human involvement.
Derivative free optimization via repeated classification
Authors: Tatsunori B. Hashimoto, Steve Yadlowsky, John C. Duchi
Date: 2018-04-11 00:00:00
Tasks: Active Learning, Classification, General Classification

We develop an algorithm for minimizing a function using $n$ batched function
value measurements at each of $T$ rounds by using classifiers to identify a
function's sublevel set. We show that sufficiently accurate classifiers can
achieve linear convergence rates, and show that the convergence rate is tied to
the difficulty of active learning sublevel sets. Further, we show that the
bootstrap is a computationally efficient approximation to the necessary
classification scheme.
  The end result is a computationally efficient derivative-free algorithm
requiring no tuning that consistently outperforms other approaches on
simulations, standard benchmarks, real-world DNA binding optimization, and
airfoil design problems whenever batched function queries are natural.
A Semantic QA-Based Approach for Text Summarization Evaluation
Authors: Ping Chen, Fei Wu, Tong Wang, Wei Ding
Date: 2017-04-21 00:00:00
Tasks: Articles, Machine Translation, Text Simplification, Text Summarization, Translation

Many Natural Language Processing and Computational Linguistics applications
involves the generation of new texts based on some existing texts, such as
summarization, text simplification and machine translation. However, there has
been a serious problem haunting these applications for decades, that is, how to
automatically and accurately assess quality of these applications. In this
paper, we will present some preliminary results on one especially useful and
challenging problem in NLP system evaluation: how to pinpoint content
differences of two text passages (especially for large pas-sages such as
articles and books). Our idea is intuitive and very different from existing
approaches. We treat one text passage as a small knowledge base, and ask it a
large number of questions to exhaustively identify all content points in it. By
comparing the correctly answered questions from two text passages, we will be
able to compare their content precisely. The experiment using 2007 DUC
summarization corpus clearly shows promising results.
Universal Successor Representations for Transfer Reinforcement Learning
Authors: Chen Ma, Junfeng Wen, Yoshua Bengio
Date: 2018-04-11 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Transfer Learning, Transfer Reinforcement Learning

The objective of transfer reinforcement learning is to generalize from a set
of previous tasks to unseen new tasks. In this work, we focus on the transfer
scenario where the dynamics among tasks are the same, but their goals differ.
Although general value function (Sutton et al., 2011) has been shown to be
useful for knowledge transfer, learning a universal value function can be
challenging in practice. To attack this, we propose (1) to use universal
successor representations (USR) to represent the transferable knowledge and (2)
a USR approximator (USRA) that can be trained by interacting with the
environment. Our experiments show that USR can be effectively applied to new
tasks, and the agent initialized by the trained USRA can achieve the goal
considerably faster than random initialization.
Kernel-based Inference of Functions over Graphs
Authors: Vassilis N. Ioannidis, Meng Ma, Athanasios N. Nikolakopoulos, Georgios B. Giannakis, Daniel Romero
Date: 2017-11-28 00:00:00

The study of networks has witnessed an explosive growth over the past decades
with several ground-breaking methods introduced. A particularly interesting --
and prevalent in several fields of study -- problem is that of inferring a
function defined over the nodes of a network. This work presents a versatile
kernel-based framework for tackling this inference problem that naturally
subsumes and generalizes the reconstruction approaches put forth recently by
the signal processing on graphs community. Both the static and the dynamic
settings are considered along with effective modeling approaches for addressing
real-world problems. The herein analytical discussion is complemented by a set
of numerical examples, which showcase the effectiveness of the presented
techniques, as well as their merits related to state-of-the-art methods.
3D Pathfinding and Collision Avoidance Using Uneven Search-space Quantization and Visual Cone Search
Authors: Diptangshu Pandit
Date: 2017-06-05 00:00:00
Tasks: Collision Avoidance, Quantization

Pathfinding is a very popular area in computer game development. While
two-dimensional (2D) pathfinding is widely applied in most of the popular game
engines, little implementation of real three-dimensional (3D) pathfinding can
be found. This research presents a dynamic search space optimization algorithm
which can be applied to tessellate 3D search space unevenly, significantly
reducing the total number of resulting nodes. The algorithm can be used with
popular pathfinding algorithms in 3D game engines. Furthermore, a simplified
standalone 3D pathfinding algorithm is proposed in this paper. The proposed
algorithm relies on ray-casting or line vision to generate a feasible path
during runtime without requiring division of the search space into a 3D grid.
Both of the proposed algorithms are simulated on Unreal Engine to show
innerworkings and resultant path comparison with A*. The advantages and
shortcomings of the proposed algorithms are also discussed along with future
directions.