pwc-archive/papers-with-abstracts huggingface dataset:
LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation
Authors: Tak-Wai Hui, Xiaoou Tang, Chen Change Loy
Date: 2018-05-18 00:00:00
Proceeding: CVPR 2018 6
Tasks: Optical Flow Estimation

FlowNet2, the state-of-the-art convolutional neural network (CNN) for optical
flow estimation, requires over 160M parameters to achieve accurate flow
estimation. In this paper we present an alternative network that outperforms
FlowNet2 on the challenging Sintel final pass and KITTI benchmarks, while being
30 times smaller in the model size and 1.36 times faster in the running speed.
This is made possible by drilling down to architectural details that might have
been missed in the current frameworks: (1) We present a more effective flow
inference approach at each pyramid level through a lightweight cascaded
network. It not only improves flow estimation accuracy through early
correction, but also permits seamless incorporation of descriptor matching in
our network. (2) We present a novel flow regularization layer to ameliorate the
issue of outliers and vague flow boundaries by using a feature-driven local
convolution. (3) Our network owns an effective structure for pyramidal feature
extraction and embraces feature warping rather than image warping as practiced
in FlowNet2. Our code and trained models are available at
https://github.com/twhui/LiteFlowNet .
Automated Process Planning for Hybrid Manufacturing
Authors: Morad Behandish, Saigopal Nelaturi, Johan de Kleer
Date: 2018-05-18 00:00:00
Tasks: Collision Avoidance, Computational Efficiency

Hybrid manufacturing (HM) technologies combine additive and subtractive
manufacturing (AM/SM) capabilities, leveraging AM's strengths in fabricating
complex geometries and SM's precision and quality to produce finished parts. We
present a systematic approach to automated computer-aided process planning
(CAPP) for HM that can identify non-trivial, qualitatively distinct, and
cost-optimal combinations of AM/SM modalities. A multimodal HM process plan is
represented by a finite Boolean expression of AM and SM manufacturing
primitives, such that the expression evaluates to an 'as-manufactured'
artifact. We show that primitives that respect spatial constraints such as
accessibility and collision avoidance may be constructed by solving inverse
configuration space problems on the 'as-designed' artifact and manufacturing
instruments. The primitives generate a finite Boolean algebra (FBA) that
enumerates the entire search space for planning. The FBA's canonical
intersection terms (i.e., 'atoms') provide the complete domain decomposition to
reframe manufacturability analysis and process planning into purely symbolic
reasoning, once a subcollection of atoms is found to be interchangeable with
the design target. The approach subsumes unimodal (all-AM or all-SM) process
planning as special cases. We demonstrate the practical potency of our
framework and its computational efficiency when applied to process planning of
complex 3D parts with dramatically different AM and SM instruments.
SemStyle: Learning to Generate Stylised Image Captions using Unaligned Text
Authors: Alexander Mathews, Lexing Xie, Xuming He
Date: 2018-05-18 00:00:00
Proceeding: CVPR 2018 6
Tasks: Descriptive, Image Captioning, Language Modeling, Language Modelling

Linguistic style is an essential part of written communication, with the
power to affect both clarity and attractiveness. With recent advances in vision
and language, we can start to tackle the problem of generating image captions
that are both visually grounded and appropriately styled. Existing approaches
either require styled training captions aligned to images or generate captions
with low relevance. We develop a model that learns to generate visually
relevant styled captions from a large corpus of styled text without aligned
images. The core idea of this model, called SemStyle, is to separate semantics
and style. One key component is a novel and concise semantic term
representation generated using natural language processing techniques and frame
semantics. In addition, we develop a unified language model that decodes
sentences with diverse word choices and syntax for different styles.
Evaluations, both automatic and manual, show captions from SemStyle preserve
image semantics, are descriptive, and are style shifted. More broadly, this
work provides possibilities to learn richer image descriptions from the
plethora of linguistic data available on the web.
Bidirectional Attention for SQL Generation
Authors: Tong Guo, Huilin Gao
Date: 2017-12-30 00:00:00
Tasks: Code Generation, Reading Comprehension

Generating structural query language (SQL) queries from natural language is a
long-standing open problem. Answering a natural language question about a
database table requires modeling complex interactions between the columns of
the table and the question. In this paper, we apply the synthesizing approach
to solve this problem. Based on the structure of SQL queries, we break down the
model to three sub-modules and design specific deep neural networks for each of
them. Taking inspiration from the similar machine reading task, we employ the
bidirectional attention mechanisms and character-level embedding with
convolutional neural networks (CNNs) to improve the result. Experimental
evaluations show that our model achieves the state-of-the-art results in
WikiSQL dataset.
Scene Understanding Networks for Autonomous Driving based on Around View Monitoring System
Authors: JeongYeol Baek, Ioana Veronica Chelu, Livia Iordache, Vlad Paunescu, HyunJoo Ryu, Alexandru Ghiuta, Andrei Petreanu, YunSung Soh, Andrei Leica, ByeongMoon Jeon
Date: 2018-05-18 00:00:00
Tasks: 3D Object Detection, Autonomous Driving, Drivable Area Detection, object-detection, Object Detection, Scene Understanding

Modern driver assistance systems rely on a wide range of sensors (RADAR,
LIDAR, ultrasound and cameras) for scene understanding and prediction. These
sensors are typically used for detecting traffic participants and scene
elements required for navigation. In this paper we argue that relying on camera
based systems, specifically Around View Monitoring (AVM) system has great
potential to achieve these goals in both parking and driving modes with
decreased costs. The contributions of this paper are as follows: we present a
new end-to-end solution for delimiting the safe drivable area for each frame by
means of identifying the closest obstacle in each direction from the driving
vehicle, we use this approach to calculate the distance to the nearest
obstacles and we incorporate it into a unified end-to-end architecture capable
of joint object detection, curb detection and safe drivable area detection.
Furthermore, we describe the family of networks for both a high accuracy
solution and a low complexity solution. We also introduce further augmentation
of the base architecture with 3D object detection.
Gated Recurrent Unit Based Acoustic Modeling with Future Context
Authors: Jie Li, Xiaorui Wang, Yuan-Yuan Zhao, Yan Li
Date: 2018-05-18 00:00:00

The use of future contextual information is typically shown to be helpful for
acoustic modeling. However, for the recurrent neural network (RNN), it's not so
easy to model the future temporal context effectively, meanwhile keep lower
model latency. In this paper, we attempt to design a RNN acoustic model that
being capable of utilizing the future context effectively and directly, with
the model latency and computation cost as low as possible. The proposed model
is based on the minimal gated recurrent unit (mGRU) with an input projection
layer inserted in it. Two context modules, temporal encoding and temporal
convolution, are specifically designed for this architecture to model the
future context. Experimental results on the Switchboard task and an internal
Mandarin ASR task show that, the proposed model performs much better than long
short-term memory (LSTM) and mGRU models, whereas enables online decoding with
a maximum latency of 170 ms. This model even outperforms a very strong
baseline, TDNN-LSTM, with smaller model latency and almost half less
parameters.
Supervising Nyström Methods via Negative Margin Support Vector Selection
Authors: Mert Al, Thee Chanyaswad, Sun-Yuan Kung
Date: 2018-05-10 00:00:00
Tasks: Classification, General Classification

The Nystr\"om methods have been popular techniques for scalable kernel based
learning. They approximate explicit, low-dimensional feature mappings for
kernel functions from the pairwise comparisons with the training data. However,
Nystr\"om methods are generally applied without the supervision provided by the
training labels in the classification/regression problems. This leads to
pairwise comparisons with randomly chosen training samples in the model.
Conversely, this work studies a supervised Nystr\"om method that chooses the
critical subsets of samples for the success of the Machine Learning model.
Particularly, we select the Nystr\"om support vectors via the negative margin
criterion, and create explicit feature maps that are more suitable for the
classification task on the data. Experimental results on six datasets show
that, without increasing the complexity over unsupervised techniques, our
method can significantly improve the classification performance achieved via
kernel approximation methods and reduce the number of features needed to reach
or exceed the performance of the full-dimensional kernel machines.
Modular Tracking Framework: A Unified Approach to Registration based Tracking
Authors: Abhineet Singh, Martin Jagersand
Date: 2016-02-29 00:00:00

This paper presents a modular, extensible and highly efficient open source
framework for registration based tracking called Modular Tracking Framework
(MTF). Targeted at robotics applications, it is implemented entirely in C++ and
designed from the ground up to easily integrate with systems that support any
of several major vision and robotics libraries including OpenCV, ROS, ViSP and
Eigen. It implements more methods, is faster, and more precise than other
existing systems. Further, the theoretical basis for its design is a new way to
conceptualize registration based trackers that decomposes them into three
constituent sub modules - Search Method (SM), Appearance Model (AM) and State
Space Model (SSM).
  In the process, we integrate many important advances published after Baker \&
Matthews' landmark work in 2004. In addition to being a practical solution for
fast and high precision tracking, MTF can also serve as a useful research tool
by allowing existing and new methods for any of the sub modules to be studied
better. When a new method is introduced for one of these, the breakdown can
help to experimentally find the combination of methods for the others that is
optimum for it. By extensive use of generic programming, MTF makes it easy to
plug in a new method for any of the sub modules so that it can not only be
tested comprehensively with existing methods but also become immediately
available for deployment in any project that uses the framework. With 16 AMs,
11 SMs and 13 SSMs implemented already, MTF provides over 2000 distinct single
layer trackers. It also allows two or more of these to be combined together in
several ways to create a practically unlimited variety of novel multi layer
trackers.
Understanding and Improving Deep Neural Network for Activity Recognition
Authors: Li Xue, Si Xiandong, Nie Lanshun, Li Jiazhen, Ding Renjie, Zhan Dechen, Chu Dianhui
Date: 2018-05-18 00:00:00
Tasks: Activity Recognition, General Classification, Human Activity Recognition

Activity recognition has become a popular research branch in the field of
pervasive computing in recent years. A large number of experiments can be
obtained that activity sensor-based data's characteristic in activity
recognition is variety, volume, and velocity. Deep learning technology,
together with its various models, is one of the most effective ways of working
on activity data. Nevertheless, there is no clear understanding of why it
performs so well or how to make it more effective. In order to solve this
problem, first, we applied convolution neural network on Human Activity
Recognition Using Smart phones Data Set. Second, we realized the visualization
of the sensor-based activity's data features extracted from the neural network.
Then we had in-depth analysis of the visualization of features, explored the
relationship between activity and features, and analyzed how Neural Networks
identify activity based on these features. After that, we extracted the
significant features related to the activities and sent the features to the
DNN-based fusion model, which improved the classification rate to 96.1%. This
is the first work to our knowledge that visualizes abstract sensor-based
activity data features. Based on the results, the method proposed in the paper
promises to realize the accurate classification of sensor- based activity
recognition.
A Tree Search Algorithm for Sequence Labeling
Authors: Yadi Lao, Jun Xu, Yanyan Lan, Jiafeng Guo, Sheng Gao, Xue-Qi Cheng
Date: 2018-04-29 00:00:00
Tasks: Chunking, Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sentence, TAG

In this paper we propose a novel reinforcement learning based model for
sequence tagging, referred to as MM-Tag. Inspired by the success and
methodology of the AlphaGo Zero, MM-Tag formalizes the problem of sequence
tagging with a Monte Carlo tree search (MCTS) enhanced Markov decision process
(MDP) model, in which the time steps correspond to the positions of words in a
sentence from left to right, and each action corresponds to assign a tag to a
word. Two long short-term memory networks (LSTM) are used to summarize the past
tag assignments and words in the sentence. Based on the outputs of LSTMs, the
policy for guiding the tag assignment and the value for predicting the whole
tagging accuracy of the whole sentence are produced. The policy and value are
then strengthened with MCTS, which takes the produced raw policy and value as
inputs, simulates and evaluates the possible tag assignments at the subsequent
positions, and outputs a better search policy for assigning tags. A
reinforcement learning algorithm is proposed to train the model parameters. Our
work is the first to apply the MCTS enhanced MDP model to the sequence tagging
task. We show that MM-Tag can accurately predict the tags thanks to the
exploratory decision making mechanism introduced by MCTS. Experimental results
show based on a chunking benchmark showed that MM-Tag outperformed the
state-of-the-art sequence tagging baselines including CRF and CRF with LSTM.
Learning Permutations with Sinkhorn Policy Gradient
Authors: Patrick Emami, Sanjay Ranka
Date: 2018-05-18 00:00:00
Tasks: Representation Learning

Many problems at the intersection of combinatorics and computer science
require solving for a permutation that optimally matches, ranks, or sorts some
data. These problems usually have a task-specific, often non-differentiable
objective function that data-driven algorithms can use as a learning signal. In
this paper, we propose the Sinkhorn Policy Gradient (SPG) algorithm for
learning policies on permutation matrices. The actor-critic neural network
architecture we introduce for SPG uniquely decouples representation learning of
the state space from the highly-structured action space of permutations with a
temperature-controlled Sinkhorn layer. The Sinkhorn layer produces continuous
relaxations of permutation matrices so that the actor-critic architecture can
be trained end-to-end. Our empirical results show that agents trained with SPG
can perform competitively on sorting, the Euclidean TSP, and matching tasks. We
also observe that SPG is significantly more data efficient at the matching task
than the baseline methods, which indicates that SPG is conducive to learning
representations that are useful for reasoning about permutations.
MDSSD: Multi-scale Deconvolutional Single Shot Detector for Small Objects
Authors: Lisha Cui, Rui Ma, Pei Lv, Xiaoheng Jiang, Zhimin Gao, Bing Zhou, Mingliang Xu
Date: 2018-05-18 00:00:00
Tasks: Object, object-detection, Object Detection, Small Object Detection

For most of the object detectors based on multi-scale feature maps, the shallow layers are rich in fine spatial information and thus mainly responsible for small object detection. The performance of small object detection, however, is still less than satisfactory because of the deficiency of semantic information on shallow feature maps. In this paper, we design a Multi-scale Deconvolutional Single Shot Detector (MDSSD), especially for small object detection. In MDSSD, multiple high-level feature maps at different scales are upsampled simultaneously to increase the spatial resolution. Afterwards, we implement the skip connections with low-level feature maps via Fusion Block. The fusion feature maps, named Fusion Module, are of strong feature representational power of small instances. It is noteworthy that these high-level feature maps utilized in Fusion Block preserve both strong semantic information and some fine details of small instances, rather than the top-most layer where the representation of fine details for small objects are potentially wiped out. The proposed framework achieves 77.6% mAP for small object detection on the challenging dataset TT100K with 512 x 512 input, outperforming other detectors with a large margin. Moreover, it can also achieve state-of-the-art results for general object detection on PASCAL VOC2007 test and MS COCO test-dev2015, especially achieving 2 to 5 points improvement on small object categories.
Hierarchical Reinforcement Learning with Deep Nested Agents
Authors: Marc Brittain, Peng Wei
Date: 2018-05-18 00:00:00
Tasks: Hierarchical Reinforcement Learning, Minecraft, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep hierarchical reinforcement learning has gained a lot of attention in
recent years due to its ability to produce state-of-the-art results in
challenging environments where non-hierarchical frameworks fail to learn useful
policies. However, as problem domains become more complex, deep hierarchical
reinforcement learning can become inefficient, leading to longer convergence
times and poor performance. We introduce the Deep Nested Agent framework, which
is a variant of deep hierarchical reinforcement learning where information from
the main agent is propagated to the low level $nested$ agent by incorporating
this information into the nested agent's state. We demonstrate the
effectiveness and performance of the Deep Nested Agent framework by applying it
to three scenarios in Minecraft with comparisons to a deep non-hierarchical
single agent framework, as well as, a deep hierarchical framework.
Spectral feature scaling method for supervised dimensionality reduction
Authors: Momo Matsuda, Keiichi Morikuni, Tetsuya Sakurai
Date: 2018-05-18 00:00:00
Tasks: Clustering, Dimensionality Reduction, General Classification, Supervised dimensionality reduction

Spectral dimensionality reduction methods enable linear separations of
complex data with high-dimensional features in a reduced space. However, these
methods do not always give the desired results due to irregularities or
uncertainties of the data. Thus, we consider aggressively modifying the scales
of the features to obtain the desired classification. Using prior knowledge on
the labels of partial samples to specify the Fiedler vector, we formulate an
eigenvalue problem of a linear matrix pencil whose eigenvector has the feature
scaling factors. The resulting factors can modify the features of entire
samples to form clusters in the reduced space, according to the known labels.
In this study, we propose new dimensionality reduction methods supervised using
the feature scaling associated with the spectral clustering. Numerical
experiments show that the proposed methods outperform well-established
supervised methods for toy problems with more samples than features, and are
more robust regarding clustering than existing methods. Also, the proposed
methods outperform existing methods regarding classification for real-world
problems with more features than samples of gene expression profiles of cancer
diseases. Furthermore, the feature scaling tends to improve the clustering and
classification accuracies of existing unsupervised methods, as the proportion
of training data increases.
Robust Task Clustering for Deep Many-Task Learning
Authors: Mo Yu, Xiaoxiao Guo, Jin-Feng Yi, Shiyu Chang, Saloni Potdar, Gerald Tesauro, Haoyu Wang, Bo-Wen Zhou
Date: 2017-08-26 00:00:00
Tasks: Clustering, Few-Shot Learning, General Classification, intent-classification, Intent Classification, Matrix Completion, Multi-Task Learning, Sentiment Analysis, Sentiment Classification

We investigate task clustering for deep-learning based multi-task and
few-shot learning in a many-task setting. We propose a new method to measure
task similarities with cross-task transfer performance matrix for the deep
learning scenario. Although this matrix provides us critical information
regarding similarity between tasks, its asymmetric property and unreliable
performance scores can affect conventional clustering methods adversely.
Additionally, the uncertain task-pairs, i.e., the ones with extremely
asymmetric transfer scores, may collectively mislead clustering algorithms to
output an inaccurate task-partition. To overcome these limitations, we propose
a novel task-clustering algorithm by using the matrix completion technique. The
proposed algorithm constructs a partially-observed similarity matrix based on
the certainty of cluster membership of the task-pairs. We then use a matrix
completion algorithm to complete the similarity matrix. Our theoretical
analysis shows that under mild constraints, the proposed algorithm will
perfectly recover the underlying "true" similarity matrix with a high
probability. Our results show that the new task clustering method can discover
task clusters for training flexible and superior neural network models in a
multi-task learning setup for sentiment classification and dialog intent
classification tasks. Our task clustering approach also extends metric-based
few-shot learning methods to adapt multiple metrics, which demonstrates
empirical advantages when the tasks are diverse.
Random active path model of deep neural networks with diluted binary synapses
Authors: Haiping Huang, Alireza Goudarzi
Date: 2017-05-02 00:00:00
Tasks: Deep Learning

Deep learning has become a powerful and popular tool for a variety of machine
learning tasks. However, it is challenging to understand the mechanism of deep
learning from a theoretical perspective. In this work, we propose a random
active path model to study collective properties of deep neural networks with
binary synapses, under the removal perturbation of connections between layers.
In the model, the path from input to output is randomly activated, and the
corresponding input unit constrains the weights along the path into the form of
a $p$-weight interaction glass model. A critical value of the perturbation is
observed to separate a spin glass regime from a paramagnetic regime, with the
transition being of the first order. The paramagnetic phase is conjectured to
have a poor generalization performance.