pwc-archive/papers-with-abstracts huggingface dataset:
MVSNet: Depth Inference for Unstructured Multi-view Stereo
Authors: Yao Yao, Zixin Luo, Shiwei Li, Tian Fang, Long Quan
Date: 2018-04-07 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Reconstruction, Point Clouds

We present an end-to-end deep learning architecture for depth map inference
from multi-view images. In the network, we first extract deep visual image
features, and then build the 3D cost volume upon the reference camera frustum
via the differentiable homography warping. Next, we apply 3D convolutions to
regularize and regress the initial depth map, which is then refined with the
reference image to generate the final output. Our framework flexibly adapts
arbitrary N-view inputs using a variance-based cost metric that maps multiple
features into one cost feature. The proposed MVSNet is demonstrated on the
large-scale indoor DTU dataset. With simple post-processing, our method not
only significantly outperforms previous state-of-the-arts, but also is several
times faster in runtime. We also evaluate MVSNet on the complex outdoor Tanks
and Temples dataset, where our method ranks first before April 18, 2018 without
any fine-tuning, showing the strong generalization ability of MVSNet.
Scalable Sentiment for Sequence-to-sequence Chatbot Response with Performance Analysis
Authors: Chih-Wei Lee, Yau-Shian Wang, Tsung-Yuan Hsu, Kuan-Yu Chen, Hung-Yi Lee, Lin-shan Lee
Date: 2018-04-07 00:00:00
Tasks: Chatbot, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Conventional seq2seq chatbot models only try to find the sentences with the
highest probabilities conditioned on the input sequences, without considering
the sentiment of the output sentences. Some research works trying to modify the
sentiment of the output sequences were reported. In this paper, we propose five
models to scale or adjust the sentiment of the chatbot response: persona-based
model, reinforcement learning, plug and play model, sentiment transformation
network and cycleGAN, all based on the conventional seq2seq model. We also
develop two evaluation metrics to estimate if the responses are reasonable
given the input. These metrics together with other two popularly used metrics
were used to analyze the performance of the five proposed models on different
aspects, and reinforcement learning and cycleGAN were shown to be very
attractive. The evaluation metrics were also found to be well correlated with
human evaluation.
Continuously Constructive Deep Neural Networks
Authors: Ozan İrsoy, Ethem Alpaydin
Date: 2018-04-07 00:00:00

Traditionally, deep learning algorithms update the network weights whereas
the network architecture is chosen manually, using a process of trial and
error. In this work, we propose two novel approaches that automatically update
the network structure while also learning its weights. The novelty of our
approach lies in our parameterization where the depth, or additional
complexity, is encapsulated continuously in the parameter space through control
parameters that add additional complexity. We propose two methods: In tunnel
networks, this selection is done at the level of a hidden unit, and in budding
perceptrons, this is done at the level of a network layer; updating this
control parameter introduces either another hidden unit or another hidden
layer. We show the effectiveness of our methods on the synthetic two-spirals
data and on two real data sets of MNIST and MIRFLICKR, where we see that our
proposed methods, with the same set of hyperparameters, can correctly adjust
the network complexity to the task complexity.
Regularizing RNNs for Caption Generation by Reconstructing The Past with The Present
Authors: Xinpeng Chen, Lin Ma, Wenhao Jiang, Jian Yao, Wei Liu
Date: 2018-03-30 00:00:00
Proceeding: CVPR 2018 6
Tasks: Caption Generation, Decoder, Image Captioning

Recently, caption generation with an encoder-decoder framework has been
extensively studied and applied in different domains, such as image captioning,
code captioning, and so on. In this paper, we propose a novel architecture,
namely Auto-Reconstructor Network (ARNet), which, coupling with the
conventional encoder-decoder framework, works in an end-to-end fashion to
generate captions. ARNet aims at reconstructing the previous hidden state with
the present one, besides behaving as the input-dependent transition operator.
Therefore, ARNet encourages the current hidden state to embed more information
from the previous one, which can help regularize the transition dynamics of
recurrent neural networks (RNNs). Extensive experimental results show that our
proposed ARNet boosts the performance over the existing encoder-decoder models
on both image captioning and source code captioning tasks. Additionally, ARNet
remarkably reduces the discrepancy between training and inference processes for
caption generation. Furthermore, the performance on permuted sequential MNIST
demonstrates that ARNet can effectively regularize RNN, especially on modeling
long-term dependencies. Our code is available at:
https://github.com/chenxinpeng/ARNet
Fine-grained Video Attractiveness Prediction Using Multimodal Deep Learning on a Large Real-world Dataset
Authors: Xinpeng Chen, Jingyuan Chen, Lin Ma, Jian Yao, Wei Liu, Jiebo Luo, Tong Zhang
Date: 2018-04-04 00:00:00
Tasks: Multimodal Deep Learning, Prediction

Nowadays, billions of videos are online ready to be viewed and shared. Among
an enormous volume of videos, some popular ones are widely viewed by online
users while the majority attract little attention. Furthermore, within each
video, different segments may attract significantly different numbers of views.
This phenomenon leads to a challenging yet important problem, namely
fine-grained video attractiveness prediction. However, one major obstacle for
such a challenging problem is that no suitable benchmark dataset currently
exists. To this end, we construct the first fine-grained video attractiveness
dataset, which is collected from one of the most popular video websites in the
world. In total, the constructed FVAD consists of 1,019 drama episodes with
780.6 hours covering different categories and a wide variety of video contents.
Apart from the large amount of videos, hundreds of millions of user behaviors
during watching videos are also included, such as "view counts",
"fast-forward", "fast-rewind", and so on, where "view counts" reflects the
video attractiveness while other engagements capture the interactions between
the viewers and videos. First, we demonstrate that video attractiveness and
different engagements present different relationships. Second, FVAD provides us
an opportunity to study the fine-grained video attractiveness prediction
problem. We design different sequential models to perform video attractiveness
prediction by relying solely on video contents. The sequential models exploit
the multimodal relationships between visual and audio components of the video
contents at different levels. Experimental results demonstrate the
effectiveness of our proposed sequential models with different visual and audio
representations, the necessity of incorporating the two modalities, and the
complementary behaviors of the sequential prediction models at different
levels.
MicronNet: A Highly Compact Deep Convolutional Neural Network Architecture for Real-time Embedded Traffic Sign Classification
Authors: Alexander Wong, Mohammad Javad Shafiee, Michael St. Jules
Date: 2018-03-28 00:00:00
Tasks: General Classification, Traffic Sign Recognition

Traffic sign recognition is a very important computer vision task for a
number of real-world applications such as intelligent transportation
surveillance and analysis. While deep neural networks have been demonstrated in
recent years to provide state-of-the-art performance traffic sign recognition,
a key challenge for enabling the widespread deployment of deep neural networks
for embedded traffic sign recognition is the high computational and memory
requirements of such networks. As a consequence, there are significant benefits
in investigating compact deep neural network architectures for traffic sign
recognition that are better suited for embedded devices. In this paper, we
introduce MicronNet, a highly compact deep convolutional neural network for
real-time embedded traffic sign recognition designed based on macroarchitecture
design principles (e.g., spectral macroarchitecture augmentation, parameter
precision optimization, etc.) as well as numerical microarchitecture
optimization strategies. The resulting overall architecture of MicronNet is
thus designed with as few parameters and computations as possible while
maintaining recognition performance, leading to optimized information density
of the proposed network. The resulting MicronNet possesses a model size of just
~1MB and ~510,000 parameters (~27x fewer parameters than state-of-the-art)
while still achieving a human performance level top-1 accuracy of 98.9% on the
German traffic sign recognition benchmark. Furthermore, MicronNet requires just
~10 million multiply-accumulate operations to perform inference, and has a
time-to-compute of just 32.19 ms on a Cortex-A53 high efficiency processor.
These experimental results show that highly compact, optimized deep neural
network architectures can be designed for real-time traffic sign recognition
that are well-suited for embedded scenarios.
Fortified Networks: Improving the Robustness of Deep Networks by Modeling the Manifold of Hidden Representations
Authors: Alex Lamb, Jonathan Binas, Anirudh Goyal, Dmitriy Serdyuk, Sandeep Subramanian, Ioannis Mitliagkas, Yoshua Bengio
Date: 2018-04-07 00:00:00
Proceeding: ICLR 2019 5

Deep networks have achieved impressive results across a variety of important
tasks. However a known weakness is a failure to perform well when evaluated on
data which differ from the training distribution, even if these differences are
very small, as is the case with adversarial examples. We propose Fortified
Networks, a simple transformation of existing networks, which fortifies the
hidden layers in a deep network by identifying when the hidden states are off
of the data manifold, and maps these hidden states back to parts of the data
manifold where the network performs well. Our principal contribution is to show
that fortifying these hidden states improves the robustness of deep networks
and our experiments (i) demonstrate improved robustness to standard adversarial
attacks in both black-box and white-box threat models; (ii) suggest that our
improvements are not primarily due to the gradient masking problem and (iii)
show the advantage of doing this fortification in the hidden layers instead of
the input space.
Approximating Hamiltonian dynamics with the Nyström method
Authors: Alessandro Rudi, Leonard Wossnig, Carlo Ciliberto, Andrea Rocchetto, Massimiliano Pontil, Simone Severini
Date: 2018-04-06 00:00:00

Simulating the time-evolution of quantum mechanical systems is BQP-hard and expected to be one of the foremost applications of quantum computers. We consider classical algorithms for the approximation of Hamiltonian dynamics using subsampling methods from randomized numerical linear algebra. We derive a simulation technique whose runtime scales polynomially in the number of qubits and the Frobenius norm of the Hamiltonian. As an immediate application, we show that sample based quantum simulation, a type of evolution where the Hamiltonian is a density matrix, can be efficiently classically simulated under specific structural conditions. Our main technical contribution is a randomized algorithm for approximating Hermitian matrix exponentials. The proof leverages a low-rank, symmetric approximation via the Nystr\"om method. Our results suggest that under strong sampling assumptions there exist classical poly-logarithmic time simulations of quantum computations.
Deep neural decoders for near term fault-tolerant experiments
Authors: Christopher Chamberland, Pooya Ronagh
Date: 2018-02-18 00:00:00

Finding efficient decoders for quantum error correcting codes adapted to
realistic experimental noise in fault-tolerant devices represents a significant
challenge. In this paper we introduce several decoding algorithms complemented
by deep neural decoders and apply them to analyze several fault-tolerant error
correction protocols such as the surface code as well as Steane and Knill error
correction. Our methods require no knowledge of the underlying noise model
afflicting the quantum device making them appealing for real-world experiments.
Our analysis is based on a full circuit-level noise model. It considers both
distance-three and five codes, and is performed near the codes pseudo-threshold
regime. Training deep neural decoders in low noise rate regimes appears to be a
challenging machine learning endeavour. We provide a detailed description of
our neural network architectures and training methodology. We then discuss both
the advantages and limitations of deep neural decoders. Lastly, we provide a
rigorous analysis of the decoding runtime of trained deep neural decoders and
compare our methods with anticipated gate times in future quantum devices.
Given the broad applications of our decoding schemes, we believe that the
methods presented in this paper could have practical applications for near term
fault-tolerant experiments.
Neural models of factuality
Authors: Rachel Rudinger, Aaron Steven White, Benjamin Van Durme
Date: 2018-04-06 00:00:00
Proceeding: NAACL 2018 6

We present two neural models for event factuality prediction, which yield
significant performance gains over previous models on three event factuality
datasets: FactBank, UW, and MEANTIME. We also present a substantial expansion
of the It Happened portion of the Universal Decompositional Semantics dataset,
yielding the largest event factuality dataset to date. We report model results
on this extended factuality dataset as well.
Visual Tracking Using Sparse Coding and Earth Mover's Distance
Authors: Gang Yao, Ashwin Dani
Date: 2018-04-06 00:00:00
Tasks: Visual Tracking

An efficient iterative Earth Mover's Distance (iEMD) algorithm for visual
tracking is proposed in this paper. The Earth Mover's Distance (EMD) is used as
the similarity measure to search for the optimal template candidates in
feature-spatial space in a video sequence. The computation of the EMD is
formulated as the transportation problem from linear programming. The
efficiency of the EMD optimization problem limits its use for visual tracking.
To alleviate this problem, a transportation-simplex method is used for EMD
optimization and a monotonically convergent iterative optimization algorithm is
developed. The local sparse representation is used as the appearance models for
the iEMD tracker. The maximum-alignment-pooling method is used for constructing
a sparse coding histogram which reduces the computational complexity of the EMD
optimization. The template update algorithm based on the EMD is also presented.
The iEMD tracking algorithm assumes small inter-frame movement in order to
guarantee convergence. When the camera is mounted on a moving robot, e.g., a
flying quadcopter, the camera could experience a sudden and rapid motion
leading to large inter-frame movements. To ensure that the tracking algorithm
converges, a gyro-aided extension of the iEMD tracker is presented, where
synchronized gyroscope information is utilized to compensate for the rotation
of the camera. The iEMD algorithm's performance is evaluated using eight
publicly available datasets. The performance of the iEMD algorithm is compared
with seven state-of-the-art tracking algorithms based on relative percentage
overlap. The robustness of this algorithm for large inter-frame displacements
is also illustrated.
Reconstructing Point Sets from Distance Distributions
Authors: Shuai Huang, Ivan Dokmanić
Date: 2018-04-06 00:00:00

We address the problem of reconstructing a set of points on a line or a loop from their unassigned noisy pairwise distances. When the points lie on a line, the problem is known as the turnpike; when they are on a loop, it is known as the beltway. We approximate the problem by discretizing the domain and representing the $N$ points via an $N$-hot encoding, which is a density supported on the discretized domain. We show how the distance distribution is then simply a collection of quadratic functionals of this density and propose to recover the point locations so that the estimated distance distribution matches the measured distance distribution. This can be cast as a constrained nonconvex optimization problem which we solve using projected gradient descent with a suitable spectral initializer. We derive conditions under which the proposed distance distribution matching approach locally converges to a global optimizer at a linear rate. Compared to the conventional backtracking approach, our method jointly reconstructs all the point locations and is robust to noise in the measurements. We substantiate these claims with state-of-the-art performance across a number of numerical experiments. Our method is the first practical approach to solve the large-scale noisy beltway problem where the points lie on a loop.
Differentiable plasticity: training plastic neural networks with backpropagation
Authors: Thomas Miconi, Jeff Clune, Kenneth O. Stanley
Date: 2018-04-06 00:00:00
Proceeding: ICML 2018 7
Tasks: Lifelong learning, Meta-Learning, Reinforcement Learning

How can we build agents that keep learning from experience, quickly and
efficiently, after their initial training? Here we take inspiration from the
main mechanism of learning in biological brains: synaptic plasticity, carefully
tuned by evolution to produce efficient lifelong learning. We show that
plasticity, just like connection weights, can be optimized by gradient descent
in large (millions of parameters) recurrent networks with Hebbian plastic
connections. First, recurrent plastic networks with more than two million
parameters can be trained to memorize and reconstruct sets of novel,
high-dimensional 1000+ pixels natural images not seen during training.
Crucially, traditional non-plastic recurrent networks fail to solve this task.
Furthermore, trained plastic networks can also solve generic meta-learning
tasks such as the Omniglot task, with competitive results and little parameter
overhead. Finally, in reinforcement learning settings, plastic networks
outperform a non-plastic equivalent in a maze exploration task. We conclude
that differentiable plasticity may provide a powerful novel approach to the
learning-to-learn problem.
Deep Person Detection in 2D Range Data
Authors: Lucas Beyer, Alexander Hermans, Timm Linder, Kai O. Arras, Bastian Leibe
Date: 2018-04-06 00:00:00
Tasks: Human Detection

Detecting humans is a key skill for mobile robots and intelligent vehicles in
a large variety of applications. While the problem is well studied for certain
sensory modalities such as image data, few works exist that address this
detection task using 2D range data. However, a widespread sensory setup for
many mobile robots in service and domestic applications contains a horizontally
mounted 2D laser scanner. Detecting people from 2D range data is challenging
due to the speed and dynamics of human leg motion and the high levels of
occlusion and self-occlusion particularly in crowds of people. While previous
approaches mostly relied on handcrafted features, we recently developed the
deep learning based wheelchair and walker detector DROW. In this paper, we show
the generalization to people, including small modifications that significantly
boost DROW's performance. Additionally, by providing a small, fully online
temporal window in our network, we further boost our score. We extend the DROW
dataset with person annotations, making this the largest dataset of person
annotations in 2D range data, recorded during several days in a real-world
environment with high diversity. Extensive experiments with three current
baseline methods indicate it is a challenging dataset, on which our improved
DROW detector beats the current state-of-the-art.
The exploding gradient problem demystified - definition, prevalence, impact, origin, tradeoffs, and solutions
Authors: George Philipp, Dawn Song, Jaime G. Carbonell
Date: 2017-12-15 00:00:00

Whereas it is believed that techniques such as Adam, batch normalization and,
more recently, SeLU nonlinearities "solve" the exploding gradient problem, we
show that this is not the case in general and that in a range of popular MLP
architectures, exploding gradients exist and that they limit the depth to which
networks can be effectively trained, both in theory and in practice. We explain
why exploding gradients occur and highlight the *collapsing domain problem*,
which can arise in architectures that avoid exploding gradients.
  ResNets have significantly lower gradients and thus can circumvent the
exploding gradient problem, enabling the effective training of much deeper
networks. We show this is a direct consequence of the Pythagorean equation. By
noticing that *any neural network is a residual network*, we devise the
*residual trick*, which reveals that introducing skip connections simplifies
the network mathematically, and that this simplicity may be the major cause for
their success.
Photographic Text-to-Image Synthesis with a Hierarchically-nested Adversarial Network
Authors: Zizhao Zhang, Yuanpu Xie, Lin Yang
Date: 2018-02-26 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Generation, Semantic Similarity, Semantic Textual Similarity

This paper presents a novel method to deal with the challenging task of
generating photographic images conditioned on semantic image descriptions. Our
method introduces accompanying hierarchical-nested adversarial objectives
inside the network hierarchies, which regularize mid-level representations and
assist generator training to capture the complex image statistics. We present
an extensile single-stream generator architecture to better adapt the jointed
discriminators and push generated images up to high resolutions. We adopt a
multi-purpose adversarial loss to encourage more effective image and text
information usage in order to improve the semantic consistency and image
fidelity simultaneously. Furthermore, we introduce a new visual-semantic
similarity measure to evaluate the semantic consistency of generated images.
With extensive experimental validation on three public datasets, our method
significantly improves previous state of the arts on all datasets over
different evaluation metrics.
Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure
Authors: Ding Liu, Shi-Ju Ran, Peter Wittek, Cheng Peng, Raul Blázquez García, Gang Su, Maciej Lewenstein
Date: 2017-10-13 00:00:00
Proceeding: ICLR 2018 1
Tasks: BIG-bench Machine Learning, Tensor Networks

The resemblance between the methods used in quantum-many body physics and in
machine learning has drawn considerable attention. In particular, tensor
networks (TNs) and deep learning architectures bear striking similarities to
the extent that TNs can be used for machine learning. Previous results used
one-dimensional TNs in image recognition, showing limited scalability and
flexibilities. In this work, we train two-dimensional hierarchical TNs to solve
image recognition problems, using a training algorithm derived from the
multi-scale entanglement renormalization ansatz. This approach introduces
mathematical connections among quantum many-body physics, quantum information
theory, and machine learning. While keeping the TN unitary in the training
phase, TN states are defined, which encode classes of images into quantum
many-body states. We study the quantum features of the TN states, including
quantum entanglement and fidelity. We find these quantities could be properties
that characterize the image classes, as well as the machine learning tasks.