pwc-archive/papers-with-abstracts huggingface dataset:
Pyramidal Gradient Matching for Optical Flow Estimation
Authors: Yuanwei Li
Date: 2017-04-11 00:00:00
Tasks: Optical Flow Estimation

Initializing optical flow field by either sparse descriptor matching or dense
patch matches has been proved to be particularly useful for capturing large
displacements. In this paper, we present a pyramidal gradient matching approach
that can provide dense matches for highly accurate and efficient optical flow
estimation. A novel contribution of our method is that image gradient is used
to describe image patches and proved to be able to produce robust matching.
Therefore, our method is more efficient than methods that adopt special
features (like SIFT) or patch distance metric. Moreover, we find that image
gradient is scalable for optical flow estimation, which means we can use
different levels of gradient feature (for example, full gradients or only
direction information of gradients) to obtain different complexity without
dramatic changes in accuracy. Another contribution is that we uncover the
secrets of limited PatchMatch through a thorough analysis and design a
pyramidal matching framework based these secrets. Our pyramidal matching
framework is aimed at robust gradient matching and effective to grow inliers
and reject outliers. In this framework, we present some special enhancements
for outlier filtering in gradient matching. By initializing EpicFlow with our
matches, experimental results show that our method is efficient and robust
(ranking 1st on both clean pass and final pass of MPI Sintel dataset among
published methods).
One-to-Many Network for Visually Pleasing Compression Artifacts Reduction
Authors: Jun Guo, Hongyang Chao
Date: 2016-11-15 00:00:00
Proceeding: CVPR 2017 7

We consider the compression artifacts reduction problem, where a compressed
image is transformed into an artifact-free image. Recent approaches for this
problem typically train a one-to-one mapping using a per-pixel $L_2$ loss
between the outputs and the ground-truths. We point out that these approaches
used to produce overly smooth results, and PSNR doesn't reflect their real
performance. In this paper, we propose a one-to-many network, which measures
output quality using a perceptual loss, a naturalness loss, and a JPEG loss. We
also avoid grid-like artifacts during deconvolution using a "shift-and-average"
strategy. Extensive experimental results demonstrate the dramatic visual
improvement of our approach over the state of the arts.
DeMoN: Depth and Motion Network for Learning Monocular Stereo
Authors: Benjamin Ummenhofer, Huizhong Zhou, Jonas Uhrig, Nikolaus Mayer, Eddy Ilg, Alexey Dosovitskiy, Thomas Brox
Date: 2016-12-07 00:00:00
Proceeding: CVPR 2017 7
Tasks: Decoder, Depth And Camera Motion, Optical Flow Estimation

In this paper we formulate structure from motion as a learning problem. We
train a convolutional network end-to-end to compute depth and camera motion
from successive, unconstrained image pairs. The architecture is composed of
multiple stacked encoder-decoder networks, the core part being an iterative
network that is able to improve its own predictions. The network estimates not
only depth and motion, but additionally surface normals, optical flow between
the images and confidence of the matching. A crucial component of the approach
is a training loss based on spatial relative differences. Compared to
traditional two-frame structure from motion methods, results are more accurate
and more robust. In contrast to the popular depth-from-single-image networks,
DeMoN learns the concept of matching and, thus, better generalizes to
structures not seen during training.
On Feature Reduction using Deep Learning for Trend Prediction in Finance
Authors: Luigi Troiano, Elena Mejuto, Pravesh Kriplani
Date: 2017-04-11 00:00:00
Tasks: Deep Learning

One of the major advantages in using Deep Learning for Finance is to embed a
large collection of information into investment decisions. A way to do that is
by means of compression, that lead us to consider a smaller feature space.
Several studies are proving that non-linear feature reduction performed by Deep
Learning tools is effective in price trend prediction. The focus has been put
mainly on Restricted Boltzmann Machines (RBM) and on output obtained by them.
Few attention has been payed to Auto-Encoders (AE) as an alternative means to
perform a feature reduction. In this paper we investigate the application of
both RBM and AE in more general terms, attempting to outline how architectural
and input space characteristics can affect the quality of prediction.
Morphological Inflection Generation with Hard Monotonic Attention
Authors: Roee Aharoni, Yoav Goldberg
Date: 2016-11-04 00:00:00
Proceeding: ACL 2017 7
Tasks: Hard Attention, Morphological Inflection

We present a neural model for morphological inflection generation which
employs a hard attention mechanism, inspired by the nearly-monotonic alignment
commonly found between the characters in a word and the characters in its
inflection. We evaluate the model on three previously studied morphological
inflection generation datasets and show that it provides state of the art
results in various setups compared to previous neural and non-neural
approaches. Finally we present an analysis of the continuous representations
learned by both the hard and soft attention \cite{bahdanauCB14} models for the
task, shedding some light on the features such models extract.
Deep Learning on Lie Groups for Skeleton-based Action Recognition
Authors: Zhiwu Huang, Chengde Wan, Thomas Probst, Luc van Gool
Date: 2016-12-18 00:00:00
Proceeding: CVPR 2017 7
Tasks: 3D Action Recognition, 3D Classification, Action Recognition, Deep Learning, Dynamic Time Warping, General Classification, Skeleton Based Action Recognition, Temporal Action Localization

In recent years, skeleton-based action recognition has become a popular 3D
classification problem. State-of-the-art methods typically first represent each
motion sequence as a high-dimensional trajectory on a Lie group with an
additional dynamic time warping, and then shallowly learn favorable Lie group
features. In this paper we incorporate the Lie group structure into a deep
network architecture to learn more appropriate Lie group features for 3D action
recognition. Within the network structure, we design rotation mapping layers to
transform the input Lie group features into desirable ones, which are aligned
better in the temporal domain. To reduce the high feature dimensionality, the
architecture is equipped with rotation pooling layers for the elements on the
Lie group. Furthermore, we propose a logarithm mapping layer to map the
resulting manifold data into a tangent space that facilitates the application
of regular output layers for the final classification. Evaluations of the
proposed network for standard 3D human action recognition datasets clearly
demonstrate its superiority over existing shallow Lie group feature learning
methods as well as most conventional deep learning methods.
Scale-invariance of ruggedness measures in fractal fitness landscapes
Authors: Hendrik Richter
Date: 2016-12-21 00:00:00

The paper deals with using chaos to direct trajectories to targets and
analyzes ruggedness and fractality of the resulting fitness landscapes. The
targeting problem is formulated as a dynamic fitness landscape and four
different chaotic maps generating such a landscape are studied. By using a
computational approach, we analyze properties of the landscapes and quantify
their fractal and rugged characteristics. In particular, it is shown that
ruggedness measures such as correlation length and information content are
scale-invariant and self-similar.
PoseAgent: Budget-Constrained 6D Object Pose Estimation via Reinforcement Learning
Authors: Alexander Krull, Eric Brachmann, Sebastian Nowozin, Frank Michel, Jamie Shotton, Carsten Rother
Date: 2016-12-12 00:00:00
Proceeding: CVPR 2017 7
Tasks: 6D Pose Estimation using RGB, Pose Estimation, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

State-of-the-art computer vision algorithms often achieve efficiency by
making discrete choices about which hypotheses to explore next. This allows
allocation of computational resources to promising candidates, however, such
decisions are non-differentiable. As a result, these algorithms are hard to
train in an end-to-end fashion. In this work we propose to learn an efficient
algorithm for the task of 6D object pose estimation. Our system optimizes the
parameters of an existing state-of-the art pose estimation system using
reinforcement learning, where the pose estimation system now becomes the
stochastic policy, parametrized by a CNN. Additionally, we present an efficient
training algorithm that dramatically reduces computation time. We show
empirically that our learned pose estimation procedure makes better use of
limited resources and improves upon the state-of-the-art on a challenging
dataset. Our approach enables differentiable end-to-end training of complex
algorithmic pipelines and learns to make optimal use of a given computational
budget.
Simplified Stochastic Feedforward Neural Networks
Authors: Kimin Lee, Jaehyung Kim, Song Chong, Jinwoo Shin
Date: 2017-04-11 00:00:00

It has been believed that stochastic feedforward neural networks (SFNNs) have
several advantages beyond deterministic deep neural networks (DNNs): they have
more expressive power allowing multi-modal mappings and regularize better due
to their stochastic nature. However, training large-scale SFNN is notoriously
harder. In this paper, we aim at developing efficient training methods for
SFNN, in particular using known architectures and pre-trained parameters of
DNN. To this end, we propose a new intermediate stochastic model, called
Simplified-SFNN, which can be built upon any baseline DNNand approximates
certain SFNN by simplifying its upper latent units above stochastic ones. The
main novelty of our approach is in establishing the connection between three
models, i.e., DNN->Simplified-SFNN->SFNN, which naturally leads to an efficient
training procedure of the stochastic models utilizing pre-trained parameters of
DNN. Using several popular DNNs, we show how they can be effectively
transferred to the corresponding stochastic models for both multi-modal and
classification tasks on MNIST, TFD, CASIA, CIFAR-10, CIFAR-100 and SVHN
datasets. In particular, we train a stochastic model of 28 layers and 36
million parameters, where training such a large-scale stochastic network is
significantly challenging without using Simplified-SFNN
ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
Authors: Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner
Date: 2017-02-14 00:00:00
Proceeding: CVPR 2017 7
Tasks: 3D Object Classification, General Classification, Retrieval, Scene Understanding, Semantic Segmentation, Surface Reconstruction

A key requirement for leveraging supervised deep learning methods is the
availability of large, labeled datasets. Unfortunately, in the context of RGB-D
scene understanding, very little data is available -- current datasets cover a
small range of scene views and have limited semantic annotations. To address
this issue, we introduce ScanNet, an RGB-D video dataset containing 2.5M views
in 1513 scenes annotated with 3D camera poses, surface reconstructions, and
semantic segmentations. To collect this data, we designed an easy-to-use and
scalable RGB-D capture system that includes automated surface reconstruction
and crowdsourced semantic annotation. We show that using this data helps
achieve state-of-the-art performance on several 3D scene understanding tasks,
including 3D object classification, semantic voxel labeling, and CAD model
retrieval. The dataset is freely available at http://www.scan-net.org.
3D Human Pose Estimation = 2D Pose Estimation + Matching
Authors: Ching-Hang Chen, Deva Ramanan
Date: 2016-12-20 00:00:00
Proceeding: CVPR 2017 7
Tasks: 2D Pose Estimation, 3D Human Pose Estimation, 3D Pose Estimation, Memorization, Pose Estimation

We explore 3D human pose estimation from a single RGB image. While many
approaches try to directly predict 3D pose from image measurements, we explore
a simple architecture that reasons through intermediate 2D pose predictions.
Our approach is based on two key observations (1) Deep neural nets have
revolutionized 2D pose estimation, producing accurate 2D predictions even for
poses with self occlusions. (2) Big-data sets of 3D mocap data are now readily
available, making it tempting to lift predicted 2D poses to 3D through simple
memorization (e.g., nearest neighbors). The resulting architecture is trivial
to implement with off-the-shelf 2D pose estimation systems and 3D mocap
libraries. Importantly, we demonstrate that such methods outperform almost all
state-of-the-art 3D pose estimation systems, most of which directly try to
regress 3D pose from 2D measurements.
Mining Object Parts from CNNs via Active Question-Answering
Authors: Quanshi Zhang, Ruiming Cao, Ying Nian Wu, Song-Chun Zhu
Date: 2017-04-11 00:00:00
Proceeding: CVPR 2017 7
Tasks: Active Learning, Object, Question Answering

Given a convolutional neural network (CNN) that is pre-trained for object
classification, this paper proposes to use active question-answering to
semanticize neural patterns in conv-layers of the CNN and mine part concepts.
For each part concept, we mine neural patterns in the pre-trained CNN, which
are related to the target part, and use these patterns to construct an And-Or
graph (AOG) to represent a four-layer semantic hierarchy of the part. As an
interpretable model, the AOG associates different CNN units with different
explicit object parts. We use an active human-computer communication to
incrementally grow such an AOG on the pre-trained CNN as follows. We allow the
computer to actively identify objects, whose neural patterns cannot be
explained by the current AOG. Then, the computer asks human about the
unexplained objects, and uses the answers to automatically discover certain CNN
patterns corresponding to the missing knowledge. We incrementally grow the AOG
to encode new knowledge discovered during the active-learning process. In
experiments, our method exhibits high learning efficiency. Our method uses
about 1/6-1/3 of the part annotations for training, but achieves similar or
better part-localization performance than fast-RCNN methods.
Deep Multimodal Representation Learning from Temporal Data
Authors: Xitong Yang, Palghat Ramesh, Radha Chitta, Sriganesh Madhvanath, Edgar A. Bernal, Jiebo Luo
Date: 2017-04-11 00:00:00
Proceeding: CVPR 2017 7
Tasks: Audio-Visual Speech Recognition, Representation Learning, speech-recognition, Speech Recognition, Time Series, Time Series Analysis, Visual Speech Recognition

In recent years, Deep Learning has been successfully applied to multimodal
learning problems, with the aim of learning useful joint representations in
data fusion applications. When the available modalities consist of time series
data such as video, audio and sensor signals, it becomes imperative to consider
their temporal structure during the fusion process. In this paper, we propose
the Correlational Recurrent Neural Network (CorrRNN), a novel temporal fusion
model for fusing multiple input modalities that are inherently temporal in
nature. Key features of our proposed model include: (i) simultaneous learning
of the joint representation and temporal dependencies between modalities, (ii)
use of multiple loss terms in the objective function, including a maximum
correlation loss term to enhance learning of cross-modal information, and (iii)
the use of an attention model to dynamically adjust the contribution of
different input modalities to the joint representation. We validate our model
via experimentation on two different tasks: video- and sensor-based activity
classification, and audio-visual speech recognition. We empirically analyze the
contributions of different components of the proposed CorrRNN model, and
demonstrate its robustness, effectiveness and state-of-the-art performance on
multiple datasets.
Multimodal Transfer: A Hierarchical Deep Convolutional Neural Network for Fast Artistic Style Transfer
Authors: Xin Wang, Geoffrey Oxholm, Da Zhang, Yuan-Fang Wang
Date: 2016-11-17 00:00:00
Proceeding: CVPR 2017 7
Tasks: Style Transfer

Transferring artistic styles onto everyday photographs has become an
extremely popular task in both academia and industry. Recently, offline
training has replaced on-line iterative optimization, enabling nearly real-time
stylization. When those stylization networks are applied directly to
high-resolution images, however, the style of localized regions often appears
less similar to the desired artistic style. This is because the transfer
process fails to capture small, intricate textures and maintain correct texture
scales of the artworks. Here we propose a multimodal convolutional neural
network that takes into consideration faithful representations of both color
and luminance channels, and performs stylization hierarchically with multiple
losses of increasing scales. Compared to state-of-the-art networks, our network
can also perform style transfer in nearly real-time by conducting much more
sophisticated training offline. By properly handling style and texture cues at
multiple scales using several modalities, we can transfer not just large-scale,
obvious style cues but also subtle, exquisite ones. That is, our scheme can
generate results that are visually pleasing and more similar to multiple
desired artistic styles with color and texture cues at multiple scales.
Federated Tensor Factorization for Computational Phenotyping
Authors: Yejin Kim, Jimeng Sun, Hwanjo Yu, Xiaoqian Jiang
Date: 2017-04-11 00:00:00
Tasks: Computational Phenotyping

Tensor factorization models offer an effective approach to convert massive
electronic health records into meaningful clinical concepts (phenotypes) for
data analysis. These models need a large amount of diverse samples to avoid
population bias. An open challenge is how to derive phenotypes jointly across
multiple hospitals, in which direct patient-level data sharing is not possible
(e.g., due to institutional policies). In this paper, we developed a novel
solution to enable federated tensor factorization for computational phenotyping
without sharing patient-level data. We developed secure data harmonization and
federated computation procedures based on alternating direction method of
multipliers (ADMM). Using this method, the multiple hospitals iteratively
update tensors and transfer secure summarized information to a central server,
and the server aggregates the information to generate phenotypes. We
demonstrated with real medical datasets that our method resembles the
centralized training model (based on combined datasets) in terms of accuracy
and phenotypes discovery while respecting privacy.
Deep Photo Style Transfer
Authors: Fujun Luan, Sylvain Paris, Eli Shechtman, Kavita Bala
Date: 2017-03-22 00:00:00
Proceeding: CVPR 2017 7
Tasks: Style Transfer

This paper introduces a deep-learning approach to photographic style transfer
that handles a large variety of image content while faithfully transferring the
reference style. Our approach builds upon the recent work on painterly transfer
that separates style from the content of an image by considering different
layers of a neural network. However, as is, this approach is not suitable for
photorealistic style transfer. Even when both the input and reference images
are photographs, the output still exhibits distortions reminiscent of a
painting. Our contribution is to constrain the transformation from the input to
the output to be locally affine in colorspace, and to express this constraint
as a custom fully differentiable energy term. We show that this approach
successfully suppresses distortion and yields satisfying photorealistic style
transfers in a broad variety of scenarios, including transfer of the time of
day, weather, season, and artistic edits.
Statistically efficient thinning of a Markov chain sampler
Authors: Art B. Owen
Date: 2015-10-27 00:00:00

It is common to subsample Markov chain output to reduce the storage burden.
Geyer (1992) shows that discarding $k-1$ out of every $k$ observations will not
improve statistical efficiency, as quantified through variance in a given
computational budget. That observation is often taken to mean that thinning
MCMC output cannot improve statistical efficiency. Here we suppose that it
costs one unit of time to advance a Markov chain and then $\theta>0$ units of
time to compute a sampled quantity of interest. For a thinned process, that
cost $\theta$ is incurred less often, so it can be advanced through more
stages. Here we provide examples to show that thinning will improve statistical
efficiency if $\theta$ is large and the sample autocorrelations decay slowly
enough. If the lag $\ell\ge1$ autocorrelations of a scalar measurement satisfy
$\rho_\ell\ge\rho_{\ell+1}\ge0$, then there is always a $\theta<\infty$ at
which thinning becomes more efficient for averages of that scalar. Many sample
autocorrelation functions resemble first order AR(1) processes with $\rho_\ell
=\rho^{|\ell|}$ for some $-1<\rho<1$. For an AR(1) process it is possible to
compute the most efficient subsampling frequency $k$. The optimal $k$ grows
rapidly as $\rho$ increases towards $1$. The resulting efficiency gain depends
primarily on $\theta$, not $\rho$. Taking $k=1$ (no thinning) is optimal when
$\rho\le0$. For $\rho>0$ it is optimal if and only if $\theta \le
(1-\rho)^2/(2\rho)$. This efficiency gain never exceeds $1+\theta$. This paper
also gives efficiency bounds for autocorrelations bounded between those of two
AR(1) processes.
DOPE: Distributed Optimization for Pairwise Energies
Authors: Jose Dolz, Ismail Ben Ayed, Christian Desrosiers
Date: 2017-04-11 00:00:00
Proceeding: CVPR 2017 7
Tasks: Distributed Optimization

We formulate an Alternating Direction Method of Mul-tipliers (ADMM) that
systematically distributes the computations of any technique for optimizing
pairwise functions, including non-submodular potentials. Such discrete
functions are very useful in segmentation and a breadth of other vision
problems. Our method decomposes the problem into a large set of small
sub-problems, each involving a sub-region of the image domain, which can be
solved in parallel. We achieve consistency between the sub-problems through a
novel constraint that can be used for a large class of pair-wise functions. We
give an iterative numerical solution that alternates between solving the
sub-problems and updating consistency variables, until convergence. We report
comprehensive experiments, which demonstrate the benefit of our general
distributed solution in the case of the popular serial algorithm of Boykov and
Kolmogorov (BK algorithm) and, also, in the context of non-submodular
functions.