pwc-archive/papers-with-abstracts huggingface dataset:
Frank-Wolfe Network: An Interpretable Deep Structure for Non-Sparse Coding
Authors: Dong Liu, Ke Sun, Zhangyang Wang, Runsheng Liu, Zheng-Jun Zha
Date: 2018-02-28 00:00:00
Tasks: Denoising, Handwritten Digit Recognition, Image Denoising, Rolling Shutter Correction, Super-Resolution

The problem of $L_p$-norm constrained coding is to convert signal into code that lies inside an $L_p$-ball and most faithfully reconstructs the signal. Previous works under the name of sparse coding considered the cases of $L_0$ and $L_1$ norms. The cases with $p>1$ values, i.e. non-sparse coding studied in this paper, remain a difficulty. We propose an interpretable deep structure namely Frank-Wolfe Network (F-W Net), whose architecture is inspired by unrolling and truncating the Frank-Wolfe algorithm for solving an $L_p$-norm constrained problem with $p\geq 1$. We show that the Frank-Wolfe solver for the $L_p$-norm constraint leads to a novel closed-form nonlinear unit, which is parameterized by $p$ and termed $pool_p$. The $pool_p$ unit links the conventional pooling, activation, and normalization operations, making F-W Net distinct from existing deep networks either heuristically designed or converted from projected gradient descent algorithms. We further show that the hyper-parameter $p$ can be made learnable instead of pre-chosen in F-W Net, which gracefully solves the non-sparse coding problem even with unknown $p$. We evaluate the performance of F-W Net on an extensive range of simulations as well as the task of handwritten digit recognition, where F-W Net exhibits strong learning capability. We then propose a convolutional version of F-W Net, and apply the convolutional F-W Net into image denoising and super-resolution tasks, where F-W Net all demonstrates impressive effectiveness, flexibility, and robustness.
Cross-modal Deep Variational Hand Pose Estimation
Authors: Adrian Spurr, Jie Song, Seonwook Park, Otmar Hilliges
Date: 2018-03-30 00:00:00
Proceeding: CVPR 2018 6
Tasks: Hand Pose Estimation, Pose Estimation

The human hand moves in complex and high-dimensional ways, making estimation
of 3D hand pose configurations from images alone a challenging task. In this
work we propose a method to learn a statistical hand model represented by a
cross-modal trained latent space via a generative deep neural network. We
derive an objective function from the variational lower bound of the VAE
framework and jointly optimize the resulting cross-modal KL-divergence and the
posterior reconstruction objective, naturally admitting a training regime that
leads to a coherent latent space across multiple modalities such as RGB images,
2D keypoint detections or 3D hand configurations. Additionally, it grants a
straightforward way of using semi-supervision. This latent space can be
directly used to estimate 3D hand poses from RGB images, outperforming the
state-of-the art in different settings. Furthermore, we show that our proposed
method can be used without changes on depth images and performs comparably to
specialized methods. Finally, the model is fully generative and can synthesize
consistent pairs of hand configurations across modalities. We evaluate our
method on both RGB and depth datasets and analyze the latent space
qualitatively.
Contrast-Oriented Deep Neural Networks for Salient Object Detection
Authors: Guanbin Li, Yizhou Yu
Date: 2018-03-30 00:00:00
Tasks: Object, object-detection, Object Detection, RGB Salient Object Detection, Salient Object Detection

Deep convolutional neural networks have become a key element in the recent
breakthrough of salient object detection. However, existing CNN-based methods
are based on either patch-wise (region-wise) training and inference or fully
convolutional networks. Methods in the former category are generally
time-consuming due to severe storage and computational redundancies among
overlapping patches. To overcome this deficiency, methods in the second
category attempt to directly map a raw input image to a predicted dense
saliency map in a single network forward pass. Though being very efficient, it
is arduous for these methods to detect salient objects of different scales or
salient regions with weak semantic information. In this paper, we develop
hybrid contrast-oriented deep neural networks to overcome the aforementioned
limitations. Each of our deep networks is composed of two complementary
components, including a fully convolutional stream for dense prediction and a
segment-level spatial pooling stream for sparse saliency inference. We further
propose an attentional module that learns weight maps for fusing the two
saliency predictions from these two streams. A tailored alternate scheme is
designed to train these deep networks by fine-tuning pre-trained baseline
models. Finally, a customized fully connected CRF model incorporating a salient
contour feature embedding can be optionally applied as a post-processing step
to improve spatial coherence and contour positioning in the fused result from
these two streams. Extensive experiments on six benchmark datasets demonstrate
that our proposed model can significantly outperform the state of the art in
terms of all popular evaluation metrics.
Intertwiners between Induced Representations (with Applications to the Theory of Equivariant Neural Networks)
Authors: Taco S. Cohen, Mario Geiger, Maurice Weiler
Date: 2018-03-28 00:00:00

Group equivariant and steerable convolutional neural networks (regular and
steerable G-CNNs) have recently emerged as a very effective model class for
learning from signal data such as 2D and 3D images, video, and other data where
symmetries are present. In geometrical terms, regular G-CNNs represent data in
terms of scalar fields ("feature channels"), whereas the steerable G-CNN can
also use vector or tensor fields ("capsules") to represent data. In algebraic
terms, the feature spaces in regular G-CNNs transform according to a regular
representation of the group G, whereas the feature spaces in Steerable G-CNNs
transform according to the more general induced representations of G. In order
to make the network equivariant, each layer in a G-CNN is required to
intertwine between the induced representations associated with its input and
output space.
  In this paper we present a general mathematical framework for G-CNNs on
homogeneous spaces like Euclidean space or the sphere. We show, using
elementary methods, that the layers of an equivariant network are convolutional
if and only if the input and output feature spaces transform according to an
induced representation. This result, which follows from G.W. Mackey's abstract
theory on induced representations, establishes G-CNNs as a universal class of
equivariant network architectures, and generalizes the important recent work of
Kondor & Trivedi on the intertwiners between regular representations.
Single Stream Parallelization of Recurrent Neural Networks for Low Power and Fast Inference
Authors: Wonyong Sung, Jinhwan Park
Date: 2018-03-30 00:00:00
Tasks: CPU

As neural network algorithms show high performance in many applications,
their efficient inference on mobile and embedded systems are of great
interests. When a single stream recurrent neural network (RNN) is executed for
a personal user in embedded systems, it demands a large amount of DRAM accesses
because the network size is usually much bigger than the cache size and the
weights of an RNN are used only once at each time step. We overcome this
problem by parallelizing the algorithm and executing it multiple time steps at
a time. This approach also reduces the power consumption by lowering the number
of DRAM accesses. QRNN (Quasi Recurrent Neural Networks) and SRU (Simple
Recurrent Unit) based recurrent neural networks are used for implementation.
The experiments for SRU showed about 300% and 930% of speed-up when the numbers
of multi time steps are 4 and 16, respectively, in an ARM CPU based system.
Facelet-Bank for Fast Portrait Manipulation
Authors: Ying-Cong Chen, Huaijia Lin, Michelle Shu, Ruiyu Li, Xin Tao, Yangang Ye, Xiaoyong Shen, Jiaya Jia
Date: 2018-03-15 00:00:00
Proceeding: CVPR 2018 6
Tasks: Facial Editing

Digital face manipulation has become a popular and fascinating way to touch
images with the prevalence of smartphones and social networks. With a wide
variety of user preferences, facial expressions, and accessories, a general and
flexible model is necessary to accommodate different types of facial editing.
In this paper, we propose a model to achieve this goal based on an end-to-end
convolutional neural network that supports fast inference, edit-effect control,
and quick partial-model update. In addition, this model learns from unpaired
image sets with different attributes. Experimental results show that our
framework can handle a wide range of expressions, accessories, and makeup
effects. It produces high-resolution and high-quality results in fast speed.
Plane-extraction from depth-data using a Gaussian mixture regression model
Authors: Richard T. Marriott, Alexander Paschevich, Radu Horaud
Date: 2017-10-05 00:00:00
Tasks: Clustering, Navigate, regression

We propose a novel algorithm for unsupervised extraction of piecewise planar
models from depth-data. Among other applications, such models are a good way of
enabling autonomous agents (robots, cars, drones, etc.) to effectively perceive
their surroundings and to navigate in three dimensions. We propose to do this
by fitting the data with a piecewise-linear Gaussian mixture regression model
whose components are skewed over planes, making them flat in appearance rather
than being ellipsoidal, by embedding an outlier-trimming process that is
formally incorporated into the proposed expectation-maximization algorithm, and
by selectively fusing contiguous, coplanar components. Part of our motivation
is an attempt to estimate more accurate plane-extraction by allowing each model
component to make use of all available data through probabilistic clustering.
The algorithm is thoroughly evaluated against a standard benchmark and is shown
to rank among the best of the existing state-of-the-art methods.
The Training of Neuromodels for Machine Comprehension of Text. Brain2Text Algorithm
Authors: A. Artemov, A. Sergeev, A. Khasenevich, A. Yuzhakov, M. Chugunov
Date: 2018-03-30 00:00:00
Tasks: Natural Language Queries, Reading Comprehension

Nowadays, the Internet represents a vast informational space, growing
exponentially and the problem of search for relevant data becomes essential as
never before. The algorithm proposed in the article allows to perform natural
language queries on content of the document and get comprehensive meaningful
answers. The problem is partially solved for English as SQuAD contains enough
data to learn on, but there is no such dataset in Russian, so the methods used
by scientists now are not applicable to Russian. Brain2 framework allows to
cope with the problem - it stands out for its ability to be applied on small
datasets and does not require impressive computing power. The algorithm is
illustrated on Sberbank of Russia Strategy's text and assumes the use of a
neuromodel consisting of 65 mln synapses. The trained model is able to
construct word-by-word answers to questions based on a given text. The existing
limitations are its current inability to identify synonyms, pronoun relations
and allegories. Nevertheless, the results of conducted experiments showed high
capacity and generalisation ability of the suggested approach.
Learning to generate classifiers
Authors: Nicholas Guttenberg, Ryota Kanai
Date: 2018-03-30 00:00:00
Tasks: General Classification

We train a network to generate mappings between training sets and
classification policies (a 'classifier generator') by conditioning on the
entire training set via an attentional mechanism. The network is directly
optimized for test set performance on an training set of related tasks, which
is then transferred to unseen 'test' tasks. We use this to optimize for
performance in the low-data and unsupervised learning regimes, and obtain
significantly better performance in the 10-50 datapoint regime than support
vector classifiers, random forests, XGBoost, and k-nearest neighbors on a range
of small datasets.
Parallel Grid Pooling for Data Augmentation
Authors: Akito Takeki, Daiki Ikami, Go Irie, Kiyoharu Aizawa
Date: 2018-03-30 00:00:00
Tasks: Data Augmentation, General Classification, Image Augmentation, image-classification, Image Classification

Convolutional neural network (CNN) architectures utilize downsampling layers,
which restrict the subsequent layers to learn spatially invariant features
while reducing computational costs. However, such a downsampling operation
makes it impossible to use the full spectrum of input features. Motivated by
this observation, we propose a novel layer called parallel grid pooling (PGP)
which is applicable to various CNN models. PGP performs downsampling without
discarding any intermediate feature. It works as data augmentation and is
complementary to commonly used data augmentation techniques. Furthermore, we
demonstrate that a dilated convolution can naturally be represented using PGP
operations, which suggests that the dilated convolution can also be regarded as
a type of data augmentation technique. Experimental results based on popular
image classification benchmarks demonstrate the effectiveness of the proposed
method. Code is available at: https://github.com/akitotakeki
Disentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognition
Authors: Feng Liu, Ronghang Zhu, Dan Zeng, Qijun Zhao, Xiaoming Liu
Date: 2018-03-30 00:00:00
Proceeding: CVPR 2018 6
Tasks: 3D Face Reconstruction, Decoder, Face Identification, Face Recognition, Face Reconstruction

This paper proposes an encoder-decoder network to disentangle shape features
during 3D face reconstruction from single 2D images, such that the tasks of
reconstructing accurate 3D face shapes and learning discriminative shape
features for face recognition can be accomplished simultaneously. Unlike
existing 3D face reconstruction methods, our proposed method directly regresses
dense 3D face shapes from single 2D images, and tackles identity and residual
(i.e., non-identity) components in 3D face shapes explicitly and separately
based on a composite 3D face shape model with latent representations. We devise
a training process for the proposed network with a joint loss measuring both
face identification error and 3D face shape reconstruction error. To construct
training data we develop a method for fitting 3D morphable model (3DMM) to
multiple 2D images of a subject. Comprehensive experiments have been done on
MICC, BU3DFE, LFW and YTF databases. The results show that our method expands
the capacity of 3DMM for capturing discriminative shape features and facial
detail, and thus outperforms existing methods both in 3D face reconstruction
accuracy and in face recognition accuracy.
Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptation
Authors: Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, Kiyoharu Aizawa
Date: 2018-03-30 00:00:00
Proceeding: CVPR 2018 6
Tasks: Domain Adaptation, object-detection, Object Detection, Weakly Supervised Object Detection

Can we detect common objects in a variety of image domains without
instance-level annotations? In this paper, we present a framework for a novel
task, cross-domain weakly supervised object detection, which addresses this
question. For this paper, we have access to images with instance-level
annotations in a source domain (e.g., natural image) and images with
image-level annotations in a target domain (e.g., watercolor). In addition, the
classes to be detected in the target domain are all or a subset of those in the
source domain. Starting from a fully supervised object detector, which is
pre-trained on the source domain, we propose a two-step progressive domain
adaptation technique by fine-tuning the detector on two types of artificially
and automatically generated samples. We test our methods on our newly collected
datasets containing three image domains, and achieve an improvement of
approximately 5 to 20 percentage points in terms of mean average precision
(mAP) compared to the best-performing baselines.
Joint Optimization Framework for Learning with Noisy Labels
Authors: Daiki Tanaka, Daiki Ikami, Toshihiko Yamasaki, Kiyoharu Aizawa
Date: 2018-03-30 00:00:00
Proceeding: CVPR 2018 6
Tasks: General Classification, image-classification, Image Classification, Learning with noisy labels

Deep neural networks (DNNs) trained on large-scale datasets have exhibited
significant performance in image classification. Many large-scale datasets are
collected from websites, however they tend to contain inaccurate labels that
are termed as noisy labels. Training on such noisy labeled datasets causes
performance degradation because DNNs easily overfit to noisy labels. To
overcome this problem, we propose a joint optimization framework of learning
DNN parameters and estimating true labels. Our framework can correct labels
during training by alternating update of network parameters and labels. We
conduct experiments on the noisy CIFAR-10 datasets and the Clothing1M dataset.
The results indicate that our approach significantly outperforms other
state-of-the-art methods.
DDRprog: A CLEVR Differentiable Dynamic Reasoning Programmer
Authors: Joseph Suarez, Justin Johnson, Fei-Fei Li
Date: 2018-03-30 00:00:00
Proceeding: ICLR 2018 1
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

We present a novel Dynamic Differentiable Reasoning (DDR) framework for
jointly learning branching programs and the functions composing them; this
resolves a significant nondifferentiability inhibiting recent dynamic
architectures. We apply our framework to two settings in two highly compact and
data efficient architectures: DDRprog for CLEVR Visual Question Answering and
DDRstack for reverse Polish notation expression evaluation. DDRprog uses a
recurrent controller to jointly predict and execute modular neural programs
that directly correspond to the underlying question logic; it explicitly forks
subprocesses to handle logical branching. By effectively leveraging additional
structural supervision, we achieve a large improvement over previous approaches
in subtask consistency and a small improvement in overall accuracy. We further
demonstrate the benefits of structural supervision in the RPN setting: the
inclusion of a stack assumption in DDRstack allows our approach to generalize
to long expressions where an LSTM fails the task.
Automatic Generation of Chinese Short Product Titles for Mobile Display
Authors: Yu Gong, Xusheng Luo, Kenny Q. Zhu, Wenwu Ou, Zhao Li, Lu Duan
Date: 2018-03-30 00:00:00
Tasks: Extractive Summarization

This paper studies the problem of automatically extracting a short title from a manually written longer description of E-commerce products for display on mobile devices. It is a new extractive summarization problem on short text inputs, for which we propose a feature-enriched network model, combining three different categories of features in parallel. Experimental results show that our framework significantly outperforms several baselines by a substantial gain of 4.5%. Moreover, we produce an extractive summarization dataset for E-commerce short texts and will release it to the research community.
Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation
Authors: Shuming Ma, Xu sun, Wei Li, Sujian Li, Wenjie Li, Xuancheng Ren
Date: 2018-03-05 00:00:00
Proceeding: NAACL 2018 6
Tasks: Abstractive Text Summarization, Decoder, Paraphrase Generation, Text Simplification, Word Embeddings

Most recent approaches use the sequence-to-sequence model for paraphrase
generation. The existing sequence-to-sequence model tends to memorize the words
and the patterns in the training dataset instead of learning the meaning of the
words. Therefore, the generated sentences are often grammatically correct but
semantically improper. In this work, we introduce a novel model based on the
encoder-decoder framework, called Word Embedding Attention Network (WEAN). Our
proposed model generates the words by querying distributed word representations
(i.e. neural word embeddings), hoping to capturing the meaning of the according
words. Following previous work, we evaluate our model on two
paraphrase-oriented tasks, namely text simplification and short text
abstractive summarization. Experimental results show that our model outperforms
the sequence-to-sequence baseline by the BLEU score of 6.3 and 5.5 on two
English text simplification datasets, and the ROUGE-2 F1 score of 5.7 on a
Chinese summarization dataset. Moreover, our model achieves state-of-the-art
performances on these three benchmark datasets.
Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks
Authors: Wei Xiong, Wenhan Luo, Lin Ma, Wei Liu, Jiebo Luo
Date: 2017-09-22 00:00:00
Proceeding: CVPR 2018 6
Tasks: Generative Adversarial Network

Taking a photo outside, can we predict the immediate future, e.g., how would
the cloud move in the sky? We address this problem by presenting a generative
adversarial network (GAN) based two-stage approach to generating realistic
time-lapse videos of high resolution. Given the first frame, our model learns
to generate long-term future frames. The first stage generates videos of
realistic contents for each frame. The second stage refines the generated video
from the first stage by enforcing it to be closer to real videos with regard to
motion dynamics. To further encourage vivid motion in the final generated
video, Gram matrix is employed to model the motion more precisely. We build a
large scale time-lapse dataset, and test our approach on this new dataset.
Using our model, we are able to generate realistic videos of up to $128\times
128$ resolution for 32 frames. Quantitative and qualitative experiment results
have demonstrated the superiority of our model over the state-of-the-art
models.
Learning View-Specific Deep Networks for Person Re-Identification
Authors: Zhanxiang Feng, Jian-Huang Lai, Xiaohua Xie
Date: 2018-03-30 00:00:00
Tasks: Person Re-Identification

In recent years, a growing body of research has focused on the problem of
person re-identification (re-id). The re-id techniques attempt to match the
images of pedestrians from disjoint non-overlapping camera views. A major
challenge of re-id is the serious intra-class variations caused by changing
viewpoints. To overcome this challenge, we propose a deep neural network-based
framework which utilizes the view information in the feature extraction stage.
The proposed framework learns a view-specific network for each camera view with
a cross-view Euclidean constraint (CV-EC) and a cross-view center loss (CV-CL).
We utilize CV-EC to decrease the margin of the features between diverse views
and extend the center loss metric to a view-specific version to better adapt
the re-id problem. Moreover, we propose an iterative algorithm to optimize the
parameters of the view-specific networks from coarse to fine. The experiments
demonstrate that our approach significantly improves the performance of the
existing deep networks and outperforms the state-of-the-art methods on the
VIPeR, CUHK01, CUHK03, SYSU-mReId, and Market-1501 benchmarks.