pwc-archive/papers-with-abstracts huggingface dataset:
Unconstrained Fashion Landmark Detection via Hierarchical Recurrent Transformer Networks
Authors: Sijie Yan, Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, Xiaoou Tang
Date: 2017-08-07 00:00:00

Fashion landmarks are functional key points defined on clothes, such as
corners of neckline, hemline, and cuff. They have been recently introduced as
an effective visual representation for fashion image understanding. However,
detecting fashion landmarks are challenging due to background clutters, human
poses, and scales. To remove the above variations, previous works usually
assumed bounding boxes of clothes are provided in training and test as
additional annotations, which are expensive to obtain and inapplicable in
practice. This work addresses unconstrained fashion landmark detection, where
clothing bounding boxes are not provided in both training and test. To this
end, we present a novel Deep LAndmark Network (DLAN), where bounding boxes and
landmarks are jointly estimated and trained iteratively in an end-to-end
manner. DLAN contains two dedicated modules, including a Selective Dilated
Convolution for handling scale discrepancies, and a Hierarchical Recurrent
Spatial Transformer for handling background clutters. To evaluate DLAN, we
present a large-scale fashion landmark dataset, namely Unconstrained Landmark
Database (ULD), consisting of 30K images. Statistics show that ULD is more
challenging than existing datasets in terms of image scales, background
clutters, and human poses. Extensive experiments demonstrate the effectiveness
of DLAN over the state-of-the-art methods. DLAN also exhibits excellent
generalization across different clothing categories and modalities, making it
extremely suitable for real-world fashion analysis.
Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images
Authors: Tribhuvanesh Orekondy, Bernt Schiele, Mario Fritz
Date: 2017-03-30 00:00:00
Proceeding: ICCV 2017 10

With an increasing number of users sharing information online, privacy
implications entailing such actions are a major concern. For explicit content,
such as user profile or GPS data, devices (e.g. mobile phones) as well as web
services (e.g. Facebook) offer to set privacy settings in order to enforce the
users' privacy preferences. We propose the first approach that extends this
concept to image content in the spirit of a Visual Privacy Advisor. First, we
categorize personal information in images into 68 image attributes and collect
a dataset, which allows us to train models that predict such information
directly from images. Second, we run a user study to understand the privacy
preferences of different users w.r.t. such attributes. Third, we propose models
that predict user specific privacy score from images in order to enforce the
users' privacy preferences. Our model is trained to predict the user specific
privacy risk and even outperforms the judgment of the users, who often fail to
follow their own privacy preferences on image data.
A Solution for Crime Scene Reconstruction using Time-of-Flight Cameras
Authors: Silvio Giancola, Daniele Piron, Pasquale Poppa, Remo Sala
Date: 2017-08-07 00:00:00
Tasks: 3D Reconstruction, Simultaneous Localization and Mapping

In this work, we propose a method for three-dimensional (3D) reconstruction
of wide crime scene, based on a Simultaneous Localization and Mapping (SLAM)
approach. We used a Kinect V2 Time-of-Flight (TOF) RGB-D camera to provide
colored dense point clouds at a 30 Hz frequency. This device is moved freely (6
degrees of freedom) during the scene exploration. The implemented SLAM solution
aligns successive point clouds using an 3D keypoints description and matching
approach. This type of approach exploits both colorimetric and geometrical
information, and permits reconstruction under poor illumination conditions. Our
solution has been tested for indoor crime scene and outdoor archaeological site
reconstruction, returning a mean error around one centimeter. It is less
precise than environmental laser scanner solution, but more practical and
portable as well as less cumbersome. Also, the hardware is definitively
cheaper.
Learning Uncertain Convolutional Features for Accurate Saliency Detection
Authors: Pingping Zhang, Dong Wang, Huchuan Lu, Hongyu Wang, Bao-Cai Yin
Date: 2017-08-07 00:00:00
Proceeding: ICCV 2017 10
Tasks: Decoder, object-detection, Object Detection, RGB Salient Object Detection, Saliency Detection, Salient Object Detection

Deep convolutional neural networks (CNNs) have delivered superior performance
in many computer vision tasks. In this paper, we propose a novel deep fully
convolutional network model for accurate salient object detection. The key
contribution of this work is to learn deep uncertain convolutional features
(UCF), which encourage the robustness and accuracy of saliency detection. We
achieve this via introducing a reformulated dropout (R-dropout) after specific
convolutional layers to construct an uncertain ensemble of internal feature
units. In addition, we propose an effective hybrid upsampling method to reduce
the checkerboard artifacts of deconvolution operators in our decoder network.
The proposed methods can also be applied to other deep convolutional networks.
Compared with existing saliency detection methods, the proposed UCF model is
able to incorporate uncertainties for more accurate object boundary inference.
Extensive experiments demonstrate that our proposed saliency model performs
favorably against state-of-the-art approaches. The uncertain feature learning
mechanism as well as the upsampling method can significantly improve
performance on other pixel-wise vision tasks.
Memory-augmented Neural Machine Translation
Authors: Yang Feng, Shiyue Zhang, Andi Zhang, Dong Wang, Andrew Abel
Date: 2017-08-07 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Machine Translation, NMT, Translation

Neural machine translation (NMT) has achieved notable success in recent
times, however it is also widely recognized that this approach has limitations
with handling infrequent words and word pairs. This paper presents a novel
memory-augmented NMT (M-NMT) architecture, which stores knowledge about how
words (usually infrequently encountered ones) should be translated in a memory
and then utilizes them to assist the neural model. We use this memory mechanism
to combine the knowledge learned from a conventional statistical machine
translation system and the rules learned by an NMT system, and also propose a
solution for out-of-vocabulary (OOV) words based on this framework. Our
experiments on two Chinese-English translation tasks demonstrated that the
M-NMT architecture outperformed the NMT baseline by $9.0$ and $2.7$ BLEU points
on the two tasks, respectively. Additionally, we found this architecture
resulted in a much more effective OOV treatment compared to competitive
methods.
Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection
Authors: Pingping Zhang, Dong Wang, Huchuan Lu, Hongyu Wang, Xiang Ruan
Date: 2017-08-07 00:00:00
Proceeding: ICCV 2017 10
Tasks: Object, object-detection, Object Detection, RGB Salient Object Detection, Salient Object Detection

Fully convolutional neural networks (FCNs) have shown outstanding performance
in many dense labeling problems. One key pillar of these successes is mining
relevant information from features in convolutional layers. However, how to
better aggregate multi-level convolutional feature maps for salient object
detection is underexplored. In this work, we present Amulet, a generic
aggregating multi-level convolutional feature framework for salient object
detection. Our framework first integrates multi-level feature maps into
multiple resolutions, which simultaneously incorporate coarse semantics and
fine details. Then it adaptively learns to combine these feature maps at each
resolution and predict saliency maps with the combined features. Finally, the
predicted results are efficiently fused to generate the final saliency map. In
addition, to achieve accurate boundary inference and semantic enhancement,
edge-aware feature maps in low-level layers and the predicted results of low
resolution features are recursively embedded into the learning framework. By
aggregating multi-level convolutional features in this efficient and flexible
manner, the proposed saliency model provides accurate salient object labeling.
Comprehensive experiments demonstrate that our method performs favorably
against state-of-the art approaches in terms of near all compared evaluation
metrics.
Identity-Aware Textual-Visual Matching with Latent Co-attention
Authors: Shuang Li, Tong Xiao, Hongsheng Li, Wei Yang, Xiaogang Wang
Date: 2017-08-07 00:00:00
Proceeding: ICCV 2017 10
Tasks: Sentence, Text based Person Retrieval

Textual-visual matching aims at measuring similarities between sentence
descriptions and images. Most existing methods tackle this problem without
effectively utilizing identity-level annotations. In this paper, we propose an
identity-aware two-stage framework for the textual-visual matching problem. Our
stage-1 CNN-LSTM network learns to embed cross-modal features with a novel
Cross-Modal Cross-Entropy (CMCE) loss. The stage-1 network is able to
efficiently screen easy incorrect matchings and also provide initial training
point for the stage-2 training. The stage-2 CNN-LSTM network refines the
matching results with a latent co-attention mechanism. The spatial attention
relates each word with corresponding image regions while the latent semantic
attention aligns different sentence structures to make the matching results
more robust to sentence structure variations. Extensive experiments on three
datasets with identity-level annotations show that our framework outperforms
state-of-the-art approaches by large margins.
Translating Phrases in Neural Machine Translation
Authors: Xing Wang, Zhaopeng Tu, Deyi Xiong, Min Zhang
Date: 2017-08-07 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Decoder, Machine Translation, Natural Language Understanding, NMT, Translation

Phrases play an important role in natural language understanding and machine
translation (Sag et al., 2002; Villavicencio et al., 2005). However, it is
difficult to integrate them into current neural machine translation (NMT) which
reads and generates sentences word by word. In this work, we propose a method
to translate phrases in NMT by integrating a phrase memory storing target
phrases from a phrase-based statistical machine translation (SMT) system into
the encoder-decoder architecture of NMT. At each decoding step, the phrase
memory is first re-written by the SMT model, which dynamically generates
relevant target phrases with contextual information provided by the NMT model.
Then the proposed model reads the phrase memory to make probability estimations
for all phrases in the phrase memory. If phrase generation is carried on, the
NMT decoder selects an appropriate phrase from the memory to perform phrase
translation and updates its decoding state by consuming the words in the
selected phrase. Otherwise, the NMT decoder generates a word from the
vocabulary as the general NMT decoder does. Experiment results on the Chinese
to English translation show that the proposed model achieves significant
improvements over the baseline on various test sets.
Correlation and Class Based Block Formation for Improved Structured Dictionary Learning
Authors: Nagendra Kumar, Rohit Sinha
Date: 2017-08-04 00:00:00
Tasks: Clustering, Dictionary Learning, General Classification, Speaker Verification

In recent years, the creation of block-structured dictionary has attracted a
lot of interest. Learning such dictionaries involve two step process: block
formation and dictionary update. Both these steps are important in producing an
effective dictionary. The existing works mostly assume that the block structure
is known a priori while learning the dictionary. For finding the unknown block
structure given a dictionary commonly sparse agglomerative clustering (SAC) is
used. It groups atoms based on their consistency in sparse coding with respect
to the unstructured dictionary. This paper explores two innovations towards
improving the reconstruction as well as the classification ability achieved
with the block-structured dictionary. First, we propose a novel block
structuring approach that makes use of the correlation among dictionary atoms.
Unlike the SAC approach, which groups diverse atoms, in the proposed approach
the blocks are formed by grouping the top most correlated atoms in the
dictionary. The proposed block clustering approach is noted to yield
significant reductions in redundancy as well as provides a direct control on
the block size when compared with the existing SAC-based block structuring.
Later, motivated by works using supervised \emph{a priori} known block
structure, we also explore the incorporation of class information in the
proposed block formation approach to further enhance the classification ability
of the block dictionary. For assessment of the reconstruction ability with
proposed innovations is done on synthetic data while the classification ability
has been evaluated in large variability speaker verification task.
Voice Conversion Using Sequence-to-Sequence Learning of Context Posterior Probabilities
Authors: Hiroyuki Miyoshi, Yuki Saito, Shinnosuke Takamichi, Hiroshi Saruwatari
Date: 2017-04-10 00:00:00
Tasks: speech-recognition, Speech Recognition, Speech Synthesis, Voice Conversion

Voice conversion (VC) using sequence-to-sequence learning of context
posterior probabilities is proposed. Conventional VC using shared context
posterior probabilities predicts target speech parameters from the context
posterior probabilities estimated from the source speech parameters. Although
conventional VC can be built from non-parallel data, it is difficult to convert
speaker individuality such as phonetic property and speaking rate contained in
the posterior probabilities because the source posterior probabilities are
directly used for predicting target speech parameters. In this work, we assume
that the training data partly include parallel speech data and propose
sequence-to-sequence learning between the source and target posterior
probabilities. The conversion models perform non-linear and variable-length
transformation from the source probability sequence to the target one. Further,
we propose a joint training algorithm for the modules. In contrast to
conventional VC, which separately trains the speech recognition that estimates
posterior probabilities and the speech synthesis that predicts target speech
parameters, our proposed method jointly trains these modules along with the
proposed probability conversion modules. Experimental results demonstrate that
our approach outperforms the conventional VC.
Accurate Light Field Depth Estimation with Superpixel Regularization over Partially Occluded Regions
Authors: Jie Chen, Junhui Hou, Yun Ni, Lap-Pui Chau
Date: 2017-08-07 00:00:00
Tasks: Depth Estimation

Depth estimation is a fundamental problem for light field photography
applications. Numerous methods have been proposed in recent years, which either
focus on crafting cost terms for more robust matching, or on analyzing the
geometry of scene structures embedded in the epipolar-plane images. Significant
improvements have been made in terms of overall depth estimation error;
however, current state-of-the-art methods still show limitations in handling
intricate occluding structures and complex scenes with multiple occlusions. To
address these challenging issues, we propose a very effective depth estimation
framework which focuses on regularizing the initial label confidence map and
edge strength weights. Specifically, we first detect partially occluded
boundary regions (POBR) via superpixel based regularization. Series of
shrinkage/reinforcement operations are then applied on the label confidence map
and edge strength weights over the POBR. We show that after weight
manipulations, even a low-complexity weighted least squares model can produce
much better depth estimation than state-of-the-art methods in terms of average
disparity error rate, occlusion boundary precision-recall rate, and the
preservation of intricate visual features.
Identifying global optimality for dictionary learning
Authors: Lei Le, Martha White
Date: 2016-04-17 00:00:00
Tasks: Dictionary Learning, Form, global-optimization, Matrix Completion

Learning new representations of input observations in machine learning is
often tackled using a factorization of the data. For many such problems,
including sparse coding and matrix completion, learning these factorizations
can be difficult, in terms of efficiency and to guarantee that the solution is
a global minimum. Recently, a general class of objectives have been
introduced-which we term induced dictionary learning models (DLMs)-that have an
induced convex form that enables global optimization. Though attractive
theoretically, this induced form is impractical, particularly for large or
growing datasets. In this work, we investigate the use of practical alternating
minimization algorithms for induced DLMs, that ensure convergence to global
optima. We characterize the stationary points of these models, and, using these
insights, highlight practical choices for the objectives. We then provide
theoretical and empirical evidence that alternating minimization, from a random
initialization, converges to global minima for a large subclass of induced
DLMs. In particular, we take advantage of the existence of the (potentially
unknown) convex induced form, to identify when stationary points are global
minima for the dictionary learning objective. We then provide an empirical
investigation into practical optimization choices for using alternating
minimization for induced DLMs, for both batch and stochastic gradient descent.
Pose-conditioned Spatio-Temporal Attention for Human Action Recognition
Authors: Fabien Baradel, Christian Wolf, Julien Mille
Date: 2017-03-29 00:00:00
Tasks: Action Recognition, Activity Recognition, Human Activity Recognition, Temporal Action Localization

We address human action recognition from multi-modal video data involving
articulated pose and RGB frames and propose a two-stream approach. The pose
stream is processed with a convolutional model taking as input a 3D tensor
holding data from a sub-sequence. A specific joint ordering, which respects the
topology of the human body, ensures that different convolutional layers
correspond to meaningful levels of abstraction. The raw RGB stream is handled
by a spatio-temporal soft-attention mechanism conditioned on features from the
pose network. An LSTM network receives input from a set of image locations at
each instant. A trainable glimpse sensor extracts features on a set of
predefined locations specified by the pose stream, namely the 4 hands of the
two people involved in the activity. Appearance features give important cues on
hand motion and on objects held in each hand. We show that it is of high
interest to shift the attention to different hands at different time steps
depending on the activity itself. Finally a temporal attention mechanism learns
how to fuse LSTM features over time. We evaluate the method on 3 datasets.
State-of-the-art results are achieved on the largest dataset for human activity
recognition, namely NTU-RGB+D, as well as on the SBU Kinect Interaction
dataset. Performance close to state-of-the-art is achieved on the smaller MSR
Daily Activity 3D dataset.
Learning Theory of Distributed Regression with Bias Corrected Regularization Kernel Network
Authors: Zheng-Chu Guo, Lei Shi, Qiang Wu
Date: 2017-08-07 00:00:00
Tasks: Learning Theory, regression

Distributed learning is an effective way to analyze big data. In distributed
regression, a typical approach is to divide the big data into multiple blocks,
apply a base regression algorithm on each of them, and then simply average the
output functions learnt from these blocks. Since the average process will
decrease the variance, not the bias, bias correction is expected to improve the
learning performance if the base regression algorithm is a biased one.
Regularization kernel network is an effective and widely used method for
nonlinear regression analysis. In this paper we will investigate a bias
corrected version of regularization kernel network. We derive the error bounds
when it is applied to a single data set and when it is applied as a base
algorithm in distributed regression. We show that, under certain appropriate
conditions, the optimal learning rates can be reached in both situations.
Analysis of Algorithms and Partial Algorithms
Authors: Andrew MacFie
Date: 2016-01-13 00:00:00
Tasks: Automated Theorem Proving

We present an alternative methodology for the analysis of algorithms, based
on the concept of expected discounted reward. This methodology naturally
handles algorithms that do not always terminate, so it can (theoretically) be
used with partial algorithms for undecidable problems, such as those found in
artificial general intelligence (AGI) and automated theorem proving. We mention
an approach to self-improving AGI enabled by this methodology.
  Aug 2017 addendum: This article was originally written with multiple
audiences in mind. It is really best put in the following terms. Goertzel,
Hutter, Legg, and others have developed a definition of an intelligence score
for a general abstract agent: expected lifetime reward in a random environment.
AIXI is generally the optimal agent according to this score, but there may be
reasons to analyze other agents and compare score values. If we want to use
this definition of intelligence in practice, perhaps we can start by analyzing
some simple agents. Common algorithms can be thought of as simple agents
(environment is input, reward is based on running time) so we take the goal of
applying the agent intelligence score to algorithms. That is, we want to find,
what are the IQ scores of algorithms? We can do some very simple analysis, but
the real answer is that even for simple algorithms, the intelligence score is
too difficult to work with in practice.
Tackling Over-pruning in Variational Autoencoders
Authors: Serena Yeung, Anitha Kannan, Yann Dauphin, Li Fei-Fei
Date: 2017-06-09 00:00:00

Variational autoencoders (VAE) are directed generative models that learn
factorial latent variables. As noted by Burda et al. (2015), these models
exhibit the problem of factor over-pruning where a significant number of
stochastic factors fail to learn anything and become inactive. This can limit
their modeling power and their ability to learn diverse and meaningful latent
representations. In this paper, we evaluate several methods to address this
problem and propose a more effective model-based approach called the epitomic
variational autoencoder (eVAE). The so-called epitomes of this model are groups
of mutually exclusive latent factors that compete to explain the data. This
approach helps prevent inactive units since each group is pressured to explain
the data. We compare the approaches with qualitative and quantitative results
on MNIST and TFD datasets. Our results show that eVAE makes efficient use of
model capacity and generalizes better than VAE.