pwc-archive/papers-with-abstracts huggingface dataset:
Finite-sample and asymptotic analysis of generalization ability with an application to penalized regression
Authors: Ning Xu, Jian Hong, Timothy C. G. Fisher
Date: 2016-09-12 00:00:00
Tasks: regression

In this paper, we study the performance of extremum estimators from the
perspective of generalization ability (GA): the ability of a model to predict
outcomes in new samples from the same population. By adapting the classical
concentration inequalities, we derive upper bounds on the empirical
out-of-sample prediction errors as a function of the in-sample errors,
in-sample data size, heaviness in the tails of the error distribution, and
model complexity. We show that the error bounds may be used for tuning key
estimation hyper-parameters, such as the number of folds $K$ in
cross-validation. We also show how $K$ affects the bias-variance trade-off for
cross-validation. We demonstrate that the $\mathcal{L}_2$-norm difference
between penalized and the corresponding un-penalized regression estimates is
directly explained by the GA of the estimates and the GA of empirical moment
conditions. Lastly, we prove that all penalized regression estimates are
$L_2$-consistent for both the $n \geqslant p$ and the $n < p$ cases.
Simulations are used to demonstrate key results.
  Keywords: generalization ability, upper bound of generalization error,
penalized regression, cross-validation, bias-variance trade-off,
$\mathcal{L}_2$ difference between penalized and unpenalized regression, lasso,
high-dimensional data.
Deep Relative Attributes
Authors: Yaser Souri, Erfan Noury, Ehsan Adeli
Date: 2015-12-13 00:00:00
Tasks: Attribute

Visual attributes are great means of describing images or scenes, in a way
both humans and computers understand. In order to establish a correspondence
between images and to be able to compare the strength of each property between
images, relative attributes were introduced. However, since their introduction,
hand-crafted and engineered features were used to learn increasingly complex
models for the problem of relative attributes. This limits the applicability of
those methods for more realistic cases. We introduce a deep neural network
architecture for the task of relative attribute prediction. A convolutional
neural network (ConvNet) is adopted to learn the features by including an
additional layer (ranking layer) that learns to rank the images based on these
features. We adopt an appropriate ranking loss to train the whole network in an
end-to-end fashion. Our proposed method outperforms the baseline and
state-of-the-art methods in relative attribute prediction on various coarse and
fine-grained datasets. Our qualitative results along with the visualization of
the saliency maps show that the network is able to learn effective features for
each specific attribute. Source code of the proposed method is available at
https://github.com/yassersouri/ghiaseddin.
A Greedy Algorithm to Cluster Specialists
Authors: Sébastien Arnold
Date: 2016-09-13 00:00:00
Tasks: Classification, Clustering, General Classification

Several recent deep neural networks experiments leverage the
generalist-specialist paradigm for classification. However, no formal study
compared the performance of different clustering algorithms for class
assignment. In this paper we perform such a study, suggest slight modifications
to the clustering procedures, and propose a novel algorithm designed to
optimize the performance of of the specialist-generalist classification system.
Our experiments on the CIFAR-10 and CIFAR-100 datasets allow us to investigate
situations for varying number of classes on similar data. We find that our
\emph{greedy pairs} clustering algorithm consistently outperforms other
alternatives, while the choice of the confusion matrix has little impact on the
final performance.
An Experimental Study of LSTM Encoder-Decoder Model for Text Simplification
Authors: Tong Wang, Ping Chen, Kevin Amaral, jipeng qiang
Date: 2016-09-13 00:00:00
Tasks: Decoder, Sentence, Text Simplification

Text simplification (TS) aims to reduce the lexical and structural complexity
of a text, while still retaining the semantic meaning. Current automatic TS
techniques are limited to either lexical-level applications or manually
defining a large amount of rules. Since deep neural networks are powerful
models that have achieved excellent performance over many difficult tasks, in
this paper, we propose to use the Long Short-Term Memory (LSTM) Encoder-Decoder
model for sentence level TS, which makes minimal assumptions about word
sequence. We conduct preliminary experiments to find that the model is able to
learn operation rules such as reversing, sorting and replacing from sequence
pairs, which shows that the model may potentially discover and apply rules such
as modifying sentence structure, substituting words, and removing words for TS.
Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
Authors: Ronan Collobert, Christian Puhrsch, Gabriel Synnaeve
Date: 2016-09-11 00:00:00
Proceeding: arXiv 2016 9
Tasks: Speech Recognition

This paper presents a simple end-to-end model for speech recognition,
combining a convolutional network based acoustic model and a graph decoding. It
is trained to output letters, with transcribed speech, without the need for
force alignment of phonemes. We introduce an automatic segmentation criterion
for training from sequence annotation without alignment that is on par with CTC
while being simpler. We show competitive results in word error rate on the
Librispeech corpus with MFCC features, and promising results from raw waveform.
ZaliQL: A SQL-Based Framework for Drawing Causal Inference from Big Data
Authors: Babak Salimi, Dan Suciu
Date: 2016-09-12 00:00:00
Tasks: Causal Inference

Causal inference from observational data is a subject of active research and
development in statistics and computer science. Many toolkits have been
developed for this purpose that depends on statistical software. However, these
toolkits do not scale to large datasets. In this paper we describe a suite of
techniques for expressing causal inference tasks from observational data in
SQL. This suite supports the state-of-the-art methods for causal inference and
run at scale within a database engine. In addition, we introduce several
optimization techniques that significantly speedup causal inference, both in
the online and offline setting. We evaluate the quality and performance of our
techniques by experiments of real datasets.
Dictionary learning based image enhancement for rarity detection
Authors: Hui Li, Xiaomeng Wang, Weifeng Liu, Yan-Jiang Wang
Date: 2013-05-04 00:00:00
Tasks: Dictionary Learning, Image Enhancement

Image enhancement is an important image processing technique that processes
images suitably for a specific application e.g. image editing. The conventional
solutions of image enhancement are grouped into two categories which are
spatial domain processing method and transform domain processing method such as
contrast manipulation, histogram equalization, homomorphic filtering. This
paper proposes a new image enhance method based on dictionary learning.
Particularly, the proposed method adjusts the image by manipulating the rarity
of dictionary atoms. Firstly, learn the dictionary through sparse coding
algorithms on divided sub-image blocks. Secondly, compute the rarity of
dictionary atoms on statistics of the corresponding sparse coefficients.
Thirdly, adjust the rarity according to specific application and form a new
dictionary. Finally, reconstruct the image using the updated dictionary and
sparse coefficients. Compared with the traditional techniques, the proposed
method enhances image based on the image content not on distribution of pixel
grey value or frequency. The advantages of the proposed method lie in that it
is in better correspondence with the response of the human visual system and
more suitable for salient objects extraction. The experimental results
demonstrate the effectiveness of the proposed image enhance method.
Joint Extraction of Events and Entities within a Document Context
Authors: Bishan Yang, Tom Mitchell
Date: 2016-09-12 00:00:00
Proceeding: NAACL 2016 6
Tasks: Entity Extraction using GAN, Event Extraction, Sentence

Events and entities are closely related; entities are often actors or
participants in events and events without entities are uncommon. The
interpretation of events and entities is highly contextually dependent.
Existing work in information extraction typically models events separately from
entities, and performs inference at the sentence level, ignoring the rest of
the document. In this paper, we propose a novel approach that models the
dependencies among variables of events, entities, and their relations, and
performs joint inference of these variables across a document. The goal is to
enable access to document-level contextual information and facilitate
context-aware predictions. We demonstrate that our approach substantially
outperforms the state-of-the-art methods for event extraction as well as a
strong baseline for entity extraction.
A Tale of Two Bases: Local-Nonlocal Regularization on Image Patches with Convolution Framelets
Authors: Rujie Yin, Tingran Gao, Yue M. Lu, Ingrid Daubechies
Date: 2016-06-04 00:00:00
Tasks: Denoising, Dimensionality Reduction, Image Denoising, Image Inpainting

We propose an image representation scheme combining the local and nonlocal
characterization of patches in an image. Our representation scheme can be shown
to be equivalent to a tight frame constructed from convolving local bases (e.g.
wavelet frames, discrete cosine transforms, etc.) with nonlocal bases (e.g.
spectral basis induced by nonlinear dimension reduction on patches), and we
call the resulting frame elements {\it convolution framelets}. Insight gained
from analyzing the proposed representation leads to a novel interpretation of a
recent high-performance patch-based image inpainting algorithm using Point
Integral Method (PIM) and Low Dimension Manifold Model (LDMM) [Osher, Shi and
Zhu, 2016]. In particular, we show that LDMM is a weighted
$\ell_2$-regularization on the coefficients obtained by decomposing images into
linear combinations of convolution framelets; based on this understanding, we
extend the original LDMM to a reweighted version that yields further improved
inpainting results. In addition, we establish the energy concentration property
of convolution framelet coefficients for the setting where the local basis is
constructed from a given nonlocal basis via a linear reconstruction framework;
a generalization of this framework to unions of local embeddings can provide a
natural setting for interpreting BM3D, one of the state-of-the-art image
denoising algorithms.
Co-active Learning to Adapt Humanoid Movement for Manipulation
Authors: Ren Mao, John S. Baras, Yezhou Yang, Cornelia Fermuller
Date: 2016-09-12 00:00:00
Tasks: Active Learning, Motion Generation

In this paper we address the problem of robot movement adaptation under
various environmental constraints interactively. Motion primitives are
generally adopted to generate target motion from demonstrations. However, their
generalization capability is weak while facing novel environments.
Additionally, traditional motion generation methods do not consider the
versatile constraints from various users, tasks, and environments. In this
work, we propose a co-active learning framework for learning to adapt robot
end-effector's movement for manipulation tasks. It is designed to adapt the
original imitation trajectories, which are learned from demonstrations, to
novel situations with various constraints. The framework also considers user's
feedback towards the adapted trajectories, and it learns to adapt movement
through human-in-the-loop interactions. The implemented system generalizes
trained motion primitives to various situations with different constraints
considering user preferences. Experiments on a humanoid platform validate the
effectiveness of our approach.
Nested Mini-Batch K-Means
Authors: James Newling, François Fleuret
Date: 2016-02-09 00:00:00
Proceeding: NeurIPS 2016 12

A new algorithm is proposed which accelerates the mini-batch k-means
algorithm of Sculley (2010) by using the distance bounding approach of Elkan
(2003). We argue that, when incorporating distance bounds into a mini-batch
algorithm, already used data should preferentially be reused. To this end we
propose using nested mini-batches, whereby data in a mini-batch at iteration t
is automatically reused at iteration t+1.
  Using nested mini-batches presents two difficulties. The first is that
unbalanced use of data can bias estimates, which we resolve by ensuring that
each data sample contributes exactly once to centroids. The second is in
choosing mini-batch sizes, which we address by balancing premature fine-tuning
of centroids with redundancy induced slow-down. Experiments show that the
resulting nmbatch algorithm is very effective, often arriving within 1% of the
empirical minimum 100 times earlier than the standard mini-batch algorithm.
Detecting Text in Natural Image with Connectionist Text Proposal Network
Authors: Zhi Tian, Weilin Huang, Tong He, Pan He, Yu Qiao
Date: 2016-09-12 00:00:00
Tasks: Scene Text Detection

We propose a novel Connectionist Text Proposal Network (CTPN) that accurately
localizes text lines in natural image. The CTPN detects a text line in a
sequence of fine-scale text proposals directly in convolutional feature maps.
We develop a vertical anchor mechanism that jointly predicts location and
text/non-text score of each fixed-width proposal, considerably improving
localization accuracy. The sequential proposals are naturally connected by a
recurrent neural network, which is seamlessly incorporated into the
convolutional network, resulting in an end-to-end trainable model. This allows
the CTPN to explore rich context information of image, making it powerful to
detect extremely ambiguous text. The CTPN works reliably on multi-scale and
multi- language text without further post-processing, departing from previous
bottom-up methods requiring multi-step post-processing. It achieves 0.88 and
0.61 F-measure on the ICDAR 2013 and 2015 benchmarks, surpass- ing recent
results [8, 35] by a large margin. The CTPN is computationally efficient with
0:14s/image, by using the very deep VGG16 model [27]. Online demo is available
at: http://textdet.com/.
Dilemma First Search for Effortless Optimization of NP-Hard Problems
Authors: Julien Weissenberg, Hayko Riemenschneider, Ralf Dragon, Luc van Gool
Date: 2016-09-12 00:00:00

To tackle the exponentiality associated with NP-hard problems, two paradigms
have been proposed. First, Branch & Bound, like Dynamic Programming, achieve
efficient exact inference but requires extensive information and analysis about
the problem at hand. Second, meta-heuristics are easier to implement but
comparatively inefficient. As a result, a number of problems have been left
unoptimized and plain greedy solutions are used. We introduce a theoretical
framework and propose a powerful yet simple search method called Dilemma First
Search (DFS). DFS exploits the decision heuristic needed for the greedy
solution for further optimization. DFS is useful when it is hard to design
efficient exact inference. We evaluate DFS on two problems: First, the Knapsack
problem, for which efficient algorithms exist, serves as a toy example. Second,
Decision Tree inference, where state-of-the-art algorithms rely on the greedy
or randomness-based solutions. We further show that decision trees benefit from
optimizations that are performed in a fraction of the iterations required by a
random-based search.
Online Data Thinning via Multi-Subspace Tracking
Authors: Xin Jiang, Rebecca Willett
Date: 2016-09-12 00:00:00
Tasks: Anomaly Detection, Clustering

In an era of ubiquitous large-scale streaming data, the availability of data
far exceeds the capacity of expert human analysts. In many settings, such data
is either discarded or stored unprocessed in datacenters. This paper proposes a
method of online data thinning, in which large-scale streaming datasets are
winnowed to preserve unique, anomalous, or salient elements for timely expert
analysis. At the heart of this proposed approach is an online anomaly detection
method based on dynamic, low-rank Gaussian mixture models. Specifically, the
high-dimensional covariances matrices associated with the Gaussian components
are associated with low-rank models. According to this model, most observations
lie near a union of subspaces. The low-rank modeling mitigates the curse of
dimensionality associated with anomaly detection for high-dimensional data, and
recent advances in subspace clustering and subspace tracking allow the proposed
method to adapt to dynamic environments. Furthermore, the proposed method
allows subsampling, is robust to missing data, and uses a mini-batch online
optimization approach. The resulting algorithms are scalable, efficient, and
are capable of operating in real time. Experiments on wide-area motion imagery
and e-mail databases illustrate the efficacy of the proposed approach.
Comment on "Why does deep and cheap learning work so well?" [arXiv:1608.08225]
Authors: David J. Schwab, Pankaj Mehta
Date: 2016-09-12 00:00:00

In a recent paper, "Why does deep and cheap learning work so well?", Lin and
Tegmark claim to show that the mapping between deep belief networks and the
variational renormalization group derived in [arXiv:1410.3831] is invalid, and
present a "counterexample" that claims to show that this mapping does not hold.
In this comment, we show that these claims are incorrect and stem from a
misunderstanding of the variational RG procedure proposed by Kadanoff. We also
explain why the "counterexample" of Lin and Tegmark is compatible with the
mapping proposed in [arXiv:1410.3831].
A Multi-Scale Cascade Fully Convolutional Network Face Detector
Authors: Zhenheng Yang, Ram Nevatia
Date: 2016-09-12 00:00:00
Tasks: Face Detection

Face detection is challenging as faces in images could be present at
arbitrary locations and in different scales. We propose a three-stage cascade
structure based on fully convolutional neural networks (FCNs). It first
proposes the approximate locations where the faces may be, then aims to find
the accurate location by zooming on to the faces. Each level of the FCN cascade
is a multi-scale fully-convolutional network, which generates scores at
different locations and in different scales. A score map is generated after
each FCN stage. Probable regions of face are selected and fed to the next
stage. The number of proposals is decreased after each level, and the areas of
regions are decreased to more precisely fit the face. Compared to passing
proposals directly between stages, passing probable regions can decrease the
number of proposals and reduce the cases where first stage doesn't propose good
bounding boxes. We show that by using FCN and score map, the FCN cascade face
detector can achieve strong performance on public datasets.
Fully-Trainable Deep Matching
Authors: James Thewlis, Shuai Zheng, Philip H. S. Torr, Andrea Vedaldi
Date: 2016-09-12 00:00:00
Tasks: Image Segmentation, Semantic Segmentation

Deep Matching (DM) is a popular high-quality method for quasi-dense image
matching. Despite its name, however, the original DM formulation does not yield
a deep neural network that can be trained end-to-end via backpropagation. In
this paper, we remove this limitation by rewriting the complete DM algorithm as
a convolutional neural network. This results in a novel deep architecture for
image matching that involves a number of new layer types and that, similar to
recent networks for image segmentation, has a U-topology. We demonstrate the
utility of the approach by improving the performance of DM by learning it
end-to-end on an image matching task.
Detailed Garment Recovery from a Single-View Image
Authors: Shan Yang, Tanya Ambert, Zherong Pan, Ke Wang, Licheng Yu, Tamara Berg, Ming C. Lin
Date: 2016-08-03 00:00:00
Tasks: parameter estimation, Semantic Parsing, Virtual Try-on

Most recent garment capturing techniques rely on acquiring multiple views of
clothing, which may not always be readily available, especially in the case of
pre-existing photographs from the web. As an alternative, we pro- pose a method
that is able to compute a rich and realistic 3D model of a human body and its
outfits from a single photograph with little human in- teraction. Our algorithm
is not only able to capture the global shape and geometry of the clothing, it
can also extract small but important details of cloth, such as occluded
wrinkles and folds. Unlike previous methods using full 3D information (i.e.
depth, multi-view images, or sampled 3D geom- etry), our approach achieves
detailed garment recovery from a single-view image by using statistical,
geometric, and physical priors and a combina- tion of parameter estimation,
semantic parsing, shape recovery, and physics- based cloth simulation. We
demonstrate the effectiveness of our algorithm by re-purposing the
reconstructed garments for virtual try-on and garment transfer applications, as
well as cloth animation for digital characters.
Examining Representational Similarity in ConvNets and the Primate Visual Cortex
Authors: Abhimanyu Dubey, Jayadeva, Sumeet Agarwal
Date: 2016-09-12 00:00:00

We compare several ConvNets with different depth and regularization
techniques with multi-unit macaque IT cortex recordings and assess the impact
of the same on representational similarity with the primate visual cortex. We
find that with increasing depth and validation performance, ConvNet features
are closer to cortical IT representations.
Deep Learning the City : Quantifying Urban Perception At A Global Scale
Authors: Abhimanyu Dubey, Nikhil Naik, Devi Parikh, Ramesh Raskar, César A. Hidalgo
Date: 2016-08-05 00:00:00
Tasks: General Classification

Computer vision methods that quantify the perception of urban environment are
increasingly being used to study the relationship between a city's physical
appearance and the behavior and health of its residents. Yet, the throughput of
current methods is too limited to quantify the perception of cities across the
world. To tackle this challenge, we introduce a new crowdsourced dataset
containing 110,988 images from 56 cities, and 1,170,000 pairwise comparisons
provided by 81,630 online volunteers along six perceptual attributes: safe,
lively, boring, wealthy, depressing, and beautiful. Using this data, we train a
Siamese-like convolutional neural architecture, which learns from a joint
classification and ranking loss, to predict human judgments of pairwise image
comparisons. Our results show that crowdsourcing combined with neural networks
can produce urban perception data at the global scale.
Optimal Encoding and Decoding for Point Process Observations: an Approximate Closed-Form Filter
Authors: Yuval Harel, Ron Meir, Manfred Opper
Date: 2016-09-12 00:00:00
Tasks: Form, State Estimation

The process of dynamic state estimation (filtering) based on point process
observations is in general intractable. Numerical sampling techniques are often
practically useful, but lead to limited conceptual insight about optimal
encoding/decoding strategies, which are of significant relevance to
Computational Neuroscience. We develop an analytically tractable Bayesian
approximation to optimal filtering based on point process observations, which
allows us to introduce distributional assumptions about sensor properties, that
greatly facilitate the analysis of optimal encoding in situations deviating
from common assumptions of uniform coding. Numerical comparison with particle
filtering demonstrate the quality of the approximation. The analytic framework
leads to insights which are difficult to obtain from numerical algorithms, and
is consistent with biological observations about the distribution of sensory
cells' tuning curve centers.