pwc-archive/papers-with-abstracts huggingface dataset:
Semi-supervised Dictionary Learning Based on Hilbert-Schmidt Independence Criterion
Authors: Mehrdad J. Gangeh, Safaa M. A. Bedawi, Ali Ghodsi, Fakhri Karray
Date: 2016-04-25 00:00:00
Tasks: Dictionary Learning

In this paper, a novel semi-supervised dictionary learning and sparse
representation (SS-DLSR) is proposed. The proposed method benefits from the
supervisory information by learning the dictionary in a space where the
dependency between the data and class labels is maximized. This maximization is
performed using Hilbert-Schmidt independence criterion (HSIC). On the other
hand, the global distribution of the underlying manifolds were learned from the
unlabeled data by minimizing the distances between the unlabeled data and the
corresponding nearest labeled data in the space of the dictionary learned. The
proposed SS-DLSR algorithm has closed-form solutions for both the dictionary
and sparse coefficients, and therefore does not have to learn the two
iteratively and alternately as is common in the literature of the DLSR. This
makes the solution for the proposed algorithm very fast. The experiments
confirm the improvement in classification performance on benchmark datasets by
including the information from both labeled and unlabeled data, particularly
when there are many unlabeled data.
End to End Learning for Self-Driving Cars
Authors: Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D. Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, Xin Zhang, Jake Zhao, Karol Zieba
Date: 2016-04-25 00:00:00
Tasks: Lane Detection, Self-Driving Cars

We trained a convolutional neural network (CNN) to map raw pixels from a
single front-facing camera directly to steering commands. This end-to-end
approach proved surprisingly powerful. With minimum training data from humans
the system learns to drive in traffic on local roads with or without lane
markings and on highways. It also operates in areas with unclear visual
guidance such as in parking lots and on unpaved roads.
  The system automatically learns internal representations of the necessary
processing steps such as detecting useful road features with only the human
steering angle as the training signal. We never explicitly trained it to
detect, for example, the outline of roads.
  Compared to explicit decomposition of the problem, such as lane marking
detection, path planning, and control, our end-to-end system optimizes all
processing steps simultaneously. We argue that this will eventually lead to
better performance and smaller systems. Better performance will result because
the internal components self-optimize to maximize overall system performance,
instead of optimizing human-selected intermediate criteria, e.g., lane
detection. Such criteria understandably are selected for ease of human
interpretation which doesn't automatically guarantee maximum system
performance. Smaller networks are possible because the system learns to solve
the problem with the minimal number of processing steps.
  We used an NVIDIA DevBox and Torch 7 for training and an NVIDIA DRIVE(TM) PX
self-driving car computer also running Torch 7 for determining where to drive.
The system operates at 30 frames per second (FPS).
Endgame Analysis of Dou Shou Qi
Authors: Jan N. van Rijn, Jonathan K. Vis
Date: 2016-04-25 00:00:00

Dou Shou Qi is a game in which two players control a number of pieces, each
of them aiming to move one of their pieces onto a given square. We implemented
an engine for analyzing the game. Moreover, we created a series of endgame
tablebases containing all configurations with up to four pieces. These
tablebases are the first steps towards theoretically solving the game. Finally,
we constructed decision trees based on the endgame tablebases. In this note we
report on some interesting patterns.
Actionness Estimation Using Hybrid Fully Convolutional Networks
Authors: Limin Wang, Yu Qiao, Xiaoou Tang, Luc van Gool
Date: 2016-04-25 00:00:00
Proceeding: CVPR 2016 6
Tasks: Action Detection, Action Recognition, Temporal Action Localization

Actionness was introduced to quantify the likelihood of containing a generic
action instance at a specific location. Accurate and efficient estimation of
actionness is important in video analysis and may benefit other relevant tasks
such as action recognition and action detection. This paper presents a new deep
architecture for actionness estimation, called hybrid fully convolutional
network (H-FCN), which is composed of appearance FCN (A-FCN) and motion FCN
(M-FCN). These two FCNs leverage the strong capacity of deep models to estimate
actionness maps from the perspectives of static appearance and dynamic motion,
respectively. In addition, the fully convolutional nature of H-FCN allows it to
efficiently process videos with arbitrary sizes. Experiments are conducted on
the challenging datasets of Stanford40, UCF Sports, and JHMDB to verify the
effectiveness of H-FCN on actionness estimation, which demonstrate that our
method achieves superior performance to previous ones. Moreover, we apply the
estimated actionness maps on action proposal generation and action detection.
Our actionness maps advance the current state-of-the-art performance of these
tasks substantially.
CMA-ES for Hyperparameter Optimization of Deep Neural Networks
Authors: Ilya Loshchilov, Frank Hutter
Date: 2016-04-25 00:00:00
Tasks: Bayesian Optimization, Hyperparameter Optimization

Hyperparameters of deep neural networks are often optimized by grid search,
random search or Bayesian optimization. As an alternative, we propose to use
the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), which is known
for its state-of-the-art performance in derivative-free optimization. CMA-ES
has some useful invariance properties and is friendly to parallel evaluations
of solutions. We provide a toy example comparing CMA-ES and state-of-the-art
Bayesian optimization algorithms for tuning the hyperparameters of a
convolutional neural network for the MNIST dataset on 30 GPUs in parallel.
Online Batch Selection for Faster Training of Neural Networks
Authors: Ilya Loshchilov, Frank Hutter
Date: 2015-11-19 00:00:00

Deep neural networks are commonly trained using stochastic non-convex
optimization procedures, which are driven by gradient information estimated on
fractions (batches) of the dataset. While it is commonly accepted that batch
size is an important parameter for offline tuning, the benefits of online
selection of batches remain poorly understood. We investigate online batch
selection strategies for two state-of-the-art methods of stochastic
gradient-based optimization, AdaDelta and Adam. As the loss function to be
minimized for the whole dataset is an aggregation of loss functions of
individual datapoints, intuitively, datapoints with the greatest loss should be
considered (selected in a batch) more frequently. However, the limitations of
this intuition and the proper control of the selection pressure over time are
open questions. We propose a simple strategy where all datapoints are ranked
w.r.t. their latest known loss value and the probability to be selected decays
exponentially as a function of rank. Our experimental results on the MNIST
dataset suggest that selecting batches speeds up both AdaDelta and Adam by a
factor of about 5.
Robust Sparse Blind Source Separation
Authors: Cecile Chenot, Jerome Bobin, Jeremy Rapin
Date: 2015-07-08 00:00:00
Tasks: blind source separation

Blind Source Separation is a widely used technique to analyze multichannel
data. In many real-world applications, its results can be significantly
hampered by the presence of unknown outliers. In this paper, a novel algorithm
coined rGMCA (robust Generalized Morphological Component Analysis) is
introduced to retrieve sparse sources in the presence of outliers. It
explicitly estimates the sources, the mixing matrix, and the outliers. It also
takes advantage of the estimation of the outliers to further implement a
weighting scheme, which provides a highly robust separation procedure.
Numerical experiments demonstrate the efficiency of rGMCA to estimate the
mixing matrix in comparison with standard BSS techniques.
Towards Reduced Reference Parametric Models for Estimating Audiovisual Quality in Multimedia Services
Authors: Edip Demirbilek, Jean-Charles Grégoire
Date: 2016-04-25 00:00:00

We have developed reduced reference parametric models for estimating
perceived quality in audiovisual multimedia services. We have created 144
unique configurations for audiovisual content including various application and
network parameters such as bitrates and distortions in terms of bandwidth,
packet loss rate and jitter. To generate the data needed for model training and
validation we have tasked 24 subjects, in a controlled environment, to rate the
overall audiovisual quality on the absolute category rating (ACR) 5-level
quality scale. We have developed models using Random Forest and Neural Network
based machine learning methods in order to estimate Mean Opinion Scores (MOS)
values. We have used information retrieved from the packet headers and side
information provided as network parameters for model training. Random Forest
based models have performed better in terms of Root Mean Square Error (RMSE)
and Pearson correlation coefficient. The side information proved to be very
effective in developing the model. We have found that, while the model
performance might be improved by replacing the side information with more
accurate bit stream level measurements, they are performing well in estimating
perceived quality in audiovisual multimedia services.
Unbiased Comparative Evaluation of Ranking Functions
Authors: Tobias Schnabel, Adith Swaminathan, Peter Frazier, Thorsten Joachims
Date: 2016-04-25 00:00:00

Eliciting relevance judgments for ranking evaluation is labor-intensive and
costly, motivating careful selection of which documents to judge. Unlike
traditional approaches that make this selection deterministically,
probabilistic sampling has shown intriguing promise since it enables the design
of estimators that are provably unbiased even when reusing data with missing
judgments. In this paper, we first unify and extend these sampling approaches
by viewing the evaluation problem as a Monte Carlo estimation task that applies
to a large number of common IR metrics. Drawing on the theoretical clarity that
this view offers, we tackle three practical evaluation scenarios: comparing two
systems, comparing $k$ systems against a baseline, and ranking $k$ systems. For
each scenario, we derive an estimator and a variance-optimizing sampling
distribution while retaining the strengths of sampling-based evaluation,
including unbiasedness, reusability despite missing data, and ease of use in
practice. In addition to the theoretical contribution, we empirically evaluate
our methods against previously used sampling heuristics and find that they
generally cut the number of required relevance judgments at least in half.
Observing and Recommending from a Social Web with Biases
Authors: Steffen Staab, Sophie Stalla-Bourdillon, Laura Carmichael
Date: 2016-04-25 00:00:00

The research question this report addresses is: how, and to what extent,
those directly involved with the design, development and employment of a
specific black box algorithm can be certain that it is not unlawfully
discriminating (directly and/or indirectly) against particular persons with
protected characteristics (e.g. gender, race and ethnicity)?
Weighted Spectral Cluster Ensemble
Authors: Muhammad Yousefnezhad, Daoqiang Zhang
Date: 2016-04-25 00:00:00
Tasks: Clustering, Community Detection, Diversity

Clustering explores meaningful patterns in the non-labeled data sets. Cluster
Ensemble Selection (CES) is a new approach, which can combine individual
clustering results for increasing the performance of the final results.
Although CES can achieve better final results in comparison with individual
clustering algorithms and cluster ensemble methods, its performance can be
dramatically affected by its consensus diversity metric and thresholding
procedure. There are two problems in CES: 1) most of the diversity metrics is
based on heuristic Shannon's entropy and 2) estimating threshold values are
really hard in practice. The main goal of this paper is proposing a robust
approach for solving the above mentioned problems. Accordingly, this paper
develops a novel framework for clustering problems, which is called Weighted
Spectral Cluster Ensemble (WSCE), by exploiting some concepts from community
detection arena and graph based clustering. Under this framework, a new version
of spectral clustering, which is called Two Kernels Spectral Clustering, is
used for generating graphs based individual clustering results. Further, by
using modularity, which is a famous metric in the community detection, on the
transformed graph representation of individual clustering results, our approach
provides an effective diversity estimation for individual clustering results.
Moreover, this paper introduces a new approach for combining the evaluated
individual clustering results without the procedure of thresholding.
Experimental study on varied data sets demonstrates that the prosed approach
achieves superior performance to state-of-the-art methods.
Protein Secondary Structure Prediction Using Cascaded Convolutional and Recurrent Neural Networks
Authors: Zhen Li, Yizhou Yu
Date: 2016-04-25 00:00:00
Tasks: Multi-Task Learning, Protein Secondary Structure Prediction

Protein secondary structure prediction is an important problem in
bioinformatics. Inspired by the recent successes of deep neural networks, in
this paper, we propose an end-to-end deep network that predicts protein
secondary structures from integrated local and global contextual features. Our
deep architecture leverages convolutional neural networks with different kernel
sizes to extract multiscale local contextual features. In addition, considering
long-range dependencies existing in amino acid sequences, we set up a
bidirectional neural network consisting of gated recurrent unit to capture
global contextual features. Furthermore, multi-task learning is utilized to
predict secondary structure labels and amino-acid solvent accessibility
simultaneously. Our proposed deep network demonstrates its effectiveness by
achieving state-of-the-art performance, i.e., 69.7% Q8 accuracy on the public
benchmark CB513, 76.9% Q8 accuracy on CASP10 and 73.1% Q8 accuracy on CASP11.
Our model and results are publicly available.
Real-time 2D/3D Registration via CNN Regression
Authors: Shun Miao, Z. Jane Wang, Rui Liao
Date: 2015-07-27 00:00:00
Tasks: Computational Efficiency, regression

In this paper, we present a Convolutional Neural Network (CNN) regression
approach for real-time 2-D/3-D registration. Different from optimization-based
methods, which iteratively optimize the transformation parameters over a
scalar-valued metric function representing the quality of the registration, the
proposed method exploits the information embedded in the appearances of the
Digitally Reconstructed Radiograph and X-ray images, and employs CNN regressors
to directly estimate the transformation parameters. The CNN regressors are
trained for local zones and applied in a hierarchical manner to break down the
complex regression task into simpler sub-tasks that can be learned separately.
Our experiment results demonstrate the advantage of the proposed method in
computational efficiency with negligible degradation of registration accuracy
compared to intensity-based methods.
Modeling the Evolution of Gene-Culture Divergence
Authors: Chris Marriott, Jobran Chebib
Date: 2016-04-25 00:00:00
Tasks: Cultural Vocal Bursts Intensity Prediction

We present a model for evolving agents using both genetic and cultural
inheritance mechanisms. Within each agent our model maintains two distinct
information stores we call the genome and the memome. Processes of adaptation
are modeled as evolutionary processes at each level of adaptation
(phylogenetic, ontogenetic, sociogenetic). We review relevant competing models
and we show how our model improves on previous attempts to model genetic and
cultural evolutionary processes. In particular we argue our model can achieve
divergent gene-culture co-evolution.
Learning to Select Pre-Trained Deep Representations with Bayesian Evidence Framework
Authors: Yong-Deok Kim, Taewoong Jang, Bohyung Han, Seungjin Choi
Date: 2015-06-08 00:00:00
Proceeding: CVPR 2016 6
Tasks: Transfer Learning

We propose a Bayesian evidence framework to facilitate transfer learning from
pre-trained deep convolutional neural networks (CNNs). Our framework is
formulated on top of a least squares SVM (LS-SVM) classifier, which is simple
and fast in both training and testing, and achieves competitive performance in
practice. The regularization parameters in LS-SVM is estimated automatically
without grid search and cross-validation by maximizing evidence, which is a
useful measure to select the best performing CNN out of multiple candidates for
transfer learning; the evidence is optimized efficiently by employing Aitken's
delta-squared process, which accelerates convergence of fixed point update. The
proposed Bayesian evidence framework also provides a good solution to identify
the best ensemble of heterogeneous CNNs through a greedy algorithm. Our
Bayesian evidence framework for transfer learning is tested on 12 visual
recognition datasets and illustrates the state-of-the-art performance
consistently in terms of prediction accuracy and modeling efficiency.
Makeup like a superstar: Deep Localized Makeup Transfer Network
Authors: Si Liu, Xinyu Ou, Ruihe Qian, Wei Wang, Xiaochun Cao
Date: 2016-04-25 00:00:00

In this paper, we propose a novel Deep Localized Makeup Transfer Network to
automatically recommend the most suitable makeup for a female and synthesis the
makeup on her face. Given a before-makeup face, her most suitable makeup is
determined automatically. Then, both the beforemakeup and the reference faces
are fed into the proposed Deep Transfer Network to generate the after-makeup
face. Our end-to-end makeup transfer network have several nice properties
including: (1) with complete functions: including foundation, lip gloss, and
eye shadow transfer; (2) cosmetic specific: different cosmetics are transferred
in different manners; (3) localized: different cosmetics are applied on
different facial regions; (4) producing naturally looking results without
obvious artifacts; (5) controllable makeup lightness: various results from
light makeup to heavy makeup can be generated. Qualitative and quantitative
experiments show that our network performs much better than the methods of [Guo
and Sim, 2009] and two variants of NerualStyle [Gatys et al., 2015a].
Deep Learning for Reward Design to Improve Monte Carlo Tree Search in ATARI Games
Authors: Xiaoxiao Guo, Satinder Singh, Richard Lewis, Honglak Lee
Date: 2016-04-24 00:00:00
Tasks: Atari Games, Decision Making, Sequential Decision Making

Monte Carlo Tree Search (MCTS) methods have proven powerful in planning for
sequential decision-making problems such as Go and video games, but their
performance can be poor when the planning depth and sampling trajectories are
limited or when the rewards are sparse. We present an adaptation of PGRD
(policy-gradient for reward-design) for learning a reward-bonus function to
improve UCT (a MCTS algorithm). Unlike previous applications of PGRD in which
the space of reward-bonus functions was limited to linear functions of
hand-coded state-action-features, we use PGRD with a multi-layer convolutional
neural network to automatically learn features from raw perception as well as
to adapt the non-linear reward-bonus function parameters. We also adopt a
variance-reducing gradient method to improve PGRD's performance. The new method
improves UCT's performance on multiple ATARI games compared to UCT without the
reward bonus. Combining PGRD and Deep Learning in this way should make adapting
rewards for MCTS algorithms far more widely and practically applicable than
before.
A Minimalistic Approach to Sum-Product Network Learning for Real Applications
Authors: Viktoriya Krakovna, Moshe Looks
Date: 2016-02-12 00:00:00
Tasks: Clustering

Sum-Product Networks (SPNs) are a class of expressive yet tractable
hierarchical graphical models. LearnSPN is a structure learning algorithm for
SPNs that uses hierarchical co-clustering to simultaneously identifying similar
entities and similar features. The original LearnSPN algorithm assumes that all
the variables are discrete and there is no missing data. We introduce a
practical, simplified version of LearnSPN, MiniSPN, that runs faster and can
handle missing data and heterogeneous features common in real applications. We
demonstrate the performance of MiniSPN on standard benchmark datasets and on
two datasets from Google's Knowledge Graph exhibiting high missingness rates
and a mix of discrete and continuous features.
Semi-supervised Vocabulary-informed Learning
Authors: Yanwei Fu, Leonid Sigal
Date: 2016-04-24 00:00:00
Proceeding: CVPR 2016 6
Tasks: Object Categorization, Open Set Learning, Zero-Shot Learning

Despite significant progress in object categorization, in recent years, a
number of important challenges remain, mainly, ability to learn from limited
labeled data and ability to recognize object classes within large, potentially
open, set of labels. Zero-shot learning is one way of addressing these
challenges, but it has only been shown to work with limited sized class
vocabularies and typically requires separation between supervised and
unsupervised classes, allowing former to inform the latter but not vice versa.
We propose the notion of semi-supervised vocabulary-informed learning to
alleviate the above mentioned challenges and address problems of supervised,
zero-shot and open set recognition using a unified framework. Specifically, we
propose a maximum margin framework for semantic manifold-based recognition that
incorporates distance constraints from (both supervised and unsupervised)
vocabulary atoms, ensuring that labeled samples are projected closest to their
correct prototypes, in the embedding space, than to others. We show that
resulting model shows improvements in supervised, zero-shot, and large open set
recognition, with up to 310K class vocabulary on AwA and ImageNet datasets.
Semi-supervised and Unsupervised Methods for Categorizing Posts in Web Discussion Forums
Authors: Krish Perumal
Date: 2016-04-01 00:00:00
Tasks: Information Retrieval, Retrieval

Web discussion forums are used by millions of people worldwide to share
information belonging to a variety of domains such as automotive vehicles,
pets, sports, etc. They typically contain posts that fall into different
categories such as problem, solution, feedback, spam, etc. Automatic
identification of these categories can aid information retrieval that is
tailored for specific user requirements. Previously, a number of supervised
methods have attempted to solve this problem; however, these depend on the
availability of abundant training data. A few existing unsupervised and
semi-supervised approaches are either focused on identifying a single category
or do not report category-specific performance. In contrast, this work proposes
unsupervised and semi-supervised methods that require no or minimal training
data to achieve this objective without compromising on performance. A
fine-grained analysis is also carried out to discuss their limitations. The
proposed methods are based on sequence models (specifically, Hidden Markov
Models) that can model language for each category using word and part-of-speech
probability distributions, and manually specified features. Empirical
evaluations across domains demonstrate that the proposed methods are better
suited for this task than existing ones.