pwc-archive/papers-with-abstracts huggingface dataset:
Weakly-supervised DCNN for RGB-D Object Recognition in Real-World Applications Which Lack Large-scale Annotated Training Data
Authors: Li Sun, Cheng Zhao, Rustam Stolkin
Date: 2017-03-19 00:00:00
Tasks: Object Recognition, Weakly-supervised Learning

This paper addresses the problem of RGBD object recognition in real-world
applications, where large amounts of annotated training data are typically
unavailable. To overcome this problem, we propose a novel, weakly-supervised
learning architecture (DCNN-GPC) which combines parametric models (a pair of
Deep Convolutional Neural Networks (DCNN) for RGB and D modalities) with
non-parametric models (Gaussian Process Classification). Our system is
initially trained using a small amount of labeled data, and then automatically
prop- agates labels to large-scale unlabeled data. We first run 3D- based
objectness detection on RGBD videos to acquire many unlabeled object proposals,
and then employ DCNN-GPC to label them. As a result, our multi-modal DCNN can
be trained end-to-end using only a small amount of human annotation. Finally,
our 3D-based objectness detection and multi-modal DCNN are integrated into a
real-time detection and recognition pipeline. In our approach, bounding-box
annotations are not required and boundary-aware detection is achieved. We also
propose a novel way to pretrain a DCNN for the depth modality, by training on
virtual depth images projected from CAD models. We pretrain our multi-modal
DCNN on public 3D datasets, achieving performance comparable to
state-of-the-art methods on Washington RGBS Dataset. We then finetune the
network by further training on a small amount of annotated data from our novel
dataset of industrial objects (nuclear waste simulants). Our weakly supervised
approach has demonstrated to be highly effective in solving a novel RGBD object
recognition application which lacks of human annotations.
Revisiting Distributed Synchronous SGD
Authors: Xinghao Pan, Jianmin Chen, Rajat Monga, Samy Bengio, Rafal Jozefowicz
Date: 2017-02-19 00:00:00
Tasks: Stochastic Optimization

Distributed training of deep learning models on large-scale training data is
typically conducted with asynchronous stochastic optimization to maximize the
rate of updates, at the cost of additional noise introduced from asynchrony. In
contrast, the synchronous approach is often thought to be impractical due to
idle time wasted on waiting for straggling workers. We revisit these
conventional beliefs in this paper, and examine the weaknesses of both
approaches. We demonstrate that a third approach, synchronous optimization with
backup workers, can avoid asynchronous noise while mitigating for the worst
stragglers. Our approach is empirically validated and shown to converge faster
and to better test accuracies.
Goal Conflict in Designing an Autonomous Artificial System
Authors: Mark Muraven
Date: 2017-03-18 00:00:00
Tasks: Ethics

Research on human self-regulation has shown that people hold many goals
simultaneously and have complex self-regulation mechanisms to deal with this
goal conflict. Artificial autonomous systems may also need to find ways to cope
with conflicting goals. Indeed, the intricate interplay among different goals
may be critical to the design as well as long-term safety and stability of
artificial autonomous systems. I discuss some of the critical features of the
human self-regulation system and how it might be applied to an artificial
system. Furthermore, the implications of goal conflict for the reliability and
stability of artificial autonomous systems and ensuring their alignment with
human goals and ethics is examined.
Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks
Authors: Zhilin Yang, Ruslan Salakhutdinov, William W. Cohen
Date: 2017-03-18 00:00:00
Tasks: Feature Engineering, Named Entity Recognition (NER), Part-Of-Speech Tagging, POS, POS Tagging, Transfer Learning

Recent papers have shown that neural networks obtain state-of-the-art
performance on several different sequence tagging tasks. One appealing property
of such systems is their generality, as excellent performance can be achieved
with a unified architecture and without task-specific feature engineering.
However, it is unclear if such systems can be used for tasks without large
amounts of training data. In this paper we explore the problem of transfer
learning for neural sequence taggers, where a source task with plentiful
annotations (e.g., POS tagging on Penn Treebank) is used to improve performance
on a target task with fewer available annotations (e.g., POS tagging for
microblogs). We examine the effects of transfer learning for deep hierarchical
recurrent networks across domains, applications, and languages, and show that
significant improvement can often be obtained. These improvements lead to
improvements over the current state-of-the-art on several well-studied tasks.
Spectrum Estimation from a Few Entries
Authors: Ashish Khetan, Sewoong Oh
Date: 2017-03-18 00:00:00
Tasks: Collaborative Filtering, Matrix Completion

Singular values of a data in a matrix form provide insights on the structure
of the data, the effective dimensionality, and the choice of hyper-parameters
on higher-level data analysis tools. However, in many practical applications
such as collaborative filtering and network analysis, we only get a partial
observation. Under such scenarios, we consider the fundamental problem of
recovering spectral properties of the underlying matrix from a sampling of its
entries. We are particularly interested in directly recovering the spectrum,
which is the set of singular values, and also in sample-efficient approaches
for recovering a spectral sum function, which is an aggregate sum of the same
function applied to each of the singular values. We propose first estimating
the Schatten $k$-norms of a matrix, and then applying Chebyshev approximation
to the spectral sum function or applying moment matching in Wasserstein
distance to recover the singular values. The main technical challenge is in
accurately estimating the Schatten norms from a sampling of a matrix. We
introduce a novel unbiased estimator based on counting small structures in a
graph and provide guarantees that match its empirical performance. Our
theoretical analysis shows that Schatten norms can be recovered accurately from
strictly smaller number of samples compared to what is needed to recover the
underlying low-rank matrix. Numerical experiments suggest that we significantly
improve upon a competing approach of using matrix completion methods.
Synthetic to Real Adaptation with Generative Correlation Alignment Networks
Authors: Xingchao Peng, Kate Saenko
Date: 2017-01-19 00:00:00
Tasks: Domain Adaptation, Object Recognition

Synthetic images rendered from 3D CAD models are useful for augmenting
training data for object recognition algorithms. However, the generated images
are non-photorealistic and do not match real image statistics. This leads to a
large domain discrepancy, causing models trained on synthetic data to perform
poorly on real domains. Recent work has shown the great potential of deep
convolutional neural networks to generate realistic images, but has not
utilized generative models to address synthetic-to-real domain adaptation. In
this work, we propose a Deep Generative Correlation Alignment Network (DGCAN)
to synthesize images using a novel domain adaption algorithm. DGCAN leverages a
shape preserving loss and a low level statistic matching loss to minimize the
domain discrepancy between synthetic and real images in deep feature space.
Experimentally, we show training off-the-shelf classifiers on the newly
generated data can significantly boost performance when testing on the real
image domains (PASCAL VOC 2007 benchmark and Office dataset), improving upon
several existing methods.
A wake-sleep algorithm for recurrent, spiking neural networks
Authors: Johannes Thiele, Peter Diehl, Matthew Cook
Date: 2017-03-18 00:00:00
Tasks: Clustering

We investigate a recently proposed model for cortical computation which
performs relational inference. It consists of several interconnected,
structurally equivalent populations of leaky integrate-and-fire (LIF) neurons,
which are trained in a self-organized fashion with spike-timing dependent
plasticity (STDP). Despite its robust learning dynamics, the model is
susceptible to a problem typical for recurrent networks which use a correlation
based (Hebbian) learning rule: if trained with high learning rates, the
recurrent connections can cause strong feedback loops in the network dynamics,
which lead to the emergence of attractor states. This causes a strong reduction
in the number of representable patterns and a decay in the inference ability of
the network. As a solution, we introduce a conceptually very simple
"wake-sleep" algorithm: during the wake phase, training is executed normally,
while during the sleep phase, the network "dreams" samples from its generative
model, which are induced by random input. This process allows us to activate
the attractor states in the network, which can then be unlearned effectively by
an anti-Hebbian mechanism. The algorithm allows us to increase learning rates
up to a factor of ten while avoiding clustering, which allows the network to
learn several times faster. Also for low learning rates, where clustering is
not an issue, it improves convergence speed and reduces the final inference
error.
Elections with Few Voters: Candidate Control Can Be Easy
Authors: Jiehua Chen, Piotr Faliszewski, Rolf Niedermeier, Nimrod Talmon
Date: 2014-11-28 00:00:00

We study the computational complexity of candidate control in elections with
few voters, that is, we consider the parameterized complexity of candidate
control in elections with respect to the number of voters as a parameter. We
consider both the standard scenario of adding and deleting candidates, where
one asks whether a given candidate can become a winner (or, in the destructive
case, can be precluded from winning) by adding or deleting few candidates, as
well as a combinatorial scenario where adding/deleting a candidate
automatically means adding or deleting a whole group of candidates. Considering
several fundamental voting rules, our results show that the parameterized
complexity of candidate control, with the number of voters as the parameter, is
much more varied than in the setting with many voters.
The N-Tuple Bandit Evolutionary Algorithm for Automatic Game Improvement
Authors: Kamolwan Kunanusont, Raluca D. Gaina, Jialin Liu, Diego Perez-Liebana, Simon M. Lucas
Date: 2017-03-18 00:00:00
Tasks: Game Design

This paper describes a new evolutionary algorithm that is especially well
suited to AI-Assisted Game Design. The approach adopted in this paper is to use
observations of AI agents playing the game to estimate the game's quality. Some
of best agents for this purpose are General Video Game AI agents, since they
can be deployed directly on a new game without game-specific tuning; these
agents tend to be based on stochastic algorithms which give robust but noisy
results and tend to be expensive to run. This motivates the main contribution
of the paper: the development of the novel N-Tuple Bandit Evolutionary
Algorithm, where a model is used to estimate the fitness of unsampled points
and a bandit approach is used to balance exploration and exploitation of the
search space. Initial results on optimising a Space Battle game variant suggest
that the algorithm offers far more robust results than the Random Mutation Hill
Climber and a Biased Mutation variant, which are themselves known to offer
competitive performance across a range of problems. Subjective observations are
also given by human players on the nature of the evolved games, which indicate
a preference towards games generated by the N-Tuple algorithm.
Evolving Game Skill-Depth using General Video Game AI Agents
Authors: Jialin Liu, Julian Togelius, Diego Perez-Liebana, Simon M. Lucas
Date: 2017-03-18 00:00:00

Most games have, or can be generalised to have, a number of parameters that
may be varied in order to provide instances of games that lead to very
different player experiences. The space of possible parameter settings can be
seen as a search space, and we can therefore use a Random Mutation Hill
Climbing algorithm or other search methods to find the parameter settings that
induce the best games. One of the hardest parts of this approach is defining a
suitable fitness function. In this paper we explore the possibility of using
one of a growing set of General Video Game AI agents to perform automatic
play-testing. This enables a very general approach to game evaluation based on
estimating the skill-depth of a game. Agent-based play-testing is
computationally expensive, so we compare two simple but efficient optimisation
algorithms: the Random Mutation Hill-Climber and the Multi-Armed Bandit Random
Mutation Hill-Climber. For the test game we use a space-battle game in order to
provide a suitable balance between simulation speed and potential skill-depth.
Results show that both algorithms are able to rapidly evolve game versions with
significant skill-depth, but that choosing a suitable resampling number is
essential in order to combat the effects of noise.
An Automated Auto-encoder Correlation-based Health-Monitoring and Prognostic Method for Machine Bearings
Authors: Ramin M. Hasani, Guodong Wang, Radu Grosu
Date: 2017-03-18 00:00:00
Tasks: Attribute

This paper studies an intelligent ultimate technique for health-monitoring
and prognostic of common rotary machine components, particularly bearings.
During a run-to-failure experiment, rich unsupervised features from vibration
sensory data are extracted by a trained sparse auto-encoder. Then, the
correlation of the extracted attributes of the initial samples (presumably
healthy at the beginning of the test) with the succeeding samples is calculated
and passed through a moving-average filter. The normalized output is named
auto-encoder correlation-based (AEC) rate which stands for an informative
attribute of the system depicting its health status and precisely identifying
the degradation starting point. We show that AEC technique well-generalizes in
several run-to-failure tests. AEC collects rich unsupervised features form the
vibration data fully autonomous. We demonstrate the superiority of the AEC over
many other state-of-the-art approaches for the health monitoring and prognostic
of machine bearings.
An Adaptive Framework to Tune the Coordinate Systems in Evolutionary Algorithms
Authors: Zhi-Zhong Liu, Yong Wang, Shengxiang Yang, Ke Tang
Date: 2017-03-18 00:00:00
Tasks: Evolutionary Algorithms

In the evolutionary computation research community, the performance of most
evolutionary algorithms (EAs) depends strongly on their implemented coordinate
system. However, the commonly used coordinate system is fixed and not well
suited for different function landscapes, EAs thus might not search
efficiently. To overcome this shortcoming, in this paper we propose a
framework, named ACoS, to adaptively tune the coordinate systems in EAs. In
ACoS, an Eigen coordinate system is established by making use of the cumulative
population distribution information, which can be obtained based on a
covariance matrix adaptation strategy and an additional archiving mechanism.
Since the population distribution information can reflect the features of the
function landscape to some extent, EAs in the Eigen coordinate system have the
capability to identify the modality of the function landscape. In addition, the
Eigen coordinate system is coupled with the original coordinate system, and
they are selected according to a probability vector. The probability vector
aims to determine the selection ratio of each coordinate system for each
individual, and is adaptively updated based on the collected information from
the offspring. ACoS has been applied to two of the most popular EA paradigms,
i.e., particle swarm optimization (PSO) and differential evolution (DE), for
solving 30 test functions with 30 and 50 dimensions at the 2014 IEEE Congress
on Evolutionary Computation. The experimental studies demonstrate its
effectiveness.
Improving training of deep neural networks via Singular Value Bounding
Authors: Kui Jia
Date: 2016-11-18 00:00:00
Proceeding: CVPR 2017 7
Tasks: General Classification, image-classification, Image Classification, object-detection, Object Detection

Deep learning methods achieve great success recently on many computer vision
problems, with image classification and object detection as the prominent
examples. In spite of these practical successes, optimization of deep networks
remains an active topic in deep learning research. In this work, we focus on
investigation of the network solution properties that can potentially lead to
good performance. Our research is inspired by theoretical and empirical results
that use orthogonal matrices to initialize networks, but we are interested in
investigating how orthogonal weight matrices perform when network training
converges. To this end, we propose to constrain the solutions of weight
matrices in the orthogonal feasible set during the whole process of network
training, and achieve this by a simple yet effective method called Singular
Value Bounding (SVB). In SVB, all singular values of each weight matrix are
simply bounded in a narrow band around the value of 1. Based on the same
motivation, we also propose Bounded Batch Normalization (BBN), which improves
Batch Normalization by removing its potential risk of ill-conditioned layer
transform. We present both theoretical and empirical results to justify our
proposed methods. Experiments on benchmark image classification datasets show
the efficacy of our proposed SVB and BBN. In particular, we achieve the
state-of-the-art results of 3.06% error rate on CIFAR10 and 16.90% on CIFAR100,
using off-the-shelf network architectures (Wide ResNets). Our preliminary
results on ImageNet also show the promise in large-scale learning.
Single image super-resolution using self-optimizing mask via fractional-order gradient interpolation and reconstruction
Authors: Qi Yang, Yanzhu Zhang, Tiebiao Zhao, YangQuan Chen
Date: 2017-03-18 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

Image super-resolution using self-optimizing mask via fractional-order
gradient interpolation and reconstruction aims to recover detailed information
from low-resolution images and reconstruct them into high-resolution images.
Due to the limited amount of data and information retrieved from low-resolution
images, it is difficult to restore clear, artifact-free images, while still
preserving enough structure of the image such as the texture. This paper
presents a new single image super-resolution method which is based on adaptive
fractional-order gradient interpolation and reconstruction. The interpolated
image gradient via optimal fractional-order gradient is first constructed
according to the image similarity and afterwards the minimum energy function is
employed to reconstruct the final high-resolution image. Fractional-order
gradient based interpolation methods provide an additional degree of freedom
which helps optimize the implementation quality due to the fact that an extra
free parameter $\alpha$-order is being used. The proposed method is able to
produce a rich texture detail while still being able to maintain structural
similarity even under large zoom conditions. Experimental results show that the
proposed method performs better than current single image super-resolution
techniques.
A Fast HOG Descriptor Using Lookup Table and Integral Image
Authors: Chunde Huang, Jiaxiang Huang
Date: 2017-03-18 00:00:00
Tasks: Object, object-detection, Object Detection, Real-Time Object Detection

The histogram of oriented gradients (HOG) is a widely used feature descriptor
in computer vision for the purpose of object detection. In the paper, a
modified HOG descriptor is described, it uses a lookup table and the method of
integral image to speed up the detection performance by a factor of 5~10. By
exploiting the special hardware features of a given platform(e.g. a digital
signal processor), further improvement can be made to the HOG descriptor in
order to have real-time object detection and tracking.
Multi-fidelity Bayesian Optimisation with Continuous Approximations
Authors: Kirthevasan Kandasamy, Gautam Dasarathy, Jeff Schneider, Barnabas Poczos
Date: 2017-03-18 00:00:00
Proceeding: ICML 2017 8
Tasks: Bayesian Optimisation

Bandit methods for black-box optimisation, such as Bayesian optimisation, are
used in a variety of applications including hyper-parameter tuning and
experiment design. Recently, \emph{multi-fidelity} methods have garnered
considerable attention since function evaluations have become increasingly
expensive in such applications. Multi-fidelity methods use cheap approximations
to the function of interest to speed up the overall optimisation process.
However, most multi-fidelity methods assume only a finite number of
approximations. In many practical applications however, a continuous spectrum
of approximations might be available. For instance, when tuning an expensive
neural network, one might choose to approximate the cross validation
performance using less data $N$ and/or few training iterations $T$. Here, the
approximations are best viewed as arising out of a continuous two dimensional
space $(N,T)$. In this work, we develop a Bayesian optimisation method, BOCA,
for this setting. We characterise its theoretical properties and show that it
achieves better regret than than strategies which ignore the approximations.
BOCA outperforms several other baselines in synthetic and real experiments.
Evaluation of Particle Swarm Optimization Algorithms for Weighted Max-Sat Problem: Technical Report
Authors: Osama Khalil
Date: 2012-12-04 00:00:00

An experimental evaluation is conducted to asses the performance of 4
different Particle Swarm Optimization neighborhood structures in solving
Max-Sat problem. The experiment has shown that none of the algorithms achieves
statistically significant performance over the others under confidence level of
0.05.
On Consistency of Graph-based Semi-supervised Learning
Authors: Chengan Du, Yunpeng Zhao, Feng Wang
Date: 2017-03-17 00:00:00

Graph-based semi-supervised learning is one of the most popular methods in
machine learning. Some of its theoretical properties such as bounds for the
generalization error and the convergence of the graph Laplacian regularizer
have been studied in computer science and statistics literatures. However, a
fundamental statistical property, the consistency of the estimator from this
method has not been proved. In this article, we study the consistency problem
under a non-parametric framework. We prove the consistency of graph-based
learning in the case that the estimated scores are enforced to be equal to the
observed responses for the labeled data. The sample sizes of both labeled and
unlabeled data are allowed to grow in this result. When the estimated scores
are not required to be equal to the observed responses, a tuning parameter is
used to balance the loss function and the graph Laplacian regularizer. We give
a counterexample demonstrating that the estimator for this case can be
inconsistent. The theoretical findings are supported by numerical studies.