pwc-archive/papers-with-abstracts huggingface dataset:
IGD Indicator-based Evolutionary Algorithm for Many-objective Optimization Problems
Authors: Yanan Sun, Gary G. Yen, Zhang Yi
Date: 2018-02-24 00:00:00
Tasks: Diversity, Evolutionary Algorithms

Inverted Generational Distance (IGD) has been widely considered as a reliable
performance indicator to concurrently quantify the convergence and diversity of
multi- and many-objective evolutionary algorithms. In this paper, an IGD
indicator-based evolutionary algorithm for solving many-objective optimization
problems (MaOPs) has been proposed. Specifically, the IGD indicator is employed
in each generation to select the solutions with favorable convergence and
diversity. In addition, a computationally efficient dominance comparison method
is designed to assign the rank values of solutions along with three newly
proposed proximity distance assignments. Based on these two designs, the
solutions are selected from a global view by linear assignment mechanism to
concern the convergence and diversity simultaneously. In order to facilitate
the accuracy of the sampled reference points for the calculation of IGD
indicator, we also propose an efficient decomposition-based nadir point
estimation method for constructing the Utopian Pareto front which is regarded
as the best approximate Pareto front for real-world MaOPs at the early stage of
the evolution. To evaluate the performance, a series of experiments is
performed on the proposed algorithm against a group of selected
state-of-the-art many-objective optimization algorithms over optimization
problems with $8$-, $15$-, and $20$-objective. Experimental results measured by
the chosen performance metrics indicate that the proposed algorithm is very
competitive in addressing MaOPs.
Auto-Conditioned Recurrent Networks for Extended Complex Human Motion Synthesis
Authors: Zimo Li, Yi Zhou, Shuangjiu Xiao, Chong He, Zeng Huang, Hao Li
Date: 2017-07-17 00:00:00
Proceeding: ICLR 2018 1
Tasks: Motion Synthesis

We present a real-time method for synthesizing highly complex human motions
using a novel training regime we call the auto-conditioned Recurrent Neural
Network (acRNN). Recently, researchers have attempted to synthesize new motion
by using autoregressive techniques, but existing methods tend to freeze or
diverge after a couple of seconds due to an accumulation of errors that are fed
back into the network. Furthermore, such methods have only been shown to be
reliable for relatively simple human motions, such as walking or running. In
contrast, our approach can synthesize arbitrary motions with highly complex
styles, including dances or martial arts in addition to locomotion. The acRNN
is able to accomplish this by explicitly accommodating for autoregressive noise
accumulation during training. Our work is the first to our knowledge that
demonstrates the ability to generate over 18,000 continuous frames (300
seconds) of new complex human motion w.r.t. different styles.
Spatially Constrained Location Prior for Scene Parsing
Authors: Ligang Zhang, Brijesh Verma, David Stockwell, Sujan Chowdhury
Date: 2018-02-24 00:00:00
Tasks: Scene Parsing

Semantic context is an important and useful cue for scene parsing in
complicated natural images with a substantial amount of variations in objects
and the environment. This paper proposes Spatially Constrained Location Prior
(SCLP) for effective modelling of global and local semantic context in the
scene in terms of inter-class spatial relationships. Unlike existing studies
focusing on either relative or absolute location prior of objects, the SCLP
effectively incorporates both relative and absolute location priors by
calculating object co-occurrence frequencies in spatially constrained image
blocks. The SCLP is general and can be used in conjunction with various visual
feature-based prediction models, such as Artificial Neural Networks and Support
Vector Machine (SVM), to enforce spatial contextual constraints on class
labels. Using SVM classifiers and a linear regression model, we demonstrate
that the incorporation of SCLP achieves superior performance compared to the
state-of-the-art methods on the Stanford background and SIFT Flow datasets.
Improved Regularity Model-based EDA for Many-objective Optimization
Authors: Yanan Sun, Gary G. Yen, Zhang Yi
Date: 2018-02-24 00:00:00
Tasks: Dimensionality Reduction, Diversity, Evolutionary Algorithms

The performance of multi-objective evolutionary algorithms deteriorates
appreciably in solving many-objective optimization problems which encompass
more than three objectives. One of the known rationales is the loss of
selection pressure which leads to the selected parents not generating promising
offspring towards Pareto-optimal front with diversity. Estimation of
distribution algorithms sample new solutions with a probabilistic model built
from the statistics extracting over the existing solutions so as to mitigate
the adverse impact of genetic operators. In this paper, an improved
regularity-based estimation of distribution algorithm is proposed to
effectively tackle unconstrained many-objective optimization problems. In the
proposed algorithm, \emph{diversity repairing mechanism} is utilized to mend
the areas where need non-dominated solutions with a closer proximity to the
Pareto-optimal front. Then \emph{favorable solutions} are generated by the
model built from the regularity of the solutions surrounding a group of
representatives. These two steps collectively enhance the selection pressure
which gives rise to the superior convergence of the proposed algorithm. In
addition, dimension reduction technique is employed in the decision space to
speed up the estimation search of the proposed algorithm. Finally, by assigning
the Pareto-optimal solutions to the uniformly distributed reference vectors, a
set of solutions with excellent diversity and convergence is obtained. To
measure the performance, NSGA-III, GrEA, MOEA/D, HypE, MBN-EDA, and RM-MEDA are
selected to perform comparison experiments over DTLZ and DTLZ$^-$ test suites
with $3$-, $5$-, $8$-, $10$-, and $15$-objective. Experimental results
quantified by the selected performance metrics reveal that the proposed
algorithm shows considerable competitiveness in addressing unconstrained
many-objective optimization problems.
Evolving Unsupervised Deep Neural Networks for Learning Meaningful Representations
Authors: Yanan Sun, Gary G. Yen, Zhang Yi
Date: 2017-12-13 00:00:00
Tasks: Evolutionary Algorithms, General Classification, parameter estimation

Deep Learning (DL) aims at learning the \emph{meaningful representations}. A
meaningful representation refers to the one that gives rise to significant
performance improvement of associated Machine Learning (ML) tasks by replacing
the raw data as the input. However, optimal architecture design and model
parameter estimation in DL algorithms are widely considered to be intractable.
Evolutionary algorithms are much preferable for complex and non-convex problems
due to its inherent characteristics of gradient-free and insensitivity to local
optimum. In this paper, we propose a computationally economical algorithm for
evolving \emph{unsupervised deep neural networks} to efficiently learn
\emph{meaningful representations}, which is very suitable in the current Big
Data era where sufficient labeled data for training is often expensive to
acquire. In the proposed algorithm, finding an appropriate architecture and the
initialized parameter values for a ML task at hand is modeled by one
computational efficient gene encoding approach, which is employed to
effectively model the task with a large number of parameters. In addition, a
local search strategy is incorporated to facilitate the exploitation search for
further improving the performance. Furthermore, a small proportion labeled data
is utilized during evolution search to guarantee the learnt representations to
be meaningful. The performance of the proposed algorithm has been thoroughly
investigated over classification tasks. Specifically, error classification rate
on MNIST with $1.15\%$ is reached by the proposed algorithm consistently, which
is a very promising result against state-of-the-art unsupervised DL algorithms.
Syntax-Directed Variational Autoencoder for Structured Data
Authors: Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, Le Song
Date: 2018-02-24 00:00:00
Proceeding: ICLR 2018 1
Tasks: Decoder, Translation, valid

Deep generative models have been enjoying success in modeling continuous
data. However it remains challenging to capture the representations for
discrete structures with formal grammars and semantics, e.g., computer programs
and molecular structures. How to generate both syntactically and semantically
correct data still remains largely an open problem. Inspired by the theory of
compiler where the syntax and semantics check is done via syntax-directed
translation (SDT), we propose a novel syntax-directed variational autoencoder
(SD-VAE) by introducing stochastic lazy attributes. This approach converts the
offline SDT check into on-the-fly generated guidance for constraining the
decoder. Comparing to the state-of-the-art methods, our approach enforces
constraints on the output space so that the output will be not only
syntactically valid, but also semantically reasonable. We evaluate the proposed
model with applications in programming language and molecules, including
reconstruction and program/molecule optimization. The results demonstrate the
effectiveness in incorporating syntactic and semantic constraints in discrete
generative models, which is significantly better than current state-of-the-art
approaches.
Facial Expression Analysis under Partial Occlusion: A Survey
Authors: Ligang Zhang, Brijesh Verma, Dian Tjondronegoro, Vinod Chandran
Date: 2018-02-24 00:00:00
Tasks: Survey

Automatic machine-based Facial Expression Analysis (FEA) has made substantial
progress in the past few decades driven by its importance for applications in
psychology, security, health, entertainment and human computer interaction. The
vast majority of completed FEA studies are based on non-occluded faces
collected in a controlled laboratory environment. Automatic expression
recognition tolerant to partial occlusion remains less understood, particularly
in real-world scenarios. In recent years, efforts investigating techniques to
handle partial occlusion for FEA have seen an increase. The context is right
for a comprehensive perspective of these developments and the state of the art
from this perspective. This survey provides such a comprehensive review of
recent advances in dataset creation, algorithm development, and investigations
of the effects of occlusion critical for robust performance in FEA systems. It
outlines existing challenges in overcoming partial occlusion and discusses
possible opportunities in advancing the technology. To the best of our
knowledge, it is the first FEA survey dedicated to occlusion and aimed at
promoting better informed and benchmarked future work.
Superpixel based Class-Semantic Texton Occurrences for Natural Roadside Vegetation Segmentation
Authors: Ligang Zhang, Brijesh Verma
Date: 2018-02-24 00:00:00
Tasks: Segmentation, Superpixels

Vegetation segmentation from roadside data is a field that has received
relatively little attention in present studies, but can be of great potentials
in a wide range of real-world applications, such as road safety assessment and
vegetation condition monitoring. In this paper, we present a novel approach
that generates class-semantic color-texture textons and aggregates superpixel
based texton occurrences for vegetation segmentation in natural roadside
images. Pixel-level class-semantic textons are first learnt by generating two
individual sets of bag-of-word visual dictionaries from color and filter-bank
texture features separately for each object class using manually cropped
training data. For a testing image, it is first oversegmented into a set of
homogeneous superpixels. The color and texture features of all pixels in each
superpixel are extracted and further mapped to one of the learnt textons using
the nearest distance metric, resulting in a color and a texture texton
occurrence matrix. The color and texture texton occurrences are aggregated
using a linear mixing method over each superpixel and the segmentation is
finally achieved using a simple yet effective majority voting strategy.
Evaluations on two public image datasets from videos collected by the
Department of Transport and Main Roads (DTMR), Queensland, Australia, and a
public roadside grass dataset show high accuracy of the proposed approach. We
also demonstrate the effectiveness of the approach for vegetation segmentation
in real-world scenarios.
Extremely Fast Decision Tree
Authors: Chaitanya Manapragada, Geoff Webb, Mahsa Salehi
Date: 2018-02-24 00:00:00
Tasks: General Classification

We introduce a novel incremental decision tree learning algorithm, Hoeffding
Anytime Tree, that is statistically more efficient than the current
state-of-the-art, Hoeffding Tree. We demonstrate that an implementation of
Hoeffding Anytime Tree---"Extremely Fast Decision Tree", a minor modification
to the MOA implementation of Hoeffding Tree---obtains significantly superior
prequential accuracy on most of the largest classification datasets from the
UCI repository. Hoeffding Anytime Tree produces the asymptotic batch tree in
the limit, is naturally resilient to concept drift, and can be used as a higher
accuracy replacement for Hoeffding Tree in most scenarios, at a small
additional computational cost.
Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis
Authors: Benjamin Shickel, Patrick Tighe, Azra Bihorac, Parisa Rashidi
Date: 2017-06-12 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning

The past decade has seen an explosion in the amount of digital information
stored in electronic health records (EHR). While primarily designed for
archiving patient clinical information and administrative healthcare tasks,
many researchers have found secondary use of these records for various clinical
informatics tasks. Over the same period, the machine learning community has
seen widespread advances in deep learning techniques, which also have been
successfully applied to the vast amount of EHR data. In this paper, we review
these deep EHR systems, examining architectures, technical aspects, and
clinical applications. We also identify shortcomings of current techniques and
discuss avenues of future research for EHR-based deep learning.
Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks
Authors: Amy Jin, Serena Yeung, Jeffrey Jopling, Jonathan Krause, Dan Azagury, Arnold Milstein, Li Fei-Fei
Date: 2018-02-24 00:00:00

Five billion people in the world lack access to quality surgical care.
Surgeon skill varies dramatically, and many surgical patients suffer
complications and avoidable harm. Improving surgical training and feedback
would help to reduce the rate of complications, half of which have been shown
to be preventable. To do this, it is essential to assess operative skill, a
process that currently requires experts and is manual, time consuming, and
subjective. In this work, we introduce an approach to automatically assess
surgeon performance by tracking and analyzing tool movements in surgical
videos, leveraging region-based convolutional neural networks. In order to
study this problem, we also introduce a new dataset, m2cai16-tool-locations,
which extends the m2cai16-tool dataset with spatial bounds of tools. While
previous methods have addressed tool presence detection, ours is the first to
not only detect presence but also spatially localize surgical tools in
real-world laparoscopic surgical videos. We show that our method both
effectively detects the spatial bounds of tools as well as significantly
outperforms existing methods on tool presence detection. We further demonstrate
the ability of our method to assess surgical quality through analysis of tool
usage patterns, movement range, and economy of motion.
Learning Differentially Private Recurrent Language Models
Authors: H. Brendan McMahan, Daniel Ramage, Kunal Talwar, Li Zhang
Date: 2017-10-18 00:00:00
Proceeding: ICLR 2018 1

We demonstrate that it is possible to train large recurrent language models
with user-level differential privacy guarantees with only a negligible cost in
predictive accuracy. Our work builds on recent advances in the training of deep
networks on user-partitioned data and privacy accounting for stochastic
gradient descent. In particular, we add user-level privacy protection to the
federated averaging algorithm, which makes "large step" updates from user-level
data. Our work demonstrates that given a dataset with a sufficiently large
number of users (a requirement easily met by even small internet-scale
datasets), achieving differential privacy comes at the cost of increased
computation, rather than in decreased utility as in most prior work. We find
that our private LSTM language models are quantitatively and qualitatively
similar to un-noised models when trained on a large dataset.
The Vertex Sample Complexity of Free Energy is Polynomial
Authors: Vishesh Jain, Frederic Koehler, Elchanan Mossel
Date: 2018-02-16 00:00:00
Tasks: LEMMA

We study the following question: given a massive Markov random field on $n$
nodes, can a small sample from it provide a rough approximation to the free
energy $\mathcal{F}_n = \log{Z_n}$?
  Results in graph limit literature by Borgs, Chayes, Lov\'asz, S\'os, and
Vesztergombi show that for Ising models on $n$ nodes and interactions of
strength $\Theta(1/n)$, an $\epsilon$ approximation to $\log Z_n / n$ can be
achieved by sampling a randomly induced model on $2^{O(1/\epsilon^2)}$ nodes.
We show that the sampling complexity of this problem is {\em polynomial in}
$1/\epsilon$. We further show a polynomial dependence on $\epsilon$ cannot be
avoided.
  Our results are very general as they apply to higher order Markov random
fields. For Markov random fields of order $r$, we obtain an algorithm that
achieves $\epsilon$ approximation using a number of samples polynomial in $r$
and $1/\epsilon$ and running time that is $2^{O(1/\epsilon^2)}$ up to
polynomial factors in $r$ and $\epsilon$. For ferromagnetic Ising models, the
running time is polynomial in $1/\epsilon$.
  Our results are intimately connected to recent research on the regularity
lemma and property testing, where the interest is in finding which properties
can tested within $\epsilon$ error in time polynomial in $1/\epsilon$. In
particular, our proofs build on results from a recent work by Alon, de la Vega,
Kannan and Karpinski, who also introduced the notion of polynomial vertex
sample complexity. Another critical ingredient of the proof is an effective
bound by the authors of the paper relating the variational free energy and the
free energy.
Empirical observations of ultraslow diffusion driven by the fractional dynamics in languages: Dynamical statistical properties of word counts of already popular words
Authors: Hayafumi Watanabe
Date: 2018-01-24 00:00:00
Tasks: Articles, Time Series, Time Series Analysis

Ultraslow diffusion (i.e. logarithmic diffusion) has been extensively studied
theoretically, but has hardly been observed empirically. In this paper,
firstly, we find the ultraslow-like diffusion of the time-series of word counts
of already popular words by analysing three different nationwide language
databases: (i) newspaper articles (Japanese), (ii) blog articles (Japanese),
and (iii) page views of Wikipedia (English, French, Chinese, and Japanese).
Secondly, we use theoretical analysis to show that this diffusion is basically
explained by the random walk model with the power-law forgetting with the
exponent $\beta \approx 0.5$, which is related to the fractional Langevin
equation. The exponent $\beta$ characterises the speed of forgetting and $\beta
\approx 0.5$ corresponds to (i) the border (or thresholds) between the
stationary and the nonstationary and (ii) the right-in-the-middle dynamics
between the IID noise for $\beta=1$ and the normal random walk for $\beta=0$.
Thirdly, the generative model of the time-series of word counts of already
popular words, which is a kind of Poisson process with the Poisson parameter
sampled by the above-mentioned random walk model, can almost reproduce not only
the empirical mean-squared displacement but also the power spectrum density and
the probability density function.
Don't Decay the Learning Rate, Increase the Batch Size
Authors: Samuel L. Smith, Pieter-Jan Kindermans, Chris Ying, Quoc V. Le
Date: 2017-11-01 00:00:00
Proceeding: ICLR 2018 1

It is common practice to decay the learning rate. Here we show one can
usually obtain the same learning curve on both training and test sets by
instead increasing the batch size during training. This procedure is successful
for stochastic gradient descent (SGD), SGD with momentum, Nesterov momentum,
and Adam. It reaches equivalent test accuracies after the same number of
training epochs, but with fewer parameter updates, leading to greater
parallelism and shorter training times. We can further reduce the number of
parameter updates by increasing the learning rate $\epsilon$ and scaling the
batch size $B \propto \epsilon$. Finally, one can increase the momentum
coefficient $m$ and scale $B \propto 1/(1-m)$, although this tends to slightly
reduce the test accuracy. Crucially, our techniques allow us to repurpose
existing training schedules for large batch training with no hyper-parameter
tuning. We train ResNet-50 on ImageNet to $76.1\%$ validation accuracy in under
30 minutes.
Energy Storage Arbitrage in Real-Time Markets via Reinforcement Learning
Authors: Hao Wang, Baosen Zhang
Date: 2017-11-08 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we derive a temporal arbitrage policy for storage via reinforcement learning. Real-time price arbitrage is an important source of revenue for storage units, but designing good strategies have proven to be difficult because of the highly uncertain nature of the prices. Instead of current model predictive or dynamic programming approaches, we use reinforcement learning to design an optimal arbitrage policy. This policy is learned through repeated charge and discharge actions performed by the storage unit through updating a value matrix. We design a reward function that does not only reflect the instant profit of charge/discharge decisions but also incorporate the history information. Simulation results demonstrate that our designed reward function leads to significant performance improvement compared with existing algorithms.
Is Generator Conditioning Causally Related to GAN Performance?
Authors: Augustus Odena, Jacob Buckman, Catherine Olsson, Tom B. Brown, Christopher Olah, Colin Raffel, Ian Goodfellow
Date: 2018-02-23 00:00:00
Proceeding: ICML 2018 7

Recent work (Pennington et al, 2017) suggests that controlling the entire
distribution of Jacobian singular values is an important design consideration
in deep learning. Motivated by this, we study the distribution of singular
values of the Jacobian of the generator in Generative Adversarial Networks
(GANs). We find that this Jacobian generally becomes ill-conditioned at the
beginning of training. Moreover, we find that the average (with z from p(z))
conditioning of the generator is highly predictive of two other ad-hoc metrics
for measuring the 'quality' of trained GANs: the Inception Score and the
Frechet Inception Distance (FID). We test the hypothesis that this relationship
is causal by proposing a 'regularization' technique (called Jacobian Clamping)
that softly penalizes the condition number of the generator Jacobian. Jacobian
Clamping improves the mean Inception Score and the mean FID for GANs trained on
several datasets. It also greatly reduces inter-run variance of the
aforementioned scores, addressing (at least partially) one of the main
criticisms of GANs.