pwc-archive/papers-with-abstracts huggingface dataset:
DropMax: Adaptive Variational Softmax
Authors: Hae Beom Lee, Juho Lee, Saehoon Kim, Eunho Yang, Sung Ju Hwang
Date: 2017-12-21 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Classification, General Classification, Variational Inference

We propose DropMax, a stochastic version of softmax classifier which at each
iteration drops non-target classes according to dropout probabilities
adaptively decided for each instance. Specifically, we overlay binary masking
variables over class output probabilities, which are input-adaptively learned
via variational inference. This stochastic regularization has an effect of
building an ensemble classifier out of exponentially many classifiers with
different decision boundaries. Moreover, the learning of dropout rates for
non-target classes on each instance allows the classifier to focus more on
classification against the most confusing classes. We validate our model on
multiple public datasets for classification, on which it obtains significantly
improved accuracy over the regular softmax classifier and other baselines.
Further analysis of the learned dropout probabilities shows that our model
indeed selects confusing classes more often when it performs classification.
NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations
Authors: Marco Ciccone, Marco Gallieri, Jonathan Masci, Christian Osendorfer, Faustino Gomez
Date: 2018-04-19 00:00:00
Proceeding: NeurIPS 2018 12

This paper introduces Non-Autonomous Input-Output Stable Network(NAIS-Net), a very deep architecture where each stacked processing block is derived from a time-invariant non-autonomous dynamical system. Non-autonomy is implemented by skip connections from the block input to each of the unrolled processing stages and allows stability to be enforced so that blocks can be unrolled adaptively to a pattern-dependent processing depth. NAIS-Net induces non-trivial, Lipschitz input-output maps, even for an infinite unroll length. We prove that the network is globally asymptotically stable so that for every initial condition there is exactly one input-dependent equilibrium assuming $tanh$ units, and incrementally stable for ReL units. An efficient implementation that enforces the stability under derived conditions for both fully-connected and convolutional layers is also presented. Experimental results show how NAIS-Net exhibits stability in practice, yielding a significant reduction in generalization gap compared to ResNets.
Learning compositionally through attentive guidance
Authors: Dieuwke Hupkes, Anand Singh, Kris Korrel, German Kruszewski, Elia Bruni
Date: 2018-05-20 00:00:00

While neural network models have been successfully applied to domains that require substantial generalisation skills, recent studies have implied that they struggle when solving the task they are trained on requires inferring its underlying compositional structure. In this paper, we introduce Attentive Guidance, a mechanism to direct a sequence to sequence model equipped with attention to find more compositional solutions. We test it on two tasks, devised precisely to assess the compositional capabilities of neural models, and we show that vanilla sequence to sequence models with attention overfit the training distribution, while the guided versions come up with compositional solutions that fit the training and testing distributions almost equally well. Moreover, the learned solutions generalise even in cases where the training and testing distributions strongly diverge. In this way, we demonstrate that sequence to sequence models are capable of finding compositional solutions without requiring extra components. These results helps to disentangle the causes for the lack of systematic compositionality in neural networks, which can in turn fuel future work.
Network Reconstruction and Controlling Based on Structural Regularity Analysis
Authors: Tao Wu, Shaojie Qiao, Xingping Xian, Xi-Zhao Wang, Wei Wang, Yanbing Liu
Date: 2018-05-20 00:00:00

From the perspective of network analysis, the ubiquitous networks are
comprised of regular and irregular components, which makes uncovering the
complexity of network structures to be a fundamental challenge. Exploring the
regular information and identifying the roles of microscopic elements in
network data can help us recognize the principle of network organization and
contribute to network data utilization. However, the intrinsic structural
properties of networks remain so far inadequately explored and theorised. With
the realistic assumption that there are consistent features across the local
structures of networks, we propose a low-rank pursuit based self-representation
network model, in which the principle of network organization can be uncovered
by a representation matrix. According to this model, original true networks can
be reconstructed based on the observed unreliable network topology. In
particular, the proposed model enables us to estimate the extent to which the
networks are regulable, i.e., measuring the reconstructability of networks. In
addition, the model is capable of measuring the importance of microscopic
network elements, i.e., nodes and links, in terms of network regularity thereby
allowing us to regulate the reconstructability of networks based on them.
Extensive experiments on disparate real-world networks demonstrate the
effectiveness of the proposed network reconstruction and regulation algorithm.
Specifically, the network regularity metric can reflect the reconstructability
of networks, and the reconstruction accuracy can be improved by removing
irregular network links. Lastly, our approach provides an unique and novel
insight into the organization exploring of complex networks.
Transitions, Losses, and Re-parameterizations: Elements of Prediction Games
Authors: Parameswaran Kamalaruban
Date: 2018-05-20 00:00:00
Tasks: Prediction

This thesis presents some geometric insights into three different types of
two player prediction games -- namely general learning task, prediction with
expert advice, and online convex optimization. These games differ in the nature
of the opponent (stochastic, adversarial, or intermediate), the order of the
players' move, and the utility function. The insights shed some light on the
understanding of the intrinsic barriers of the prediction problems and the
design of computationally efficient learning algorithms with strong theoretical
guarantees (such as generalizability, statistical consistency, and constant
regret etc.).
STS Classification with Dual-stream CNN
Authors: Shuchen Weng, Wenbo Li, Yi Zhang, Siwei Lyu
Date: 2018-05-20 00:00:00
Tasks: Activity Recognition, Classification, General Classification, STS, Time Series, Time Series Analysis

The structured time series (STS) classification problem requires the modeling
of interweaved spatiotemporal dependency. most previous STS classification
methods model the spatial and temporal dependencies independently. Due to the
complexity of the STS data, we argue that a desirable STS classification method
should be a holistic framework that can be made as adaptive and flexible as
possible. This motivates us to design a deep neural network with such merits.
Inspired by the dual-stream hypothesis in neural science, we propose a novel
dual-stream framework for modeling the interweaved spatiotemporal dependency,
and develop a convolutional neural network within this framework that aims to
achieve high adaptability and flexibility in STS configurations from various
diagonals, i.e., sequential order, dependency range and features. The proposed
architecture is highly modularized and scalable, making it easy to be adapted
to specific tasks. The effectiveness of our model is demonstrated through
experiments on synthetic data as well as benchmark datasets for skeleton based
activity recognition.
Exp-Concavity of Proper Composite Losses
Authors: Parameswaran Kamalaruban, Robert C. Williamson, Xinhua Zhang
Date: 2018-05-20 00:00:00
Tasks: Computational Efficiency

The goal of online prediction with expert advice is to find a decision
strategy which will perform almost as well as the best expert in a given pool
of experts, on any sequence of outcomes. This problem has been widely studied
and $O(\sqrt{T})$ and $O(\log{T})$ regret bounds can be achieved for convex
losses (\cite{zinkevich2003online}) and strictly convex losses with bounded
first and second derivatives (\cite{hazan2007logarithmic}) respectively. In
special cases like the Aggregating Algorithm (\cite{vovk1995game}) with mixable
losses and the Weighted Average Algorithm (\cite{kivinen1999averaging}) with
exp-concave losses, it is possible to achieve $O(1)$ regret bounds.
\cite{van2012exp} has argued that mixability and exp-concavity are roughly
equivalent under certain conditions. Thus by understanding the underlying
relationship between these two notions we can gain the best of both algorithms
(strong theoretical performance guarantees of the Aggregating Algorithm and the
computational efficiency of the Weighted Average Algorithm). In this paper we
provide a complete characterization of the exp-concavity of any proper
composite loss. Using this characterization and the mixability condition of
proper losses (\cite{van2012mixability}), we show that it is possible to
transform (re-parameterize) any $\beta$-mixable binary proper loss into a
$\beta$-exp-concave composite loss with the same $\beta$. In the multi-class
case, we propose an approximation approach for this transformation.
Nonlinear Distributional Gradient Temporal-Difference Learning
Authors: Chao Qu, Shie Mannor, Huan Xu
Date: 2018-05-20 00:00:00
Tasks: Distributional Reinforcement Learning, Reinforcement Learning

We devise a distributional variant of gradient temporal-difference (TD)
learning. Distributional reinforcement learning has been demonstrated to
outperform the regular one in the recent study
\citep{bellemare2017distributional}. In the policy evaluation setting, we
design two new algorithms called distributional GTD2 and distributional TDC
using the Cram{\'e}r distance on the distributional version of the Bellman
error objective function, which inherits advantages of both the nonlinear
gradient TD algorithms and the distributional RL approach. In the control
setting, we propose the distributional Greedy-GQ using the similar derivation.
We prove the asymptotic almost-sure convergence of distributional GTD2 and TDC
to a local optimal solution for general smooth function approximators, which
includes neural networks that have been widely used in recent study to solve
the real-life RL problems. In each step, the computational complexities of
above three algorithms are linear w.r.t.\ the number of the parameters of the
function approximator, thus can be implemented efficiently for neural networks.
Generating High-Quality Surface Realizations Using Data Augmentation and Factored Sequence Models
Authors: Henry Elder, Chris Hokamp
Date: 2018-05-20 00:00:00
Proceeding: WS 2018 7
Tasks: Data Augmentation

This work presents a new state of the art in reconstruction of surface
realizations from obfuscated text. We identify the lack of sufficient training
data as the major obstacle to training high-performing models, and solve this
issue by generating large amounts of synthetic training data. We also propose
preprocessing techniques which make the structure contained in the input
features more accessible to sequence models. Our models were ranked first on
all evaluation metrics in the English portion of the 2018 Surface Realization
shared task.
Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning
Authors: Pinxin Long, Tingxiang Fan, Xinyi Liao, Wenxi Liu, Hao Zhang, Jia Pan
Date: 2017-09-28 00:00:00
Tasks: Collision Avoidance, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Developing a safe and efficient collision avoidance policy for multiple
robots is challenging in the decentralized scenarios where each robot generate
its paths without observing other robots' states and intents. While other
distributed multi-robot collision avoidance systems exist, they often require
extracting agent-level features to plan a local collision-free action, which
can be computationally prohibitive and not robust. More importantly, in
practice the performance of these methods are much lower than their centralized
counterparts.
  We present a decentralized sensor-level collision avoidance policy for
multi-robot systems, which directly maps raw sensor measurements to an agent's
steering commands in terms of movement velocity. As a first step toward
reducing the performance gap between decentralized and centralized methods, we
present a multi-scenario multi-stage training framework to find an optimal
policy which is trained over a large number of robots on rich, complex
environments simultaneously using a policy gradient based reinforcement
learning algorithm. We validate the learned sensor-level collision avoidance
policy in a variety of simulated scenarios with thorough performance
evaluations and show that the final learned policy is able to find time
efficient, collision-free paths for a large-scale robot system. We also
demonstrate that the learned policy can be well generalized to new scenarios
that do not appear in the entire training period, including navigating a
heterogeneous group of robots and a large-scale scenario with 100 robots.
Videos are available at https://sites.google.com/view/drlmaca
Human-guided data exploration using randomisation
Authors: Kai Puolamäki, Emilia Oikarinen, Buse Atli, Andreas Henelius
Date: 2018-05-20 00:00:00

An explorative data analysis system should be aware of what the user already
knows and what the user wants to know of the data: otherwise the system cannot
provide the user with the most informative and useful views of the data. We
propose a principled way to do exploratory data analysis, where the user's
background knowledge is modeled by a distribution parametrised by subsets of
rows and columns in the data, called tiles. The user can also use tiles to
describe his or her interests concerning relations in the data. We provide a
computationally efficient implementation of this concept based on constrained
randomisation. The implementation is used to model both the background
knowledge and the user's information request and is a necessary prerequisite
for any interactive system. Furthermore, we describe a novel linear projection
pursuit method to find and show the views most informative to the user, which
at the limit of no background knowledge and with generic objectives reduces to
PCA. We show that our method is robust under noise and fast enough for
interactive use. We also show that the method gives understandable and useful
results when analysing real-world data sets. We will release an open source
library implementing the idea, including the experiments presented in this
paper. We show that our method can outperform standard projection pursuit
visualisation methods in exploration tasks. Our framework makes it possible to
construct human-guided data exploration systems which are fast, powerful, and
give results that are easy to comprehend.
Minimax Lower Bounds for Cost Sensitive Classification
Authors: Parameswaran Kamalaruban, Robert C. Williamson
Date: 2018-05-20 00:00:00
Tasks: BIG-bench Machine Learning, Binary Classification, Classification, General Classification

The cost-sensitive classification problem plays a crucial role in
mission-critical machine learning applications, and differs with traditional
classification by taking the misclassification costs into consideration.
Although being studied extensively in the literature, the fundamental limits of
this problem are still not well understood. We investigate the hardness of this
problem by extending the standard minimax lower bound of balanced binary
classification problem (due to \cite{massart2006risk}), and emphasize the
impact of cost terms on the hardness.
Task-Agnostic Meta-Learning for Few-shot Learning
Authors: Muhammad Abdullah Jamal, Guo-Jun Qi, Mubarak Shah
Date: 2018-05-20 00:00:00
Tasks: Classification, Few-Shot Learning, General Classification, Meta-Learning, Reinforcement Learning

Meta-learning approaches have been proposed to tackle the few-shot learning
problem.Typically, a meta-learner is trained on a variety of tasks in the hopes
of being generalizable to new tasks. However, the generalizability on new tasks
of a meta-learner could be fragile when it is over-trained on existing tasks
during meta-training phase. In other words, the initial model of a meta-learner
could be too biased towards existing tasks to adapt to new tasks, especially
when only very few examples are available to update the model. To avoid a
biased meta-learner and improve its generalizability, we propose a novel
paradigm of Task-Agnostic Meta-Learning (TAML) algorithms. Specifically, we
present an entropy-based approach that meta-learns an unbiased initial model
with the largest uncertainty over the output labels by preventing it from
over-performing in classification tasks. Alternatively, a more general
inequality-minimization TAML is presented for more ubiquitous scenarios by
directly minimizing the inequality of initial losses beyond the classification
tasks wherever a suitable loss can be defined.Experiments on benchmarked
datasets demonstrate that the proposed approaches outperform compared
meta-learning algorithms in both few-shot classification and reinforcement
learning tasks.
Piecewise Flat Embedding for Image Segmentation
Authors: Chaowei Fang, Zicheng Liao, Yizhou Yu
Date: 2018-02-09 00:00:00
Proceeding: ICCV 2015 12
Tasks: Clustering, Contour Detection, Image Segmentation, Segmentation, Semantic Segmentation

We introduce a new multi-dimensional nonlinear embedding -- Piecewise Flat
Embedding (PFE) -- for image segmentation. Based on the theory of sparse signal
recovery, piecewise flat embedding with diverse channels attempts to recover a
piecewise constant image representation with sparse region boundaries and
sparse cluster value scattering. The resultant piecewise flat embedding
exhibits interesting properties such as suppressing slowly varying signals, and
offers an image representation with higher region identifiability which is
desirable for image segmentation or high-level semantic analysis tasks. We
formulate our embedding as a variant of the Laplacian Eigenmap embedding with
an $L_{1,p} (0<p\leq1)$ regularization term to promote sparse solutions. First,
we devise a two-stage numerical algorithm based on Bregman iterations to
compute $L_{1,1}$-regularized piecewise flat embeddings. We further generalize
this algorithm through iterative reweighting to solve the general
$L_{1,p}$-regularized problem. To demonstrate its efficacy, we integrate PFE
into two existing image segmentation frameworks, segmentation based on
clustering and hierarchical segmentation based on contour detection.
Experiments on four major benchmark datasets, BSDS500, MSRC, Stanford
Background Dataset, and PASCAL Context, show that segmentation algorithms
incorporating our embedding achieve significantly improved results.
A Deep Structure of Person Re-Identification using Multi-Level Gaussian Models
Authors: Dinesh Kumar Vishwakarma, Sakshi Upadhyay
Date: 2018-05-20 00:00:00
Tasks: Metric Learning, Person Re-Identification

Person re-identification is being widely used in the forensic, and security
and surveillance system, but person re-identification is a challenging task in
real life scenario. Hence, in this work, a new feature descriptor model has
been proposed using a multilayer framework of Gaussian distribution model on
pixel features, which include color moments, color space values and Schmid
filter responses. An image of a person usually consists of distinct body
regions, usually with differentiable clothing followed by local colors and
texture patterns. Thus, the image is evaluated locally by dividing the image
into overlapping regions. Each region is further fragmented into a set of local
Gaussians on small patches. A global Gaussian encodes, these local Gaussians
for each region creating a multi-level structure. Hence, the global picture of
a person is described by local level information present in it, which is often
ignored. Also, we have analyzed the efficiency of earlier metric learning
methods on this descriptor. The performance of the descriptor is evaluated on
four public available challenging datasets and the highest accuracy achieved on
these datasets are compared with similar state-of-the-arts, which demonstrate
the superior performance.
Balancing Shared Autonomy with Human-Robot Communication
Authors: Rosario Scalise, Yonatan Bisk, Maxwell Forbes, Daqing Yi, Yejin Choi, Siddhartha Srinivasa
Date: 2018-05-20 00:00:00

Robotic agents that share autonomy with a human should leverage human domain
knowledge and account for their preferences when completing a task. This extra
knowledge can dramatically improve plan efficiency and user-satisfaction, but
these gains are lost if communicating with a robot is taxing and unnatural. In
this paper, we show how viewing humanrobot language through the lens of shared
autonomy explains the efficiency versus cognitive load trade-offs humans make
when deciding how cooperative and explicit to make their instructions.
Wavelet Convolutional Neural Networks
Authors: Shin Fujieda, Kohei Takayama, Toshiya Hachisuka
Date: 2018-05-20 00:00:00
Tasks: General Classification, image-classification, Image Classification, Object Recognition, Texture Classification

Spatial and spectral approaches are two major approaches for image processing
tasks such as image classification and object recognition. Among many such
algorithms, convolutional neural networks (CNNs) have recently achieved
significant performance improvement in many challenging tasks. Since CNNs
process images directly in the spatial domain, they are essentially spatial
approaches. Given that spatial and spectral approaches are known to have
different characteristics, it will be interesting to incorporate a spectral
approach into CNNs. We propose a novel CNN architecture, wavelet CNNs, which
combines a multiresolution analysis and CNNs into one model. Our insight is
that a CNN can be viewed as a limited form of a multiresolution analysis. Based
on this insight, we supplement missing parts of the multiresolution analysis
via wavelet transform and integrate them as additional components in the entire
architecture. Wavelet CNNs allow us to utilize spectral information which is
mostly lost in conventional CNNs but useful in most image processing tasks. We
evaluate the practical performance of wavelet CNNs on texture classification
and image annotation. The experiments show that wavelet CNNs can achieve better
accuracy in both tasks than existing models while having significantly fewer
parameters than conventional CNNs.