pwc-archive/papers-with-abstracts huggingface dataset:
Single Image Super-Resolution Using Lightweight CNN with Maxout Units
Authors: Jae-Seok Choi, Munchurl Kim
Date: 2017-11-07 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

Rectified linear units (ReLU) are well-known to be helpful in obtaining
faster convergence and thus higher performance for many deep-learning-based
applications. However, networks with ReLU tend to perform poorly when the
number of filter parameters is constrained to a small number. To overcome it,
in this paper, we propose a novel network utilizing maxout units (MU), and show
its effectiveness on super-resolution (SR) applications. In general, the MU has
been known to make the filter sizes doubled in generating the feature maps of
the same sizes in classification problems. In this paper, we first reveal that
the MU can even make the filter sizes halved in restoration problems thus
leading to compaction of the network sizes. To show this, our SR network is
designed without increasing the filter sizes with MU, which outperforms the
state of the art SR methods with a smaller number of filter parameters. To the
best of our knowledge, we are the first to incorporate MU into SR applications
and show promising performance results. In MU, feature maps from a previous
convolutional layer are divided into two parts along channels, which are then
compared element-wise and only their max values are passed to a next layer.
Along with some interesting properties of MU to be analyzed, we further
investigate other variants of MU and their effects. In addition, while ReLU
have a trouble for learning in networks with a very small number of
convolutional filter parameters, MU do not. For SR applications, our MU-based
network reconstructs high-resolution images with comparable quality compared to
previous deep-learning-based SR methods, with lower filter parameters.
Medoids in almost linear time via multi-armed bandits
Authors: Vivek Bagaria, Govinda M. Kamath, Vasilis Ntranos, Martin J. Zhang, David Tse
Date: 2017-11-02 00:00:00
Tasks: Multi-Armed Bandits

Computing the medoid of a large number of points in high-dimensional space is
an increasingly common operation in many data science problems. We present an
algorithm Med-dit which uses O(n log n) distance evaluations to compute the
medoid with high probability. Med-dit is based on a connection with the
multi-armed bandit problem. We evaluate the performance of Med-dit empirically
on the Netflix-prize and the single-cell RNA-Seq datasets, containing hundreds
of thousands of points living in tens of thousands of dimensions, and observe a
5-10x improvement in performance over the current state of the art. Med-dit is
available at https://github.com/bagavi/Meddit
DeepRain: ConvLSTM Network for Precipitation Prediction using Multichannel Radar Data
Authors: Seongchan Kim, Seungkyun Hong, Minsu Joh, Sa-kwang Song
Date: 2017-11-07 00:00:00
Tasks: Time Series, Time Series Analysis

Accurate rainfall forecasting is critical because it has a great impact on
people's social and economic activities. Recent trends on various literatures
show that Deep Learning (Neural Network) is a promising methodology to tackle
many challenging tasks. In this study, we introduce a brand-new data-driven
precipitation prediction model called DeepRain. This model predicts the amount
of rainfall from weather radar data, which is three-dimensional and
four-channel data, using convolutional LSTM (ConvLSTM). ConvLSTM is a variant
of LSTM (Long Short-Term Memory) containing a convolution operation inside the
LSTM cell. For the experiment, we used radar reflectivity data for a two-year
period whose input is in a time series format in units of 6 min divided into 15
records. The output is the predicted rainfall information for the input data.
Experimental results show that two-stacked ConvLSTM reduced RMSE by 23.0%
compared to linear regression.
Learning the distribution with largest mean: two bandit frameworks
Authors: Emilie Kaufmann, Aurélien Garivier
Date: 2017-01-31 00:00:00
Tasks: BIG-bench Machine Learning, Vocal Bursts Valence Prediction

Over the past few years, the multi-armed bandit model has become increasingly
popular in the machine learning community, partly because of applications
including online content optimization. This paper reviews two different
sequential learning tasks that have been considered in the bandit literature ;
they can be formulated as (sequentially) learning which distribution has the
highest mean among a set of distributions, with some constraints on the
learning process. For both of them (regret minimization and best arm
identification) we present recent, asymptotically optimal algorithms. We
compare the behaviors of the sampling rule of each algorithm as well as the
complexity terms associated to each problem.
Learning Overcomplete HMMs
Authors: Vatsal Sharan, Sham Kakade, Percy Liang, Gregory Valiant
Date: 2017-11-07 00:00:00
Proceeding: NeurIPS 2017 12

We study the problem of learning overcomplete HMMs---those that have many
hidden states but a small output alphabet. Despite having significant practical
importance, such HMMs are poorly understood with no known positive or negative
results for efficient learning. In this paper, we present several new
results---both positive and negative---which help define the boundaries between
the tractable and intractable settings. Specifically, we show positive results
for a large subclass of HMMs whose transition matrices are sparse,
well-conditioned, and have small probability mass on short cycles. On the other
hand, we show that learning is impossible given only a polynomial number of
samples for HMMs with a small output alphabet and whose transition matrices are
random regular graphs with large degree. We also discuss these results in the
context of learning HMMs which can capture long-term dependencies.
Machine Learning Approach to RF Transmitter Identification
Authors: K. Youssef, Louis-S. Bouchard, K. Z. Haigh, H. Krovi, J. Silovsky, C. P. Vander Valk
Date: 2017-11-05 00:00:00
Tasks: BIG-bench Machine Learning, General Classification

With the development and widespread use of wireless devices in recent years
(mobile phones, Internet of Things, Wi-Fi), the electromagnetic spectrum has
become extremely crowded. In order to counter security threats posed by rogue
or unknown transmitters, it is important to identify RF transmitters not by the
data content of the transmissions but based on the intrinsic physical
characteristics of the transmitters. RF waveforms represent a particular
challenge because of the extremely high data rates involved and the potentially
large number of transmitters present in a given location. These factors outline
the need for rapid fingerprinting and identification methods that go beyond the
traditional hand-engineered approaches. In this study, we investigate the use
of machine learning (ML) strategies to the classification and identification
problems, and the use of wavelets to reduce the amount of data required. Four
different ML strategies are evaluated: deep neural nets (DNN), convolutional
neural nets (CNN), support vector machines (SVM), and multi-stage training
(MST) using accelerated Levenberg-Marquardt (A-LM) updates. The A-LM MST method
preconditioned by wavelets was by far the most accurate, achieving 100%
classification accuracy of transmitters, as tested using data originating from
12 different transmitters. We discuss strategies for extension of MST to a much
larger number of transmitters.
Quality-Efficiency Trade-offs in Machine Learning for Text Processing
Authors: Ricardo Baeza-Yates, Zeinab Liaghat
Date: 2017-11-07 00:00:00
Tasks: BIG-bench Machine Learning, Document Classification, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Sentiment Analysis, Small Data Image Classification

Data mining, machine learning, and natural language processing are powerful
techniques that can be used together to extract information from large texts.
Depending on the task or problem at hand, there are many different approaches
that can be used. The methods available are continuously being optimized, but
not all these methods have been tested and compared in a set of problems that
can be solved using supervised machine learning algorithms. The question is
what happens to the quality of the methods if we increase the training data
size from, say, 100 MB to over 1 GB? Moreover, are quality gains worth it when
the rate of data processing diminishes? Can we trade quality for time
efficiency and recover the quality loss by just being able to process more
data? We attempt to answer these questions in a general way for text processing
tasks, considering the trade-offs involving training data size, learning time,
and quality obtained. We propose a performance trade-off framework and apply it
to three important text processing problems: Named Entity Recognition,
Sentiment Analysis and Document Classification. These problems were also chosen
because they have different levels of object granularity: words, paragraphs,
and documents. For each problem, we selected several supervised machine
learning algorithms and we evaluated the trade-offs of them on large publicly
available data sets (news, reviews, patents). To explore these trade-offs, we
use different data subsets of increasing size ranging from 50 MB to several GB.
We also consider the impact of the data set and the evaluation technique. We
find that the results do not change significantly and that most of the time the
best algorithms is the fastest. However, we also show that the results for
small data (say less than 100 MB) are different from the results for big data
and in those cases the best algorithm is much harder to determine.
Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net
Authors: Anirudh Goyal, Nan Rosemary Ke, Surya Ganguli, Yoshua Bengio
Date: 2017-11-07 00:00:00
Proceeding: NeurIPS 2017 12

We propose a novel method to directly learn a stochastic transition operator
whose repeated application provides generated samples. Traditional undirected
graphical models approach this problem indirectly by learning a Markov chain
model whose stationary distribution obeys detailed balance with respect to a
parameterized energy function. The energy function is then modified so the
model and data distributions match, with no guarantee on the number of steps
required for the Markov chain to converge. Moreover, the detailed balance
condition is highly restrictive: energy based models corresponding to neural
networks must have symmetric weights, unlike biological neural circuits. In
contrast, we develop a method for directly learning arbitrarily parameterized
transition operators capable of expressing non-equilibrium stationary
distributions that violate detailed balance, thereby enabling us to learn more
biologically plausible asymmetric neural networks and more general non-energy
based dynamical systems. The proposed training objective, which we derive via
principled variational methods, encourages the transition operator to "walk
back" in multi-step trajectories that start at data-points, as quickly as
possible back to the original data points. We present a series of experimental
results illustrating the soundness of the proposed approach, Variational
Walkback (VW), on the MNIST, CIFAR-10, SVHN and CelebA datasets, demonstrating
superior samples compared to earlier attempts to learn a transition operator.
We also show that although each rapid training trajectory is limited to a
finite but variable number of steps, our transition operator continues to
generate good samples well past the length of such trajectories, thereby
demonstrating the match of its non-equilibrium stationary distribution to the
data distribution. Source Code: http://github.com/anirudh9119/walkback_nips17
SSDH: Semi-supervised Deep Hashing for Large Scale Image Retrieval
Authors: Jian Zhang, Yuxin Peng
Date: 2016-07-28 00:00:00
Tasks: Deep Hashing, graph construction, Image Retrieval, Retrieval, Semantic Similarity, Semantic Textual Similarity

Hashing methods have been widely used for efficient similarity retrieval on
large scale image database. Traditional hashing methods learn hash functions to
generate binary codes from hand-crafted features, which achieve limited
accuracy since the hand-crafted features cannot optimally represent the image
content and preserve the semantic similarity. Recently, several deep hashing
methods have shown better performance because the deep architectures generate
more discriminative feature representations. However, these deep hashing
methods are mainly designed for supervised scenarios, which only exploit the
semantic similarity information, but ignore the underlying data structures. In
this paper, we propose the semi-supervised deep hashing (SSDH) approach, to
perform more effective hash function learning by simultaneously preserving
semantic similarity and underlying data structures. The main contributions are
as follows: (1) We propose a semi-supervised loss to jointly minimize the
empirical error on labeled data, as well as the embedding error on both labeled
and unlabeled data, which can preserve the semantic similarity and capture the
meaningful neighbors on the underlying data structures for effective hashing.
(2) A semi-supervised deep hashing network is designed to extensively exploit
both labeled and unlabeled data, in which we propose an online graph
construction method to benefit from the evolving deep features during training
to better capture semantic neighbors. To the best of our knowledge, the
proposed deep network is the first deep hashing method that can perform hash
code learning and feature learning simultaneously in a semi-supervised fashion.
Experimental results on 5 widely-used datasets show that our proposed approach
outperforms the state-of-the-art hashing methods.
The Scaling Limit of High-Dimensional Online Independent Component Analysis
Authors: Chuang Wang, Yue M. Lu
Date: 2017-10-15 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Vocal Bursts Intensity Prediction

We analyze the dynamics of an online algorithm for independent component
analysis in the high-dimensional scaling limit. As the ambient dimension tends
to infinity, and with proper time scaling, we show that the time-varying joint
empirical measure of the target feature vector and the estimates provided by
the algorithm will converge weakly to a deterministic measured-valued process
that can be characterized as the unique solution of a nonlinear PDE. Numerical
solutions of this PDE, which involves two spatial variables and one time
variable, can be efficiently obtained. These solutions provide detailed
information about the performance of the ICA algorithm, as many practical
performance metrics are functionals of the joint empirical measures. Numerical
simulations show that our asymptotic analysis is accurate even for moderate
dimensions. In addition to providing a tool for understanding the performance
of the algorithm, our PDE analysis also provides useful insight. In particular,
in the high-dimensional limit, the original coupled dynamics associated with
the algorithm will be asymptotically "decoupled", with each coordinate
independently solving a 1-D effective minimization problem via stochastic
gradient descent. Exploiting this insight to design new algorithms for
achieving optimal trade-offs between computational and statistical efficiency
may prove an interesting line of future research.
Convolutional Normalizing Flows
Authors: Guoqing Zheng, Yiming Yang, Jaime Carbonell
Date: 2017-11-07 00:00:00
Proceeding: ICLR 2018 1
Tasks: Variational Inference

Bayesian posterior inference is prevalent in various machine learning
problems. Variational inference provides one way to approximate the posterior
distribution, however its expressive power is limited and so is the accuracy of
resulting approximation. Recently, there has a trend of using neural networks
to approximate the variational posterior distribution due to the flexibility of
neural network architecture. One way to construct flexible variational
distribution is to warp a simple density into a complex by normalizing flows,
where the resulting density can be analytically evaluated. However, there is a
trade-off between the flexibility of normalizing flow and computation cost for
efficient transformation. In this paper, we propose a simple yet effective
architecture of normalizing flows, ConvFlow, based on convolution over the
dimensions of random input vector. Experiments on synthetic and real world
posterior inference problems demonstrate the effectiveness and efficiency of
the proposed method.
Challenges in Disentangling Independent Factors of Variation
Authors: Attila Szabó, Qiyang Hu, Tiziano Portenier, Matthias Zwicker, Paolo Favaro
Date: 2017-11-07 00:00:00
Proceeding: ICLR 2018 1
Tasks: Image Generation

We study the problem of building models that disentangle independent factors
of variation. Such models could be used to encode features that can efficiently
be used for classification and to transfer attributes between different images
in image synthesis. As data we use a weakly labeled training set. Our weak
labels indicate what single factor has changed between two data samples,
although the relative value of the change is unknown. This labeling is of
particular interest as it may be readily available without annotation costs. To
make use of weak labels we introduce an autoencoder model and train it through
constraints on image pairs and triplets. We formally prove that without
additional knowledge there is no guarantee that two images with the same factor
of variation will be mapped to the same feature. We call this issue the
reference ambiguity. Moreover, we show the role of the feature dimensionality
and adversarial training. We demonstrate experimentally that the proposed model
can successfully transfer attributes on several datasets, but show also cases
when the reference ambiguity occurs.
Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
Authors: Junhyuk Oh, Satinder Singh, Honglak Lee, Pushmeet Kohli
Date: 2017-06-15 00:00:00
Proceeding: ICML 2017 8
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

As a step towards developing zero-shot task generalization capabilities in
reinforcement learning (RL), we introduce a new RL problem where the agent
should learn to execute sequences of instructions after learning useful skills
that solve subtasks. In this problem, we consider two types of generalizations:
to previously unseen instructions and to longer sequences of instructions. For
generalization over unseen instructions, we propose a new objective which
encourages learning correspondences between similar subtasks by making
analogies. For generalization over sequential instructions, we present a
hierarchical architecture where a meta controller learns to use the acquired
skills for executing the instructions. To deal with delayed reward, we propose
a new neural architecture in the meta controller that learns when to update the
subtask, which makes learning more efficient. Experimental results on a
stochastic 3D domain show that the proposed ideas are crucial for
generalization to longer instructions as well as unseen instructions.
Visually-Aware Fashion Recommendation and Design with Generative Image Models
Authors: Wang-Cheng Kang, Chen Fang, Zhaowen Wang, Julian McAuley
Date: 2017-11-07 00:00:00
Tasks: Recommendation Systems

Building effective recommender systems for domains like fashion is
challenging due to the high level of subjectivity and the semantic complexity
of the features involved (i.e., fashion styles). Recent work has shown that
approaches to `visual' recommendation (e.g.~clothing, art, etc.) can be made
more accurate by incorporating visual signals directly into the recommendation
objective, using `off-the-shelf' feature representations derived from deep
networks. Here, we seek to extend this contribution by showing that
recommendation performance can be significantly improved by learning `fashion
aware' image representations directly, i.e., by training the image
representation (from the pixel level) and the recommender system jointly; this
contribution is related to recent work using Siamese CNNs, though we are able
to show improvements over state-of-the-art recommendation techniques such as
BPR and variants that make use of pre-trained visual features. Furthermore, we
show that our model can be used \emph{generatively}, i.e., given a user and a
product category, we can generate new images (i.e., clothing items) that are
most consistent with their personal taste. This represents a first step towards
building systems that go beyond recommending existing items from a product
corpus, but which can be used to suggest styles and aid the design of new
products.
Unsupervised Transformation Learning via Convex Relaxations
Authors: Tatsunori B. Hashimoto, John C. Duchi, Percy Liang
Date: 2017-11-06 00:00:00
Proceeding: NeurIPS 2017 12

Our goal is to extract meaningful transformations from raw images, such as
varying the thickness of lines in handwriting or the lighting in a portrait. We
propose an unsupervised approach to learn such transformations by attempting to
reconstruct an image from a linear combination of transformations of its
nearest neighbors. On handwritten digits and celebrity portraits, we show that
even with linear transformations, our method generates visually high-quality
modified images. Moreover, since our method is semiparametric and does not
model the data distribution, the learned transformations extrapolate off the
training data and can be applied to new types of images.
Value Prediction Network
Authors: Junhyuk Oh, Satinder Singh, Honglak Lee
Date: 2017-07-11 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Atari Games, Deep Reinforcement Learning, Prediction, Reinforcement Learning, Reinforcement Learning (RL), Value prediction

This paper proposes a novel deep reinforcement learning (RL) architecture,
called Value Prediction Network (VPN), which integrates model-free and
model-based RL methods into a single neural network. In contrast to typical
model-based RL methods, VPN learns a dynamics model whose abstract states are
trained to make option-conditional predictions of future values (discounted sum
of rewards) rather than of future observations. Our experimental results show
that VPN has several advantages over both model-free and model-based baselines
in a stochastic environment where careful planning is required but building an
accurate observation-prediction model is difficult. Furthermore, VPN
outperforms Deep Q-Network (DQN) on several Atari games even with
short-lookahead planning, demonstrating its potential as a new way of learning
a good state representation.
Image Segmentation of Multi-Shaped Overlapping Objects
Authors: Kumar Abhinav, Jaideep Singh Chauhan, Debasis Sarkar
Date: 2017-11-06 00:00:00
Tasks: Image Segmentation, Semantic Segmentation

In this work, we propose a new segmentation algorithm for images containing
convex objects present in multiple shapes with a high degree of overlap. The
proposed algorithm is carried out in two steps, first we identify the visible
contours, segment them using concave points and finally group the segments
belonging to the same object. The next step is to assign a shape identity to
these grouped contour segments. For images containing objects in multiple
shapes we begin first by identifying shape classes of the contours followed by
assigning a shape entity to these classes. We provide a comprehensive
experimentation of our algorithm on two crystal image datasets. One dataset
comprises of images containing objects in multiple shapes overlapping each
other and the other dataset contains standard images with objects present in a
single shape. We test our algorithm against two baselines, with our proposed
algorithm outperforming both the baselines.