pwc-archive/papers-with-abstracts huggingface dataset:
Episodic Multi-armed Bandits
Authors: Cem Tekin, Mihaela van der Schaar
Date: 2015-08-04 00:00:00
Tasks: Multi-Armed Bandits, Reinforcement Learning

We introduce a new class of reinforcement learning methods referred to as
{\em episodic multi-armed bandits} (eMAB). In eMAB the learner proceeds in {\em
episodes}, each composed of several {\em steps}, in which it chooses an action
and observes a feedback signal. Moreover, in each step, it can take a special
action, called the $stop$ action, that ends the current episode. After the
$stop$ action is taken, the learner collects a terminal reward, and observes
the costs and terminal rewards associated with each step of the episode. The
goal of the learner is to maximize its cumulative gain (i.e., the terminal
reward minus costs) over all episodes by learning to choose the best sequence
of actions based on the feedback. First, we define an {\em oracle} benchmark,
which sequentially selects the actions that maximize the expected immediate
gain. Then, we propose our online learning algorithm, named {\em FeedBack
Adaptive Learning} (FeedBAL), and prove that its regret with respect to the
benchmark is bounded with high probability and increases logarithmically in
expectation. Moreover, the regret only has polynomial dependence on the number
of steps, actions and states. eMAB can be used to model applications that
involve humans in the loop, ranging from personalized medical screening to
personalized web-based education, where sequences of actions are taken in each
episode, and optimal behavior requires adapting the chosen actions based on the
feedback.
Deep Dictionary Learning: A PARametric NETwork Approach
Authors: Shahin Mahdizadehaghdam, Ashkan Panahi, Hamid Krim, Liyi Dai
Date: 2018-03-11 00:00:00
Tasks: Classification, Dictionary Learning, General Classification, image-classification, Image Classification

Deep dictionary learning seeks multiple dictionaries at different image
scales to capture complementary coherent characteristics. We propose a method
for learning a hierarchy of synthesis dictionaries with an image classification
goal. The dictionaries and classification parameters are trained by a
classification objective, and the sparse features are extracted by reducing a
reconstruction loss in each layer. The reconstruction objectives in some sense
regularize the classification problem and inject source signal information in
the extracted features. The performance of the proposed hierarchical method
increases by adding more layers, which consequently makes this model easier to
tune and adapt. The proposed algorithm furthermore, shows remarkably lower
fooling rate in presence of adversarial perturbation. The validation of the
proposed approach is based on its classification performance using four
benchmark datasets and is compared to a CNN of similar size.
Data-Augmented Contact Model for Rigid Body Simulation
Authors: Yifeng Jiang, Jiazheng Sun, C. Karen Liu
Date: 2018-03-11 00:00:00
Tasks: model

Accurately modeling contact behaviors for real-world, near-rigid materials remains a grand challenge for existing rigid-body physics simulators. This paper introduces a data-augmented contact model that incorporates analytical solutions with observed data to predict the 3D contact impulse which could result in rigid bodies bouncing, sliding or spinning in all directions. Our method enhances the expressiveness of the standard Coulomb contact model by learning the contact behaviors from the observed data, while preserving the fundamental contact constraints whenever possible. For example, a classifier is trained to approximate the transitions between static and dynamic frictions, while non-penetration constraint during collision is enforced analytically. Our method computes the aggregated effect of contact for the entire rigid body, instead of predicting the contact force for each contact point individually, maintaining same simulation speed as the number of contact points increases for detailed geometries. Supplemental video: https://shorturl.at/eilwX Keywords: Physics Simulation Algorithms, Dynamics Learning, Contact Learning
Multi-objective Contextual Bandit Problem with Similarity Information
Authors: Eralp Turğay, Doruk Öner, Cem Tekin
Date: 2018-03-11 00:00:00

In this paper we propose the multi-objective contextual bandit problem with
similarity information. This problem extends the classical contextual bandit
problem with similarity information by introducing multiple and possibly
conflicting objectives. Since the best arm in each objective can be different
given the context, learning the best arm based on a single objective can
jeopardize the rewards obtained from the other objectives. In order to evaluate
the performance of the learner in this setup, we use a performance metric
called the contextual Pareto regret. Essentially, the contextual Pareto regret
is the sum of the distances of the arms chosen by the learner to the context
dependent Pareto front. For this problem, we develop a new online learning
algorithm called Pareto Contextual Zooming (PCZ), which exploits the idea of
contextual zooming to learn the arms that are close to the Pareto front for
each observed context by adaptively partitioning the joint context-arm set
according to the observed rewards and locations of the context-arm pairs
selected in the past. Then, we prove that PCZ achieves $\tilde O
(T^{(1+d_p)/(2+d_p)})$ Pareto regret where $d_p$ is the Pareto zooming
dimension that depends on the size of the set of near-optimal context-arm
pairs. Moreover, we show that this regret bound is nearly optimal by providing
an almost matching $\Omega (T^{(1+d_p)/(2+d_p)})$ lower bound.
Multi-Armed Bandits for Correlated Markovian Environments with Smoothed Reward Feedback
Authors: Tanner Fiez, Shreyas Sekar, Lillian J. Ratliff
Date: 2018-03-11 00:00:00
Tasks: Multi-Armed Bandits, Q-Learning

We study a multi-armed bandit problem in a dynamic environment where arm
rewards evolve in a correlated fashion according to a Markov chain. Different
than much of the work on related problems, in our formulation a learning
algorithm does not have access to either a priori information or observations
of the state of the Markov chain and only observes smoothed reward feedback
following time intervals we refer to as epochs. We demonstrate that existing
methods such as UCB and $\varepsilon$-greedy can suffer linear regret in such
an environment. Employing mixing-time bounds on Markov chains, we develop
algorithms called EpochUCB and EpochGreedy that draw inspiration from the
aforementioned methods, yet which admit sublinear regret guarantees for the
problem formulation. Our proposed algorithms proceed in epochs in which an arm
is played repeatedly for a number of iterations that grows linearly as a
function of the number of times an arm has been played in the past. We analyze
these algorithms under two types of smoothed reward feedback at the end of each
epoch: a reward that is the discount-average of the discounted rewards within
an epoch, and a reward that is the time-average of the rewards within an epoch.
Preparing Bengali-English Code-Mixed Corpus for Sentiment Analysis of Indian Languages
Authors: Soumil Mandal, Sainik Kumar Mahata, Dipankar Das
Date: 2018-03-11 00:00:00
Tasks: Sentiment Analysis, TAG

Analysis of informative contents and sentiments of social users has been
attempted quite intensively in the recent past. Most of the systems are usable
only for monolingual data and fails or gives poor results when used on data
with code-mixing property. To gather attention and encourage researchers to
work on this crisis, we prepared gold standard Bengali-English code-mixed data
with language and polarity tag for sentiment analysis purposes. In this paper,
we discuss the systems we prepared to collect and filter raw Twitter data. In
order to reduce manual work while annotation, hybrid systems combining rule
based and supervised models were developed for both language and sentiment
tagging. The final corpus was annotated by a group of annotators following a
few guidelines. The gold standard corpus thus obtained has impressive
inter-annotator agreement obtained in terms of Kappa values. Various metrics
like Code-Mixed Index (CMI), Code-Mixed Factor (CF) along with various aspects
(language and emotion) also qualitatively polled the code-mixed and sentiment
properties of the corpus.
BEBP: An Poisoning Method Against Machine Learning Based IDSs
Authors: Pan Li, Qiang Liu, Wentao Zhao, Dongxu Wang, Siqi Wang
Date: 2018-03-11 00:00:00
Tasks: BIG-bench Machine Learning, Intrusion Detection

In big data era, machine learning is one of fundamental techniques in
intrusion detection systems (IDSs). However, practical IDSs generally update
their decision module by feeding new data then retraining learning models in a
periodical way. Hence, some attacks that comprise the data for training or
testing classifiers significantly challenge the detecting capability of machine
learning-based IDSs. Poisoning attack, which is one of the most recognized
security threats towards machine learning-based IDSs, injects some adversarial
samples into the training phase, inducing data drifting of training data and a
significant performance decrease of target IDSs over testing data. In this
paper, we adopt the Edge Pattern Detection (EPD) algorithm to design a novel
poisoning method that attack against several machine learning algorithms used
in IDSs. Specifically, we propose a boundary pattern detection algorithm to
efficiently generate the points that are near to abnormal data but considered
to be normal ones by current classifiers. Then, we introduce a Batch-EPD
Boundary Pattern (BEBP) detection algorithm to overcome the limitation of the
number of edge pattern points generated by EPD and to obtain more useful
adversarial samples. Based on BEBP, we further present a moderate but effective
poisoning method called chronic poisoning attack. Extensive experiments on
synthetic and three real network data sets demonstrate the performance of the
proposed poisoning method against several well-known machine learning
algorithms and a practical intrusion detection method named FMIFS-LSSVM-IDS.
BTS-DSN: Deeply Supervised Neural Network with Short Connections for Retinal Vessel Segmentation
Authors: Song Guo, Kai Wang, Hong Kang, Yujun Zhang, Yingqi Gao, Tao Li
Date: 2018-03-11 00:00:00
Tasks: Retinal Vessel Segmentation, Segmentation, Specificity

Background and Objective: The condition of vessel of the human eye is an important factor for the diagnosis of ophthalmological diseases. Vessel segmentation in fundus images is a challenging task due to complex vessel structure, the presence of similar structures such as microaneurysms and hemorrhages, micro-vessel with only one to several pixels wide, and requirements for finer results. Methods:In this paper, we present a multi-scale deeply supervised network with short connections (BTS-DSN) for vessel segmentation. We used short connections to transfer semantic information between side-output layers. Bottom-top short connections pass low level semantic information to high level for refining results in high-level side-outputs, and top-bottom short connection passes much structural information to low level for reducing noises in low-level side-outputs. In addition, we employ cross-training to show that our model is suitable for real world fundus images. Results: The proposed BTS-DSN has been verified on DRIVE, STARE and CHASE_DB1 datasets, and showed competitive performance over other state-of-the-art methods. Specially, with patch level input, the network achieved 0.7891/0.8212 sensitivity, 0.9804/0.9843 specificity, 0.9806/0.9859 AUC, and 0.8249/0.8421 F1-score on DRIVE and STARE, respectively. Moreover, our model behaves better than other methods in cross-training experiments. Conclusions: BTS-DSN achieves competitive performance in vessel segmentation task on three public datasets. It is suitable for vessel segmentation. The source code of our method is available at https://github.com/guomugong/BTS-DSN.
Empirical bounds for functions with weak interactions
Authors: Andreas Maurer, Massimiliano Pontil
Date: 2018-03-11 00:00:00

We provide sharp empirical estimates of expectation, variance and normal
approximation for a class of statistics whose variation in any argument does
not change too much when another argument is modified. Examples of such weak
interactions are furnished by U- and V-statistics, Lipschitz L-statistics and
various error functionals of L2-regularized algorithms and Gibbs algorithms.
Cubic Range Error Model for Stereo Vision with Illuminators
Authors: Marius Huber, Timo Hinzmann, Roland Siegwart, Larry H. Matthies
Date: 2018-03-11 00:00:00
Tasks: Scheduling

Use of low-cost depth sensors, such as a stereo camera setup with
illuminators, is of particular interest for numerous applications ranging from
robotics and transportation to mixed and augmented reality. The ability to
quantify noise is crucial for these applications, e.g., when the sensor is used
for map generation or to develop a sensor scheduling policy in a multi-sensor
setup. Range error models provide uncertainty estimates and help weigh the data
correctly in instances where range measurements are taken from different
vantage points or with different sensors. The weighing is important to fuse
range data into a map in a meaningful way, i.e., the high confidence data is
relied on most heavily. Such a model is derived in this work. We show that the
range error for stereo systems with integrated illuminators is cubic and
validate the proposed model experimentally with an off-the-shelf structured
light stereo system. The experiments confirm the validity of the model and
simplify the application of this type of sensor in robotics. The proposed error
model is relevant to any stereo system with low ambient light where the main
light source is located at the camera system. Among others, this is the case
for structured light stereo systems and night stereo systems with headlights.
In this work, we propose that the range error is cubic in range for stereo
systems with integrated illuminators. Experimental validation with an
off-the-shelf structured light stereo system shows that the exponent is between
2.4 and 2.6. The deviation is attributed to our model considering only shot
noise.
Soft-Robust Actor-Critic Policy-Gradient
Authors: Esther Derman, Daniel J. Mankowitz, Timothy A. Mann, Shie Mannor
Date: 2018-03-11 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Robust Reinforcement Learning aims to derive optimal behavior that accounts
for model uncertainty in dynamical systems. However, previous studies have
shown that by considering the worst case scenario, robust policies can be
overly conservative. Our soft-robust framework is an attempt to overcome this
issue. In this paper, we present a novel Soft-Robust Actor-Critic algorithm
(SR-AC). It learns an optimal policy with respect to a distribution over an
uncertainty set and stays robust to model uncertainty but avoids the
conservativeness of robust strategies. We show the convergence of SR-AC and
test the efficiency of our approach on different domains by comparing it
against regular learning methods and their robust formulations.
Channel-Recurrent Autoencoding for Image Modeling
Authors: Wenling Shang, Kihyuk Sohn, Yuandong Tian
Date: 2017-06-12 00:00:00

Despite recent successes in synthesizing faces and bedrooms, existing
generative models struggle to capture more complex image types, potentially due
to the oversimplification of their latent space constructions. To tackle this
issue, building on Variational Autoencoders (VAEs), we integrate recurrent
connections across channels to both inference and generation steps, allowing
the high-level features to be captured in global-to-local, coarse-to-fine
manners. Combined with adversarial loss, our channel-recurrent VAE-GAN
(crVAE-GAN) outperforms VAE-GAN in generating a diverse spectrum of high
resolution images while maintaining the same level of computational efficacy.
Our model produces interpretable and expressive latent representations to
benefit downstream tasks such as image completion. Moreover, we propose two
novel regularizations, namely the KL objective weighting scheme over time steps
and mutual information maximization between transformed latent variables and
the outputs, to enhance the training.
Deep reinforcement learning for time series: playing idealized trading games
Authors: Xiang Gao
Date: 2018-03-11 00:00:00
Tasks: Deep Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Time Series, Time Series Analysis

Deep Q-learning is investigated as an end-to-end solution to estimate the
optimal strategies for acting on time series input. Experiments are conducted
on two idealized trading games. 1) Univariate: the only input is a wave-like
price time series, and 2) Bivariate: the input includes a random stepwise price
time series and a noisy signal time series, which is positively correlated with
future price changes. The Univariate game tests whether the agent can capture
the underlying dynamics, and the Bivariate game tests whether the agent can
utilize the hidden relation among the inputs. Stacked Gated Recurrent Unit
(GRU), Long Short-Term Memory (LSTM) units, Convolutional Neural Network (CNN),
and multi-layer perceptron (MLP) are used to model Q values. For both games,
all agents successfully find a profitable strategy. The GRU-based agents show
best overall performance in the Univariate game, while the MLP-based agents
outperform others in the Bivariate game.
Controllable Top-down Feature Transformer
Authors: Zhiwei Jia, Haoshen Hong, Siyang Wang, Kwonjoon Lee, Zhuowen Tu
Date: 2017-12-06 00:00:00
Tasks: Data Augmentation, Style Transfer

We study the intrinsic transformation of feature maps across convolutional
network layers with explicit top-down control. To this end, we develop top-down
feature transformer (TFT), under controllable parameters, that are able to
account for the hidden layer transformation while maintaining the overall
consistency across layers. The learned generators capture the underlying
feature transformation processes that are independent of particular training
images. Our proposed TFT framework brings insights to and helps the
understanding of, an important problem of studying the CNN internal feature
representation and transformation under the top-down processes. In the case of
spatial transformations, we demonstrate the significant advantage of TFT over
existing data-driven approaches in building data-independent transformations.
We also show that it can be adopted in other applications such as data
augmentation and image style transfer.
A pathway-based kernel boosting method for sample classification using genomic data
Authors: Li Zeng, Zhaolong Yu, Hongyu Zhao
Date: 2018-03-11 00:00:00
Tasks: General Classification

The analysis of cancer genomic data has long suffered "the curse of
dimensionality". Sample sizes for most cancer genomic studies are a few
hundreds at most while there are tens of thousands of genomic features studied.
Various methods have been proposed to leverage prior biological knowledge, such
as pathways, to more effectively analyze cancer genomic data. Most of the
methods focus on testing marginal significance of the associations between
pathways and clinical phenotypes. They can identify relevant pathways, but do
not involve predictive modeling. In this article, we propose a Pathway-based
Kernel Boosting (PKB) method for integrating gene pathway information for
sample classification, where we use kernel functions calculated from each
pathway as base learners and learn the weights through iterative optimization
of the classification loss function. We apply PKB and several competing methods
to three cancer studies with pathological and clinical information, including
tumor grade, stage, tumor sites, and metastasis status. Our results show that
PKB outperforms other methods, and identifies pathways relevant to the outcome
variables.
Density Estimation via Discrepancy Based Adaptive Sequential Partition
Authors: Dangna Li, Kun Yang, Wing Hung Wong
Date: 2014-04-05 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Density Estimation

Given $iid$ observations from an unknown absolute continuous distribution
defined on some domain $\Omega$, we propose a nonparametric method to learn a
piecewise constant function to approximate the underlying probability density
function. Our density estimate is a piecewise constant function defined on a
binary partition of $\Omega$. The key ingredient of the algorithm is to use
discrepancy, a concept originates from Quasi Monte Carlo analysis, to control
the partition process. The resulting algorithm is simple, efficient, and has a
provable convergence rate. We empirically demonstrate its efficiency as a
density estimation method. We present its applications on a wide range of
tasks, including finding good initializations for k-means.
Path of Vowel Raising in Chengdu Dialect of Mandarin
Authors: Hai Hu, Yiwen Zhang
Date: 2018-03-11 00:00:00

He and Rao (2013) reported a raising phenomenon of /a/ in /Xan/ (X being a
consonant or a vowel) in Chengdu dialect of Mandarin, i.e. /a/ is realized as
[epsilon] for young speakers but [ae] for older speakers, but they offered no
acoustic analysis. We designed an acoustic study that examined the realization
of /Xan/ in speakers of different age (old vs. young) and gender (male vs.
female) groups, where X represents three conditions: 1) unaspirated consonants:
C ([p], [t], [k]), 2) aspirated consonants: Ch ([ph], [th], [kh]), and 3) high
vowels: V ([i], [y], [u]). 17 native speakers were asked to read /Xan/
characters and the F1 values are extracted for comparison. Our results
confirmed the raising effect in He and Rao (2013), i.e., young speakers realize
/a/ as [epsilon] in /an/, whereas older speakers in the most part realize it as
[ae]. Also, female speakers raise more than male speakers within the same age
group. Interestingly, within the /Van/ condition, older speakers do raise /a/
in /ian/ and /yan/. We interpret this as /a/ first assimilates to its preceding
front high vowels /i/ and /y/ for older speakers, which then becomes
phonologized in younger speakers in all conditions, including /Chan/ and /Can/.
This shows a possible trajectory of the ongoing sound change in the Chengdu
dialect.
Bidding Machine: Learning to Bid for Directly Optimizing Profits in Display Advertising
Authors: Kan Ren, Wei-Nan Zhang, Ke Chang, Yifei Rong, Yong Yu, Jun Wang
Date: 2018-03-01 00:00:00
Tasks: BIG-bench Machine Learning

Real-time bidding (RTB) based display advertising has become one of the key
technological advances in computational advertising. RTB enables advertisers to
buy individual ad impressions via an auction in real-time and facilitates the
evaluation and the bidding of individual impressions across multiple
advertisers. In RTB, the advertisers face three main challenges when optimizing
their bidding strategies, namely (i) estimating the utility (e.g., conversions,
clicks) of the ad impression, (ii) forecasting the market value (thus the cost)
of the given ad impression, and (iii) deciding the optimal bid for the given
auction based on the first two. Previous solutions assume the first two are
solved before addressing the bid optimization problem. However, these
challenges are strongly correlated and dealing with any individual problem
independently may not be globally optimal. In this paper, we propose Bidding
Machine, a comprehensive learning to bid framework, which consists of three
optimizers dealing with each challenge above, and as a whole, jointly optimizes
these three parts. We show that such a joint optimization would largely
increase the campaign effectiveness and the profit. From the learning
perspective, we show that the bidding machine can be updated smoothly with both
offline periodical batch or online sequential training schemes. Our extensive
offline empirical study and online A/B testing verify the high effectiveness of
the proposed bidding machine.