pwc-archive/papers-with-abstracts huggingface dataset:
Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and Perspectives
Authors: Jing Han, Zixing Zhang, NIcholas Cummins, Björn Schuller
Date: 2018-09-21 00:00:00
Tasks: Sentiment Analysis

Over the past few years, adversarial training has become an extremely active
research topic and has been successfully applied to various Artificial
Intelligence (AI) domains. As a potentially crucial technique for the
development of the next generation of emotional AI systems, we herein provide a
comprehensive overview of the application of adversarial training to affective
computing and sentiment analysis. Various representative adversarial training
algorithms are explained and discussed accordingly, aimed at tackling diverse
challenges associated with emotional AI systems. Further, we highlight a range
of potential future research directions. We expect that this overview will help
facilitate the development of adversarial training for affective computing and
sentiment analysis in both the academic and industrial communities.
Understanding Compressive Adversarial Privacy
Authors: Xiao Chen, Peter Kairouz, Ram Rajagopal
Date: 2018-09-21 00:00:00

Designing a data sharing mechanism without sacrificing too much privacy can
be considered as a game between data holders and malicious attackers. This
paper describes a compressive adversarial privacy framework that captures the
trade-off between the data privacy and utility. We characterize the optimal
data releasing mechanism through convex optimization when assuming that both
the data holder and attacker can only modify the data using linear
transformations. We then build a more realistic data releasing mechanism that
can rely on a nonlinear compression model while the attacker uses a neural
network. We demonstrate in a series of empirical applications that this
framework, consisting of compressive adversarial privacy, can preserve
sensitive information.
Constrained Exploration and Recovery from Experience Shaping
Authors: Tu-Hoa Pham, Giovanni De Magistris, Don Joven Agravante, Subhajit Chaudhury, Asim Munawar, Ryuki Tachibana
Date: 2018-09-21 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We consider the problem of reinforcement learning under safety requirements,
in which an agent is trained to complete a given task, typically formalized as
the maximization of a reward signal over time, while concurrently avoiding
undesirable actions or states, associated to lower rewards, or penalties. The
construction and balancing of different reward components can be difficult in
the presence of multiple objectives, yet is crucial for producing a satisfying
policy. For example, in reaching a target while avoiding obstacles, low
collision penalties can lead to reckless movements while high penalties can
discourage exploration. To circumvent this limitation, we examine the effect of
past actions in terms of safety to estimate which are acceptable or should be
avoided in the future. We then actively reshape the action space of the agent
during reinforcement learning, so that reward-driven exploration is constrained
within safety limits. We propose an algorithm enabling the learning of such
safety constraints in parallel with reinforcement learning and demonstrate its
effectiveness in terms of both task completion and training time.
Finite Sample Analysis of the GTD Policy Evaluation Algorithms in Markov Setting
Authors: Yue Wang, Wei Chen, Yu-Ting Liu, Zhi-Ming Ma, Tie-Yan Liu
Date: 2018-09-21 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In reinforcement learning (RL) , one of the key components is policy
evaluation, which aims to estimate the value function (i.e., expected long-term
accumulated reward) of a policy. With a good policy evaluation method, the RL
algorithms will estimate the value function more accurately and find a better
policy. When the state space is large or continuous \emph{Gradient-based
Temporal Difference(GTD)} policy evaluation algorithms with linear function
approximation are widely used. Considering that the collection of the
evaluation data is both time and reward consuming, a clear understanding of the
finite sample performance of the policy evaluation algorithms is very important
to reinforcement learning. Under the assumption that data are i.i.d. generated,
previous work provided the finite sample analysis of the GTD algorithms with
constant step size by converting them into convex-concave saddle point
problems. However, it is well-known that, the data are generated from Markov
processes rather than i.i.d. in RL problems.. In this paper, in the realistic
Markov setting, we derive the finite sample bounds for the general
convex-concave saddle point problems, and hence for the GTD algorithms. We have
the following discussions based on our bounds. (1) With variants of step size,
GTD algorithms converge. (2) The convergence rate is determined by the step
size, with the mixing time of the Markov process as the coefficient. The faster
the Markov processes mix, the faster the convergence. (3) We explain that the
experience replay trick is effective by improving the mixing property of the
Markov process. To the best of our knowledge, our analysis is the first to
provide finite sample bounds for the GTD algorithms in Markov setting.
Target Transfer Q-Learning and Its Convergence Analysis
Authors: Yue Wang, Qi Meng, Wei Cheng, Yuting Liug, Zhi-Ming Ma, Tie-Yan Liu
Date: 2018-09-21 00:00:00
Tasks: Q-Learning, Reinforcement Learning, Reinforcement Learning (RL), Transfer Learning

Q-learning is one of the most popular methods in Reinforcement Learning (RL).
Transfer Learning aims to utilize the learned knowledge from source tasks to
help new tasks to improve the sample complexity of the new tasks. Considering
that data collection in RL is both more time and cost consuming and Q-learning
converges slowly comparing to supervised learning, different kinds of transfer
RL algorithms are designed. However, most of them are heuristic with no
theoretical guarantee of the convergence rate. Therefore, it is important for
us to clearly understand when and how will transfer learning help RL method and
provide the theoretical guarantee for the improvement of the sample complexity.
In this paper, we propose to transfer the Q-function learned in the source task
to the target of the Q-learning in the new task when certain safe conditions
are satisfied. We call this new transfer Q-learning method target transfer
Q-Learning. The safe conditions are necessary to avoid the harm to the new
tasks and thus ensure the convergence of the algorithm. We study the
convergence rate of the target transfer Q-learning. We prove that if the two
tasks are similar with respect to the MDPs, the optimal Q-functions in the
source and new RL tasks are similar which means the error of the transferred
target Q-function in new MDP is small. Also, the convergence rate analysis
shows that the target transfer Q-Learning will converge faster than Q-learning
if the error of the transferred target Q-function is smaller than the current
Q-function in the new task. Based on our theoretical results, we design the
safe condition as the Bellman error of the transferred target Q-function is
less than the current Q-function. Our experiments are consistent with our
theoretical founding and verified the effectiveness of our proposed target
transfer Q-learning method.
Context-Aware Systems for Sequential Item Recommendation
Authors: Moin Nadeem, Dustin Stansbury, Shane Mooney
Date: 2018-09-21 00:00:00
Tasks: Recommendation Systems

Quizlet is the most popular online learning tool in the United States, and is
used by over 2/3 of high school students, and 1/2 of college students. With
more than 95% of Quizlet users reporting improved grades as a result, the
platform has become the de-facto tool used in millions of classrooms. In this
paper, we explore the task of recommending suitable content for a student to
study, given their prior interests, as well as what their peers are studying.
We propose a novel approach, i.e. Neural Educational Recommendation Engine
(NERE), to recommend educational content by leveraging student behaviors rather
than ratings. We have found that this approach better captures social factors
that are more aligned with learning. NERE is based on a recurrent neural
network that includes collaborative and content-based approaches for
recommendation, and takes into account any particular student's speed, mastery,
and experience to recommend the appropriate task. We train NERE by jointly
learning the user embeddings and content embeddings, and attempt to predict the
content embedding for the final timestamp. We also develop a confidence
estimator for our neural network, which is a crucial requirement for
productionizing this model. We apply NERE to Quizlet's proprietary dataset, and
present our results. We achieved an R^2 score of 0.81 in the content embedding
space, and a recall score of 54% on our 100 nearest neighbors. This vastly
exceeds the recall@100 score of 12% that a standard matrix-factorization
approach provides. We conclude with a discussion on how NERE will be deployed,
and position our work as one of the first educational recommender systems for
the K-12 space.
Neural network approach to classifying alarming student responses to online assessment
Authors: Christopher M. Ormerod, Amy E. Harris
Date: 2018-09-20 00:00:00

Automated scoring engines are increasingly being used to score the free-form
text responses that students give to questions. Such engines are not designed
to appropriately deal with responses that a human reader would find alarming
such as those that indicate an intention to self-harm or harm others, responses
that allude to drug abuse or sexual abuse or any response that would elicit
concern for the student writing the response. Our neural network models have
been designed to help identify these anomalous responses from a large
collection of typical responses that students give. The responses identified by
the neural network can be assessed for urgency, severity, and validity more
quickly by a team of reviewers than otherwise possible. Given the anomalous
nature of these types of responses, our goal is to maximize the chance of
flagging these responses for review given the constraint that only a fixed
percentage of responses can viably be assessed by a team of reviewers.
Permutation Invariant Gaussian Matrix Models
Authors: Sanjaye Ramgoolam
Date: 2018-09-20 00:00:00

Permutation invariant Gaussian matrix models were recently developed for applications in computational linguistics. A 5-parameter family of models was solved. In this paper, we use a representation theoretic approach to solve the general 13-parameter Gaussian model, which can be viewed as a zero-dimensional quantum field theory. We express the two linear and eleven quadratic terms in the action in terms of representation theoretic parameters. These parameters are coefficients of simple quadratic expressions in terms of appropriate linear combinations of the matrix variables transforming in specific irreducible representations of the symmetric group $S_D$ where $D$ is the size of the matrices. They allow the identification of constraints which ensure a convergent Gaussian measure and well-defined expectation values for polynomial functions of the random matrix at all orders. A graph-theoretic interpretation is known to allow the enumeration of permutation invariants of matrices at linear, quadratic and higher orders. We express the expectation values of all the quadratic graph-basis invariants and a selection of cubic and quartic invariants in terms of the representation theoretic parameters of the model.
Neural Speech Synthesis with Transformer Network
Authors: Naihan Li, Shujie Liu, Yanqing Liu, Sheng Zhao, Ming Liu, Ming Zhou
Date: 2018-09-19 00:00:00
Tasks: Decoder, Machine Translation, NMT, Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis

Although end-to-end neural text-to-speech (TTS) methods (such as Tacotron2)
are proposed and achieve state-of-the-art performance, they still suffer from
two problems: 1) low efficiency during training and inference; 2) hard to model
long dependency using current recurrent neural networks (RNNs). Inspired by the
success of Transformer network in neural machine translation (NMT), in this
paper, we introduce and adapt the multi-head attention mechanism to replace the
RNN structures and also the original attention mechanism in Tacotron2. With the
help of multi-head self-attention, the hidden states in the encoder and decoder
are constructed in parallel, which improves the training efficiency. Meanwhile,
any two inputs at different times are connected directly by self-attention
mechanism, which solves the long range dependency problem effectively. Using
phoneme sequences as input, our Transformer TTS network generates mel
spectrograms, followed by a WaveNet vocoder to output the final audio results.
Experiments are conducted to test the efficiency and performance of our new
network. For the efficiency, our Transformer TTS network can speed up the
training about 4.25 times faster compared with Tacotron2. For the performance,
rigorous human tests show that our proposed model achieves state-of-the-art
performance (outperforms Tacotron2 with a gap of 0.048) and is very close to
human quality (4.39 vs 4.44 in MOS).
Monolingual sentence matching for text simplification
Authors: Yonghui Huang, Yunhui Li, Yi Luan
Date: 2018-09-19 00:00:00
Tasks: Sentence, Text Simplification

This work improves monolingual sentence alignment for text simplification,
specifically for text in standard and simple Wikipedia. We introduce a
convolutional neural network structure to model similarity between two
sentences. Due to the limitation of available parallel corpora, the model is
trained in a semi-supervised way, by using the output of a knowledge-based high
performance aligning system. We apply the resulting similarity score to rescore
the knowledge-based output, and adapt the model by a small hand-aligned
dataset. Experiments show that both rescoring and adaptation improve the
performance of knowledge-based method.
Language Identification with Deep Bottleneck Features
Authors: Zhanyu Ma, Hong Yu
Date: 2018-09-18 00:00:00
Tasks: Language Identification, Speech Language Identification, Transfer Learning

In this paper we proposed an end-to-end short utterances speech language identification(SLD) approach based on a Long Short Term Memory (LSTM) neural network which is special suitable for SLD application in intelligent vehicles. Features used for LSTM learning are generated by a transfer learning method. Bottle-neck features of a deep neural network (DNN) which are trained for mandarin acoustic-phonetic classification are used for LSTM training. In order to improve the SLD accuracy of short utterances a phase vocoder based time-scale modification(TSM) method is used to reduce and increase speech rated of the test utterance. By splicing the normal, speech rate reduced and increased utterances, we can extend length of test utterances so as to improved improved the performance of the SLD system. The experimental results on AP17-OLR database shows that the proposed methods can improve the performance of SLD, especially on short utterance with 1s and 3s durations.
Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design
Authors: Niranjan Srinivas, Andreas Krause, Sham M. Kakade, Matthias Seeger
Date: 2009-12-21 00:00:00
Tasks: Experimental Design

Many applications require optimizing an unknown, noisy function that is
expensive to evaluate. We formalize this task as a multi-armed bandit problem,
where the payoff function is either sampled from a Gaussian process (GP) or has
low RKHS norm. We resolve the important open problem of deriving regret bounds
for this setting, which imply novel convergence rates for GP optimization. We
analyze GP-UCB, an intuitive upper-confidence based algorithm, and bound its
cumulative regret in terms of maximal information gain, establishing a novel
connection between GP optimization and experimental design. Moreover, by
bounding the latter in terms of operator spectra, we obtain explicit sublinear
regret bounds for many commonly used covariance functions. In some important
cases, our bounds have surprisingly weak dependence on the dimensionality. In
our experiments on real sensor data, GP-UCB compares favorably with other
heuristical GP optimization approaches.
Object Detection from Scratch with Deep Supervision
Authors: Zhiqiang Shen, Zhuang Liu, Jianguo Li, Yu-Gang Jiang, Yurong Chen, xiangyang xue
Date: 2018-09-25 00:00:00
Tasks: General Classification, Object, object-detection, Object Detection

We propose Deeply Supervised Object Detectors (DSOD), an object detection
framework that can be trained from scratch. Recent advances in object detection
heavily depend on the off-the-shelf models pre-trained on large-scale
classification datasets like ImageNet and OpenImage. However, one problem is
that adopting pre-trained models from classification to detection task may
incur learning bias due to the different objective function and diverse
distributions of object categories. Techniques like fine-tuning on detection
task could alleviate this issue to some extent but are still not fundamental.
Furthermore, transferring these pre-trained models across discrepant domains
will be more difficult (e.g., from RGB to depth images). Thus, a better
solution to handle these critical problems is to train object detectors from
scratch, which motivates our proposed method. Previous efforts on this
direction mainly failed by reasons of the limited training data and naive
backbone network structures for object detection. In DSOD, we contribute a set
of design principles for learning object detectors from scratch. One of the key
principles is the deep supervision, enabled by layer-wise dense connections in
both backbone networks and prediction layers, plays a critical role in learning
good detectors from scratch. After involving several other principles, we build
our DSOD based on the single-shot detection framework (SSD). We evaluate our
method on PASCAL VOC 2007, 2012 and COCO datasets. DSOD achieves consistently
better results than the state-of-the-art methods with much more compact models.
Specifically, DSOD outperforms baseline method SSD on all three benchmarks,
while requiring only 1/2 parameters. We also observe that DSOD can achieve
comparable/slightly better results than Mask RCNN + FPN (under similar input
size) with only 1/3 parameters, using no extra data or pre-trained models.
A generalized Bayesian framework for the analysis of subscription based businesses
Authors: Rahul Madhavan, Ankit Baraskar
Date: 2017-04-12 00:00:00
Tasks: Decision Making, Form

We have created a framework for analyzing subscription based businesses in
terms of a unified metric which we call SCV (single customer value). The major
advance in this paper is to model customer churn as an exponential decay
variable, which directly follows from experimental data relating to
subscription based businesses. This Bayesian probabilistic model was used to
compute an expected value for the revenue contribution of a single user. We
obtain an exact closed-form solution for the constant churn model, and an
approximate closed-form solution for the exponential decay model. In addition,
we define a general methodology for decision making processes using sensitivity
analysis of the model equation, which we illustrate with a real-life case study
for a food based subscription business.
Semantic and structural image segmentation for prosthetic vision
Authors: Melani Sanchez-Garcia, Ruben Martinez-Cantin, Jose J. Guerrero
Date: 2018-09-25 00:00:00
Tasks: Image Segmentation, Object, object-detection, Object Detection, Object Recognition, Scene Recognition, Scene Understanding, Semantic Segmentation

Prosthetic vision is being applied to partially recover the retinal stimulation of visually impaired people. However, the phosphenic images produced by the implants have very limited information bandwidth due to the poor resolution and lack of color or contrast. The ability of object recognition and scene understanding in real environments is severely restricted for prosthetic users. Computer vision can play a key role to overcome the limitations and to optimize the visual information in the simulated prosthetic vision, improving the amount of information that is presented. We present a new approach to build a schematic representation of indoor environments for phosphene images. The proposed method combines a variety of convolutional neural networks for extracting and conveying relevant information about the scene such as structural informative edges of the environment and silhouettes of segmented objects. Experiments were conducted with normal sighted subjects with a Simulated Prosthetic Vision system. The results show good accuracy for object recognition and room identification tasks for indoor scenes using the proposed approach, compared to other image processing methods.
A Re-ranker Scheme for Integrating Large Scale NLU models
Authors: Chengwei Su, Rahul Gupta, Shankar Ananthakrishnan, Spyros Matsoukas
Date: 2018-09-25 00:00:00
Tasks: Natural Language Understanding

Large scale Natural Language Understanding (NLU) systems are typically
trained on large quantities of data, requiring a fast and scalable training
strategy. A typical design for NLU systems consists of domain-level NLU modules
(domain classifier, intent classifier and named entity recognizer). Hypotheses
(NLU interpretations consisting of various intent+slot combinations) from these
domain specific modules are typically aggregated with another downstream
component. The re-ranker integrates outputs from domain-level recognizers,
returning a scored list of cross domain hypotheses. An ideal re-ranker will
exhibit the following two properties: (a) it should prefer the most relevant
hypothesis for the given input as the top hypothesis and, (b) the
interpretation scores corresponding to each hypothesis produced by the
re-ranker should be calibrated. Calibration allows the final NLU interpretation
score to be comparable across domains. We propose a novel re-ranker strategy
that addresses these aspects, while also maintaining domain specific
modularity. We design optimization loss functions for such a modularized
re-ranker and present results on decreasing the top hypothesis error rate as
well as maintaining the model calibration. We also experiment with an extension
involving training the domain specific re-rankers on datasets curated
independently by each domain to allow further asynchronization. %The proposed
re-ranker design showcases the following: (i) improved NLU performance over an
unweighted aggregation strategy, (ii) cross-domain calibrated performance and,
(iii) support for use cases involving training each re-ranker on datasets
curated by each domain independently.