pwc-archive/papers-with-abstracts huggingface dataset:
NRC-Canada at SMM4H Shared Task: Classifying Tweets Mentioning Adverse Drug Reactions and Medication Intake
Authors: Svetlana Kiritchenko, Saif M. Mohammad, Jason Morin, Berry de Bruijn
Date: 2018-05-11 00:00:00
Tasks: General Classification, Task 2, Word Embeddings

Our team, NRC-Canada, participated in two shared tasks at the AMIA-2017
Workshop on Social Media Mining for Health Applications (SMM4H): Task 1 -
classification of tweets mentioning adverse drug reactions, and Task 2 -
classification of tweets describing personal medication intake. For both tasks,
we trained Support Vector Machine classifiers using a variety of surface-form,
sentiment, and domain-specific features. With nine teams participating in each
task, our submissions ranked first on Task 1 and third on Task 2. Handling
considerable class imbalance proved crucial for Task 1. We applied an
under-sampling technique to reduce class imbalance (from about 1:10 to 1:2).
Standard n-gram features, n-grams generalized over domain terms, as well as
general-domain and domain-specific word embeddings had a substantial impact on
the overall performance in both tasks. On the other hand, including sentiment
lexicon features did not result in any improvement.
MAT: A Multi-strength Adversarial Training Method to Mitigate Adversarial Attacks
Authors: Chang Song, Hsin-Pai Cheng, Huanrui Yang, Sicheng Li, Chunpeng Wu, Qing Wu, Hai Li, Yiran Chen
Date: 2017-05-27 00:00:00

Some recent works revealed that deep neural networks (DNNs) are vulnerable to
so-called adversarial attacks where input examples are intentionally perturbed
to fool DNNs. In this work, we revisit the DNN training process that includes
adversarial examples into the training dataset so as to improve DNN's
resilience to adversarial attacks, namely, adversarial training. Our
experiments show that different adversarial strengths, i.e., perturbation
levels of adversarial examples, have different working zones to resist the
attack. Based on the observation, we propose a multi-strength adversarial
training method (MAT) that combines the adversarial training examples with
different adversarial strengths to defend adversarial attacks. Two training
structures - mixed MAT and parallel MAT - are developed to facilitate the
tradeoffs between training time and memory occupation. Our results show that
MAT can substantially minimize the accuracy degradation of deep learning
systems to adversarial attacks on MNIST, CIFAR-10, CIFAR-100, and SVHN.
Sentiment Composition of Words with Opposing Polarities
Authors: Svetlana Kiritchenko, Saif M. Mohammad
Date: 2018-05-11 00:00:00
Proceeding: NAACL 2016 6

In this paper, we explore sentiment composition in phrases that have at least
one positive and at least one negative word---phrases like 'happy accident' and
'best winter break'. We compiled a dataset of such opposing polarity phrases
and manually annotated them with real-valued scores of sentiment association.
Using this dataset, we analyze the linguistic patterns present in opposing
polarity phrases. Finally, we apply several unsupervised and supervised
techniques of sentiment composition to determine their efficacy on this
dataset. Our best system, which incorporates information from the phrase's
constituents, their parts of speech, their sentiment association scores, and
their embedding vectors, obtains an accuracy of over 80% on the opposing
polarity phrases.
Online Bandit Linear Optimization: A Study
Authors: Vikram Mullachery, Samarth Tiwari
Date: 2018-05-11 00:00:00

This article introduces the concepts around Online Bandit Linear Optimization
and explores an efficient setup called SCRiBLe (Self-Concordant Regularization
in Bandit Learning) created by Abernethy et. al.\cite{abernethy}. The SCRiBLe
setup and algorithm yield a $O(\sqrt{T})$ regret bound and polynomial run time
complexity bound on the dimension of the input space. In this article we build
up to the bandit linear optimization case and study SCRiBLe.
Phase Transitions and a Model Order Selection Criterion for Spectral Graph Clustering
Authors: Pin-Yu Chen, Alfred O. Hero
Date: 2016-04-11 00:00:00
Tasks: Clustering, Graph Clustering, Model Selection, Spectral Graph Clustering

One of the longstanding open problems in spectral graph clustering (SGC) is
the so-called model order selection problem: automated selection of the correct
number of clusters. This is equivalent to the problem of finding the number of
connected components or communities in an undirected graph. We propose
automated model order selection (AMOS), a solution to the SGC model selection
problem under a random interconnection model (RIM) using a novel selection
criterion that is based on an asymptotic phase transition analysis. AMOS can
more generally be applied to discovering hidden block diagonal structure in
symmetric non-negative matrices. Numerical experiments on simulated graphs
validate the phase transition analysis, and real-world network data is used to
validate the performance of the proposed model selection procedure.
A volumetric deep Convolutional Neural Network for simulation of mock dark matter halo catalogues
Authors: Philippe Berger, George Stein
Date: 2018-05-11 00:00:00

For modern large-scale structure survey techniques it has become standard
practice to test data analysis pipelines on large suites of mock simulations, a
task which is currently prohibitively expensive for full N-body simulations.
Instead of calculating this costly gravitational evolution, we have trained a
three-dimensional deep Convolutional Neural Network (CNN) to identify dark
matter protohalos directly from the cosmological initial conditions. Training
on halo catalogues from the Peak Patch semi-analytic code, we test various CNN
architectures and find they generically achieve a Dice coefficient of ~92% in
only 24 hours of training. We present a simple and fast geometric halo finding
algorithm to extract halos from this powerful pixel-wise binary classifier and
find that the predicted catalogues match the mass function and power spectra of
the ground truth simulations to within ~10%. We investigate the effect of
long-range tidal forces on an object-by-object basis and find that the
network's predictions are consistent with the non-linear ellipsoidal collapse
equations used explicitly by the Peak Patch algorithm.
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems
Authors: Svetlana Kiritchenko, Saif M. Mohammad
Date: 2018-05-11 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Sentiment Analysis, Vocal Bursts Valence Prediction

Automatic machine learning systems can inadvertently accentuate and
perpetuate inappropriate human biases. Past work on examining inappropriate
biases has largely focused on just individual systems. Further, there is no
benchmark dataset for examining inappropriate biases in systems. Here for the
first time, we present the Equity Evaluation Corpus (EEC), which consists of
8,640 English sentences carefully chosen to tease out biases towards certain
races and genders. We use the dataset to examine 219 automatic sentiment
analysis systems that took part in a recent shared task, SemEval-2018 Task 1
'Affect in Tweets'. We find that several of the systems show statistically
significant bias; that is, they consistently provide slightly higher sentiment
intensity predictions for one race or one gender. We make the EEC freely
available.
On the Direction of Discrimination: An Information-Theoretic Analysis of Disparate Impact in Machine Learning
Authors: Hao Wang, Berk Ustun, Flavio P. Calmon
Date: 2018-01-16 00:00:00
Tasks: Attribute, BIG-bench Machine Learning, Binary Classification

In the context of machine learning, disparate impact refers to a form of
systematic discrimination whereby the output distribution of a model depends on
the value of a sensitive attribute (e.g., race or gender). In this paper, we
propose an information-theoretic framework to analyze the disparate impact of a
binary classification model. We view the model as a fixed channel, and quantify
disparate impact as the divergence in output distributions over two groups. Our
aim is to find a correction function that can perturb the input distributions
of each group to align their output distributions. We present an optimization
problem that can be solved to obtain a correction function that will make the
output distributions statistically indistinguishable. We derive closed-form
expressions to efficiently compute the correction function, and demonstrate the
benefits of our framework on a recidivism prediction problem based on the
ProPublica COMPAS dataset.
RNN-based counterfactual prediction, with an application to homestead policy and public schooling
Authors: Jason Poulos, Shuxi Zeng
Date: 2017-12-10 00:00:00
Tasks: Causal Inference, counterfactual, Counterfactual Inference, Time Series Analysis, Time Series Prediction

This paper proposes a method for estimating the effect of a policy intervention on an outcome over time. We train recurrent neural networks (RNNs) on the history of control unit outcomes to learn a useful representation for predicting future outcomes. The learned representation of control units is then applied to the treated units for predicting counterfactual outcomes. RNNs are specifically structured to exploit temporal dependencies in panel data, and are able to learn negative and nonlinear interactions between control unit outcomes. We apply the method to the problem of estimating the long-run impact of U.S. homestead policy on public school spending.
Augmented Skeleton Space Transfer for Depth-based Hand Pose Estimation
Authors: Seungryul Baek, Kwang In Kim, Tae-Kyun Kim
Date: 2018-05-11 00:00:00
Proceeding: CVPR 2018 6
Tasks: Hand Pose Estimation, Pose Estimation

Crucial to the success of training a depth-based 3D hand pose estimator (HPE)
is the availability of comprehensive datasets covering diverse camera
perspectives, shapes, and pose variations. However, collecting such annotated
datasets is challenging. We propose to complete existing databases by
generating new database entries. The key idea is to synthesize data in the
skeleton space (instead of doing so in the depth-map space) which enables an
easy and intuitive way of manipulating data entries. Since the skeleton entries
generated in this way do not have the corresponding depth map entries, we
exploit them by training a separate hand pose generator (HPG) which synthesizes
the depth map from the skeleton entries. By training the HPG and HPE in a
single unified optimization framework enforcing that 1) the HPE agrees with the
paired depth and skeleton entries; and 2) the HPG-HPE combination satisfies the
cyclic consistency (both the input and the output of HPG-HPE are skeletons)
observed via the newly generated unpaired skeletons, our algorithm constructs a
HPE which is robust to variations that go beyond the coverage of the existing
database. Our training algorithm adopts the generative adversarial networks
(GAN) training process. As a by-product, we obtain a hand pose discriminator
(HPD) that is capable of picking out realistic hand poses. Our algorithm
exploits this capability to refine the initial skeleton estimates in testing,
further improving the accuracy. We test our algorithm on four challenging
benchmark datasets (ICVL, MSRA, NYU and Big Hand 2.2M datasets) and demonstrate
that our approach outperforms or is on par with state-of-the-art methods
quantitatively and qualitatively.
The unreasonable effectiveness of the forget gate
Authors: Jos van der Westhuizen, Joan Lasenby
Date: 2018-04-13 00:00:00

Given the success of the gated recurrent unit, a natural question is whether
all the gates of the long short-term memory (LSTM) network are necessary.
Previous research has shown that the forget gate is one of the most important
gates in the LSTM. Here we show that a forget-gate-only version of the LSTM
with chrono-initialized biases, not only provides computational savings but
outperforms the standard LSTM on multiple benchmark datasets and competes with
some of the best contemporary models. Our proposed network, the JANET, achieves
accuracies of 99% and 92.5% on the MNIST and pMNIST datasets, outperforming the
standard LSTM which yields accuracies of 98.5% and 91%.
Interactive Reinforcement Learning with Dynamic Reuse of Prior Knowledge from Human/Agent's Demonstration
Authors: Zhaodong Wang, Matthew E. Taylor
Date: 2018-05-11 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning has enjoyed multiple successes in recent years.
However, these successes typically require very large amounts of data before an
agent achieves acceptable performance. This paper introduces a novel way of
combating such requirements by leveraging existing (human or agent) knowledge.
In particular, this paper uses demonstrations from agents and humans, allowing
an untrained agent to quickly achieve high performance. We empirically compare
with, and highlight the weakness of, HAT and CHAT, methods of transferring
knowledge from a source agent/human to a target agent. This paper introduces an
effective transfer approach, DRoP, combining the offline knowledge
(demonstrations recorded before learning) with online confidence-based
performance analysis. DRoP dynamically involves the demonstrator's knowledge,
integrating it into the reinforcement learning agent's online learning loop to
achieve efficient and robust learning.
Non-Stationary Texture Synthesis by Adversarial Expansion
Authors: Yang Zhou, Zhen Zhu, Xiang Bai, Dani Lischinski, Daniel Cohen-Or, Hui Huang
Date: 2018-05-11 00:00:00
Tasks: Generative Adversarial Network, Texture Synthesis

The real world exhibits an abundance of non-stationary textures. Examples
include textures with large-scale structures, as well as spatially variant and
inhomogeneous textures. While existing example-based texture synthesis methods
can cope well with stationary textures, non-stationary textures still pose a
considerable challenge, which remains unresolved. In this paper, we propose a
new approach for example-based non-stationary texture synthesis. Our approach
uses a generative adversarial network (GAN), trained to double the spatial
extent of texture blocks extracted from a specific texture exemplar. Once
trained, the fully convolutional generator is able to expand the size of the
entire exemplar, as well as of any of its sub-blocks. We demonstrate that this
conceptually simple approach is highly effective for capturing large-scale
structures, as well as other non-stationary attributes of the input exemplar.
As a result, it can cope with challenging textures, which, to our knowledge, no
other existing method can handle.
Information Assisted Dictionary Learning for fMRI data analysis
Authors: Manuel Morante, Yannis Kopsinis, Sergios Theodoridis, Athanassios Protopapas
Date: 2018-02-05 00:00:00
Tasks: Dictionary Learning, Experimental Design

In this paper, the task-related fMRI problem is treated in its matrix factorization formulation, focused on the Dictionary Learning (DL) approach. The new method allows the incorporation of a priori knowledge associated both with the experimental design as well as with available brain Atlases. Moreover, the proposed method can efficiently cope with uncertainties related to the HRF modeling. In addition, the proposed method bypasses one of the major drawbacks that are associated with DL methods; that is, the selection of the sparsity-related regularization parameters. In our formulation, an alternative sparsity promoting constraint is employed, that bears a direct relation to the number of voxels in the spatial maps. Hence, the related parameters can be tuned using information that is available from brain atlases. The proposed method is evaluated against several other popular techniques, including GLM. The obtained performance gains are reported via a novel realistic synthetic fMRI dataset as well as real data that are related to a challenging experimental design.
Non-stationary Stochastic Optimization under $L_{p,q}$-Variation Measures
Authors: Xi Chen, Yining Wang, Yu-Xiang Wang
Date: 2017-08-09 00:00:00
Tasks: Stochastic Optimization

We consider a non-stationary sequential stochastic optimization problem, in
which the underlying cost functions change over time under a variation budget
constraint. We propose an $L_{p,q}$-variation functional to quantify the
change, which yields less variation for dynamic function sequences whose
changes are constrained to short time periods or small subsets of input domain.
Under the $L_{p,q}$-variation constraint, we derive both upper and matching
lower regret bounds for smooth and strongly convex function sequences, which
generalize previous results in Besbes et al. (2015). Furthermore, we provide an
upper bound for general convex function sequences with noisy gradient feedback,
which matches the optimal rate as $p\to\infty$. Our results reveal some
surprising phenomena under this general variation functional, such as the curse
of dimensionality of the function domain. The key technical novelties in our
analysis include affinity lemmas that characterize the distance of the
minimizers of two convex functions with bounded Lp difference, and a cubic
spline based construction that attains matching lower bounds.
Extracting Action Sequences from Texts Based on Deep Reinforcement Learning
Authors: Wenfeng Feng, Hankz Hankui Zhuo, Subbarao Kambhampati
Date: 2018-03-07 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), World Knowledge

Extracting action sequences from natural language texts is challenging, as it
requires commonsense inferences based on world knowledge. Although there has
been work on extracting action scripts, instructions, navigation actions, etc.,
they require that either the set of candidate actions be provided in advance,
or that action descriptions are restricted to a specific form, e.g.,
description templates. In this paper, we aim to extract action sequences from
texts in free natural language, i.e., without any restricted templates,
provided the candidate set of actions is unknown. We propose to extract action
sequences from texts based on the deep reinforcement learning framework.
Specifically, we view "selecting" or "eliminating" words from texts as
"actions", and the texts associated with actions as "states". We then build
Q-networks to learn the policy of extracting actions and extract plans from the
labeled texts. We demonstrate the effectiveness of our approach on several
datasets with comparison to state-of-the-art approaches, including online
experiments interacting with humans.
Bootstrapping Multilingual Intent Models via Machine Translation for Dialog Automation
Authors: Nicholas Ruiz, Srinivas Bangalore, John Chen
Date: 2018-05-11 00:00:00
Tasks: Machine Translation, Translation

With the resurgence of chat-based dialog systems in consumer and enterprise
applications, there has been much success in developing data-driven and
rule-based natural language models to understand human intent. Since these
models require large amounts of data and in-domain knowledge, expanding an
equivalent service into new markets is disrupted by language barriers that
inhibit dialog automation.
  This paper presents a user study to evaluate the utility of out-of-the-box
machine translation technology to (1) rapidly bootstrap multilingual spoken
dialog systems and (2) enable existing human analysts to understand foreign
language utterances. We additionally evaluate the utility of machine
translation in human assisted environments, where a portion of the traffic is
processed by analysts. In English->Spanish experiments, we observe a high
potential for dialog automation, as well as the potential for human analysts to
process foreign language utterances with high accuracy.
Improved Predictive Models for Acute Kidney Injury with IDEAs: Intraoperative Data Embedded Analytics
Authors: Lasith Adhikari, Tezcan Ozrazgat-Baslanti, Paul Thottakkara, Ashkan Ebadi, Amir Motaei, Parisa Rashidi, Xiaolin Li, Azra Bihorac
Date: 2018-05-11 00:00:00
Tasks: Time Series, Time Series Analysis

Acute kidney injury (AKI) is a common and serious complication after a
surgery which is associated with morbidity and mortality. The majority of
existing perioperative AKI risk score prediction models are limited in their
generalizability and do not fully utilize the physiological intraoperative
time-series data. Thus, there is a need for intelligent, accurate, and robust
systems, able to leverage information from large-scale data to predict
patient's risk of developing postoperative AKI. A retrospective single-center
cohort of 2,911 adult patients who underwent surgery at the University of
Florida Health has been used for this study. We used machine learning and
statistical analysis techniques to develop perioperative models to predict the
risk of AKI (risk during the first 3 days, 7 days, and until the discharge day)
before and after the surgery. In particular, we examined the improvement in
risk prediction by incorporating three intraoperative physiologic time series
data, i.e., mean arterial blood pressure, minimum alveolar concentration, and
heart rate. For an individual patient, the preoperative model produces a
probabilistic AKI risk score, which will be enriched by integrating
intraoperative statistical features through a machine learning stacking
approach inside a random forest classifier. We compared the performance of our
model based on the area under the receiver operating characteristics curve
(AUROC), accuracy and net reclassification improvement (NRI). The predictive
performance of the proposed model is better than the preoperative data only
model. For AKI-7day outcome: The AUC was 0.86 (accuracy was 0.78) in the
proposed model, while the preoperative AUC was 0.84 (accuracy 0.76).
Furthermore, with the integration of intraoperative features, we were able to
classify patients who were misclassified in the preoperative model.
Cross-lingual Document Retrieval using Regularized Wasserstein Distance
Authors: Georgios Balikas, Charlotte Laclau, Ievgen Redko, Massih-Reza Amini
Date: 2018-05-11 00:00:00
Tasks: Information Retrieval, Retrieval

Many information retrieval algorithms rely on the notion of a good distance
that allows to efficiently compare objects of different nature. Recently, a new
promising metric called Word Mover's Distance was proposed to measure the
divergence between text passages. In this paper, we demonstrate that this
metric can be extended to incorporate term-weighting schemes and provide more
accurate and computationally efficient matching between documents using
entropic regularization. We evaluate the benefits of both extensions in the
task of cross-lingual document retrieval (CLDR). Our experimental results on
eight CLDR problems suggest that the proposed methods achieve remarkable
improvements in terms of Mean Reciprocal Rank compared to several baselines.