pwc-archive/papers-with-abstracts huggingface dataset:
Generalizing Word Embeddings using Bag of Subwords
Authors: Jinman Zhao, Sidharth Mudgal, YIngyu Liang
Date: 2018-09-12 00:00:00
Proceeding: EMNLP 2018 10
Tasks: TAG, Word Embeddings, Word Similarity

We approach the problem of generalizing pre-trained word embeddings beyond
fixed-size vocabularies without using additional contextual information. We
propose a subword-level word vector generation model that views words as bags
of character $n$-grams. The model is simple, fast to train and provides good
vectors for rare or unseen words. Experiments show that our model achieves
state-of-the-art performances in English word similarity task and in joint
prediction of part-of-speech tag and morphosyntactic attributes in 23
languages, suggesting our model's ability in capturing the relationship between
words' textual representations and their embeddings.
An Ontology-Based Artificial Intelligence Model for Medicine Side-Effect Prediction: Taking Traditional Chinese Medicine as An Example
Authors: Yuanzhe Yao, Zeheng Wang, Liang Li, Kun Lu, Runyu Liu, Zhiyuan Liu, Jing Yan
Date: 2018-09-12 00:00:00
Tasks: Prediction

In this work, an ontology-based model for AI-assisted medicine side-effect (SE) prediction is developed, where three main components, including the drug model, the treatment model, and the AI-assisted prediction model, of proposed model are presented. To validate the proposed model, an ANN structure is established and trained by two hundred and forty-two TCM prescriptions. These data are gathered and classified from the most famous ancient TCM book and more than one thousand SE reports, in which two ontology-based attributions, hot and cold, are introduced to evaluate whether the prescription will cause SE or not. The results preliminarily reveal that it is a relationship between the ontology-based attributions and the corresponding predicted indicator that can be learnt by AI for predicting the SE, which suggests the proposed model has a potential in AI-assisted SE prediction. However, it should be noted that, the proposed model highly depends on the sufficient clinic data, and hereby, much deeper exploration is important for enhancing the accuracy of the prediction.
A Fast Globally Linearly Convergent Algorithm for the Computation of Wasserstein Barycenters
Authors: Lei Yang, Jia Li, Defeng Sun, Kim-Chuan Toh
Date: 2018-09-12 00:00:00

We consider the problem of computing a Wasserstein barycenter for a set of discrete probability distributions with finite supports, which finds many applications in areas such as statistics, machine learning and image processing. When the support points of the barycenter are pre-specified, this problem can be modeled as a linear programming (LP) problem whose size can be extremely large. To handle this large-scale LP, we analyse the structure of its dual problem, which is conceivably more tractable and can be reformulated as a well-structured convex problem with 3 kinds of block variables and a coupling linear equality constraint. We then adapt a symmetric Gauss-Seidel based alternating direction method of multipliers (sGS-ADMM) to solve the resulting dual problem and establish its global convergence and global linear convergence rate. As a critical component for efficient computation, we also show how all the subproblems involved can be solved exactly and efficiently. This makes our method suitable for computing a Wasserstein barycenter on a large-scale data set, without introducing an entropy regularization term as is commonly practiced. In addition, our sGS-ADMM can be used as a subroutine in an alternating minimization method to compute a barycenter when its support points are not pre-specified. Numerical results on synthetic data sets and image data sets demonstrate that our method is highly competitive for solving large-scale Wasserstein barycenter problems, in comparison to two existing representative methods and the commercial software Gurobi.
Sampled in Pairs and Driven by Text: A New Graph Embedding Framework
Authors: Liheng Chen, Yanru Qu, Zhenghui Wang, Lin Qiu, Wei-Nan Zhang, Ken Chen, Shaodian Zhang, Yong Yu
Date: 2018-09-12 00:00:00
Tasks: Graph Embedding, Link Prediction

In graphs with rich texts, incorporating textual information with structural information would benefit constructing expressive graph embeddings. Among various graph embedding models, random walk (RW)-based is one of the most popular and successful groups. However, it is challenged by two issues when applied on graphs with rich texts: (i) sampling efficiency: deriving from the training objective of RW-based models (e.g., DeepWalk and node2vec), we show that RW-based models are likely to generate large amounts of redundant training samples due to three main drawbacks. (ii) text utilization: these models have difficulty in dealing with zero-shot scenarios where graph embedding models have to infer graph structures directly from texts. To solve these problems, we propose a novel framework, namely Text-driven Graph Embedding with Pairs Sampling (TGE-PS). TGE-PS uses Pairs Sampling (PS) to improve the sampling strategy of RW, being able to reduce ~99% training samples while preserving competitive performance. TGE-PS uses Text-driven Graph Embedding (TGE), an inductive graph embedding approach, to generate node embeddings from texts. Since each node contains rich texts, TGE is able to generate high-quality embeddings and provide reasonable predictions on existence of links to unseen nodes. We evaluate TGE-PS on several real-world datasets, and experiment results demonstrate that TGE-PS produces state-of-the-art results on both traditional and zero-shot link prediction tasks.
Safe Exploration in Markov Decision Processes with Time-Variant Safety using Spatio-Temporal Gaussian Process
Authors: Akifumi Wachi, Hiroshi Kajino, Asim Munawar
Date: 2018-09-12 00:00:00
Tasks: Reinforcement Learning, Robot Navigation, Safe Exploration

In many real-world applications (e.g., planetary exploration, robot
navigation), an autonomous agent must be able to explore a space with
guaranteed safety. Most safe exploration algorithms in the field of
reinforcement learning and robotics have been based on the assumption that the
safety features are a priori known and time-invariant. This paper presents a
learning algorithm called ST-SafeMDP for exploring Markov decision processes
(MDPs) that is based on the assumption that the safety features are a priori
unknown and time-variant. In this setting, the agent explores MDPs while
constraining the probability of entering unsafe states defined by a safety
function being below a threshold. The unknown and time-variant safety values
are modeled using a spatio-temporal Gaussian process. However, there remains an
issue that an agent may have no viable action in a shrinking true safe space.
To address this issue, we formulate a problem maximizing the cumulative number
of safe states in the worst case scenario with respect to future observations.
The effectiveness of this approach was demonstrated in two simulation settings,
including one using real lunar terrain data.
EEG-based video identification using graph signal modeling and graph convolutional neural network
Authors: Soobeom Jang, Seong-Eun Moon, Jong-Seok Lee
Date: 2018-09-12 00:00:00
Tasks: EEG, Electroencephalogram (EEG)

This paper proposes a novel graph signal-based deep learning method for
electroencephalography (EEG) and its application to EEG-based video
identification. We present new methods to effectively represent EEG data as
signals on graphs, and learn them using graph convolutional neural networks.
Experimental results for video identification using EEG responses obtained
while watching videos show the effectiveness of the proposed approach in
comparison to existing methods. Effective schemes for graph signal
representation of EEG are also discussed.
Ensemble of Convolutional Neural Networks for Automatic Grading of Diabetic Retinopathy and Macular Edema
Authors: Avinash Kori, Sai Saketh Chennamsetty, Mohammed Safwan K. P., Varghese Alex
Date: 2018-09-12 00:00:00
Tasks: Transfer Learning

In this manuscript, we automate the procedure of grading of diabetic
retinopathy and macular edema from fundus images using an ensemble of
convolutional neural networks. The availability of limited amount of labeled
data to perform supervised learning was circumvented by using transfer learning
approach. The models in the ensemble were pre-trained on a large dataset
comprising natural images and were later fine-tuned with the limited data for
the task of choice. For an image, the ensemble of classifiers generate multiple
predictions, and a max-voting based approach was utilized to attain the final
grade of the anomaly in the image. For the task of grading DR, on the test data
(n=56), the ensemble achieved an accuracy of 83.9\%, while for the task for
grading macular edema the network achieved an accuracy of 95.45% (n=44).
On Markov Chain Gradient Descent
Authors: Tao Sun, Yuejiao Sun, Wotao Yin
Date: 2018-09-12 00:00:00
Proceeding: NeurIPS 2018 12

Stochastic gradient methods are the workhorse (algorithms) of large-scale
optimization problems in machine learning, signal processing, and other
computational sciences and engineering. This paper studies Markov chain
gradient descent, a variant of stochastic gradient descent where the random
samples are taken on the trajectory of a Markov chain. Existing results of this
method assume convex objectives and a reversible Markov chain and thus have
their limitations. We establish new non-ergodic convergence under wider step
sizes, for nonconvex problems, and for non-reversible finite-state Markov
chains. Nonconvexity makes our method applicable to broader problem classes.
Non-reversible finite-state Markov chains, on the other hand, can mix
substatially faster. To obtain these results, we introduce a new technique that
varies the mixing levels of the Markov chains. The reported numerical results
validate our contributions.
Automatic, Personalized, and Flexible Playlist Generation using Reinforcement Learning
Authors: Shun-Yao Shih, Heng-Yu Chi
Date: 2018-09-12 00:00:00
Tasks: Diversity, Language Modeling, Language Modelling, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Songs can be well arranged by professional music curators to form a riveting
playlist that creates engaging listening experiences. However, it is
time-consuming for curators to timely rearrange these playlists for fitting
trends in future. By exploiting the techniques of deep learning and
reinforcement learning, in this paper, we consider music playlist generation as
a language modeling problem and solve it by the proposed attention language
model with policy gradient. We develop a systematic and interactive approach so
that the resulting playlists can be tuned flexibly according to user
preferences. Considering a playlist as a sequence of words, we first train our
attention RNN language model on baseline recommended playlists. By optimizing
suitable imposed reward functions, the model is thus refined for corresponding
preferences. The experimental results demonstrate that our approach not only
generates coherent playlists automatically but is also able to flexibly
recommend personalized playlists for diversity, novelty and freshness.
Hyperspectral Image Classification in the Presence of Noisy Labels
Authors: Junjun Jiang, Jiayi Ma, Zheng Wang, Chen Chen, Xian-Ming Liu
Date: 2018-09-12 00:00:00
Tasks: Classification, General Classification, Hyperspectral Image Classification, image-classification, Image Classification

Label information plays an important role in supervised hyperspectral image
classification problem. However, current classification methods all ignore an
important and inevitable problem---labels may be corrupted and collecting clean
labels for training samples is difficult, and often impractical. Therefore, how
to learn from the database with noisy labels is a problem of great practical
importance. In this paper, we study the influence of label noise on
hyperspectral image classification, and develop a random label propagation
algorithm (RLPA) to cleanse the label noise. The key idea of RLPA is to exploit
knowledge (e.g., the superpixel based spectral-spatial constraints) from the
observed hyperspectral images and apply it to the process of label propagation.
Specifically, RLPA first constructs a spectral-spatial probability transfer
matrix (SSPTM) that simultaneously considers the spectral similarity and
superpixel based spatial information. It then randomly chooses some training
samples as "clean" samples and sets the rest as unlabeled samples, and
propagates the label information from the "clean" samples to the rest unlabeled
samples with the SSPTM. By repeating the random assignment (of "clean" labeled
samples and unlabeled samples) and propagation, we can obtain multiple labels
for each training sample. Therefore, the final propagated label can be
calculated by a majority vote algorithm. Experimental studies show that RLPA
can reduce the level of noisy label and demonstrates the advantages of our
proposed method over four major classifiers with a significant margin---the
gains in terms of the average OA, AA, Kappa are impressive, e.g., 9.18%, 9.58%,
and 0.1043. The Matlab source code is available at
https://github.com/junjun-jiang/RLPA
Convolutional Neural Network Approach for EEG-based Emotion Recognition using Brain Connectivity and its Spatial Information
Authors: Seong-Eun Moon, Soobeom Jang, Jong-Seok Lee
Date: 2018-09-12 00:00:00
Tasks: Deep Learning, EEG, Electroencephalogram (EEG), Emotion Recognition

Emotion recognition based on electroencephalography (EEG) has received
attention as a way to implement human-centric services. However, there is still
much room for improvement, particularly in terms of the recognition accuracy.
In this paper, we propose a novel deep learning approach using convolutional
neural networks (CNNs) for EEG-based emotion recognition. In particular, we
employ brain connectivity features that have not been used with deep learning
models in previous studies, which can account for synchronous activations of
different brain regions. In addition, we develop a method to effectively
capture asymmetric brain activity patterns that are important for emotion
recognition. Experimental results confirm the effectiveness of our approach.
Temporal Pattern Attention for Multivariate Time Series Forecasting
Authors: Shun-Yao Shih, Fan-Keng Sun, Hung-Yi Lee
Date: 2018-09-12 00:00:00
Tasks: Multivariate Time Series Forecasting, Time Series, Time Series Analysis, Time Series Forecasting, Univariate Time Series Forecasting

Forecasting multivariate time series data, such as prediction of electricity consumption, solar power production, and polyphonic piano pieces, has numerous valuable applications. However, complex and non-linear interdependencies between time steps and series complicate the task. To obtain accurate prediction, it is crucial to model long-term dependency in time series data, which can be achieved to some good extent by recurrent neural network (RNN) with attention mechanism. Typical attention mechanism reviews the information at each previous time step and selects the relevant information to help generate the outputs, but it fails to capture the temporal patterns across multiple time steps. In this paper, we propose to use a set of filters to extract time-invariant temporal patterns, which is similar to transforming time series data into its "frequency domain". Then we proposed a novel attention mechanism to select relevant time series, and use its "frequency domain" information for forecasting. We applied the proposed model on several real-world tasks and achieved state-of-the-art performance in all of them with only one exception.
Multimodal neural pronunciation modeling for spoken languages with logographic origin
Authors: Minh Nguyen, Gia H. Ngo, Nancy F. Chen
Date: 2018-09-12 00:00:00
Proceeding: EMNLP 2018 10

Graphemes of most languages encode pronunciation, though some are more
explicit than others. Languages like Spanish have a straightforward mapping
between its graphemes and phonemes, while this mapping is more convoluted for
languages like English. Spoken languages such as Cantonese present even more
challenges in pronunciation modeling: (1) they do not have a standard written
form, (2) the closest graphemic origins are logographic Han characters, of
which only a subset of these logographic characters implicitly encodes
pronunciation. In this work, we propose a multimodal approach to predict the
pronunciation of Cantonese logographic characters, using neural networks with a
geometric representation of logographs and pronunciation of cognates in
historically related languages. The proposed framework improves performance by
18.1% and 25.0% respective to unimodal and multimodal baselines.
Zoom: SSD-based Vector Search for Optimizing Accuracy, Latency and Memory
Authors: Minjia Zhang, Yuxiong He
Date: 2018-09-11 00:00:00
Tasks: Information Retrieval, Retrieval

With the advancement of machine learning and deep learning, vector search
becomes instrumental to many information retrieval systems, to search and find
best matches to user queries based on their semantic similarities.These online
services require the search architecture to be both effective with high
accuracy and efficient with low latency and memory footprint, which existing
work fails to offer. We develop, Zoom, a new vector search solution that
collaboratively optimizes accuracy, latency and memory based on a multiview
approach. (1) A "preview" step generates a small set of good candidates,
leveraging compressed vectors in memory for reduced footprint and fast lookup.
(2) A "fullview" step on SSDs reranks those candidates with their full-length
vector, striking high accuracy. Our evaluation shows that, Zoom achieves an
order of magnitude improvements on efficiency while attaining equal or higher
accuracy, comparing with the state-of-the-art.
Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals
Authors: Andrew Cotter, Heinrich Jiang, Serena Wang, Taman Narayan, Maya Gupta, Seungil You, Karthik Sridharan
Date: 2018-09-11 00:00:00
Tasks: 2D Cyclist Detection, Fairness

We show that many machine learning goals, such as improved fairness metrics,
can be expressed as constraints on the model's predictions, which we call rate
constraints. We study the problem of training non-convex models subject to
these rate constraints (or any non-convex and non-differentiable constraints).
In the non-convex setting, the standard approach of Lagrange multipliers may
fail. Furthermore, if the constraints are non-differentiable, then one cannot
optimize the Lagrangian with gradient-based methods. To solve these issues, we
introduce the proxy-Lagrangian formulation. This new formulation leads to an
algorithm that produces a stochastic classifier by playing a two-player
non-zero-sum game solving for what we call a semi-coarse correlated
equilibrium, which in turn corresponds to an approximately optimal and feasible
solution to the constrained optimization problem. We then give a procedure
which shrinks the randomized solution down to one that is a mixture of at most
$m+1$ deterministic solutions, given $m$ constraints. This culminates in
algorithms that can solve non-convex constrained optimization problems with
possibly non-differentiable and non-convex constraints with theoretical
guarantees. We provide extensive experimental results enforcing a wide range of
policy goals including different fairness metrics, and other goals on accuracy,
coverage, recall, and churn.
Change-Point Detection on Hierarchical Circadian Models
Authors: Pablo Moreno-Muñoz, David Ramírez, Antonio Artés-Rodríguez
Date: 2018-09-11 00:00:00
Tasks: Change Point Detection

This paper addresses the problem of change-point detection on sequences of
high-dimensional and heterogeneous observations, which also possess a periodic
temporal structure. Due to the dimensionality problem, when the time between
change-points is on the order of the dimension of the model parameters, drifts
in the underlying distribution can be misidentified as changes. To overcome
this limitation, we assume that the observations lie in a lower-dimensional
manifold that admits a latent variable representation. In particular, we
propose a hierarchical model that is computationally feasible, widely
applicable to heterogeneous data and robust to missing instances. Additionally,
the observations' periodic dependencies are captured by non-stationary periodic
covariance functions. The proposed technique is particularly fitted to (and
motivated by) the problem of detecting changes in human behavior using
smartphones and its application to relapse detection in psychiatric patients.
Finally, we validate the technique on synthetic examples and we demonstrate its
utility in the detection of behavioral changes using real data acquired by
smartphones.
Clinically Deployed Distributed Magnetic Resonance Imaging Reconstruction: Application to Pediatric Knee Imaging
Authors: Michael J. Anderson, Jonathan I. Tamir, Javier S. Turek, Marcus T. Alley, Theodore L. Willke, Shreyas S. Vasanawala, Michael Lustig
Date: 2018-09-11 00:00:00
Tasks: compressed sensing

Magnetic resonance imaging is capable of producing volumetric images without
ionizing radiation. Nonetheless, long acquisitions lead to prohibitively long
exams. Compressed sensing (CS) can enable faster scanning via sub-sampling with
reduced artifacts. However, CS requires significantly higher reconstruction
computation, limiting current clinical applications to 2D/3D or
limited-resolution dynamic imaging. Here we analyze the practical limitations
to T2 Shuffling, a four-dimensional CS-based acquisition, which provides sharp
3D-isotropic-resolution and multi-contrast images in a single scan. Our
improvements to the pipeline on a single machine provide a 3x overall
reconstruction speedup, which allowed us to add algorithmic changes improving
image quality. Using four machines, we achieved additional 2.1x improvement
through distributed parallelization. Our solution reduced the reconstruction
time in the hospital to 90 seconds on a 4-node cluster, enabling its use
clinically. To understand the implications of scaling this application, we
simulated running our reconstructions with a multiple scanner setup typical in
hospitals.