pwc-archive/papers-with-abstracts huggingface dataset:
Learning in Memristive Neural Network Architectures using Analog Backpropagation Circuits
Authors: Olga Krestinskaya, Khaled Nabil Salama, Alex Pappachen James
Date: 2018-08-31 00:00:00

The on-chip implementation of learning algorithms would speed-up the training
of neural networks in crossbar arrays. The circuit level design and
implementation of backpropagation algorithm using gradient descent operation
for neural network architectures is an open problem. In this paper, we proposed
the analog backpropagation learning circuits for various memristive learning
architectures, such as Deep Neural Network (DNN), Binary Neural Network (BNN),
Multiple Neural Network (MNN), Hierarchical Temporal Memory (HTM) and
Long-Short Term Memory (LSTM). The circuit design and verification is done
using TSMC 180nm CMOS process models, and TiO2 based memristor models. The
application level validations of the system are done using XOR problem, MNIST
character and Yale face image databases
Retrieve-and-Read: Multi-task Learning of Information Retrieval and Reading Comprehension
Authors: Kyosuke Nishida, Itsumi Saito, Atsushi Otsuka, Hisako Asano, Junji Tomita
Date: 2018-08-31 00:00:00
Tasks: Information Retrieval, Multi-Task Learning, Reading Comprehension, Retrieval

This study considers the task of machine reading at scale (MRS) wherein,
given a question, a system first performs the information retrieval (IR) task
of finding relevant passages in a knowledge source and then carries out the
reading comprehension (RC) task of extracting an answer span from the passages.
Previous MRS studies, in which the IR component was trained without considering
answer spans, struggled to accurately find a small number of relevant passages
from a large set of passages. In this paper, we propose a simple and effective
approach that incorporates the IR and RC tasks by using supervised multi-task
learning in order that the IR component can be trained by considering answer
spans. Experimental results on the standard benchmark, answering SQuAD
questions using the full Wikipedia as the knowledge source, showed that our
model achieved state-of-the-art performance. Moreover, we thoroughly evaluated
the individual contributions of our model components with our new Japanese
dataset and SQuAD. The results showed significant improvements in the IR task
and provided a new perspective on IR for RC: it is effective to teach which
part of the passage answers the question rather than to give only a relevance
score to the whole passage.
Do Language Models Understand Anything? On the Ability of LSTMs to Understand Negative Polarity Items
Authors: Jaap Jumelet, Dieuwke Hupkes
Date: 2018-08-31 00:00:00
Proceeding: WS 2018 11
Tasks: Language Modeling, Language Modelling, Sentence

In this paper, we attempt to link the inner workings of a neural language
model to linguistic theory, focusing on a complex phenomenon well discussed in
formal linguis- tics: (negative) polarity items. We briefly discuss the leading
hypotheses about the licensing contexts that allow negative polarity items and
evaluate to what extent a neural language model has the ability to correctly
process a subset of such constructions. We show that the model finds a relation
between the licensing context and the negative polarity item and appears to be
aware of the scope of this context, which we extract from a parse tree of the
sentence. With this research, we hope to pave the way for other studies linking
formal linguistics to deep learning.
Explicit State Tracking with Semi-Supervision for Neural Dialogue Generation
Authors: Xisen Jin, Wenqiang Lei, Zhaochun Ren, Hongshen Chen, Shangsong Liang, Yihong Zhao, Dawei Yin
Date: 2018-08-31 00:00:00
Tasks: Decoder, Dialogue Generation, Dialogue State Tracking

The task of dialogue generation aims to automatically provide responses given
previous utterances. Tracking dialogue states is an important ingredient in
dialogue generation for estimating users' intention. However, the
\emph{expensive nature of state labeling} and the \emph{weak interpretability}
make the dialogue state tracking a challenging problem for both task-oriented
and non-task-oriented dialogue generation: For generating responses in
task-oriented dialogues, state tracking is usually learned from manually
annotated corpora, where the human annotation is expensive for training; for
generating responses in non-task-oriented dialogues, most of existing work
neglects the explicit state tracking due to the unlimited number of dialogue
states.
  In this paper, we propose the \emph{semi-supervised explicit dialogue state
tracker} (SEDST) for neural dialogue generation. To this end, our approach has
two core ingredients: \emph{CopyFlowNet} and \emph{posterior regularization}.
Specifically, we propose an encoder-decoder architecture, named
\emph{CopyFlowNet}, to represent an explicit dialogue state with a
probabilistic distribution over the vocabulary space. To optimize the training
procedure, we apply a posterior regularization strategy to integrate indirect
supervision. Extensive experiments conducted on both task-oriented and
non-task-oriented dialogue corpora demonstrate the effectiveness of our
proposed model. Moreover, we find that our proposed semi-supervised dialogue
state tracker achieves a comparable performance as state-of-the-art supervised
learning baselines in state tracking procedure.
Proximity Forest: An effective and scalable distance-based classifier for time series
Authors: Benjamin Lucas, Ahmed Shifaz, Charlotte Pelletier, Lachlan O'Neill, Nayyar Zaidi, Bart Goethals, Francois Petitjean, Geoffrey I. Webb
Date: 2018-08-31 00:00:00
Tasks: Attribute, Earth Observation, General Classification, Time Series, Time Series Analysis, Time Series Classification

Research into the classification of time series has made enormous progress in
the last decade. The UCR time series archive has played a significant role in
challenging and guiding the development of new learners for time series
classification. The largest dataset in the UCR archive holds 10 thousand time
series only; which may explain why the primary research focus has been in
creating algorithms that have high accuracy on relatively small datasets.
  This paper introduces Proximity Forest, an algorithm that learns accurate
models from datasets with millions of time series, and classifies a time series
in milliseconds. The models are ensembles of highly randomized Proximity Trees.
Whereas conventional decision trees branch on attribute values (and usually
perform poorly on time series), Proximity Trees branch on the proximity of time
series to one exemplar time series or another; allowing us to leverage the
decades of work into developing relevant measures for time series. Proximity
Forest gains both efficiency and accuracy by stochastic selection of both
exemplars and similarity measures.
  Our work is motivated by recent time series applications that provide orders
of magnitude more time series than the UCR benchmarks. Our experiments
demonstrate that Proximity Forest is highly competitive on the UCR archive: it
ranks among the most accurate classifiers while being significantly faster. We
demonstrate on a 1M time series Earth observation dataset that Proximity Forest
retains this accuracy on datasets that are many orders of magnitude greater
than those in the UCR repository, while learning its models at least 100,000
times faster than current state of the art models Elastic Ensemble and COTE.
Ensemble Sequence Level Training for Multimodal MT: OSU-Baidu WMT18 Multimodal Machine Translation System Report
Authors: Renjie Zheng, Yilin Yang, Mingbo Ma, Liang Huang
Date: 2018-08-31 00:00:00
Proceeding: WS 2018 10
Tasks: Decoder, Machine Translation, Multimodal Machine Translation, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Translation

This paper describes multimodal machine translation systems developed jointly
by Oregon State University and Baidu Research for WMT 2018 Shared Task on
multimodal translation. In this paper, we introduce a simple approach to
incorporate image information by feeding image features to the decoder side. We
also explore different sequence level training methods including scheduled
sampling and reinforcement learning which lead to substantial improvements. Our
systems ensemble several models using different architectures and training
methods and achieve the best performance for three subtasks: En-De and En-Cs in
task 1 and (En+De+Fr)-Cs task 1B.
On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data
Authors: Nan Lu, Gang Niu, Aditya Krishna Menon, Masashi Sugiyama
Date: 2018-08-31 00:00:00
Proceeding: ICLR 2019 5

Empirical risk minimization (ERM), with proper loss function and
regularization, is the common practice of supervised classification. In this
paper, we study training arbitrary (from linear to deep) binary classifier from
only unlabeled (U) data by ERM. We prove that it is impossible to estimate the
risk of an arbitrary binary classifier in an unbiased manner given a single set
of U data, but it becomes possible given two sets of U data with different
class priors. These two facts answer a fundamental question---what the minimal
supervision is for training any binary classifier from only U data. Following
these findings, we propose an ERM-based learning method from two sets of U
data, and then prove it is consistent. Experiments demonstrate the proposed
method could train deep models and outperform state-of-the-art methods for
learning from two sets of U data.
Learning to Describe Differences Between Pairs of Similar Images
Authors: Harsh Jhamtani, Taylor Berg-Kirkpatrick
Date: 2018-08-31 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Sentence

In this paper, we introduce the task of automatically generating text to
describe the differences between two similar images. We collect a new dataset
by crowd-sourcing difference descriptions for pairs of image frames extracted
from video-surveillance footage. Annotators were asked to succinctly describe
all the differences in a short paragraph. As a result, our novel dataset
provides an opportunity to explore models that align language and vision, and
capture visual salience. The dataset may also be a useful benchmark for
coherent multi-sentence generation. We perform a firstpass visual analysis that
exposes clusters of differing pixels as a proxy for object-level differences.
We propose a model that captures visual salience by using a latent variable to
align clusters of differing pixels with output sentences. We find that, for
both single-sentence generation and as well as multi-sentence generation, the
proposed model outperforms the models that use attention alone.
AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scale
Authors: Jiayu Du, Xingyu Na, Xuechen Liu, Hui Bu
Date: 2018-08-31 00:00:00
Tasks: Chinese Word Segmentation, speech-recognition, Speech Recognition, Transfer Learning

AISHELL-1 is by far the largest open-source speech corpus available for
Mandarin speech recognition research. It was released with a baseline system
containing solid training and testing pipelines for Mandarin ASR. In AISHELL-2,
1000 hours of clean read-speech data from iOS is published, which is free for
academic usage. On top of AISHELL-2 corpus, an improved recipe is developed and
released, containing key components for industrial applications, such as
Chinese word segmentation, flexible vocabulary expension and phone set
transformation etc. Pipelines support various state-of-the-art techniques, such
as time-delayed neural networks and Lattic-Free MMI objective funciton. In
addition, we also release dev and test data from other channels(Android and
Mic). For research community, we hope that AISHELL-2 corpus can be a solid
resource for topics like transfer learning and robust ASR. For industry, we
hope AISHELL-2 recipe can be a helpful reference for building meaningful
industrial systems and products.
Multi-Hop Knowledge Graph Reasoning with Reward Shaping
Authors: Xi Victoria Lin, Richard Socher, Caiming Xiong
Date: 2018-08-31 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Knowledge Graph Embeddings, Knowledge Graphs, Reinforcement Learning, Reinforcement Learning (RL)

Multi-hop reasoning is an effective approach for query answering (QA) over
incomplete knowledge graphs (KGs). The problem can be formulated in a
reinforcement learning (RL) setup, where a policy-based agent sequentially
extends its inference path until it reaches a target. However, in an incomplete
KG environment, the agent receives low-quality rewards corrupted by false
negatives in the training data, which harms generalization at test time.
Furthermore, since no golden action sequence is used for training, the agent
can be misled by spurious search trajectories that incidentally lead to the
correct answer. We propose two modeling advances to address both issues: (1) we
reduce the impact of false negative supervision by adopting a pretrained
one-hop embedding model to estimate the reward of unobserved facts; (2) we
counter the sensitivity to spurious paths of on-policy RL by forcing the agent
to explore a diverse set of paths using randomly generated edge masks. Our
approach significantly improves over existing path-based KGQA models on several
benchmark datasets and is comparable or better than embedding-based models.
Multi-Cell Multi-Task Convolutional Neural Networks for Diabetic Retinopathy Grading
Authors: Kang Zhou, Zaiwang Gu, Wen Liu, Weixin Luo, Jun Cheng, Shenghua Gao, Jiang Liu
Date: 2018-08-31 00:00:00
Tasks: Diabetic Retinopathy Grading, General Classification, Multi-Task Learning

Diabetic Retinopathy (DR) is a non-negligible eye disease among patients with
Diabetes Mellitus, and automatic retinal image analysis algorithm for the DR
screening is in high demand. Considering the resolution of retinal image is
very high, where small pathological tissues can be detected only with large
resolution image and large local receptive field are required to identify those
late stage disease, but directly training a neural network with very deep
architecture and high resolution image is both time computational expensive and
difficult because of gradient vanishing/exploding problem, we propose a
\textbf{Multi-Cell} architecture which gradually increases the depth of deep
neural network and the resolution of input image, which both boosts the
training time but also improves the classification accuracy. Further,
considering the different stages of DR actually progress gradually, which means
the labels of different stages are related. To considering the relationships of
images with different stages, we propose a \textbf{Multi-Task} learning
strategy which predicts the label with both classification and regression.
Experimental results on the Kaggle dataset show that our method achieves a
Kappa of 0.841 on test set which is the 4-th rank of all state-of-the-arts
methods. Further, our Multi-Cell Multi-Task Convolutional Neural Networks
(M$^2$CNN) solution is a general framework, which can be readily integrated
with many other deep neural network architectures.
Speaker Fluency Level Classification Using Machine Learning Techniques
Authors: Alan Preciado-Grijalva, Ramon F. Brena
Date: 2018-08-31 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification

Level assessment for foreign language students is necessary for putting them
in the right level group, furthermore, interviewing students is a very
time-consuming task, so we propose to automate the evaluation of speaker
fluency level by implementing machine learning techniques. This work presents
an audio processing system capable of classifying the level of fluency of
non-native English speakers using five different machine learning models. As a
first step, we have built our own dataset, which consists of labeled audio
conversations in English between people ranging in different fluency
domains/classes (low, intermediate, high). We segment the audio conversations
into 5s non-overlapped audio clips to perform feature extraction on them. We
start by extracting Mel cepstral coefficients from the audios, selecting 20
coefficients is an appropriate quantity for our data. We thereafter extracted
zero-crossing rate, root mean square energy and spectral flux features, proving
that this improves model performance. Out of a total of 1424 audio segments,
with 70% training data and 30% test data, one of our trained models (support
vector machine) achieved a classification accuracy of 94.39%, whereas the other
four models passed an 89% classification accuracy threshold.
Directed Exploration in PAC Model-Free Reinforcement Learning
Authors: Min-hwan Oh, Garud Iyengar
Date: 2018-08-31 00:00:00
Tasks: Efficient Exploration, model, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We study an exploration method for model-free RL that generalizes the
counter-based exploration bonus methods and takes into account long term
exploratory value of actions rather than a single step look-ahead. We propose a
model-free RL method that modifies Delayed Q-learning and utilizes the
long-term exploration bonus with provable efficiency. We show that our proposed
method finds a near-optimal policy in polynomial time (PAC-MDP), and also
provide experimental evidence that our proposed algorithm is an efficient
exploration method.
Dynamic mode decomposition in vector-valued reproducing kernel Hilbert spaces for extracting dynamical structure among observables
Authors: Keisuke Fujii, Yoshinobu Kawahara
Date: 2018-08-30 00:00:00

Understanding nonlinear dynamical systems (NLDSs) is challenging in a variety of engineering and scientific fields. Dynamic mode decomposition (DMD), which is a numerical algorithm for the spectral analysis of Koopman operators, has been attracting attention as a way of obtaining global modal descriptions of NLDSs without requiring explicit prior knowledge. However, since existing DMD algorithms are in principle formulated based on the concatenation of scalar observables, it is not directly applicable to data with dependent structures among observables, which take, for example, the form of a sequence of graphs. In this paper, we formulate Koopman spectral analysis for NLDSs with structures among observables and propose an estimation algorithm for this problem. This method can extract and visualize the underlying low-dimensional global dynamics of NLDSs with structures among observables from data, which can be useful in understanding the underlying dynamics of such NLDSs. To this end, we first formulate the problem of estimating spectra of the Koopman operator defined in vector-valued reproducing kernel Hilbert spaces, and then develop an estimation procedure for this problem by reformulating tensor-based DMD. As a special case of our method, we propose the method named as Graph DMD, which is a numerical algorithm for Koopman spectral analysis of graph dynamical systems, using a sequence of adjacency matrices. We investigate the empirical performance of our method by using synthetic and real-world data.
Fair Algorithms for Learning in Allocation Problems
Authors: Hadi Elzayn, Shahin Jabbari, Christopher Jung, Michael Kearns, Seth Neel, Aaron Roth, Zachary Schutzman
Date: 2018-08-30 00:00:00
Tasks: Fairness

Settings such as lending and policing can be modeled by a centralized agent
allocating a resource (loans or police officers) amongst several groups, in
order to maximize some objective (loans given that are repaid or criminals that
are apprehended). Often in such problems fairness is also a concern. A natural
notion of fairness, based on general principles of equality of opportunity,
asks that conditional on an individual being a candidate for the resource, the
probability of actually receiving it is approximately independent of the
individual's group. In lending this means that equally creditworthy individuals
in different racial groups have roughly equal chances of receiving a loan. In
policing it means that two individuals committing the same crime in different
districts would have roughly equal chances of being arrested.
  We formalize this fairness notion for allocation problems and investigate its
algorithmic consequences. Our main technical results include an efficient
learning algorithm that converges to an optimal fair allocation even when the
frequency of candidates (creditworthy individuals or criminals) in each group
is unknown. The algorithm operates in a censored feedback model in which only
the number of candidates who received the resource in a given allocation can be
observed, rather than the true number of candidates. This models the fact that
we do not learn the creditworthiness of individuals we do not give loans to nor
learn about crimes committed if the police presence in a district is low.
  As an application of our framework, we consider the predictive policing
problem. The learning algorithm is trained on arrest data gathered from its own
deployments on previous days, resulting in a potential feedback loop that our
algorithm provably overcomes. We empirically investigate the performance of our
algorithm on the Philadelphia Crime Incidents dataset.
LUCSS: Language-based User-customized Colourization of Scene Sketches
Authors: Changqing Zou, Haoran Mo, Ruofei Du, Xing Wu, Chengying Gao, Hongbo Fu
Date: 2018-08-30 00:00:00
Tasks: Colorization

We introduce LUCSS, a language-based system for interactive col- orization of
scene sketches, based on their semantic understanding. LUCSS is built upon deep
neural networks trained via a large-scale repository of scene sketches and
cartoon-style color images with text descriptions. It con- sists of three
sequential modules. First, given a scene sketch, the segmenta- tion module
automatically partitions an input sketch into individual object instances.
Next, the captioning module generates the text description with spatial
relationships based on the instance-level segmentation results. Fi- nally, the
interactive colorization module allows users to edit the caption and produce
colored images based on the altered caption. Our experiments show the
effectiveness of our approach and the desirability of its compo- nents to
alternative choices.
A Self-Attention Network for Hierarchical Data Structures with an Application to Claims Management
Authors: Leander Löw, Martin Spindler, Eike Brechmann
Date: 2018-08-30 00:00:00
Tasks: Fraud Detection, Management

Insurance companies must manage millions of claims per year. While most of
these claims are non-fraudulent, fraud detection is core for insurance
companies. The ultimate goal is a predictive model to single out the fraudulent
claims and pay out the non-fraudulent ones immediately. Modern machine learning
methods are well suited for this kind of problem. Health care claims often have
a data structure that is hierarchical and of variable length. We propose one
model based on piecewise feed forward neural networks (deep learning) and
another model based on self-attention neural networks for the task of claim
management. We show that the proposed methods outperform bag-of-words based
models, hand designed features, and models based on convolutional neural
networks, on a data set of two million health care claims. The proposed
self-attention method performs the best.