pwc-archive/papers-with-abstracts huggingface dataset:
Adversarial Training of End-to-end Speech Recognition Using a Criticizing Language Model
Authors: Alexander H. Liu, Hung-Yi Lee, Lin-shan Lee
Date: 2018-11-02 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, speech-recognition, Speech Recognition

In this paper we proposed a novel Adversarial Training (AT) approach for
end-to-end speech recognition using a Criticizing Language Model (CLM). In this
way the CLM and the automatic speech recognition (ASR) model can challenge and
learn from each other iteratively to improve the performance. Since the CLM
only takes the text as input, huge quantities of unpaired text data can be
utilized in this approach within end-to-end training. Moreover, AT can be
applied to any end-to-end ASR model using any deep-learning-based language
modeling frameworks, and compatible with any existing end-to-end decoding
method. Initial results with an example experimental setup demonstrated the
proposed approach is able to gain consistent improvements efficiently from
auxiliary text data under different scenarios.
Deep Optimisation: Solving Combinatorial Optimisation Problems using Deep Neural Networks
Authors: J. R. Caldwell, R. A. Watson, C. Thies, J. D. Knowles
Date: 2018-11-02 00:00:00
Tasks: Evolutionary Algorithms

Deep Optimisation (DO) combines evolutionary search with Deep Neural Networks
(DNNs) in a novel way - not for optimising a learning algorithm, but for
finding a solution to an optimisation problem. Deep learning has been
successfully applied to classification, regression, decision and generative
tasks and in this paper we extend its application to solving optimisation
problems. Model Building Optimisation Algorithms (MBOAs), a branch of
evolutionary algorithms, have been successful in combining machine learning
methods and evolutionary search but, until now, they have not utilised DNNs. DO
is the first algorithm to use a DNN to learn and exploit the problem structure
to adapt the variation operator (changing the neighbourhood structure of the
search process). We demonstrate the performance of DO using two theoretical
optimisation problems within the MAXSAT class. The Hierarchical Transformation
Optimisation Problem (HTOP) has controllable deep structure that provides a
clear evaluation of how DO works and why using a layerwise technique is
essential for learning and exploiting problem structure. The Parity Modular
Constraint Problem (MCparity) is a simplistic example of a problem containing
higher-order dependencies (greater than pairwise) which DO can solve and state
of the art MBOAs cannot. Further, we show that DO can exploit deep structure in
TSP instances. Together these results show that there exists problems that DO
can find and exploit deep problem structure that other algorithms cannot.
Making this connection between DNNs and optimisation allows for the utilisation
of advanced tools applicable to DNNs that current MBOAs are unable to use.
Abstractive Summarization of Reddit Posts with Multi-level Memory Networks
Authors: Byeongchang Kim, Hyunwoo Kim, Gunhee Kim
Date: 2018-11-02 00:00:00
Proceeding: NAACL 2019 6
Tasks: Abstractive Text Summarization, Articles

We address the problem of abstractive summarization in two directions:
proposing a novel dataset and a new model. First, we collect Reddit TIFU
dataset, consisting of 120K posts from the online discussion forum Reddit. We
use such informal crowd-generated posts as text source, in contrast with
existing datasets that mostly use formal documents as source such as news
articles. Thus, our dataset could less suffer from some biases that key
sentences usually locate at the beginning of the text and favorable summary
candidates are already inside the text in similar forms. Second, we propose a
novel abstractive summarization model named multi-level memory networks (MMN),
equipped with multi-level memory to store the information of text from
different levels of abstraction. With quantitative evaluation and user studies
via Amazon Mechanical Turk, we show the Reddit TIFU dataset is highly
abstractive and the MMN outperforms the state-of-the-art summarization models.
A Bayesian Approach for Sequence Tagging with Crowds
Authors: Edwin Simpson, Iryna Gurevych
Date: 2018-11-02 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Active Learning, Argument Mining, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

Current methods for sequence tagging, a core task in NLP, are data hungry, which motivates the use of crowdsourcing as a cheap way to obtain labelled data. However, annotators are often unreliable and current aggregation methods cannot capture common types of span annotation errors. To address this, we propose a Bayesian method for aggregating sequence tags that reduces errors by modelling sequential dependencies between the annotations as well as the ground-truth labels. By taking a Bayesian approach, we account for uncertainty in the model due to both annotator errors and the lack of data for modelling annotators who complete few tasks. We evaluate our model on crowdsourced data for named entity recognition, information extraction and argument mining, showing that our sequential model outperforms the previous state of the art. We also find that our approach can reduce crowdsourcing costs through more effective active learning, as it better captures uncertainty in the sequence labels when there are few annotations.
Towards the AlexNet Moment for Homomorphic Encryption: HCNN, theFirst Homomorphic CNN on Encrypted Data with GPUs
Authors: Ahmad Al Badawi, Jin Chao, Jie Lin, Chan Fook Mun, Jun Jie Sim, Benjamin Hong Meng Tan, Xiao Nan, Khin Mi Mi Aung, Vijay Ramaseshan Chandrasekhar
Date: 2018-11-02 00:00:00
Tasks: Privacy Preserving

Deep Learning as a Service (DLaaS) stands as a promising solution for cloud-based inference applications. In this setting, the cloud has a pre-learned model whereas the user has samples on which she wants to run the model. The biggest concern with DLaaS is user privacy if the input samples are sensitive data. We provide here an efficient privacy-preserving system by employing high-end technologies such as Fully Homomorphic Encryption (FHE), Convolutional Neural Networks (CNNs) and Graphics Processing Units (GPUs). FHE, with its widely-known feature of computing on encrypted data, empowers a wide range of privacy-concerned applications. This comes at high cost as it requires enormous computing power. In this paper, we show how to accelerate the performance of running CNNs on encrypted data with GPUs. We evaluated two CNNs to classify homomorphically the MNIST and CIFAR-10 datasets. Our solution achieved a sufficient security level (> 80 bit) and reasonable classification accuracy (99%) and (77.55%) for MNIST and CIFAR-10, respectively. In terms of latency, we could classify an image in 5.16 seconds and 304.43 seconds for MNIST and CIFAR-10, respectively. Our system can also classify a batch of images (> 8,000) without extra overhead.
A Survey on Natural Language Processing for Fake News Detection
Authors: Ray Oshikawa, Jing Qian, William Yang Wang
Date: 2018-11-02 00:00:00
Proceeding: LREC 2020 5
Tasks: Fake News Detection

Fake news detection is a critical yet challenging problem in Natural Language Processing (NLP). The rapid rise of social networking platforms has not only yielded a vast increase in information accessibility but has also accelerated the spread of fake news. Thus, the effect of fake news has been growing, sometimes extending to the offline world and threatening public safety. Given the massive amount of Web content, automatic fake news detection is a practical NLP problem useful to all online content providers, in order to reduce the human time and effort to detect and prevent the spread of fake news. In this paper, we describe the challenges involved in fake news detection and also describe related tasks. We systematically review and compare the task formulations, datasets and NLP solutions that have been developed for this task, and also discuss the potentials and limitations of them. Based on our insights, we outline promising research directions, including more fine-grained, detailed, fair, and practical detection models. We also highlight the difference between fake news detection and other related tasks, and the importance of NLP solutions for fake news detection.
Adaptive Ranking Based Constraint Handling for Explicitly Constrained Black-Box Optimization
Authors: Naoki Sakamoto, Youhei Akimoto
Date: 2018-11-02 00:00:00

We propose a novel constraint-handling technique for the covariance matrix adaptation evolution strategy (CMA-ES). The proposed technique is aimed at solving explicitly constrained black-box continuous optimization problems, in which the explicit constraint is a constraint whereby the computational time for the constraint violation and its (numerical) gradient are negligible compared to that for the objective function. This method is designed to realize two invariance properties: invariance to the affine transformation of the search space, and invariance to the increasing transformation of the objective and constraint functions. The CMA-ES is designed to possess these properties for handling difficulties that appear in black-box optimization problems, such as non-separability, ill-conditioning, ruggedness, and the different orders of magnitude in the objective. The proposed constraint-handling technique (CHT), known as ARCH, modifies the underlying CMA-ES only in terms of the ranking of the candidate solutions. It employs a repair operator and an adaptive ranking aggregation strategy to compute the ranking. We developed test problems to evaluate the effects of the invariance properties, and performed experiments to empirically verify the invariance of the algorithm. We compared the proposed method with other CHTs on the CEC 2006 constrained optimization benchmark suite to demonstrate its efficacy. Empirical studies reveal that ARCH is able to exploit the explicitness of the constraint functions effectively, sometimes even more efficiently than an existing box-constraint handling technique on box-constrained problems, while exhibiting the invariance properties. Moreover, ARCH overwhelmingly outperforms CHTs by not exploiting the explicit constraints in terms of the number of objective function calls.
ChemBoost: A chemical language based approach for protein-ligand binding affinity prediction
Authors: Rıza Özçelik, Hakime Öztürk, Arzucan Özgür, Elif Ozkirimli
Date: 2018-11-02 00:00:00
Tasks: Drug Discovery, Word Embeddings

Identification of high affinity drug-target interactions is a major research question in drug discovery. Proteins are generally represented by their structures or sequences. However, structures are available only for a small subset of biomolecules and sequence similarity is not always correlated with functional similarity. We propose ChemBoost, a chemical language based approach for affinity prediction using SMILES syntax. We hypothesize that SMILES is a codified language and ligands are documents composed of chemical words. These documents can be used to learn chemical word vectors that represent words in similar contexts with similar vectors. In ChemBoost, the ligands are represented via chemical word embeddings, while the proteins are represented through sequence-based features and/or chemical words of their ligands. Our aim is to process the patterns in SMILES as a language to predict protein-ligand affinity, even when we cannot infer the function from the sequence. We used eXtreme Gradient Boosting to predict protein-ligand affinities in KIBA and BindingDB data sets. ChemBoost was able to predict drug-target binding affinity as well as or better than state-of-the-art machine learning systems. When powered with ligand-centric representations, ChemBoost was more robust to the changes in protein sequence similarity and successfully captured the interactions between a protein and a ligand, even if the protein has low sequence similarity to the known targets of the ligand.
A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes
Authors: Jialin Song, Yuxin Chen, Yisong Yue
Date: 2018-11-02 00:00:00
Tasks: Bayesian Optimization, Gaussian Processes, global-optimization

How can we efficiently gather information to optimize an unknown function,
when presented with multiple, mutually dependent information sources with
different costs? For example, when optimizing a robotic system, intelligently
trading off computer simulations and real robot testings can lead to
significant savings. Existing methods, such as multi-fidelity GP-UCB or Entropy
Search-based approaches, either make simplistic assumptions on the interaction
among different fidelities or use simple heuristics that lack theoretical
guarantees. In this paper, we study multi-fidelity Bayesian optimization with
complex structural dependencies among multiple outputs, and propose
MF-MI-Greedy, a principled algorithmic framework for addressing this problem.
In particular, we model different fidelities using additive Gaussian processes
based on shared latent structures with the target function. Then we use
cost-sensitive mutual information gain for efficient Bayesian global
optimization. We propose a simple notion of regret which incorporates the cost
of different fidelities, and prove that MF-MI-Greedy achieves low regret. We
demonstrate the strong empirical performance of our algorithm on both synthetic
and real-world datasets.
Risk-Stratify: Confident Stratification Of Patients Based On Risk
Authors: Kartik Ahuja, Mihaela van der Schaar
Date: 2018-11-02 00:00:00
Tasks: Prognosis

A clinician desires to use a risk-stratification method that achieves
confident risk-stratification - the risk estimates of the different patients
reflect the true risks with a high probability. This allows him/her to use
these risks to make accurate predictions about prognosis and decisions about
screening, treatments for the current patient. We develop Risk-stratify - a two
phase algorithm that is designed to achieve confident risk-stratification. In
the first phase, we grow a tree to partition the covariate space. Each node in
the tree is split using statistical tests that determine if the risks of the
child nodes are different or not. The choice of the statistical tests depends
on whether the data is censored (Log-rank test) or not (U-test). The set of the
leaves of the tree form a partition. The risk distribution of patients that
belong to a leaf is different from the sibling leaf but not the rest of the
leaves. Therefore, some of the leaves that have similar underlying risks are
incorrectly specified to have different risks. In the second phase, we develop
a novel recursive graph decomposition approach to address this problem. We
merge the leaves of the tree that have similar risks to form new leaves that
form the final output. We apply Risk-stratify on a cohort of patients (with no
history of cardiovascular disease) from UK Biobank and assess their risk for
cardiovascular disease. Risk-stratify significantly improves
risk-stratification, i.e., a lower fraction of the groups have over/under
estimated risks (measured in terms of false discovery rate; 33% reduction) in
comparison to state-of-the-art methods for cardiovascular prediction (Random
forests, Cox model, etc.). We find that the Cox model significantly over
estimates the risk of 21,621 patients out of 216,211 patients. Risk-stratify
can accurately categorize 2,987 of these 21,621 patients as low-risk
individuals.
Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition
Authors: Hui Li, Peng Wang, Chunhua Shen, Guyu Zhang
Date: 2018-11-02 00:00:00
Tasks: Decoder, Irregular Text Recognition, Optical Character Recognition (OCR), Scene Text Recognition

Recognizing irregular text in natural scene images is challenging due to the
large variance in text appearance, such as curvature, orientation and
distortion. Most existing approaches rely heavily on sophisticated model
designs and/or extra fine-grained annotations, which, to some extent, increase
the difficulty in algorithm implementation and data collection. In this work,
we propose an easy-to-implement strong baseline for irregular scene text
recognition, using off-the-shelf neural network components and only word-level
annotations. It is composed of a $31$-layer ResNet, an LSTM-based
encoder-decoder framework and a 2-dimensional attention module. Despite its
simplicity, the proposed method is robust and achieves state-of-the-art
performance on both regular and irregular scene text recognition benchmarks.
Code is available at: https://tinyurl.com/ShowAttendRead
Effective Learning of Probabilistic Models for Clinical Predictions from Longitudinal Data
Authors: Shuo Yang
Date: 2018-11-02 00:00:00
Tasks: BIG-bench Machine Learning, Medical Diagnosis, Relational Reasoning

With the expeditious advancement of information technologies, health-related
data presented unprecedented potentials for medical and health discoveries but
at the same time significant challenges for machine learning techniques both in
terms of size and complexity. Those challenges include: the structured data
with various storage formats and value types caused by heterogeneous data
sources; the uncertainty widely existing in every aspect of medical diagnosis
and treatments; the high dimensionality of the feature space; the longitudinal
medical records data with irregular intervals between adjacent observations;
the richness of relations existing among objects with similar genetic factors,
location or socio-demographic background. This thesis aims to develop advanced
Statistical Relational Learning approaches in order to effectively exploit such
health-related data and facilitate the discoveries in medical research. It
presents the work on cost-sensitive statistical relational learning for mining
structured imbalanced data, the first continuous-time probabilistic logic model
for predicting sequential events from longitudinal structured data as well as
hybrid probabilistic relational models for learning from heterogeneous
structured data. It also demonstrates the outstanding performance of these
proposed models as well as other state of the art machine learning models when
applied to medical research problems and other real-world large-scale systems,
reveals the great potential of statistical relational learning for exploring
the structured health-related data to facilitate medical research.
Confiding in and Listening to Virtual Agents: The Effect of Personality
Authors: Jingyi Li, Michelle X. Zhou, Huahai Yang, Gloria Mark
Date: 2018-11-02 00:00:00

We present an intelligent virtual interviewer that engages with a user in a
text-based conversation and automatically infers the user's psychological
traits, such as personality. We investigate how the personality of a virtual
interviewer influences a user's behavior from two perspectives: the user's
willingness to confide in, and listen to, a virtual interviewer. We have
developed two virtual interviewers with distinct personalities and deployed
them in a real-world recruiting event. We present findings from completed
interviews with 316 actual job applicants. Notably, users are more willing to
confide in and listen to a virtual interviewer with a serious, assertive
personality. Moreover, users' personality traits, inferred from their chat
text, influence their perception of a virtual interviewer, and their
willingness to confide in and listen to a virtual interviewer. Finally, we
discuss the implications of our work on building hyper-personalized,
intelligent agents based on user traits.
Unique Identification of Macaques for Population Monitoring and Control
Authors: Ankita Shukla, Gullal Singh Cheema, Saket Anand, Qamar Qureshi, Yadvendradev Jhala
Date: 2018-11-02 00:00:00
Tasks: Face Identification

Despite loss of natural habitat due to development and urbanization, certain
species like the Rhesus macaque have adapted well to the urban environment.
With abundant food and no predators, macaque populations have increased
substantially in urban areas, leading to frequent conflicts with humans.
Overpopulated areas often witness macaques raiding crops, feeding on bird and
snake eggs as well as destruction of nests, thus adversely affecting other
species in the ecosystem. In order to mitigate these adverse effects,
sterilization has emerged as a humane and effective way of population control
of macaques. As sterilization requires physical capture of individuals or
groups, their unique identification is integral to such control measures. In
this work, we propose the Macaque Face Identification (MFID), an image based,
non-invasive tool that relies on macaque facial recognition to identify
individuals, and can be used to verify if they are sterilized. Our primary
contribution is a robust facial recognition and verification module designed
for Rhesus macaques, but extensible to other non-human primate species. We
evaluate the performance of MFID on a dataset of 93 monkeys under closed set,
open set and verification evaluation protocols. Finally, we also report state
of the art results when evaluating our proposed model on endangered primate
species.
Stronger Data Poisoning Attacks Break Data Sanitization Defenses
Authors: Pang Wei Koh, Jacob Steinhardt, Percy Liang
Date: 2018-11-02 00:00:00
Tasks: Data Poisoning, Sentiment Analysis, Sentiment Classification, Spam detection

Machine learning models trained on data from the outside world can be corrupted by data poisoning attacks that inject malicious points into the models' training sets. A common defense against these attacks is data sanitization: first filter out anomalous training points before training the model. In this paper, we develop three attacks that can bypass a broad range of common data sanitization defenses, including anomaly detectors based on nearest neighbors, training loss, and singular-value decomposition. By adding just 3% poisoned data, our attacks successfully increase test error on the Enron spam detection dataset from 3% to 24% and on the IMDB sentiment classification dataset from 12% to 29%. In contrast, existing attacks which do not explicitly account for these data sanitization defenses are defeated by them. Our attacks are based on two ideas: (i) we coordinate our attacks to place poisoned points near one another, and (ii) we formulate each attack as a constrained optimization problem, with constraints designed to ensure that the poisoned points evade detection. As this optimization involves solving an expensive bilevel problem, our three attacks correspond to different ways of approximating this problem, based on influence functions; minimax duality; and the Karush-Kuhn-Tucker (KKT) conditions. Our results underscore the need to develop more robust defenses against data poisoning attacks.
Efficient Metropolitan Traffic Prediction Based on Graph Recurrent Neural Network
Authors: Xiaoyu Wang, Cailian Chen, Yang Min, Jianping He, Bo Yang, Yang Zhang
Date: 2018-11-02 00:00:00
Tasks: Traffic Prediction

Traffic prediction is a fundamental and vital task in Intelligence
Transportation System (ITS), but it is very challenging to get high accuracy
while containing low computational complexity due to the spatiotemporal
characteristics of traffic flow, especially under the metropolitan
circumstances. In this work, a new topological framework, called Linkage
Network, is proposed to model the road networks and present the propagation
patterns of traffic flow. Based on the Linkage Network model, a novel online
predictor, named Graph Recurrent Neural Network (GRNN), is designed to learn
the propagation patterns in the graph. It could simultaneously predict traffic
flow for all road segments based on the information gathered from the whole
graph, which thus reduces the computational complexity significantly from O(nm)
to O(n+m), while keeping the high accuracy. Moreover, it can also predict the
variations of traffic trends. Experiments based on real-world data demonstrate
that the proposed method outperforms the existing prediction methods.