pwc-archive/papers-with-abstracts huggingface dataset:
Effective Feature Representation for Clinical Text Concept Extraction
Authors: Yifeng Tao, Bruno Godefroy, Guillaume Genthial, Christopher Potts
Date: 2018-10-31 00:00:00
Proceeding: WS 2019 6

Crucial information about the practice of healthcare is recorded only in
free-form text, which creates an enormous opportunity for high-impact NLP.
However, annotated healthcare datasets tend to be small and expensive to
obtain, which raises the question of how to make maximally efficient uses of
the available data. To this end, we develop an LSTM-CRF model for combining
unsupervised word representations and hand-built feature representations
derived from publicly available healthcare ontologies. We show that this
combined model yields superior performance on five datasets of diverse kinds of
healthcare text (clinical, social, scientific, commercial). Each involves the
labeling of complex, multi-word spans that pick out different healthcare
concepts. We also introduce a new labeled dataset for identifying the treatment
relations between drugs and diseases.
Aligning Very Small Parallel Corpora Using Cross-Lingual Word Embeddings and a Monogamy Objective
Authors: Nina Poerner, Masoud Jalili Sabet, Benjamin Roth, Hinrich Schütze
Date: 2018-10-31 00:00:00
Tasks: Cross-Lingual Word Embeddings, Word Alignment, Word Embeddings

Count-based word alignment methods, such as the IBM models or fast-align,
struggle on very small parallel corpora. We therefore present an alternative
approach based on cross-lingual word embeddings (CLWEs), which are trained on
purely monolingual data. Our main contribution is an unsupervised objective to
adapt CLWEs to parallel corpora. In experiments on between 25 and 500
sentences, our method outperforms fast-align. We also show that our fine-tuning
objective consistently improves a CLWE-only baseline.
An Interdisciplinary Comparison of Sequence Modeling Methods for Next-Element Prediction
Authors: Niek Tax, Irene Teinemaa, Sebastiaan J. van Zelst
Date: 2018-10-31 00:00:00
Tasks: BIG-bench Machine Learning, Descriptive

Data of sequential nature arise in many application domains in forms of, e.g.
textual data, DNA sequences, and software execution traces. Different research
disciplines have developed methods to learn sequence models from such datasets:
(i) in the machine learning field methods such as (hidden) Markov models and
recurrent neural networks have been developed and successfully applied to a
wide-range of tasks, (ii) in process mining process discovery techniques aim to
generate human-interpretable descriptive models, and (iii) in the grammar
inference field the focus is on finding descriptive models in the form of
formal grammars. Despite their different focuses, these fields share a common
goal - learning a model that accurately describes the behavior in the
underlying data. Those sequence models are generative, i.e, they can predict
what elements are likely to occur after a given unfinished sequence. So far,
these fields have developed mainly in isolation from each other and no
comparison exists. This paper presents an interdisciplinary experimental
evaluation that compares sequence modeling techniques on the task of
next-element prediction on four real-life sequence datasets. The results
indicate that machine learning techniques that generally have no aim at
interpretability in terms of accuracy outperform techniques from the process
mining and grammar inference fields that aim to yield interpretable models.
A Mixture of Expert Approach for Low-Cost Customization of Deep Neural Networks
Authors: Boyu Zhang, Azadeh Davoodi, Yu-Hen Hu
Date: 2018-10-31 00:00:00
Tasks: Mixture-of-Experts

The ability to customize a trained Deep Neural Network (DNN) locally using
user-specific data may greatly enhance user experiences, reduce development
costs, and protect user's privacy. In this work, we propose to incorporate a
novel Mixture of Experts (MOE) approach to accomplish this goal. This
architecture comprises of a Global Expert (GE), a Local Expert (LE) and a
Gating Network (GN). The GE is a trained DNN developed on a large training
dataset representative of many potential users. After deployment on an embedded
edge device, GE will be subject to customized, user-specific data (e.g., accent
in speech) and its performance may suffer. This problem may be alleviated by
training a local DNN (the local expert, LE) on a small size customized training
data to correct the errors made by GE. A gating network then will be trained to
determine whether an incoming data should be handled by GE or LE. Since the
customized dataset is in general very small, the cost of training LE and GN
would be much lower than that of re-training of GE. The training of LE and GN
thus can be performed at local device, properly protecting the privacy of
customized training data. In this work, we developed a prototype MOE
architecture for handwritten alphanumeric character recognition task. We use
EMNIST as the generic dataset, LeNet5 as GE, and handwritings of 10 users as
the customized dataset. We show that with the LE and GN, the classification
accuracy is significantly enhanced over the customized dataset with almost no
degradation of accuracy over the generic dataset. In terms of energy and
network size, the overhead of LE and GN is around 2.5% compared to those of GE.
DEEPGONET: Multi-label Prediction of GO Annotation for Protein from Sequence Using Cascaded Convolutional and Recurrent Network
Authors: Sheikh Muhammad Saiful Islam, Md Mahedi Hasan
Date: 2018-10-31 00:00:00
Tasks: Efficient Exploration

The present gap between the amount of available protein sequence due to the
development of next generation sequencing technology (NGS) and slow and
expensive experimental extraction of useful information like annotation of
protein sequence in different functional aspects, is ever widening, which can
be reduced by employing automatic function prediction (AFP) approaches. Gene
Ontology (GO), comprising of more than 40, 000 classes, defines three aspects
of protein function names Biological Process (BP), Cellular Component (CC),
Molecular Function (MF). Multiple functions of a single protein, has made
automatic function prediction a large-scale, multi-class, multi-label task. In
this paper, we present DEEPGONET, a novel cascaded convolutional and recurrent
neural network, to predict the top-level hierarchy of GO ontology. The network
takes the primary sequence of protein as input which makes it more useful than
other prevailing state-of-the-art deep learning based methods with multi-modal
input, making them less applicable for proteins where only primary sequence is
available. All predictions of different protein functions of our network are
performed by the same architecture, a proof of better generalization as
demonstrated by promising performance on a variety of organisms while trained
on Homo sapiens only, which is made possible by efficient exploration of vast
output space by leveraging hierarchical relationship among GO classes. The
promising performance of our model makes it a potential avenue for directing
experimental protein functions exploration efficiently by vastly eliminating
possible routes which is done by the exploring only the suggested routes from
our model. Our proposed model is also very simple and efficient in terms of
computational time and space compared to other architectures in literature.
Some New Layer Architectures for Graph CNN
Authors: Shrey Gadiya, Deepak Anand, Amit Sethi
Date: 2018-10-31 00:00:00
Tasks: General Classification

While convolutional neural networks (CNNs) have recently made great strides
in supervised classification of data structured on a grid (e.g. images composed
of pixel grids), in several interesting datasets, the relations between
features can be better represented as a general graph instead of a regular
grid. Although recent algorithms that adapt CNNs to graphs have shown promising
results, they mostly neglect learning explicit operations for edge features
while focusing on vertex features alone. We propose new formulations for
convolutional, pooling, and fully connected layers for neural networks that
make more comprehensive use of the information available in multi-dimensional
graphs. Using these layers led to an improvement in classification accuracy
over the state-of-the-art methods on benchmark graph datasets.
Generating Texts with Integer Linear Programming
Authors: Gerasimos Lampouras, Ion Androutsopoulos
Date: 2018-10-31 00:00:00
Tasks: Concept-To-Text Generation, Referring Expression, Referring expression generation, Sentence, Text Generation

Concept-to-text generation typically employs a pipeline architecture, which
often leads to suboptimal texts. Content selection, for example, may greedily
select the most important facts, which may require, however, too many words to
express, and this may be undesirable when space is limited or expensive.
Selecting other facts, possibly only slightly less important, may allow the
lexicalization stage to use much fewer words, or to report more facts in the
same space. Decisions made during content selection and lexicalization may also
lead to more or fewer sentence aggregation opportunities, affecting the length
and readability of the resulting texts. Building upon on a publicly available
state of the art natural language generator for Semantic Web ontologies, this
article presents an Integer Linear Programming model that, unlike pipeline
architectures, jointly considers choices available in content selection,
lexicalization, and sentence aggregation to avoid greedy local decisions and
produce more compact texts, i.e., texts that report more facts per word.
Compact texts are desirable, for example, when generating advertisements to be
included in Web search results, or when summarizing structured information in
limited space. An extended version of the proposed model also considers a
limited form of referring expression generation and avoids redundant sentences.
An approximation of the two models can be used when longer texts need to be
generated. Experiments with three ontologies confirm that the proposed models
lead to more compact texts, compared to pipeline systems, with no deterioration
or with improvements in the perceived quality of the generated texts.
Technical Note on Transcription Factor Motif Discovery from Importance Scores (TF-MoDISco) version 0.5.6.5
Authors: Avanti Shrikumar, Katherine Tian, Žiga Avsec, Anna Shcherbina, Abhimanyu Banerjee, Mahfuza Sharmin, Surag Nair, Anshul Kundaje
Date: 2018-10-31 00:00:00

TF-MoDISco (Transcription Factor Motif Discovery from Importance Scores) is an algorithm for identifying motifs from basepair-level importance scores computed on genomic sequence data. This technical note focuses on version v0.5.6.5. The implementation is available at https://github.com/kundajelab/tfmodisco/tree/v0.5.6.5
Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness
Authors: Raphael Suter, Đorđe Miladinović, Bernhard Schölkopf, Stefan Bauer
Date: 2018-10-31 00:00:00
Tasks: Disentanglement, Representation Learning

The ability to learn disentangled representations that split underlying sources of variation in high dimensional, unstructured data is important for data efficient and robust use of neural networks. While various approaches aiming towards this goal have been proposed in recent times, a commonly accepted definition and validation procedure is missing. We provide a causal perspective on representation learning which covers disentanglement and domain shift robustness as special cases. Our causal framework allows us to introduce a new metric for the quantitative evaluation of deep latent variable models. We show how this metric can be estimated from labeled observational data and further provide an efficient estimation algorithm that scales linearly in the dataset size.
Low-Dimensional Bottleneck Features for On-Device Continuous Speech Recognition
Authors: David B. Ramsay, Kevin Kilgour, Dominik Roblek, Matthew Sharifi
Date: 2018-10-31 00:00:00
Tasks: speech-recognition, Speech Recognition

Low power digital signal processors (DSPs) typically have a very limited
amount of memory in which to cache data. In this paper we develop efficient
bottleneck feature (BNF) extractors that can be run on a DSP, and retrain a
baseline large-vocabulary continuous speech recognition (LVCSR) system to use
these BNFs with only a minimal loss of accuracy. The small BNFs allow the DSP
chip to cache more audio features while the main application processor is
suspended, thereby reducing the overall battery usage. Our presented system is
able to reduce the footprint of standard, fixed point DSP spectral features by
a factor of 10 without any loss in word error rate (WER) and by a factor of 64
with only a 5.8% relative increase in WER.
Deep Net Features for Complex Emotion Recognition
Authors: Bhalaji Nagarajan, V Ramana Murthy Oruganti
Date: 2018-10-31 00:00:00
Tasks: automatic-speech-translation, Emotion Recognition, Translation

This paper investigates the influence of different acoustic features,
audio-events based features and automatic speech translation based lexical
features in complex emotion recognition such as curiosity. Pretrained networks,
namely, AudioSet Net, VoxCeleb Net and Deep Speech Net trained extensively for
different speech based applications are studied for this objective. Information
from deep layers of these networks are considered as descriptors and encoded
into feature vectors. Experimental results on the EmoReact dataset consisting
of 8 complex emotions show the effectiveness, yielding highest F1 score of 0.85
as against the baseline of 0.69 in the literature.
Quantum principal component analysis only achieves an exponential speedup because of its state preparation assumptions
Authors: Ewin Tang
Date: 2018-10-31 00:00:00
Tasks: Clustering, Recommendation Systems

A central roadblock to analyzing quantum algorithms on quantum states is the lack of a comparable input model for classical algorithms. Inspired by recent work of the author [E. Tang, STOC'19], we introduce such a model, where we assume we can efficiently perform $\ell^2$-norm samples of input data, a natural analogue to quantum algorithms that assume efficient state preparation of classical data. Though this model produces less practical algorithms than the (stronger) standard model of classical computation, it captures versions of many of the features and nuances of quantum linear algebra algorithms. With this model, we describe classical analogues to Lloyd, Mohseni, and Rebentrost's quantum algorithms for principal component analysis [Nat. Phys. 10, 631 (2014)] and nearest-centroid clustering [arXiv:1307.0411]. Since they are only polynomially slower, these algorithms suggest that the exponential speedups of their quantum counterparts are simply an artifact of state preparation assumptions.
WaveGlow: A Flow-based Generative Network for Speech Synthesis
Authors: Ryan Prenger, Rafael Valle, Bryan Catanzaro
Date: 2018-10-31 00:00:00
Tasks: Audio Synthesis, GPU, regression, Speech Synthesis

In this paper we propose WaveGlow: a flow-based network capable of generating
high quality speech from mel-spectrograms. WaveGlow combines insights from Glow
and WaveNet in order to provide fast, efficient and high-quality audio
synthesis, without the need for auto-regression. WaveGlow is implemented using
only a single network, trained using only a single cost function: maximizing
the likelihood of the training data, which makes the training procedure simple
and stable. Our PyTorch implementation produces audio samples at a rate of more
than 500 kHz on an NVIDIA V100 GPU. Mean Opinion Scores show that it delivers
audio quality as good as the best publicly available WaveNet implementation.
All code will be made publicly available online.
Non-linear Attributed Graph Clustering by Symmetric NMF with PU Learning
Authors: Seiji Maekawa, Koh Takeuch, Makoto Onizuka
Date: 2018-09-21 00:00:00
Tasks: Attribute, Clustering, Graph Clustering

We consider the clustering problem of attributed graphs. Our challenge is how
we can design an effective and efficient clustering method that precisely
captures the hidden relationship between the topology and the attributes in
real-world graphs. We propose Non-linear Attributed Graph Clustering by
Symmetric Non-negative Matrix Factorization with Positive Unlabeled Learning.
The features of our method are three holds. 1) it learns a non-linear
projection function between the different cluster assignments of the topology
and the attributes of graphs so as to capture the complicated relationship
between the topology and the attributes in real-world graphs, 2) it leverages
the positive unlabeled learning to take the effect of partially observed
positive edges into the cluster assignment, and 3) it achieves efficient
computational complexity, $O((n^2+mn)kt)$, where $n$ is the vertex size, $m$ is
the attribute size, $k$ is the number of clusters, and $t$ is the number of
iterations for learning the cluster assignment. We conducted experiments
extensively for various clustering methods with various real datasets to
validate that our method outperforms the former clustering methods regarding
the clustering quality.
Graph Degree Linkage: Agglomerative Clustering on a Directed Graph
Authors: Wei Zhang, Xiaogang Wang, Deli Zhao, Xiaoou Tang
Date: 2012-08-25 00:00:00
Tasks: Clustering, Computational Efficiency, Image Clustering

This paper proposes a simple but effective graph-based agglomerative
algorithm, for clustering high-dimensional data. We explore the different roles
of two fundamental concepts in graph theory, indegree and outdegree, in the
context of clustering. The average indegree reflects the density near a sample,
and the average outdegree characterizes the local geometry around a sample.
Based on such insights, we define the affinity measure of clusters via the
product of average indegree and average outdegree. The product-based affinity
makes our algorithm robust to noise. The algorithm has three main advantages:
good performance, easy implementation, and high computational efficiency. We
test the algorithm on two fundamental computer vision problems: image
clustering and object matching. Extensive experiments demonstrate that it
outperforms the state-of-the-arts in both applications.
AIS-BN: An Adaptive Importance Sampling Algorithm for Evidential Reasoning in Large Bayesian Networks
Authors: J. Cheng, M. J. Druzdzel
Date: 2011-06-01 00:00:00
Tasks: Pathfinder

Stochastic sampling algorithms, while an attractive alternative to exact
algorithms in very large Bayesian network models, have been observed to perform
poorly in evidential reasoning with extremely unlikely evidence. To address
this problem, we propose an adaptive importance sampling algorithm, AIS-BN,
that shows promising convergence rates even under extreme conditions and seems
to outperform the existing sampling algorithms consistently. Three sources of
this performance improvement are (1) two heuristics for initialization of the
importance function that are based on the theoretical properties of importance
sampling in finite-dimensional integrals and the structural advantages of
Bayesian networks, (2) a smooth learning method for the importance function,
and (3) a dynamic weighting function for combining samples from different
stages of the algorithm. We tested the performance of the AIS-BN algorithm
along with two state of the art general purpose sampling algorithms, likelihood
weighting (Fung and Chang, 1989; Shachter and Peot, 1989) and self-importance
sampling (Shachter and Peot, 1989). We used in our tests three large real
Bayesian network models available to the scientific community: the CPCS network
(Pradhan et al., 1994), the PathFinder network (Heckerman, Horvitz, and
Nathwani, 1990), and the ANDES network (Conati, Gertner, VanLehn, and Druzdzel,
1997), with evidence as unlikely as 10^-41. While the AIS-BN algorithm always
performed better than the other two algorithms, in the majority of the test
cases it achieved orders of magnitude improvement in precision of the results.
Improvement in speed given a desired precision is even more dramatic, although
we are unable to report numerical results here, as the other algorithms almost
never achieved the precision reached even by the first few iterations of the
AIS-BN algorithm.
Training neural audio classifiers with few data
Authors: Jordi Pons, Joan Serrà, Xavier Serra
Date: 2018-10-24 00:00:00
Tasks: Acoustic Scene Classification, General Classification, Scene Classification, Transfer Learning

We investigate supervised learning strategies that improve the training of
neural network audio classifiers on small annotated collections. In particular,
we study whether (i) a naive regularization of the solution space, (ii)
prototypical networks, (iii) transfer learning, or (iv) their combination, can
foster deep learning models to better leverage a small amount of training
examples. To this end, we evaluate (i-iv) for the tasks of acoustic event
recognition and acoustic scene classification, considering from 1 to 100
labeled examples per class. Results indicate that transfer learning is a
powerful strategy in such scenarios, but prototypical networks show promising
results when one does not count with external or validation data.
Efficient Neural Network Robustness Certification with General Activation Functions
Authors: Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, Luca Daniel
Date: 2018-11-02 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Computational Efficiency, Efficient Neural Network

Finding minimum distortion of adversarial examples and thus certifying
robustness in neural network classifiers for given data points is known to be a
challenging problem. Nevertheless, recently it has been shown to be possible to
give a non-trivial certified lower bound of minimum adversarial distortion, and
some recent progress has been made towards this direction by exploiting the
piece-wise linear nature of ReLU activations. However, a generic robustness
certification for general activation functions still remains largely
unexplored. To address this issue, in this paper we introduce CROWN, a general
framework to certify robustness of neural networks with general activation
functions for given input data points. The novelty in our algorithm consists of
bounding a given activation function with linear and quadratic functions, hence
allowing it to tackle general activation functions including but not limited to
four popular choices: ReLU, tanh, sigmoid and arctan. In addition, we
facilitate the search for a tighter certified lower bound by adaptively
selecting appropriate surrogates for each neuron activation. Experimental
results show that CROWN on ReLU networks can notably improve the certified
lower bounds compared to the current state-of-the-art algorithm Fast-Lin, while
having comparable computational efficiency. Furthermore, CROWN also
demonstrates its effectiveness and flexibility on networks with general
activation functions, including tanh, sigmoid and arctan.
Simple Attention-Based Representation Learning for Ranking Short Social Media Posts
Authors: Peng Shi, Jinfeng Rao, Jimmy Lin
Date: 2018-11-02 00:00:00
Proceeding: NAACL 2019 6
Tasks: Representation Learning

This paper explores the problem of ranking short social media posts with respect to user queries using neural networks. Instead of starting with a complex architecture, we proceed from the bottom up and examine the effectiveness of a simple, word-level Siamese architecture augmented with attention-based mechanisms for capturing semantic "soft" matches between query and post tokens. Extensive experiments on datasets from the TREC Microblog Tracks show that our simple models not only achieve better effectiveness than existing approaches that are far more complex or exploit a more diverse set of relevance signals, but are also much faster. Implementations of our samCNN (Simple Attention-based Matching CNN) models are shared with the community to support future work.