pwc-archive/papers-with-abstracts huggingface dataset:
Neural and Synaptic Array Transceiver: A Brain-Inspired Computing Framework for Embedded Learning
Authors: Georgios Detorakis, Sadique Sheik, Charles Augustine, Somnath Paul, Bruno U. Pedroni, Nikil Dutt, Jeffrey Krichmar, Gert Cauwenberghs, Emre Neftci
Date: 2017-09-29 00:00:00
Tasks: Continual Learning, Reinforcement Learning

Embedded, continual learning for autonomous and adaptive behavior is a key
application of neuromorphic hardware. However, neuromorphic implementations of
embedded learning at large scales that are both flexible and efficient have
been hindered by a lack of a suitable algorithmic framework. As a result, the
most neuromorphic hardware is trained off-line on large clusters of dedicated
processors or GPUs and transferred post hoc to the device. We address this by
introducing the neural and synaptic array transceiver (NSAT), a neuromorphic
computational framework facilitating flexible and efficient embedded learning
by matching algorithmic requirements and neural and synaptic dynamics. NSAT
supports event-driven supervised, unsupervised and reinforcement learning
algorithms including deep learning. We demonstrate the NSAT in a wide range of
tasks, including the simulation of Mihalas-Niebur neuron, dynamic neural
fields, event-driven random back-propagation for event-based deep learning,
event-based contrastive divergence for unsupervised learning, and voltage-based
learning rules for sequence learning. We anticipate that this contribution will
establish the foundation for a new generation of devices enabling adaptive
mobile systems, wearable devices, and robots with data-driven autonomy.
Machine learning modeling of superconducting critical temperature
Authors: Valentin Stanev, Corey Oses, A. Gilad Kusne, Efrain Rodriguez, Johnpierre Paglione, Stefano Curtarolo, Ichiro Takeuchi
Date: 2017-09-08 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, regression

Superconductivity has been the focus of enormous research effort since its
discovery more than a century ago. Yet, some features of this unique phenomenon
remain poorly understood; prime among these is the connection between
superconductivity and chemical/structural properties of materials. To bridge
the gap, several machine learning schemes are developed herein to model the
critical temperatures ($T_{\mathrm{c}}$) of the 12,000+ known superconductors
available via the SuperCon database. Materials are first divided into two
classes based on their $T_{\mathrm{c}}$ values, above and below 10 K, and a
classification model predicting this label is trained. The model uses
coarse-grained features based only on the chemical compositions. It shows
strong predictive power, with out-of-sample accuracy of about 92%. Separate
regression models are developed to predict the values of $T_{\mathrm{c}}$ for
cuprate, iron-based, and "low-$T_{\mathrm{c}}$" compounds. These models also
demonstrate good performance, with learned predictors offering potential
insights into the mechanisms behind superconductivity in different families of
materials. To improve the accuracy and interpretability of these models, new
features are incorporated using materials data from the AFLOW Online
Repositories. Finally, the classification and regression models are combined
into a single integrated pipeline and employed to search the entire Inorganic
Crystallographic Structure Database (ICSD) for potential new superconductors.
We identify more than 30 non-cuprate and non-iron-based oxides as candidate
materials.
Bag-Level Aggregation for Multiple Instance Active Learning in Instance Classification Problems
Authors: Marc-André Carbonneau, Eric Granger, Ghyslain Gagnon
Date: 2017-10-06 00:00:00
Tasks: Active Learning, General Classification, Informativeness, Medical Image Analysis, Multiple Instance Learning

A growing number of applications, e.g. video surveillance and medical image
analysis, require training recognition systems from large amounts of weakly
annotated data while some targeted interactions with a domain expert are
allowed to improve the training process. In such cases, active learning (AL)
can reduce labeling costs for training a classifier by querying the expert to
provide the labels of most informative instances. This paper focuses on AL
methods for instance classification problems in multiple instance learning
(MIL), where data is arranged into sets, called bags, that are weakly labeled.
Most AL methods focus on single instance learning problems. These methods are
not suitable for MIL problems because they cannot account for the bag structure
of data. In this paper, new methods for bag-level aggregation of instance
informativeness are proposed for multiple instance active learning (MIAL). The
\textit{aggregated informativeness} method identifies the most informative
instances based on classifier uncertainty, and queries bags incorporating the
most information. The other proposed method, called \textit{cluster-based
aggregative sampling}, clusters data hierarchically in the instance space. The
informativeness of instances is assessed by considering bag labels, inferred
instance labels, and the proportion of labels that remain to be discovered in
clusters. Both proposed methods significantly outperform reference methods in
extensive experiments using benchmark data from several application domains.
Results indicate that using an appropriate strategy to address MIAL problems
yields a significant reduction in the number of queries needed to achieve the
same level of performance as single instance AL methods.
Low-resource bilingual lexicon extraction using graph based word embeddings
Authors: Ximena Gutierrez-Vasques, Victor Mijangos
Date: 2017-10-06 00:00:00
Tasks: Translation, Word Alignment, Word Embeddings

In this work we focus on the task of automatically extracting bilingual
lexicon for the language pair Spanish-Nahuatl. This is a low-resource setting
where only a small amount of parallel corpus is available. Most of the
downstream methods do not work well under low-resources conditions. This is
specially true for the approaches that use vectorial representations like
Word2Vec. Our proposal is to construct bilingual word vectors from a graph.
This graph is generated using translation pairs obtained from an unsupervised
word alignment method.
  We show that, in a low-resource setting, these type of vectors are successful
in representing words in a bilingual semantic space. Moreover, when a linear
transformation is applied to translate words from one language to another, our
graph based representations considerably outperform the popular setting that
uses Word2Vec.
CAMREP- Concordia Action and Motion Repository
Authors: Kaustubha Mendhurwar, Qing Gu, Vladimir de la Cruz, Sudhir Mudur, Tiberiu Popa
Date: 2017-10-06 00:00:00
Tasks: Action Recognition, Temporal Action Localization

Action recognition, motion classification, gait analysis and synthesis are
fundamental problems in a number of fields such as computer graphics,
bio-mechanics and human computer interaction that generate a large body of
research. This type of data is complex because it is inherently
multidimensional and has multiple modalities such as video, motion capture
data, accelerometer data, etc. While some of this data, such as monocular video
are easy to acquire, others are much more difficult and expensive such as
motion capture data or multi-view video. This creates a large barrier of entry
in the research community for data driven research. We have embarked on
creating a new large repository of motion and action data (CAMREP) consisting
of several motion and action databases. What makes this database unique is that
we use a variety of modalities, enabling multi-modal analysis. Presently, the
size of datasets varies with some having a large number of subjects while
others having smaller numbers. We have also acquired long capture sequences in
a number of cases, making some datasets rather large.
A Continuous Relaxation of Beam Search for End-to-end Training of Neural Sequence Models
Authors: Kartik Goyal, Graham Neubig, Chris Dyer, Taylor Berg-Kirkpatrick
Date: 2017-08-01 00:00:00
Tasks: CCG Supertagging, Motion Segmentation, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

Beam search is a desirable choice of test-time decoding algorithm for neural
sequence models because it potentially avoids search errors made by simpler
greedy methods. However, typical cross entropy training procedures for these
models do not directly consider the behaviour of the final decoding method. As
a result, for cross-entropy trained models, beam decoding can sometimes yield
reduced test performance when compared with greedy decoding. In order to train
models that can more effectively make use of beam search, we propose a new
training procedure that focuses on the final loss metric (e.g. Hamming loss)
evaluated on the output of beam search. While well-defined, this "direct loss"
objective is itself discontinuous and thus difficult to optimize. Hence, in our
approach, we form a sub-differentiable surrogate objective by introducing a
novel continuous approximation of the beam search decoding procedure. In
experiments, we show that optimizing this new training objective yields
substantially better results on two sequence tasks (Named Entity Recognition
and CCG Supertagging) when compared with both cross entropy trained greedy
decoding and cross entropy trained beam decoding baselines.
The DIRHA-English corpus and related tasks for distant-speech recognition in domestic environments
Authors: Mirco Ravanelli, Maurizio Omologo
Date: 2017-10-06 00:00:00
Tasks: Articles, Distant Speech Recognition, speech-recognition, Speech Recognition

This paper introduces the contents and the possible usage of the
DIRHA-ENGLISH multi-microphone corpus, recently realized under the EC DIRHA
project. The reference scenario is a domestic environment equipped with a large
number of microphones and microphone arrays distributed in space.
  The corpus is composed of both real and simulated material, and it includes
12 US and 12 UK English native speakers. Each speaker uttered different sets of
phonetically-rich sentences, newspaper articles, conversational speech,
keywords, and commands. From this material, a large set of 1-minute sequences
was generated, which also includes typical domestic background noise as well as
inter/intra-room reverberation effects. Dev and test sets were derived, which
represent a very precious material for different studies on multi-microphone
speech processing and distant-speech recognition. Various tasks and
corresponding Kaldi recipes have already been developed.
  The paper reports a first set of baseline results obtained using different
techniques, including Deep Neural Networks (DNN), aligned with the
state-of-the-art at international level.
HDLTex: Hierarchical Deep Learning for Text Classification
Authors: Kamran Kowsari, Donald E. Brown, Mojtaba Heidarysafa, Kiana Jafari Meimandi, Matthew S. Gerber, Laura E. Barnes
Date: 2017-09-24 00:00:00
Tasks: Classification, Deep Learning, Document Classification, General Classification, Multi-class Classification, text-classification, Text Classification

The continually increasing number of documents produced each year
necessitates ever improving information processing methods for searching,
retrieving, and organizing text. Central to these information processing
methods is document classification, which has become an important application
for supervised learning. Recently the performance of these traditional
classifiers has degraded as the number of documents has increased. This is
because along with this growth in the number of documents has come an increase
in the number of categories. This paper approaches this problem differently
from current document classification methods that view the problem as
multi-class classification. Instead we perform hierarchical classification
using an approach we call Hierarchical Deep Learning for Text classification
(HDLTex). HDLTex employs stacks of deep learning architectures to provide
specialized understanding at each level of the document hierarchy.
Contrastive Learning for Image Captioning
Authors: Bo Dai, Dahua Lin
Date: 2017-10-06 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Contrastive Learning, Image Captioning

Image captioning, a popular topic in computer vision, has achieved
substantial progress in recent years. However, the distinctiveness of natural
descriptions is often overlooked in previous work. It is closely related to the
quality of captions, as distinctive captions are more likely to describe images
with their unique aspects. In this work, we propose a new learning method,
Contrastive Learning (CL), for image captioning. Specifically, via two
constraints formulated on top of a reference model, the proposed method can
encourage distinctiveness, while maintaining the overall quality of the
generated captions. We tested our method on two challenging datasets, where it
improves the baseline model by significant margins. We also showed in our
studies that the proposed method is generic and can be used for models with
various structures.
On the Challenges of Sentiment Analysis for Dynamic Events
Authors: Monireh Ebrahimi, Amir Hossein Yazdavar, Amit Sheth
Date: 2017-10-06 00:00:00
Tasks: Emotion Recognition, Marketing, Sentiment Analysis

With the proliferation of social media over the last decade, determining
people's attitude with respect to a specific topic, document, interaction or
events has fueled research interest in natural language processing and
introduced a new channel called sentiment and emotion analysis. For instance,
businesses routinely look to develop systems to automatically understand their
customer conversations by identifying the relevant content to enhance marketing
their products and managing their reputations. Previous efforts to assess
people's sentiment on Twitter have suggested that Twitter may be a valuable
resource for studying political sentiment and that it reflects the offline
political landscape. According to a Pew Research Center report, in January 2016
44 percent of US adults stated having learned about the presidential election
through social media. Furthermore, 24 percent reported use of social media
posts of the two candidates as a source of news and information, which is more
than the 15 percent who have used both candidates' websites or emails combined.
The first presidential debate between Trump and Hillary was the most tweeted
debate ever with 17.1 million tweets.
Performance Prediction and Optimization of Solar Water Heater via a Knowledge-Based Machine Learning Method
Authors: Hao Li, Zhijian Liu
Date: 2017-10-06 00:00:00
Tasks: BIG-bench Machine Learning, Prediction

Measuring the performance of solar energy and heat transfer systems requires
a lot of time, economic cost and manpower. Meanwhile, directly predicting their
performance is challenging due to the complicated internal structures.
Fortunately, a knowledge-based machine learning method can provide a promising
prediction and optimization strategy for the performance of energy systems. In
this Chapter, the authors will show how they utilize the machine learning
models trained from a large experimental database to perform precise prediction
and optimization on a solar water heater (SWH) system. A new energy system
optimization strategy based on a high-throughput screening (HTS) process is
proposed. This Chapter consists of: i) Comparative studies on varieties of
machine learning models (artificial neural networks (ANNs), support vector
machine (SVM) and extreme learning machine (ELM)) to predict the performances
of SWHs; ii) Development of an ANN-based software to assist the quick
prediction and iii) Introduction of a computational HTS method to design a
high-performance SWH system.
The Voynich Manuscript is Written in Natural Language: The Pahlavi Hypothesis
Authors: J. Michael Herrmann
Date: 2017-09-06 00:00:00
Tasks: Relation, Translation

The late medieval Voynich Manuscript (VM) has resisted decryption and was
considered a meaningless hoax or an unsolvable cipher. Here, we provide
evidence that the VM is written in natural language by establishing a relation
of the Voynich alphabet and the Iranian Pahlavi script. Many of the Voynich
characters are upside-down versions of their Pahlavi counterparts, which may be
an effect of different writing directions. Other Voynich letters can be
explained as ligatures or departures from Pahlavi with the intent to cope with
known problems due to the stupendous ambiguity of Pahlavi text. While a
translation of the VM text is not attempted here, we can confirm the
Voynich-Pahlavi relation at the character level by the transcription of many
words from the VM illustrations and from parts of the main text. Many of the
transcribed words can be identified as terms from Zoroastrian cosmology which
is in line with the use of Pahlavi script in Zoroastrian communities from
medieval times.
Machine Learning for Drug Overdose Surveillance
Authors: Daniel B. Neill, William Herlands
Date: 2017-10-06 00:00:00
Tasks: Anomaly Detection, BIG-bench Machine Learning

We describe two recently proposed machine learning approaches for discovering
emerging trends in fatal accidental drug overdoses. The Gaussian Process Subset
Scan enables early detection of emerging patterns in spatio-temporal data,
accounting for both the non-iid nature of the data and the fact that detecting
subtle patterns requires integration of information across multiple spatial
areas and multiple time steps. We apply this approach to 17 years of
county-aggregated data for monthly opioid overdose deaths in the New York City
metropolitan area, showing clear advantages in the utility of discovered
patterns as compared to typical anomaly detection approaches.
  To detect and characterize emerging overdose patterns that differentially
affect a subpopulation of the data, including geographic, demographic, and
behavioral patterns (e.g., which combinations of drugs are involved), we apply
the Multidimensional Tensor Scan to 8 years of case-level overdose data from
Allegheny County, PA. We discover previously unidentified overdose patterns
which reveal unusual demographic clusters, show impacts of drug legislation,
and demonstrate potential for early detection and targeted intervention. These
approaches to early detection of overdose patterns can inform prevention and
response efforts, as well as understanding the effects of policy changes.
Dictionary-Free MRI PERK: Parameter Estimation via Regression with Kernels
Authors: Gopal Nataraj, Jon-Fredrik Nielsen, Clayton Scott, Jeffrey A. Fessler
Date: 2017-10-06 00:00:00
Tasks: parameter estimation, regression

This paper introduces a fast, general method for dictionary-free parameter
estimation in quantitative magnetic resonance imaging (QMRI) via regression
with kernels (PERK). PERK first uses prior distributions and the nonlinear MR
signal model to simulate many parameter-measurement pairs. Inspired by machine
learning, PERK then takes these parameter-measurement pairs as labeled training
points and learns from them a nonlinear regression function using kernel
functions and convex optimization. PERK admits a simple implementation as
per-voxel nonlinear lifting of MRI measurements followed by linear minimum
mean-squared error regression. We demonstrate PERK for $T_1,T_2$ estimation, a
well-studied application where it is simple to compare PERK estimates against
dictionary-based grid search estimates. Numerical simulations as well as
single-slice phantom and in vivo experiments demonstrate that PERK and grid
search produce comparable $T_1,T_2$ estimates in white and gray matter, but
PERK is consistently at least $23\times$ faster. This acceleration factor will
increase by several orders of magnitude for full-volume QMRI estimation
problems involving more latent parameters per voxel.
Learning Word Embeddings for Hyponymy with Entailment-Based Distributional Semantics
Authors: James Henderson
Date: 2017-10-06 00:00:00
Tasks: Learning Word Embeddings, Lexical Entailment, Word Embeddings

Lexical entailment, such as hyponymy, is a fundamental issue in the semantics
of natural language. This paper proposes distributional semantic models which
efficiently learn word embeddings for entailment, using a recently-proposed
framework for modelling entailment in a vector-space. These models postulate a
latent vector for a pseudo-phrase containing two neighbouring word vectors. We
investigate both modelling words as the evidence they contribute about this
phrase vector, or as the posterior distribution of a one-word phrase vector,
and find that the posterior vectors perform better. The resulting word
embeddings outperform the best previous results on predicting hyponymy between
words, in unsupervised and semi-supervised experiments.
Accumulated Gradient Normalization
Authors: Joeri Hermans, Gerasimos Spanakis, Rico Möckel
Date: 2017-10-06 00:00:00

This work addresses the instability in asynchronous data parallel
optimization. It does so by introducing a novel distributed optimizer which is
able to efficiently optimize a centralized model under communication
constraints. The optimizer achieves this by pushing a normalized sequence of
first-order gradients to a parameter server. This implies that the magnitude of
a worker delta is smaller compared to an accumulated gradient, and provides a
better direction towards a minimum compared to first-order gradients, which in
turn also forces possible implicit momentum fluctuations to be more aligned
since we make the assumption that all workers contribute towards a single
minima. As a result, our approach mitigates the parameter staleness problem
more effectively since staleness in asynchrony induces (implicit) momentum, and
achieves a better convergence rate compared to other optimizers such as
asynchronous EASGD and DynSGD, which we show empirically.
Predicting Remaining Useful Life using Time Series Embeddings based on Recurrent Neural Networks
Authors: Narendhar Gugulothu, Vishnu Tv, Pankaj Malhotra, Lovekesh Vig, Puneet Agarwal, Gautam Shroff
Date: 2017-09-04 00:00:00
Tasks: Missing Values, Time Series, Time Series Analysis

We consider the problem of estimating the remaining useful life (RUL) of a
system or a machine from sensor data. Many approaches for RUL estimation based
on sensor data make assumptions about how machines degrade. Additionally,
sensor data from machines is noisy and often suffers from missing values in
many practical settings. We propose Embed-RUL: a novel approach for RUL
estimation from sensor data that does not rely on any degradation-trend
assumptions, is robust to noise, and handles missing values. Embed-RUL utilizes
a sequence-to-sequence model based on Recurrent Neural Networks (RNNs) to
generate embeddings for multivariate time series subsequences. The embeddings
for normal and degraded machines tend to be different, and are therefore found
to be useful for RUL estimation. We show that the embeddings capture the
overall pattern in the time series while filtering out the noise, so that the
embeddings of two machines with similar operational behavior are close to each
other, even when their sensor readings have significant and varying levels of
noise content. We perform experiments on publicly available turbofan engine
dataset and a proprietary real-world dataset, and demonstrate that Embed-RUL
outperforms the previously reported state-of-the-art on several metrics.