pwc-archive/papers-with-abstracts huggingface dataset:
Investigating the Working of Text Classifiers
Authors: Devendra Singh Sachan, Manzil Zaheer, Ruslan Salakhutdinov
Date: 2018-01-19 00:00:00
Proceeding: COLING 2018 8
Tasks: Classification, General Classification, text-classification, Text Classification

Text classification is one of the most widely studied tasks in natural
language processing. Motivated by the principle of compositionality, large
multilayer neural network models have been employed for this task in an attempt
to effectively utilize the constituent expressions. Almost all of the reported
work train large networks using discriminative approaches, which come with a
caveat of no proper capacity control, as they tend to latch on to any signal
that may not generalize. Using various recent state-of-the-art approaches for
text classification, we explore whether these models actually learn to compose
the meaning of the sentences or still just focus on some keywords or lexicons
for classifying the document. To test our hypothesis, we carefully construct
datasets where the training and test splits have no direct overlap of such
lexicons, but overall language structure would be similar. We study various
text classifiers and observe that there is a big performance drop on these
datasets. Finally, we show that even simple models with our proposed
regularization techniques, which disincentivize focusing on key lexicons, can
substantially improve classification accuracy.
Grammar Induction for Minimalist Grammars using Variational Bayesian Inference : A Technical Report
Authors: Eva Portelance, Amelia Bruno, Daniel Harasim, Leon Bergen, Timothy J. O'Donnell
Date: 2017-10-31 00:00:00
Tasks: Bayesian Inference, parameter estimation

The following technical report presents a formal approach to probabilistic minimalist grammar parameter estimation. We describe a formalization of a minimalist grammar. We then present an algorithm for the application of variational Bayesian inference to this formalization.
Reasoning about multiple aspects in DLs: Semantics and Closure Construction
Authors: Laura Giordano, Valentina Gliozzi
Date: 2018-01-18 00:00:00

Starting from the observation that rational closure has the undesirable
property of being an "all or nothing" mechanism, we here propose a
multipreferential semantics, which enriches the preferential semantics
underlying rational closure in order to separately deal with the inheritance of
different properties in an ontology with exceptions. We provide a
multipreference closure mechanism which is sound with respect to the
multipreference semantics.
Overpruning in Variational Bayesian Neural Networks
Authors: Brian Trippe, Richard Turner
Date: 2018-01-18 00:00:00
Tasks: Variational Inference

The motivations for using variational inference (VI) in neural networks
differ significantly from those in latent variable models. This has a
counter-intuitive consequence; more expressive variational approximations can
provide significantly worse predictions as compared to those with less
expressive families. In this work we make two contributions. First, we identify
a cause of this performance gap, variational over-pruning. Second, we introduce
a theoretically grounded explanation for this phenomenon. Our perspective sheds
light on several related published results and provides intuition into the
design of effective variational approximations of neural networks.
Sentiment Predictability for Stocks
Authors: Jordan Prosky, Xingyou Song, Andrew Tan, Michael Zhao
Date: 2017-12-15 00:00:00
Tasks: Event Extraction, Prediction, Sentiment Analysis, Stock Market Prediction

In this work, we present our findings and experiments for stock-market
prediction using various textual sentiment analysis tools, such as mood
analysis and event extraction, as well as prediction models, such as LSTMs and
specific convolutional architectures.
SIMLR: A Tool for Large-Scale Genomic Analyses by Multi-Kernel Learning
Authors: Bo Wang, Daniele Ramazzotti, Luca De Sano, Junjie Zhu, Emma Pierson, Serafim Batzoglou
Date: 2017-03-21 00:00:00
Tasks: Clustering, Dimensionality Reduction

We here present SIMLR (Single-cell Interpretation via Multi-kernel LeaRning),
an open-source tool that implements a novel framework to learn a
sample-to-sample similarity measure from expression data observed for
heterogenous samples. SIMLR can be effectively used to perform tasks such as
dimension reduction, clustering, and visualization of heterogeneous populations
of samples. SIMLR was benchmarked against state-of-the-art methods for these
three tasks on several public datasets, showing it to be scalable and capable
of greatly improving clustering performance, as well as providing valuable
insights by making the data more interpretable via better a visualization.
Availability and Implementation
  SIMLR is available on GitHub in both R and MATLAB implementations.
Furthermore, it is also available as an R package on http://bioconductor.org.
Connectome Smoothing via Low-rank Approximations
Authors: Runze Tang, Michael Ketcha, Alexandra Badea, Evan D. Calabrese, Daniel S. Margulies, Joshua T. Vogelstein, Carey E. Priebe, Daniel L. Sussman
Date: 2016-09-06 00:00:00

In statistical connectomics, the quantitative study of brain networks,
estimating the mean of a population of graphs based on a sample is a core
problem. Often, this problem is especially difficult because the sample or
cohort size is relatively small, sometimes even a single subject. While using
the element-wise sample mean of the adjacency matrices is a common approach,
this method does not exploit any underlying structural properties of the
graphs. We propose using a low-rank method which incorporates tools for
dimension selection and diagonal augmentation to smooth the estimates and
improve performance over the naive methodology for small sample sizes.
Theoretical results for the stochastic blockmodel show that this method offers
major improvements when there are many vertices. Similarly, we demonstrate that
the low-rank methods outperform the standard sample mean for a variety of
independent edge distributions as well as human connectome data derived from
magnetic resonance imaging, especially when sample sizes are small. Moreover,
the low-rank methods yield "eigen-connectomes", which correlate with the
lobe-structure of the human brain and superstructures of the mouse brain. These
results indicate that low-rank methods are an important part of the tool box
for researchers studying populations of graphs in general, and statistical
connectomics in particular.
Contextual and Position-Aware Factorization Machines for Sentiment Classification
Authors: Shuai Wang, Mianwei Zhou, Geli Fei, Yi Chang, Bing Liu
Date: 2018-01-18 00:00:00
Tasks: Classification, General Classification, Position, Recommendation Systems, Sentence, Sentiment Analysis, Sentiment Classification, Word Embeddings

While existing machine learning models have achieved great success for
sentiment classification, they typically do not explicitly capture
sentiment-oriented word interaction, which can lead to poor results for
fine-grained analysis at the snippet level (a phrase or sentence).
Factorization Machine provides a possible approach to learning element-wise
interaction for recommender systems, but they are not directly applicable to
our task due to the inability to model contexts and word sequences. In this
work, we develop two Position-aware Factorization Machines which consider word
interaction, context and position information. Such information is jointly
encoded in a set of sentiment-oriented word interaction vectors. Compared to
traditional word embeddings, SWI vectors explicitly capture sentiment-oriented
word interaction and simplify the parameter learning. Experimental results show
that while they have comparable performance with state-of-the-art methods for
document-level classification, they benefit the snippet/sentence-level
sentiment analysis.
When Does Stochastic Gradient Algorithm Work Well?
Authors: Lam M. Nguyen, Nam H. Nguyen, Dzung T. Phan, Jayant R. Kalagnanam, Katya Scheinberg
Date: 2018-01-18 00:00:00
Tasks: General Classification, regression, Stochastic Optimization

In this paper, we consider a general stochastic optimization problem which is
often at the core of supervised learning, such as deep learning and linear
classification. We consider a standard stochastic gradient descent (SGD) method
with a fixed, large step size and propose a novel assumption on the objective
function, under which this method has the improved convergence rates (to a
neighborhood of the optimal solutions). We then empirically demonstrate that
these assumptions hold for logistic regression and standard deep neural
networks on classical data sets. Thus our analysis helps to explain when
efficient behavior can be expected from the SGD method in training
classification models and deep neural networks.
PAG2ADMG: An Algorithm for the Complete Causal Enumeration of a Markov Equivalence Class
Authors: Nishant Subramani
Date: 2016-12-01 00:00:00
Tasks: Model Selection

Causal graphs, such as directed acyclic graphs (DAGs) and partial ancestral
graphs (PAGs), represent causal relationships among variables in a model.
Methods exist for learning DAGs and PAGs from data and for converting DAGs to
PAGs. However, these methods are significantly limited in that they only output
a single causal graph consistent with the independencies and dependencies (the
Markov equivalence class $M$) estimated from the data. This is problematic and
insufficient because many distinct graphs may be consistent with $M$. A data
modeler may wish to select among these numerous consistent graphs using domain
knowledge or other model selection algorithms. Enumeration of the set of
consistent graphs is the bottleneck. In this paper, we present a method that
makes this desired enumeration possible. We introduce PAG2ADMG, the first
algorithm for enumerating all causal graphs consistent with $M$. PAG2ADMG
converts a given PAG into the complete set of acyclic directed mixed graphs
(ADMGs) consistent with $M$. We prove the correctness of the approach and
demonstrate its efficiency relative to brute-force enumeration.
Bridging the Gap Between Neural Networks and Neuromorphic Hardware with A Neural Network Compiler
Authors: Yu Ji, Youhui Zhang, WenGuang Chen, Yuan Xie
Date: 2017-11-15 00:00:00

Different from developing neural networks (NNs) for general-purpose
processors, the development for NN chips usually faces with some
hardware-specific restrictions, such as limited precision of network signals
and parameters, constrained computation scale, and limited types of non-linear
functions.
  This paper proposes a general methodology to address the challenges. We
decouple the NN applications from the target hardware by introducing a compiler
that can transform an existing trained, unrestricted NN into an equivalent
network that meets the given hardware's constraints. We propose multiple
techniques to make the transformation adaptable to different kinds of NN chips,
and reliable for restrict hardware constraints.
  We have built such a software tool that supports both spiking neural networks
(SNNs) and traditional artificial neural networks (ANNs). We have demonstrated
its effectiveness with a fabricated neuromorphic chip and a
processing-in-memory (PIM) design. Tests show that the inference error caused
by this solution is insignificant and the transformation time is much shorter
than the retraining time. Also, we have studied the parameter-sensitivity
evaluations to explore the tradeoffs between network error and resource
utilization for different transformation strategies, which could provide
insights for co-design optimization of neuromorphic hardware and software.
Robust Kronecker Component Analysis
Authors: Mehdi Bahri, Yannis Panagakis, Stefanos Zafeiriou
Date: 2018-01-18 00:00:00
Tasks: Denoising, Dictionary Learning, Dimensionality Reduction, Image Denoising

Dictionary learning and component analysis models are fundamental for
learning compact representations that are relevant to a given task (feature
extraction, dimensionality reduction, denoising, etc.). The model complexity is
encoded by means of specific structure, such as sparsity, low-rankness, or
nonnegativity. Unfortunately, approaches like K-SVD - that learn dictionaries
for sparse coding via Singular Value Decomposition (SVD) - are hard to scale to
high-volume and high-dimensional visual data, and fragile in the presence of
outliers. Conversely, robust component analysis methods such as the Robust
Principal Component Analysis (RPCA) are able to recover low-complexity (e.g.,
low-rank) representations from data corrupted with noise of unknown magnitude
and support, but do not provide a dictionary that respects the structure of the
data (e.g., images), and also involve expensive computations. In this paper, we
propose a novel Kronecker-decomposable component analysis model, coined as
Robust Kronecker Component Analysis (RKCA), that combines ideas from sparse
dictionary learning and robust component analysis. RKCA has several appealing
properties, including robustness to gross corruption; it can be used for
low-rank modeling, and leverages separability to solve significantly smaller
problems. We design an efficient learning algorithm by drawing links with a
restricted form of tensor factorization, and analyze its optimality and
low-rankness properties. The effectiveness of the proposed approach is
demonstrated on real-world applications, namely background subtraction and
image denoising and completion, by performing a thorough comparison with the
current state of the art.
Upgrading from Gaussian Processes to Student's-T Processes
Authors: Brendan D. Tracey, David H. Wolpert
Date: 2018-01-18 00:00:00
Tasks: Bayesian Optimization, Gaussian Processes

Gaussian process priors are commonly used in aerospace design for performing
Bayesian optimization. Nonetheless, Gaussian processes suffer two significant
drawbacks: outliers are a priori assumed unlikely, and the posterior variance
conditioned on observed data depends only on the locations of those data, not
the associated sample values. Student's-T processes are a generalization of
Gaussian processes, founded on the Student's-T distribution instead of the
Gaussian distribution. Student's-T processes maintain the primary advantages of
Gaussian processes (kernel function, analytic update rule) with additional
benefits beyond Gaussian processes. The Student's-T distribution has higher
Kurtosis than a Gaussian distribution and so outliers are much more likely, and
the posterior variance increases or decreases depending on the variance of
observed data sample values. Here, we describe Student's-T processes, and
discuss their advantages in the context of aerospace optimization. We show how
to construct a Student's-T process using a kernel function and how to update
the process given new samples. We provide a clear derivation of
optimization-relevant quantities such as expected improvement, and contrast
with the related computations for Gaussian processes. Finally, we compare the
performance of Student's-T processes against Gaussian process on canonical test
problems in Bayesian optimization, and apply the Student's-T process to the
optimization of an aerostructural design problem.
Latitude: A Model for Mixed Linear-Tropical Matrix Factorization
Authors: Sanjar Karaev, James Hook, Pauli Miettinen
Date: 2018-01-18 00:00:00

Nonnegative matrix factorization (NMF) is one of the most frequently-used
matrix factorization models in data analysis. A significant reason to the
popularity of NMF is its interpretability and the `parts of whole'
interpretation of its components. Recently, max-times, or subtropical, matrix
factorization (SMF) has been introduced as an alternative model with equally
interpretable `winner takes it all' interpretation. In this paper we propose a
new mixed linear--tropical model, and a new algorithm, called Latitude, that
combines NMF and SMF, being able to smoothly alternate between the two. In our
model, the data is modeled using the latent factors and latent parameters that
control whether the factors are interpreted as NMF or SMF features, or their
mixtures. We present an algorithm for our novel matrix factorization. Our
experiments show that our algorithm improves over both baselines, and can yield
interpretable results that reveal more of the latent structure than either NMF
or SMF alone.
Probabilistic RGB-D Odometry based on Points, Lines and Planes Under Depth Uncertainty
Authors: Pedro F. Proenca, Yang Gao
Date: 2017-06-13 00:00:00
Tasks: Visual Odometry

This work proposes a robust visual odometry method for structured
environments that combines point features with line and plane segments,
extracted through an RGB-D camera. Noisy depth maps are processed by a
probabilistic depth fusion framework based on Mixtures of Gaussians to denoise
and derive the depth uncertainty, which is then propagated throughout the
visual odometry pipeline. Probabilistic 3D plane and line fitting solutions are
used to model the uncertainties of the feature parameters and pose is estimated
by combining the three types of primitives based on their uncertainties.
Performance evaluation on RGB-D sequences collected in this work and two public
RGB-D datasets: TUM and ICL-NUIM show the benefit of using the proposed depth
fusion framework and combining the three feature-types, particularly in scenes
with low-textured surfaces, dynamic objects and missing depth measurements.
Multivariate Gaussian and Student$-t$ Process Regression for Multi-output Prediction
Authors: Zexun Chen, Bo wang, Alexander N. Gorban
Date: 2017-03-13 00:00:00
Tasks: GPR, Prediction, regression

Gaussian process model for vector-valued function has been shown to be useful
for multi-output prediction. The existing method for this model is to
re-formulate the matrix-variate Gaussian distribution as a multivariate normal
distribution. Although it is effective in many cases, re-formulation is not
always workable and is difficult to apply to other distributions because not
all matrix-variate distributions can be transformed to respective multivariate
distributions, such as the case for matrix-variate Student$-t$ distribution. In
this paper, we propose a unified framework which is used not only to introduce
a novel multivariate Student$-t$ process regression model (MV-TPR) for
multi-output prediction, but also to reformulate the multivariate Gaussian
process regression (MV-GPR) that overcomes some limitations of the existing
methods. Both MV-GPR and MV-TPR have closed-form expressions for the marginal
likelihoods and predictive distributions under this unified framework and thus
can adopt the same optimization approaches as used in the conventional GPR. The
usefulness of the proposed methods is illustrated through several simulated and
real data examples. In particular, we verify empirically that MV-TPR has
superiority for the datasets considered, including air quality prediction and
bike rent prediction. At last, the proposed methods are shown to produce
profitable investment strategies in the stock markets.
Segmentation of nearly isotropic overlapped tracks in photomicrographs using successive erosions as watershed markers
Authors: Alexandre Fioravante de Siqueira, Wagner Massayuki Nakasuga, Sandro Guedes, Lothar Ratschbacher
Date: 2017-06-10 00:00:00

The major challenges of automatic track counting are distinguishing tracks
and material defects, identifying small tracks and defects of similar size, and
detecting overlapping tracks. Here we address the latter issue using WUSEM, an
algorithm which combines the watershed transform, morphological erosions and
labeling to separate regions in photomicrographs. WUSEM shows reliable results
when used in photomicrographs presenting almost isotropic objects. We tested
this method in two datasets of diallyl phthalate (DAP) photomicrographs and
compared the results when counting manually and using the classic watershed.
The mean automatic/manual efficiency ratio when using WUSEM in the test
datasets is 0.97 +/- 0.11.
DNA Steganalysis Using Deep Recurrent Neural Networks
Authors: Ho Bae, Byunghan Lee, Sunyoung Kwon, Sungroh Yoon
Date: 2017-04-27 00:00:00
Tasks: Steganalysis

Recent advances in next-generation sequencing technologies have facilitated
the use of deoxyribonucleic acid (DNA) as a novel covert channels in
steganography. There are various methods that exist in other domains to detect
hidden messages in conventional covert channels. However, they have not been
applied to DNA steganography. The current most common detection approaches,
namely frequency analysis-based methods, often overlook important signals when
directly applied to DNA steganography because those methods depend on the
distribution of the number of sequence characters. To address this limitation,
we propose a general sequence learning-based DNA steganalysis framework. The
proposed approach learns the intrinsic distribution of coding and non-coding
sequences and detects hidden messages by exploiting distribution variations
after hiding these messages. Using deep recurrent neural networks (RNNs), our
framework identifies the distribution variations by using the classification
score to predict whether a sequence is to be a coding or non-coding sequence.
We compare our proposed method to various existing methods and biological
sequence analysis methods implemented on top of our framework. According to our
experimental results, our approach delivers a robust detection performance
compared to other tools.
Natural Language Multitasking: Analyzing and Improving Syntactic Saliency of Hidden Representations
Authors: Gino Brunner, Yuyi Wang, Roger Wattenhofer, Michael Weigelt
Date: 2018-01-18 00:00:00
Tasks: Sentence, Translation

We train multi-task autoencoders on linguistic tasks and analyze the learned
hidden sentence representations. The representations change significantly when
translation and part-of-speech decoders are added. The more decoders a model
employs, the better it clusters sentences according to their syntactic
similarity, as the representation space becomes less entangled. We explore the
structure of the representation space by interpolating between sentences, which
yields interesting pseudo-English sentences, many of which have recognizable
syntactic structure. Lastly, we point out an interesting property of our
models: The difference-vector between two sentences can be added to change a
third sentence with similar features in a meaningful way.