pwc-archive/papers-with-abstracts huggingface dataset:
Exact Cluster Recovery via Classical Multidimensional Scaling
Authors: Anna Little, Yuying Xie, Qiang Sun
Date: 2018-12-31 00:00:00
Tasks: Clustering, Dimensionality Reduction

Classical multidimensional scaling is an important dimension reduction technique. Yet few theoretical results characterizing its statistical performance exist. This paper provides a theoretical framework for analyzing the quality of embedded samples produced by classical multidimensional scaling. This lays the foundation for various downstream statistical analyses, and we focus on clustering noisy data. Our results provide scaling conditions on the sample size, ambient dimensionality, between-class distance, and noise level under which classical multidimensional scaling followed by a distance-based clustering algorithm can recover the cluster labels of all samples with high probability. Numerical simulations confirm these scaling conditions are near-sharp. Applications to both human genomics data and natural language data lend strong support to the methodology and theory.
Pull out all the stops: Textual analysis via punctuation sequences
Authors: Alexandra N. M. Darmon, Marya Bazzi, Sam D. Howison, Mason A. Porter
Date: 2018-12-31 00:00:00
Tasks: All

Whether enjoying the lucid prose of a favorite author or slogging through some other writer's cumbersome, heavy-set prattle (full of parentheses, em dashes, compound adjectives, and Oxford commas), readers will notice stylistic signatures not only in word choice and grammar, but also in punctuation itself. Indeed, visual sequences of punctuation from different authors produce marvelously different (and visually striking) sequences. Punctuation is a largely overlooked stylistic feature in "stylometry", the quantitative analysis of written text. In this paper, we examine punctuation sequences in a corpus of literary documents and ask the following questions: Are the properties of such sequences a distinctive feature of different authors? Is it possible to distinguish literary genres based on their punctuation sequences? Do the punctuation styles of authors evolve over time? Are we on to something interesting in trying to do stylometry without words, or are we full of sound and fury (signifying nothing)?
Learning to Design RNA
Authors: Frederic Runge, Danny Stoll, Stefan Falkner, Frank Hutter
Date: 2018-12-31 00:00:00
Proceeding: ICLR 2019 5
Tasks: CPU, Deep Reinforcement Learning, Meta-Learning, Reinforcement Learning

Designing RNA molecules has garnered recent interest in medicine, synthetic
biology, biotechnology and bioinformatics since many functional RNA molecules
were shown to be involved in regulatory processes for transcription,
epigenetics and translation. Since an RNA's function depends on its structural
properties, the RNA Design problem is to find an RNA sequence which satisfies
given structural constraints. Here, we propose a new algorithm for the RNA
Design problem, dubbed LEARNA. LEARNA uses deep reinforcement learning to train
a policy network to sequentially design an entire RNA sequence given a
specified target structure. By meta-learning across 65000 different RNA Design
tasks for one hour on 20 CPU cores, our extension Meta-LEARNA constructs an RNA
Design policy that can be applied out of the box to solve novel RNA Design
tasks. Methodologically, for what we believe to be the first time, we jointly
optimize over a rich space of architectures for the policy network, the
hyperparameters of the training procedure and the formulation of the decision
process. Comprehensive empirical results on two widely-used RNA Design
benchmarks, as well as a third one that we introduce, show that our approach
achieves new state-of-the-art performance on the former while also being orders
of magnitudes faster in reaching the previous state-of-the-art performance. In
an ablation study, we analyze the importance of our method's different
components.
Image Super-Resolution via RL-CSC: When Residual Learning Meets Convolutional Sparse Coding
Authors: Menglei Zhang, Zhou Liu, Lei Yu
Date: 2018-12-31 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

We propose a simple yet effective model for Single Image Super-Resolution
(SISR), by combining the merits of Residual Learning and Convolutional Sparse
Coding (RL-CSC). Our model is inspired by the Learned Iterative
Shrinkage-Threshold Algorithm (LISTA). We extend LISTA to its convolutional
version and build the main part of our model by strictly following the
convolutional form, which improves the network's interpretability.
Specifically, the convolutional sparse codings of input feature maps are
learned in a recursive manner, and high-frequency information can be recovered
from these CSCs. More importantly, residual learning is applied to alleviate
the training difficulty when the network goes deeper. Extensive experiments on
benchmark datasets demonstrate the effectiveness of our method. RL-CSC (30
layers) outperforms several recent state-of-the-arts, e.g., DRRN (52 layers)
and MemNet (80 layers) in both accuracy and visual qualities. Codes and more
results are available at https://github.com/axzml/RL-CSC.
Differentiable Satisfiability and Differentiable Answer Set Programming for Sampling-Based Multi-Model Optimization
Authors: Matthias Nickles
Date: 2018-12-31 00:00:00
Tasks: Model Optimization

We propose Differentiable Satisfiability and Differentiable Answer Set
Programming (Differentiable SAT/ASP) for multi-model optimization. Models
(answer sets or satisfying truth assignments) are sampled using a novel SAT/ASP
solving approach which uses a gradient descent-based branching mechanism.
Sampling proceeds until the value of a user-defined multi-model cost function
reaches a given threshold. As major use cases for our approach we propose
distribution-aware model sampling and expressive yet scalable probabilistic
logic programming. As our main algorithmic approach to Differentiable SAT/ASP,
we introduce an enhancement of the state-of-the-art CDNL/CDCL algorithm for
SAT/ASP solving. Additionally, we present alternative algorithms which use an
unmodified ASP solver (Clingo/clasp) and map the optimization task to
conventional answer set optimization or use so-called propagators. We also
report on the open source software DelSAT, a recent prototype implementation of
our main algorithm, and on initial experimental results which indicate that
DelSATs performance is, when applied to the use case of probabilistic logic
inference, on par with Markov Logic Network (MLN) inference performance,
despite having advantageous properties compared to MLNs, such as the ability to
express inductive definitions and to work with probabilities as weights
directly in all cases. Our experiments also indicate that our main algorithm is
strongly superior in terms of performance compared to the presented alternative
approaches which reduce a common instance of the general problem to regular
SAT/ASP.
High Quality Monocular Depth Estimation via Transfer Learning
Authors: Ibraheem Alhashim, Peter Wonka
Date: 2018-12-31 00:00:00
Tasks: Action Classification, Decoder, Depth Estimation, Monocular Depth Estimation, Transfer Learning, Vocal Bursts Intensity Prediction

Accurate depth estimation from images is a fundamental task in many
applications including scene understanding and reconstruction. Existing
solutions for depth estimation often produce blurry approximations of low
resolution. This paper presents a convolutional neural network for computing a
high-resolution depth map given a single RGB image with the help of transfer
learning. Following a standard encoder-decoder architecture, we leverage
features extracted using high performing pre-trained networks when initializing
our encoder along with augmentation and training strategies that lead to more
accurate results. We show how, even for a very simple decoder, our method is
able to achieve detailed high-resolution depth maps. Our network, with fewer
parameters and training iterations, outperforms state-of-the-art on two
datasets and also produces qualitatively better results that capture object
boundaries more faithfully. Code and corresponding pre-trained weights are made
publicly available.
Two "correlation games" for a nonlinear network with Hebbian excitatory neurons and anti-Hebbian inhibitory neurons
Authors: H. Sebastian Seung
Date: 2018-12-31 00:00:00

A companion paper introduces a nonlinear network with Hebbian excitatory (E)
neurons that are reciprocally coupled with anti-Hebbian inhibitory (I) neurons
and also receive Hebbian feedforward excitation from sensory (S) afferents. The
present paper derives the network from two normative principles that are
mathematically equivalent but conceptually different. The first principle
formulates unsupervised learning as a constrained optimization problem:
maximization of S-E correlations subject to a copositivity constraint on E-E
correlations. A combination of Legendre and Lagrangian duality yields a
zero-sum continuous game between excitatory and inhibitory connections that is
solved by the neural network. The second principle defines a zero-sum game
between E and I cells. E cells want to maximize S-E correlations and minimize
E-I correlations, while I cells want to maximize I-E correlations and minimize
power. The conflict between I and E objectives effectively forces the E cells
to decorrelate from each other, although only incompletely. Legendre duality
yields the neural network.
Advancing Acoustic-to-Word CTC Model with Attention and Mixed-Units
Authors: Amit Das, Jinyu Li, Guoli Ye, Rui Zhao, Yifan Gong
Date: 2018-12-31 00:00:00
Tasks: Decoder, Language Modeling, Language Modelling

The acoustic-to-word model based on the Connectionist Temporal Classification (CTC) criterion is a natural end-to-end (E2E) system directly targeting word as output unit. Two issues exist in the system: first, the current output of the CTC model relies on the current input and does not account for context weighted inputs. This is the hard alignment issue. Second, the word-based CTC model suffers from the out-of-vocabulary (OOV) issue. This means it can model only frequently occurring words while tagging the remaining words as OOV. Hence, such a model is limited in its capacity in recognizing only a fixed set of frequent words. In this study, we propose addressing these problems using a combination of attention mechanism and mixed-units. In particular, we introduce Attention CTC, Self-Attention CTC, Hybrid CTC, and Mixed-unit CTC. First, we blend attention modeling capabilities directly into the CTC network using Attention CTC and Self-Attention CTC. Second, to alleviate the OOV issue, we present Hybrid CTC which uses a word and letter CTC with shared hidden layers. The Hybrid CTC consults the letter CTC when the word CTC emits an OOV. Then, we propose a much better solution by training a Mixed-unit CTC which decomposes all the OOV words into sequences of frequent words and multi-letter units. Evaluated on a 3400 hours Microsoft Cortana voice assistant task, our final acoustic-to-word solution using attention and mixed-units achieves a relative reduction in word error rate (WER) over the vanilla word CTC by 12.09\%. Such an E2E model without using any language model (LM) or complex decoder also outperforms a traditional context-dependent (CD) phoneme CTC with strong LM and decoder by 6.79% relative.
Learning RUMs: Reducing Mixture to Single Component via PCA
Authors: Devavrat Shah, Dogyoon Song
Date: 2018-12-31 00:00:00
Tasks: Clustering, Missing Values

We consider the problem of learning a mixture of Random Utility Models (RUMs). Despite the success of RUMs in various domains and the versatility of mixture RUMs to capture the heterogeneity in preferences, there has been only limited progress in learning a mixture of RUMs from partial data such as pairwise comparisons. In contrast, there have been significant advances in terms of learning a single component RUM using pairwise comparisons. In this paper, we aim to bridge this gap between mixture learning and single component learning of RUM by developing a `reduction' procedure. We propose to utilize PCA-based spectral clustering that simultaneously `de-noises' pairwise comparison data. We prove that our algorithm manages to cluster the partial data correctly (i.e., comparisons from the same RUM component are grouped in the same cluster) with high probability even when data is generated from a possibly {\em heterogeneous} mixture of well-separated {\em generic} RUMs. Both the time and the sample complexities scale polynomially in model parameters including the number of items. Two key features in the analysis are in establishing (1) a meaningful upper bound on the sub-Gaussian norm for RUM components embedded into the vector space of pairwise marginals and (2) the robustness of PCA with missing values in the $L_{2, \infty}$ sense, which might be of interest in their own right.
Large-Scale Object Detection of Images from Network Cameras in Variable Ambient Lighting Conditions
Authors: Caleb Tung, Matthew R. Kelleher, Ryan J. Schlueter, Binhan Xu, Yung-Hsiang Lu, George K. Thiruvathukal, Yen-Kuang Chen, Yang Lu
Date: 2018-12-31 00:00:00
Tasks: object-detection, Object Detection

Computer vision relies on labeled datasets for training and evaluation in
detecting and recognizing objects. The popular computer vision program, YOLO
("You Only Look Once"), has been shown to accurately detect objects in many
major image datasets. However, the images found in those datasets, are
independent of one another and cannot be used to test YOLO's consistency at
detecting the same object as its environment (e.g. ambient lighting) changes.
This paper describes a novel effort to evaluate YOLO's consistency for
large-scale applications. It does so by working (a) at large scale and (b) by
using consecutive images from a curated network of public video cameras
deployed in a variety of real-world situations, including traffic
intersections, national parks, shopping malls, university campuses, etc. We
specifically examine YOLO's ability to detect objects in different scenarios
(e.g., daytime vs. night), leveraging the cameras' ability to rapidly retrieve
many successive images for evaluating detection consistency. Using our camera
network and advanced computing resources (supercomputers), we analyzed more
than 5 million images captured by 140 network cameras in 24 hours. Compared
with labels marked by humans (considered as "ground truth"), YOLO struggles to
consistently detect the same humans and cars as their positions change from one
frame to the next; it also struggles to detect objects at night time. Our
findings suggest that state-of-the art vision solutions should be trained by
data from network camera with contextual information before they can be
deployed in applications that demand high consistency on object detection.
Accurate, Data-Efficient, Unconstrained Text Recognition with Convolutional Neural Networks
Authors: Mohamed Yousef, Khaled F. Hussain, Usama S. Mohammed
Date: 2018-12-31 00:00:00
Tasks: Handwriting Recognition, License Plate Recognition, Optical Character Recognition (OCR), Scene Text Recognition

Unconstrained text recognition is an important computer vision task,
featuring a wide variety of different sub-tasks, each with its own set of
challenges. One of the biggest promises of deep neural networks has been the
convergence and automation of feature extractors from input raw signals,
allowing for the highest possible performance with minimum required domain
knowledge. To this end, we propose a data-efficient, end-to-end neural network
model for generic, unconstrained text recognition. In our proposed architecture
we strive for simplicity and efficiency without sacrificing recognition
accuracy. Our proposed architecture is a fully convolutional network without
any recurrent connections trained with the CTC loss function. Thus it operates
on arbitrary input sizes and produces strings of arbitrary length in a very
efficient and parallelizable manner. We show the generality and superiority of
our proposed text recognition architecture by achieving state of the art
results on seven public benchmark datasets, covering a wide spectrum of text
recognition tasks, namely: Handwriting Recognition, CAPTCHA recognition, OCR,
License Plate Recognition, and Scene Text Recognition. Our proposed
architecture has won the ICFHR2018 Competition on Automated Text Recognition on
a READ Dataset.
How did Donald Trump Surprisingly Win the 2016 United States Presidential Election? an Information-Theoretic Perspective (Clean Sensing for Big Data Analytics:Optimal Strategies,Estimation Error Bounds Tighter than the Cramér-Rao Bound)
Authors: Weiyu Xu, Lifeng Lai, Amin Khajehnejad
Date: 2018-12-31 00:00:00
Tasks: parameter estimation

Donald Trump was lagging behind in nearly all opinion polls leading up to the
2016 US presidential election, but he surprisingly won the election. This
raises the following important questions: 1) why most opinion polls were not
accurate in 2016? and 2) how to improve the accuracies of opinion polls? In
this paper, we study the inaccuracies of opinion polls in the 2016 election
through the lens of information theory. We first propose a general framework of
parameter estimation, called clean sensing (polling), which performs optimal
parameter estimation with sensing cost constraints, from heterogeneous and
potentially distorted data sources. We then cast the opinion polling as a
problem of parameter estimation from potentially distorted heterogeneous data
sources, and derive the optimal polling strategy using heterogenous and
possibly distorted data under cost constraints. Our results show that a larger
number of data samples do not necessarily lead to better polling accuracy,
which give a possible explanation of the inaccuracies of opinion polls in 2016.
The optimal sensing strategy should instead optimally allocate sensing
resources over heterogenous data sources according to several factors including
data quality, and, moreover, for a particular data source, it should strike an
optimal balance between the quality of data samples, and the quantity of data
samples.
  As a byproduct of this research, in a general setting, we derive a group of
new lower bounds on the mean-squared errors of general unbiased and biased
parameter estimators. These new lower bounds can be tighter than the classical
Cram\'{e}r-Rao bound (CRB) and Chapman-Robbins bound. Our derivations are via
studying the Lagrange dual problems of certain convex programs. The classical
Cram\'{e}r-Rao bound and Chapman-Robbins bound follow naturally from our
results for special cases of these convex programs.
Sentence-Level Sentiment Analysis of Financial News Using Distributed Text Representations and Multi-Instance Learning
Authors: Bernhard Lutz, Nicolas Pröllochs, Dirk Neumann
Date: 2018-12-31 00:00:00
Tasks: Decision Making, Sentence, Sentiment Analysis

Researchers and financial professionals require robust computerized tools
that allow users to rapidly operationalize and assess the semantic textual
content in financial news. However, existing methods commonly work at the
document-level while deeper insights into the actual structure and the
sentiment of individual sentences remain blurred. As a result, investors are
required to apply the utmost attention and detailed, domain-specific knowledge
in order to assess the information on a fine-grained basis. To facilitate this
manual process, this paper proposes the use of distributed text representations
and multi-instance learning to transfer information from the document-level to
the sentence-level. Compared to alternative approaches, this method features
superior predictive performance while preserving context and interpretability.
Our analysis of a manually-labeled dataset yields a predictive accuracy of up
to 69.90%, exceeding the performance of alternative approaches by at least 3.80
percentage points. Accordingly, this study not only benefits investors with
regard to their financial decision-making, but also helps companies to
communicate their messages as intended.
Unary and Binary Classification Approaches and their Implications for Authorship Verification
Authors: Oren Halvani, Christian Winter, Lukas Graner
Date: 2018-12-31 00:00:00
Tasks: Authorship Verification, Binary Classification, Classification, General Classification, Information Retrieval, Retrieval

Retrieving indexed documents, not by their topical content but their writing
style opens the door for a number of applications in information retrieval
(IR). One application is to retrieve textual content of a certain author X,
where the queried IR system is provided beforehand with a set of reference
texts of X. Authorship verification (AV), which is a research subject in the
field of digital text forensics, is suitable for this purpose. The task of AV
is to determine if two documents (i.e. an indexed and a reference document)
have been written by the same author X. Even though AV represents a unary
classification problem, a number of existing approaches consider it as a binary
classification task. However, the underlying classification model of an AV
method has a number of serious implications regarding its prerequisites,
evaluability, and applicability. In our comprehensive literature review, we
observed several misunderstandings regarding the differentiation of unary and
binary AV approaches that require consideration. The objective of this paper
is, therefore, to clarify these by proposing clear criteria and new properties
that aim to improve the characterization of existing and future AV approaches.
Given both, we investigate the applicability of eleven existing unary and
binary AV methods as well as four generic unary classification algorithms on
two self-compiled corpora. Furthermore, we highlight an important issue
concerning the evaluation of AV methods based on fixed decision criterions,
which has not been paid attention in previous AV studies.
Fast Perceptual Image Enhancement
Authors: Etienne de Stoutz, Andrey Ignatov, Nikolay Kobyshev, Radu Timofte, Luc van Gool
Date: 2018-12-31 00:00:00
Tasks: CPU, Image Enhancement

The vast majority of photos taken today are by mobile phones. While their
quality is rapidly growing, due to physical limitations and cost constraints,
mobile phone cameras struggle to compare in quality with DSLR cameras. This
motivates us to computationally enhance these images. We extend upon the
results of Ignatov et al., where they are able to translate images from compact
mobile cameras into images with comparable quality to high-resolution photos
taken by DSLR cameras. However, the neural models employed require large
amounts of computational resources and are not lightweight enough to run on
mobile devices. We build upon the prior work and explore different network
architectures targeting an increase in image quality and speed. With an
efficient network architecture which does most of its processing in a lower
spatial resolution, we achieve a significantly higher mean opinion score (MOS)
than the baseline while speeding up the computation by 6.3 times on a
consumer-grade CPU. This suggests a promising direction for
neural-network-based photo enhancement using the phone hardware of the future.