pwc-archive/papers-with-abstracts huggingface dataset:
Efficient Parameter-free Clustering Using First Neighbor Relations
Authors: M. Saquib Sarfraz, Vivek Sharma, Rainer Stiefelhagen
Date: 2019-02-28 00:00:00
Tasks: Clustering

We present a new clustering method in the form of a single clustering
equation that is able to directly discover groupings in the data. The main
proposition is that the first neighbor of each sample is all one needs to
discover large chains and finding the groups in the data. In contrast to most
existing clustering algorithms our method does not require any
hyper-parameters, distance thresholds and/or the need to specify the number of
clusters. The proposed algorithm belongs to the family of hierarchical
agglomerative methods. The technique has a very low computational overhead, is
easily scalable and applicable to large practical problems. Evaluation on well
known datasets from different domains ranging between 1077 and 8.1 million
samples shows substantial performance gains when compared to the existing
clustering techniques.
NOODL: Provable Online Dictionary Learning and Sparse Coding
Authors: Sirisha Rambhatla, Xingguo Li, Jarvis Haupt
Date: 2019-02-28 00:00:00
Tasks: Dictionary Learning

We consider the dictionary learning problem, where the aim is to model the given data as a linear combination of a few columns of a matrix known as a dictionary, where the sparse weights forming the linear combination are known as coefficients. Since the dictionary and coefficients, parameterizing the linear model are unknown, the corresponding optimization is inherently non-convex. This was a major challenge until recently, when provable algorithms for dictionary learning were proposed. Yet, these provide guarantees only on the recovery of the dictionary, without explicit recovery guarantees on the coefficients. Moreover, any estimation error in the dictionary adversely impacts the ability to successfully localize and estimate the coefficients. This potentially limits the utility of existing provable dictionary learning methods in applications where coefficient recovery is of interest. To this end, we develop NOODL: a simple Neurally plausible alternating Optimization-based Online Dictionary Learning algorithm, which recovers both the dictionary and coefficients exactly at a geometric rate, when initialized appropriately. Our algorithm, NOODL, is also scalable and amenable for large scale distributed implementations in neural architectures, by which we mean that it only involves simple linear and non-linear operations. Finally, we corroborate these theoretical results via experimental evaluation of the proposed algorithm with the current state-of-the-art techniques. Keywords: dictionary learning, provable dictionary learning, online dictionary learning, non-convex, sparse coding, support recovery, iterative hard thresholding, matrix factorization, neural architectures, neural networks, noodl, sparse representations, sparse signal processing.
Distributed Learning with Sublinear Communication
Authors: Jayadev Acharya, Christopher De Sa, Dylan J. Foster, Karthik Sridharan
Date: 2019-02-28 00:00:00
Tasks: Quantization

In distributed statistical learning, $N$ samples are split across $m$
machines and a learner wishes to use minimal communication to learn as well as
if the examples were on a single machine. This model has received substantial
interest in machine learning due to its scalability and potential for parallel
speedup. However, in high-dimensional settings, where the number examples is
smaller than the number of features ("dimension"), the speedup afforded by
distributed learning may be overshadowed by the cost of communicating a single
example. This paper investigates the following question: When is it possible to
learn a $d$-dimensional model in the distributed setting with total
communication sublinear in $d$?
  Starting with a negative result, we show that for learning $\ell_1$-bounded
or sparse linear models, no algorithm can obtain optimal error until
communication is linear in dimension. Our main result is that that by slightly
relaxing the standard boundedness assumptions for linear models, we can obtain
distributed algorithms that enjoy optimal error with communication logarithmic
in dimension. This result is based on a family of algorithms that combine
mirror descent with randomized sparsification/quantization of iterates, and
extends to the general stochastic convex optimization model.
Incorporating End-to-End Speech Recognition Models for Sentiment Analysis
Authors: Egor Lakomkin, Mohammad Ali Zamani, Cornelius Weber, Sven Magg, Stefan Wermter
Date: 2019-02-28 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Emotion Recognition, Sentiment Analysis, Sentiment Classification, speech-recognition, Speech Recognition

Previous work on emotion recognition demonstrated a synergistic effect of
combining several modalities such as auditory, visual, and transcribed text to
estimate the affective state of a speaker. Among these, the linguistic modality
is crucial for the evaluation of an expressed emotion. However, manually
transcribed spoken text cannot be given as input to a system practically. We
argue that using ground-truth transcriptions during training and evaluation
phases leads to a significant discrepancy in performance compared to real-world
conditions, as the spoken text has to be recognized on the fly and can contain
speech recognition mistakes. In this paper, we propose a method of integrating
an automatic speech recognition (ASR) output with a character-level recurrent
neural network for sentiment recognition. In addition, we conduct several
experiments investigating sentiment recognition for human-robot interaction in
a noise-realistic scenario which is challenging for the ASR systems. We
quantify the improvement compared to using only the acoustic modality in
sentiment recognition. We demonstrate the effectiveness of this approach on the
Multimodal Corpus of Sentiment Intensity (MOSI) by achieving 73,6% accuracy in
a binary sentiment classification task, exceeding previously reported results
that use only acoustic input. In addition, we set a new state-of-the-art
performance on the MOSI dataset (80.4% accuracy, 2% absolute improvement).
The principles of adaptation in organisms and machines I: machine learning, information theory, and thermodynamics
Authors: Hideaki Shimazaki
Date: 2019-02-28 00:00:00
Tasks: BIG-bench Machine Learning

How do organisms recognize their environment by acquiring knowledge about the
world, and what actions do they take based on this knowledge? This article
examines hypotheses about organisms' adaptation to the environment from machine
learning, information-theoretic, and thermodynamic perspectives. We start with
constructing a hierarchical model of the world as an internal model in the
brain, and review standard machine learning methods to infer causes by
approximately learning the model under the maximum likelihood principle. This
in turn provides an overview of the free energy principle for an organism, a
hypothesis to explain perception and action from the principle of least
surprise. Treating this statistical learning as communication between the world
and brain, learning is interpreted as a process to maximize information about
the world. We investigate how the classical theories of perception such as the
infomax principle relates to learning the hierarchical model. We then present
an approach to the recognition and learning based on thermodynamics, showing
that adaptation by causal learning results in the second law of thermodynamics
whereas inference dynamics that fuses observation with prior knowledge forms a
thermodynamic process. These provide a unified view on the adaptation of
organisms to the environment.
B-Script: Transcript-based B-roll Video Editing with Recommendations
Authors: Bernd Huber, Hijung Valentina Shin, Bryan Russell, Oliver Wang, Gautham J. Mysore
Date: 2019-02-28 00:00:00
Tasks: Video Editing

In video production, inserting B-roll is a widely used technique to enrich
the story and make a video more engaging. However, determining the right
content and positions of B-roll and actually inserting it within the main
footage can be challenging, and novice producers often struggle to get both
timing and content right. We present B-Script, a system that supports B-roll
video editing via interactive transcripts. B-Script has a built-in
recommendation system trained on expert-annotated data, recommending users
B-roll position and content. To evaluate the system, we conducted a
within-subject user study with 110 participants, and compared three interface
variations: a timeline-based editor, a transcript-based editor, and a
transcript-based editor with recommendations. Users found it easier and were
faster to insert B-roll using the transcript-based interface, and they created
more engaging videos when recommendations were provided.
No Padding Please: Efficient Neural Handwriting Recognition
Authors: Gideon Maillette de Buy Wenniger, Lambert Schomaker, Andy Way
Date: 2019-02-28 00:00:00
Tasks: Handwriting Recognition, Handwritten Text Recognition

Neural handwriting recognition (NHR) is the recognition of handwritten text
with deep learning models, such as multi-dimensional long short-term memory
(MDLSTM) recurrent neural networks. Models with MDLSTM layers have achieved
state-of-the art results on handwritten text recognition tasks. While
multi-directional MDLSTM-layers have an unbeaten ability to capture the
complete context in all directions, this strength limits the possibilities for
parallelization, and therefore comes at a high computational cost. In this work
we develop methods to create efficient MDLSTM-based models for NHR,
particularly a method aimed at eliminating computation waste that results from
padding. This proposed method, called example-packing, replaces wasteful
stacking of padded examples with efficient tiling in a 2-dimensional grid. For
word-based NHR this yields a speed improvement of factor 6.6 over an already
efficient baseline of minimal padding for each batch separately. For line-based
NHR the savings are more modest, but still significant. In addition to
example-packing, we propose: 1) a technique to optimize parallelization for
dynamic graph definition frameworks including PyTorch, using convolutions with
grouping, 2) a method for parallelization across GPUs for variable-length
example batches. All our techniques are thoroughly tested on our own PyTorch
re-implementation of MDLSTM-based NHR models. A thorough evaluation on the IAM
dataset shows that our models are performing similar to earlier implementations
of state-of-the-art models. Our efficient NHR model and some of the reusable
techniques discussed with it offer ways to realize relatively efficient models
for the omnipresent scenario of variable-length inputs in deep learning.
Jointly Optimizing Diversity and Relevance in Neural Response Generation
Authors: Xiang Gao, Sungjin Lee, Yizhe Zhang, Chris Brockett, Michel Galley, Jianfeng Gao, Bill Dolan
Date: 2019-02-28 00:00:00
Proceeding: NAACL 2019 6
Tasks: Chatbot, Dialogue Generation, Diversity, Response Generation

Although recent neural conversation models have shown great potential, they
often generate bland and generic responses. While various approaches have been
explored to diversify the output of the conversation model, the improvement
often comes at the cost of decreased relevance. In this paper, we propose a
SpaceFusion model to jointly optimize diversity and relevance that essentially
fuses the latent space of a sequence-to-sequence model and that of an
autoencoder model by leveraging novel regularization terms. As a result, our
approach induces a latent space in which the distance and direction from the
predicted response vector roughly match the relevance and diversity,
respectively. This property also lends itself well to an intuitive
visualization of the latent space. Both automatic and human evaluation results
demonstrate that the proposed approach brings significant improvement compared
to strong baselines in both diversity and relevance.
Two-phase Hair Image Synthesis by Self-Enhancing Generative Model
Authors: Haonan Qiu, Chuan Wang, Hang Zhu, Xiangyu Zhu, Jinjin Gu, Xiaoguang Han
Date: 2019-02-28 00:00:00
Tasks: Image Generation, Image-to-Image Translation, Super-Resolution, Translation, Vocal Bursts Valence Prediction

Generating plausible hair image given limited guidance, such as sparse
sketches or low-resolution image, has been made possible with the rise of
Generative Adversarial Networks (GANs). Traditional image-to-image translation
networks can generate recognizable results, but finer textures are usually lost
and blur artifacts commonly exist. In this paper, we propose a two-phase
generative model for high-quality hair image synthesis. The two-phase pipeline
first generates a coarse image by an existing image translation model, then
applies a re-generating network with self-enhancing capability to the coarse
image. The self-enhancing capability is achieved by a proposed structure
extraction layer, which extracts the texture and orientation map from a hair
image. Extensive experiments on two tasks, Sketch2Hair and Hair
Super-Resolution, demonstrate that our approach is able to synthesize plausible
hair image with finer details, and outperforms the state-of-the-art.
Active Exploration in Markov Decision Processes
Authors: Jean Tarbouriech, Alessandro Lazaric
Date: 2019-02-28 00:00:00

We introduce the active exploration problem in Markov decision processes
(MDPs). Each state of the MDP is characterized by a random value and the
learner should gather samples to estimate the mean value of each state as
accurately as possible. Similarly to active exploration in multi-armed bandit
(MAB), states may have different levels of noise, so that the higher the noise,
the more samples are needed. As the noise level is initially unknown, we need
to trade off the exploration of the environment to estimate the noise and the
exploitation of these estimates to compute a policy maximizing the accuracy of
the mean predictions. We introduce a novel learning algorithm to solve this
problem showing that active exploration in MDPs may be significantly more
difficult than in MAB. We also derive a heuristic procedure to mitigate the
negative effect of slowly mixing policies. Finally, we validate our findings on
simple numerical simulations.
Semantics of higher-order probabilistic programs with conditioning
Authors: Fredrik Dahlqvist, Dexter Kozen
Date: 2019-02-28 00:00:00

We present a denotational semantics for higher-order probabilistic programs
in terms of linear operators between Banach spaces. Our semantics is rooted in
the classical theory of Banach spaces and their tensor products, but bears
similarities with the well-known Scott semantics of higher-order programs
through the use ordered Banach spaces which allow definitions in terms of fixed
points. Being based on a monoidal rather than cartesian closed structure, our
semantics effectively treats randomness as a resource.
FaceLiveNet+: A Holistic Networks For Face Authentication Based On Dynamic Multi-task Convolutional Neural Networks
Authors: Zuheng Ming, Junshi Xia, Muhammad Muzzamil Luqman, Jean-Christophe Burie, Kaixing Zhao
Date: 2019-02-28 00:00:00
Tasks: Face Verification, Facial Expression Recognition, Facial Expression Recognition (FER), Multi-Task Learning

This paper proposes a holistic multi-task Convolutional Neural Networks
(CNNs) with the dynamic weights of the tasks,namely FaceLiveNet+, for face
authentication. FaceLiveNet+ can employ face verification and facial expression
recognition as a solution of liveness control simultaneously. Comparing to the
single-task learning, the proposed multi-task learning can better capture the
feature representation for all of the tasks. The experimental results show the
superiority of the multi-task learning to the single-task learning for both the
face verification task and facial expression recognition task. Rather using a
conventional multi-task learning with fixed weights for the tasks, this work
proposes a so called dynamic-weight-unit to automatically learn the weights of
the tasks. The experiments have shown the effectiveness of the dynamic weights
for training the networks. Finally, the holistic evaluation for face
authentication based on the proposed protocol has shown the feasibility to
apply the FaceLiveNet+ for face authentication.
One-Shot Federated Learning
Authors: Neel Guha, Ameet Talwalkar, Virginia Smith
Date: 2019-02-28 00:00:00
Tasks: Ensemble Learning, Federated Learning

We present one-shot federated learning, where a central server learns a
global model over a network of federated devices in a single round of
communication. Our approach - drawing on ensemble learning and knowledge
aggregation - achieves an average relative gain of 51.5% in AUC over local
baselines and comes within 90.1% of the (unattainable) global ideal. We discuss
these methods and identify several promising directions of future work.
On the convex geometry of blind deconvolution and matrix completion
Authors: Felix Krahmer, Dominik Stöger
Date: 2019-02-28 00:00:00
Tasks: Matrix Completion

Low-rank matrix recovery from structured measurements has been a topic of intense study in the last decade and many important problems like matrix completion and blind deconvolution have been formulated in this framework. An important benchmark method to solve these problems is to minimize the nuclear norm, a convex proxy for the rank. A common approach to establish recovery guarantees for this convex program relies on the construction of a so-called approximate dual certificate. However, this approach provides only limited insight in various respects. Most prominently, the noise bounds exhibit seemingly suboptimal dimension factors. In this paper we take a novel, more geometric viewpoint to analyze both the matrix completion and the blind deconvolution scenario. We find that for both these applications the dimension factors in the noise bounds are not an artifact of the proof, but the problems are intrinsically badly conditioned. We show, however, that bad conditioning only arises for very small noise levels: Under mild assumptions that include many realistic noise levels we derive near-optimal error estimates for blind deconvolution under adversarial noise.
ROVO: Robust Omnidirectional Visual Odometry for Wide-baseline Wide-FOV Camera Systems
Authors: Hochang Seok, Jongwoo Lim
Date: 2019-02-28 00:00:00
Tasks: Motion Estimation, Pose Estimation, Visual Odometry

In this paper we propose a robust visual odometry system for a wide-baseline
camera rig with wide field-of-view (FOV) fisheye lenses, which provides full
omnidirectional stereo observations of the environment. For more robust and
accurate ego-motion estimation we adds three components to the standard VO
pipeline, 1) the hybrid projection model for improved feature matching, 2)
multi-view P3P RANSAC algorithm for pose estimation, and 3) online update of
rig extrinsic parameters. The hybrid projection model combines the perspective
and cylindrical projection to maximize the overlap between views and minimize
the image distortion that degrades feature matching performance. The multi-view
P3P RANSAC algorithm extends the conventional P3P RANSAC to multi-view images
so that all feature matches in all views are considered in the inlier counting
for robust pose estimation. Finally the online extrinsic calibration is
seamlessly integrated in the backend optimization framework so that the changes
in camera poses due to shocks or vibrations can be corrected automatically. The
proposed system is extensively evaluated with synthetic datasets with
ground-truth and real sequences of highly dynamic environment, and its superior
performance is demonstrated.
Adversarial Training for Satire Detection: Controlling for Confounding Variables
Authors: Robert McHardy, Heike Adel, Roman Klinger
Date: 2019-02-28 00:00:00
Proceeding: NAACL 2019 6
Tasks: General Classification, Knowledge Base Population, Satire Detection

The automatic detection of satire vs. regular news is relevant for downstream
applications (for instance, knowledge base population) and to improve the
understanding of linguistic characteristics of satire. Recent approaches build
upon corpora which have been labeled automatically based on article sources. We
hypothesize that this encourages the models to learn characteristics for
different publication sources (e.g., "The Onion" vs. "The Guardian") rather
than characteristics of satire, leading to poor generalization performance to
unseen publication sources. We therefore propose a novel model for satire
detection with an adversarial component to control for the confounding variable
of publication source. On a large novel data set collected from German news
(which we make available to the research community), we observe comparable
satire classification performance and, as desired, a considerable drop in
publication classification performance with adversarial training. Our analysis
shows that the adversarial component is crucial for the model to learn to pay
attention to linguistic properties of satire.
Citation Needed: A Taxonomy and Algorithmic Assessment of Wikipedia's Verifiability
Authors: Miriam Redi, Besnik Fetahu, Jonathan Morgan, Dario Taraborelli
Date: 2019-02-28 00:00:00
Tasks: Articles, Fact Checking

Wikipedia is playing an increasingly central role on the web,and the policies
its contributors follow when sourcing and fact-checking content affect million
of readers. Among these core guiding principles, verifiability policies have a
particularly important role. Verifiability requires that information included
in a Wikipedia article be corroborated against reliable secondary sources.
Because of the manual labor needed to curate and fact-check Wikipedia at scale,
however, its contents do not always evenly comply with these policies.
Citations (i.e. reference to external sources) may not conform to verifiability
requirements or may be missing altogether, potentially weakening the
reliability of specific topic areas of the free encyclopedia. In this paper, we
aim to provide an empirical characterization of the reasons why and how
Wikipedia cites external sources to comply with its own verifiability
guidelines. First, we construct a taxonomy of reasons why inline citations are
required by collecting labeled data from editors of multiple Wikipedia language
editions. We then collect a large-scale crowdsourced dataset of Wikipedia
sentences annotated with categories derived from this taxonomy. Finally, we
design and evaluate algorithmic models to determine if a statement requires a
citation, and to predict the citation reason based on our taxonomy. We evaluate
the robustness of such models across different classes of Wikipedia articles of
varying quality, as well as on an additional dataset of claims annotated for
fact-checking purposes.
Tensor-variate Mixture of Experts for Proportional Myographic Control of a Robotic Hand
Authors: Noémie Jaquier, Robert Haschke, Sylvain Calinon
Date: 2019-02-28 00:00:00
Tasks: Mixture-of-Experts, regression

When data are organized in matrices or arrays of higher dimensions (tensors), classical regression methods first transform these data into vectors, therefore ignoring the underlying structure of the data and increasing the dimensionality of the problem. This flattening operation typically leads to overfitting when only few training data is available. In this paper, we present a mixture-of-experts model that exploits tensorial representations for regression of tensor-valued data. The proposed formulation takes into account the underlying structure of the data and remains efficient when few training data are available. Evaluation on artificially generated data, as well as offline and real-time experiments recognizing hand movements from tactile myography prove the effectiveness of the proposed approach.