pwc-archive/papers-with-abstracts huggingface dataset:
An Online Learning Algorithm for Neuromorphic Hardware Implementation
Authors: Chetan Singh Thakur, Runchun Wang, Saeed Afshar, Gregory Cohen, Tara Julia Hamilton, Jonathan Tapson, Andre van Schaik
Date: 2015-05-11 00:00:00
Tasks: regression

We propose a sign-based online learning (SOL) algorithm for a neuromorphic
hardware framework called Trainable Analogue Block (TAB). The TAB framework
utilises the principles of neural population coding, implying that it encodes
the input stimulus using a large pool of nonlinear neurons. The SOL algorithm
is a simple weight update rule that employs the sign of the hidden layer
activation and the sign of the output error, which is the difference between
the target output and the predicted output. The SOL algorithm is easily
implementable in hardware, and can be used in any artificial neural network
framework that learns weights by minimising a convex cost function. We show
that the TAB framework can be trained for various regression tasks using the
SOL algorithm.
Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs
Authors: Yu. A. Malkov, D. A. Yashunin
Date: 2016-03-30 00:00:00

We present a new approach for the approximate K-nearest neighbor search based
on navigable small world graphs with controllable hierarchy (Hierarchical NSW,
HNSW). The proposed solution is fully graph-based, without any need for
additional search structures, which are typically used at the coarse search
stage of the most proximity graph techniques. Hierarchical NSW incrementally
builds a multi-layer structure consisting from hierarchical set of proximity
graphs (layers) for nested subsets of the stored elements. The maximum layer in
which an element is present is selected randomly with an exponentially decaying
probability distribution. This allows producing graphs similar to the
previously studied Navigable Small World (NSW) structures while additionally
having the links separated by their characteristic distance scales. Starting
search from the upper layer together with utilizing the scale separation boosts
the performance compared to NSW and allows a logarithmic complexity scaling.
Additional employment of a heuristic for selecting proximity graph neighbors
significantly increases performance at high recall and in case of highly
clustered data. Performance evaluation has demonstrated that the proposed
general metric space search index is able to strongly outperform previous
opensource state-of-the-art vector-only approaches. Similarity of the algorithm
to the skip list structure allows straightforward balanced distributed
implementation.
Joint Named Entity Recognition and Stance Detection in Tweets
Authors: Dilek Küçük
Date: 2017-07-30 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Sentiment Analysis, Stance Detection

Named entity recognition (NER) is a well-established task of information
extraction which has been studied for decades. More recently, studies reporting
NER experiments on social media texts have emerged. On the other hand, stance
detection is a considerably new research topic usually considered within the
scope of sentiment analysis. Stance detection studies are mostly applied to
texts of online debates where the stance of the text owner for a particular
target, either explicitly or implicitly mentioned in text, is explored. In this
study, we investigate the possible contribution of named entities to the stance
detection task in tweets. We report the evaluation results of NER experiments
as well as that of the subsequent stance detection experiments using named
entities, on a publicly-available stance-annotated data set of tweets. Our
results indicate that named entities obtained with a high-performance NER
system can contribute to stance detection performance on tweets.
An Augmented Lagrangian Method for Piano Transcription using Equal Loudness Thresholding and LSTM-based Decoding
Authors: Sebastian Ewert, Mark B. Sandler
Date: 2017-07-01 00:00:00
Tasks: Music Transcription, parameter estimation

A central goal in automatic music transcription is to detect individual note
events in music recordings. An important variant is instrument-dependent music
transcription where methods can use calibration data for the instruments in
use. However, despite the additional information, results rarely exceed an
f-measure of 80%. As a potential explanation, the transcription problem can be
shown to be badly conditioned and thus relies on appropriate regularization. A
recently proposed method employs a mixture of simple, convex regularizers (to
stabilize the parameter estimation process) and more complex terms (to
encourage more meaningful structure). In this paper, we present two extensions
to this method. First, we integrate a computational loudness model to better
differentiate real from spurious note detections. Second, we employ
(Bidirectional) Long Short Term Memory networks to re-weight the likelihood of
detected note constellations. Despite their simplicity, our two extensions lead
to a drop of about 35% in note error rate compared to the state-of-the-art.
Occlusion Handling using Semantic Segmentation and Visibility-Based Rendering for Mixed Reality
Authors: Menandro Roxas, Tomoki Hori, Taiki Fukiage, Yasuhide Okamoto, Takeshi Oishi
Date: 2017-07-30 00:00:00
Tasks: Depth Estimation, Mixed Reality, Occlusion Handling, Optical Flow Estimation, Segmentation, Semantic Segmentation

Real-time occlusion handling is a major problem in outdoor mixed reality
system because it requires great computational cost mainly due to the
complexity of the scene. Using only segmentation, it is difficult to accurately
render a virtual object occluded by complex objects such as trees, bushes etc.
In this paper, we propose a novel occlusion handling method for real-time,
outdoor, and omni-directional mixed reality system using only the information
from a monocular image sequence. We first present a semantic segmentation
scheme for predicting the amount of visibility for different type of objects in
the scene. We also simultaneously calculate a foreground probability map using
depth estimation derived from optical flow. Finally, we combine the
segmentation result and the probability map to render the computer generated
object and the real scene using a visibility-based rendering method. Our
results show great improvement in handling occlusions compared to existing
blending based methods.
Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization
Authors: Xun Huang, Serge Belongie
Date: 2017-03-20 00:00:00
Proceeding: ICCV 2017 10
Tasks: Style Transfer

Gatys et al. recently introduced a neural algorithm that renders a content
image in the style of another image, achieving so-called style transfer.
However, their framework requires a slow iterative optimization process, which
limits its practical application. Fast approximations with feed-forward neural
networks have been proposed to speed up neural style transfer. Unfortunately,
the speed improvement comes at a cost: the network is usually tied to a fixed
set of styles and cannot adapt to arbitrary new styles. In this paper, we
present a simple yet effective approach that for the first time enables
arbitrary style transfer in real-time. At the heart of our method is a novel
adaptive instance normalization (AdaIN) layer that aligns the mean and variance
of the content features with those of the style features. Our method achieves
speed comparable to the fastest existing approach, without the restriction to a
pre-defined set of styles. In addition, our approach allows flexible user
controls such as content-style trade-off, style interpolation, color & spatial
controls, all using a single feed-forward neural network.
ScanNet: A Fast and Dense Scanning Framework for Metastatic Breast Cancer Detection from Whole-Slide Images
Authors: Huangjing Lin, Hao Chen, Qi Dou, Liansheng Wang, Jing Qin, Pheng-Ann Heng
Date: 2017-07-30 00:00:00
Tasks: Breast Cancer Detection, Diagnostic, whole slide images

Lymph node metastasis is one of the most significant diagnostic indicators in
breast cancer, which is traditionally observed under the microscope by
pathologists. In recent years, computerized histology diagnosis has become one
of the most rapidly expanding fields in medical image computing, which
alleviates pathologists' workload and reduces misdiagnosis rate. However,
automatic detection of lymph node metastases from whole slide images remains a
challenging problem, due to the large-scale data with enormous resolutions and
existence of hard mimics. In this paper, we propose a novel framework by
leveraging fully convolutional networks for efficient inference to meet the
speed requirement for clinical practice, while reconstructing dense predictions
under different offsets for ensuring accurate detection on both micro- and
macro-metastases. Incorporating with the strategies of asynchronous sample
prefetching and hard negative mining, the network can be effectively trained.
Extensive experiments on the benchmark dataset of 2016 Camelyon Grand Challenge
corroborated the efficacy of our method. Compared with the state-of-the-art
methods, our method achieved superior performance with a faster speed on the
tumor localization task and surpassed human performance on the WSI
classification task.
Robust, Deep and Inductive Anomaly Detection
Authors: Raghavendra Chalapathy, Aditya Krishna Menon, Sanjay Chawla
Date: 2017-04-22 00:00:00
Tasks: Anomaly Detection

PCA is a classical statistical technique whose simplicity and maturity has
seen it find widespread use as an anomaly detection technique. However, it is
limited in this regard by being sensitive to gross perturbations of the input,
and by seeking a linear subspace that captures normal behaviour. The first
issue has been dealt with by robust PCA, a variant of PCA that explicitly
allows for some data points to be arbitrarily corrupted, however, this does not
resolve the second issue, and indeed introduces the new issue that one can no
longer inductively find anomalies on a test set. This paper addresses both
issues in a single model, the robust autoencoder. This method learns a
nonlinear subspace that captures the majority of data points, while allowing
for some data to have arbitrary corruption. The model is simple to train and
leverages recent advances in the optimisation of deep neural networks.
Experiments on a range of real-world datasets highlight the model's
effectiveness.
Discover and Learn New Objects from Documentaries
Authors: Kai Chen, Hang Song, Chen Change Loy, Dahua Lin
Date: 2017-07-30 00:00:00
Proceeding: CVPR 2017 7
Tasks: Object, Weakly-supervised Learning

Despite the remarkable progress in recent years, detecting objects in a new
context remains a challenging task. Detectors learned from a public dataset can
only work with a fixed list of categories, while training from scratch usually
requires a large amount of training data with detailed annotations. This work
aims to explore a novel approach -- learning object detectors from documentary
films in a weakly supervised manner. This is inspired by the observation that
documentaries often provide dedicated exposition of certain object categories,
where visual presentations are aligned with subtitles. We believe that object
detectors can be learned from such a rich source of information. Towards this
goal, we develop a joint probabilistic framework, where individual pieces of
information, including video frames and subtitles, are brought together via
both visual and linguistic links. On top of this formulation, we further derive
a weakly supervised learning algorithm, where object model learning and
training set mining are unified in an optimization procedure. Experimental
results on a real world dataset demonstrate that this is an effective approach
to learning new object detectors.
Deep Functional Maps: Structured Prediction for Dense Shape Correspondence
Authors: Or Litany, Tal Remez, Emanuele Rodolà, Alex M. Bronstein, Michael M. Bronstein
Date: 2017-04-27 00:00:00
Proceeding: ICCV 2017 10
Tasks: Structured Prediction

We introduce a new framework for learning dense correspondence between
deformable 3D shapes. Existing learning based approaches model shape
correspondence as a labelling problem, where each point of a query shape
receives a label identifying a point on some reference domain; the
correspondence is then constructed a posteriori by composing the label
predictions of two input shapes. We propose a paradigm shift and design a
structured prediction model in the space of functional maps, linear operators
that provide a compact representation of the correspondence. We model the
learning process via a deep residual network which takes dense descriptor
fields defined on two shapes as input, and outputs a soft map between the two
given objects. The resulting correspondence is shown to be accurate on several
challenging benchmarks comprising multiple categories, synthetic models, real
scans with acquisition artifacts, topological noise, and partiality.
Virtual PET Images from CT Data Using Deep Convolutional Networks: Initial Results
Authors: Avi Ben-Cohen, Eyal Klang, Stephen P. Raskin, Michal Marianne Amitai, Hayit Greenspan
Date: 2017-07-30 00:00:00

In this work we present a novel system for PET estimation using CT scans. We
explore the use of fully convolutional networks (FCN) and conditional
generative adversarial networks (GAN) to export PET data from CT data. Our
dataset includes 25 pairs of PET and CT scans where 17 were used for training
and 8 for testing. The system was tested for detection of malignant tumors in
the liver region. Initial results look promising showing high detection
performance with a TPR of 92.3% and FPR of 0.25 per case. Future work entails
expansion of the current system to the entire body using a much larger dataset.
Such a system can be used for tumor detection and drug treatment evaluation in
a CT-only environment instead of the expansive and radioactive PET-CT scan.
A Multivariate Hawkes Process with Gaps in Observations
Authors: Triet M Le
Date: 2016-08-03 00:00:00
Tasks: Point Processes

Given a collection of entities (or nodes) in a network and our intermittent
observations of activities from each entity, an important problem is to learn
the hidden edges depicting directional relationships among these entities.
Here, we study causal relationships (excitations) that are realized by a
multivariate Hawkes process. The multivariate Hawkes process (MHP) and its
variations (spatio-temporal point processes) have been used to study contagion
in earthquakes, crimes, neural spiking activities, the stock and foreign
exchange markets, etc. In this paper, we consider the multivariate Hawkes
process with gaps in observations (MHPG). We propose a variational problem for
detecting sparsely hidden relationships with a multivariate Hawkes process that
takes into account the gaps from each entity. We bypass the problem of dealing
with a large amount of missing events by introducing a small number of unknown
boundary conditions. In the case where our observations are sparse (e.g. from
10% to 30%), we show through numerical simulations that robust recovery with
MHPG is still possible even if the lengths of the observed intervals are small
but they are chosen accordingly. The numerical results also show that the
knowledge of gaps and imposing the right boundary conditions are very crucial
in discovering the underlying patterns and hidden relationships.
Comparing Information-Theoretic Measures of Complexity in Boltzmann Machines
Authors: Maxinder S. Kanwal, Joshua A. Grochow, Nihat Ay
Date: 2017-06-29 00:00:00

In the past three decades, many theoretical measures of complexity have been
proposed to help understand complex systems. In this work, for the first time,
we place these measures on a level playing field, to explore the qualitative
similarities and differences between them, and their shortcomings.
Specifically, using the Boltzmann machine architecture (a fully connected
recurrent neural network) with uniformly distributed weights as our model of
study, we numerically measure how complexity changes as a function of network
dynamics and network parameters. We apply an extension of one such
information-theoretic measure of complexity to understand incremental Hebbian
learning in Hopfield networks, a fully recurrent architecture model of
autoassociative memory. In the course of Hebbian learning, the total
information flow reflects a natural upward trend in complexity as the network
attempts to learn more and more patterns.
Learning Language Representations for Typology Prediction
Authors: Chaitanya Malaviya, Graham Neubig, Patrick Littell
Date: 2017-07-29 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Machine Translation, NMT, Prediction, Translation

One central mystery of neural NLP is what neural models "know" about their
subject matter. When a neural machine translation system learns to translate
from one language to another, does it learn the syntax or semantics of the
languages? Can this knowledge be extracted from the system to fill holes in
human scientific knowledge? Existing typological databases contain relatively
full feature specifications for only a few hundred languages. Exploiting the
existence of parallel texts in more than a thousand languages, we build a
massive many-to-one neural machine translation (NMT) system from 1017 languages
into English, and use this to predict information missing from typological
databases. Experiments show that the proposed method is able to infer not only
syntactic, but also phonological and phonetic inventory features, and improves
over a baseline that has access to information about the languages' geographic
and phylogenetic neighbors.
Fully Decentralized Policies for Multi-Agent Systems: An Information Theoretic Approach
Authors: Roel Dobbe, David Fridovich-Keil, Claire Tomlin
Date: 2017-07-20 00:00:00
Proceeding: NeurIPS 2017 12

Learning cooperative policies for multi-agent systems is often challenged by
partial observability and a lack of coordination. In some settings, the
structure of a problem allows a distributed solution with limited
communication. Here, we consider a scenario where no communication is
available, and instead we learn local policies for all agents that collectively
mimic the solution to a centralized multi-agent static optimization problem.
Our main contribution is an information theoretic framework based on rate
distortion theory which facilitates analysis of how well the resulting fully
decentralized policies are able to reconstruct the optimal solution. Moreover,
this framework provides a natural extension that addresses which nodes an agent
should communicate with to improve the performance of its individual policy.
MIHash: Online Hashing with Mutual Information
Authors: Fatih Cakir, Kun He, Sarah Adel Bargal, Stan Sclaroff
Date: 2017-03-27 00:00:00
Proceeding: ICCV 2017 10
Tasks: Image Retrieval, Retrieval

Learning-based hashing methods are widely used for nearest neighbor
retrieval, and recently, online hashing methods have demonstrated good
performance-complexity trade-offs by learning hash functions from streaming
data. In this paper, we first address a key challenge for online hashing: the
binary codes for indexed data must be recomputed to keep pace with updates to
the hash functions. We propose an efficient quality measure for hash functions,
based on an information-theoretic quantity, mutual information, and use it
successfully as a criterion to eliminate unnecessary hash table updates. Next,
we also show how to optimize the mutual information objective using stochastic
gradient descent. We thus develop a novel hashing method, MIHash, that can be
used in both online and batch settings. Experiments on image retrieval
benchmarks (including a 2.5M image dataset) confirm the effectiveness of our
formulation, both in reducing hash table recomputations and in learning
high-quality hash functions.
Fine-Gray competing risks model with high-dimensional covariates: estimation and Inference
Authors: Jue Hou, Jelena Bradic, Ronghui Xu
Date: 2017-07-29 00:00:00

The purpose of this paper is to construct confidence intervals for the
regression coefficients in the Fine-Gray model for competing risks data with
random censoring, where the number of covariates can be larger than the sample
size. Despite strong motivation from biomedical applications, a
high-dimensional Fine-Gray model has attracted relatively little attention
among the methodological or theoretical literature. We fill in this gap by
developing confidence intervals based on a one-step bias-correction for a
regularized estimation. We develop a theoretical framework for the partial
likelihood, which does not have independent and identically distributed entries
and therefore presents many technical challenges. We also study the
approximation error from the weighting scheme under random censoring for
competing risks and establish new concentration results for time-dependent
processes. In addition to the theoretical results and algorithms, we present
extensive numerical experiments and an application to a study of non-cancer
mortality among prostate cancer patients using the linked Medicare-SEER data.
KNN Ensembles for Tweedie Regression: The Power of Multiscale Neighborhoods
Authors: Colleen M. Farrelly
Date: 2017-07-29 00:00:00
Tasks: Outlier Detection, regression, Topological Data Analysis

Very few K-nearest-neighbor (KNN) ensembles exist, despite the efficacy of
this approach in regression, classification, and outlier detection. Those that
do exist focus on bagging features, rather than varying k or bagging
observations; it is unknown whether varying k or bagging observations can
improve prediction. Given recent studies from topological data analysis,
varying k may function like multiscale topological methods, providing stability
and better prediction, as well as increased ensemble diversity.
  This paper explores 7 KNN ensemble algorithms combining bagged features,
bagged observations, and varied k to understand how each of these contribute to
model fit. Specifically, these algorithms are tested on Tweedie regression
problems through simulations and 6 real datasets; results are compared to
state-of-the-art machine learning models including extreme learning machines,
random forest, boosted regression, and Morse-Smale regression.
  Results on simulations suggest gains from varying k above and beyond bagging
features or samples, as well as the robustness of KNN ensembles to the curse of
dimensionality. KNN regression ensembles perform favorably against
state-of-the-art algorithms and dramatically improve performance over KNN
regression. Further, real dataset results suggest varying k is a good strategy
in general (particularly for difficult Tweedie regression problems) and that
KNN regression ensembles often outperform state-of-the-art methods.
  These results for k-varying ensembles echo recent theoretical results in
topological data analysis, where multidimensional filter functions and
multiscale coverings provide stability and performance gains over
single-dimensional filters and single-scale covering. This opens up the
possibility of leveraging multiscale neighborhoods and multiple measures of
local geometry in ensemble methods.