pwc-archive/papers-with-abstracts huggingface dataset:
Norm-Range Partition: A Universal Catalyst for LSH based Maximum Inner Product Search (MIPS)
Authors: Xiao Yan, Xinyan Dai, Jie Liu, Kaiwen Zhou, James Cheng
Date: 2018-10-22 00:00:00

Recently, locality sensitive hashing (LSH) was shown to be effective for MIPS
and several algorithms including $L_2$-ALSH, Sign-ALSH and Simple-LSH have been
proposed. In this paper, we introduce the norm-range partition technique, which
partitions the original dataset into sub-datasets containing items with similar
2-norms and builds hash index independently for each sub-dataset. We prove that
norm-range partition reduces the query processing complexity for all existing
LSH based MIPS algorithms under mild conditions. The key to performance
improvement is that norm-range partition allows to use smaller normalization
factor most sub-datasets. For efficient query processing, we also formulate a
unified framework to rank the buckets from the hash indexes of different
sub-datasets. Experiments on real datasets show that norm-range partition
significantly reduces the number of probed for LSH based MIPS algorithms when
achieving the same recall.
Greedy Actor-Critic: A New Conditional Cross-Entropy Method for Policy Improvement
Authors: Samuel Neumann, Sungsu Lim, Ajin Joseph, Yangchen Pan, Adam White, Martha White
Date: 2018-10-22 00:00:00
Tasks: Policy Gradient Methods, Q-Learning

Many policy gradient methods are variants of Actor-Critic (AC), where a value function (critic) is learned to facilitate updating the parameterized policy (actor). The update to the actor involves a log-likelihood update weighted by the action-values, with the addition of entropy regularization for soft variants. In this work, we explore an alternative update for the actor, based on an extension of the cross entropy method (CEM) to condition on inputs (states). The idea is to start with a broader policy and slowly concentrate around maximal actions, using a maximum likelihood update towards actions in the top percentile per state. The speed of this concentration is controlled by a proposal policy, that concentrates at a slower rate than the actor. We first provide a policy improvement result in an idealized setting, and then prove that our conditional CEM (CCEM) strategy tracks a CEM update per state, even with changing action-values. We empirically show that our Greedy AC algorithm, that uses CCEM for the actor update, performs better than Soft Actor-Critic and is much less sensitive to entropy-regularization.
Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?
Authors: Nitin Bansal, Xiaohan Chen, Zhangyang Wang
Date: 2018-10-22 00:00:00
Proceeding: NeurIPS 2018

This paper seeks to answer the question: as the (near-) orthogonality of
weights is found to be a favorable property for training deep convolutional
neural networks, how can we enforce it in more effective and easy-to-use ways?
We develop novel orthogonality regularizations on training deep CNNs, utilizing
various advanced analytical tools such as mutual coherence and restricted
isometry property. These plug-and-play regularizations can be conveniently
incorporated into training almost any CNN without extra hassle. We then
benchmark their effects on state-of-the-art models: ResNet, WideResNet, and
ResNeXt, on several most popular computer vision datasets: CIFAR-10, CIFAR-100,
SVHN and ImageNet. We observe consistent performance gains after applying those
proposed regularizations, in terms of both the final accuracies achieved, and
faster and more stable convergences. We have made our codes and pre-trained
models publicly available:
https://github.com/nbansal90/Can-we-Gain-More-from-Orthogonality.
Stochastic Gradient MCMC for State Space Models
Authors: Christopher Aicher, Yi-An Ma, Nicholas J. Foti, Emily B. Fox
Date: 2018-10-22 00:00:00
Tasks: Bayesian Inference, State Space Models, Time Series, Time Series Analysis

State space models (SSMs) are a flexible approach to modeling complex time series. However, inference in SSMs is often computationally prohibitive for long time series. Stochastic gradient MCMC (SGMCMC) is a popular method for scalable Bayesian inference for large independent data. Unfortunately when applied to dependent data, such as in SSMs, SGMCMC's stochastic gradient estimates are biased as they break crucial temporal dependencies. To alleviate this, we propose stochastic gradient estimators that control this bias by performing additional computation in a `buffer' to reduce breaking dependencies. Furthermore, we derive error bounds for this bias and show a geometric decay under mild conditions. Using these estimators, we develop novel SGMCMC samplers for discrete, continuous and mixed-type SSMs with analytic message passing. Our experiments on real and synthetic data demonstrate the effectiveness of our SGMCMC algorithms compared to batch MCMC, allowing us to scale inference to long time series with millions of time points.
Axiomatic Interpretability for Multiclass Additive Models
Authors: Xuezhou Zhang, Sarah Tan, Paul Koch, Yin Lou, Urszula Chajewska, Rich Caruana
Date: 2018-10-22 00:00:00
Tasks: Additive models, Binary Classification, regression

Generalized additive models (GAMs) are favored in many regression and binary classification problems because they are able to fit complex, nonlinear functions while still remaining interpretable. In the first part of this paper, we generalize a state-of-the-art GAM learning algorithm based on boosted trees to the multiclass setting, and show that this multiclass algorithm outperforms existing GAM learning algorithms and sometimes matches the performance of full complexity models such as gradient boosted trees. In the second part, we turn our attention to the interpretability of GAMs in the multiclass setting. Surprisingly, the natural interpretability of GAMs breaks down when there are more than two classes. Naive interpretation of multiclass GAMs can lead to false conclusions. Inspired by binary GAMs, we identify two axioms that any additive model must satisfy in order to not be visually misleading. We then develop a technique called Additive Post-Processing for Interpretability (API), that provably transforms a pre-trained additive model to satisfy the interpretability axioms without sacrificing accuracy. The technique works not just on models trained with our learning algorithm, but on any multiclass additive model, including multiclass linear and logistic regression. We demonstrate the effectiveness of API on a 12-class infant mortality dataset.
SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation
Authors: Xiaolin Zhang, Yunchao Wei, Yi Yang, Thomas Huang
Date: 2018-10-22 00:00:00
Tasks: Few-Shot Semantic Segmentation, Segmentation, Semantic Segmentation

One-shot image semantic segmentation poses a challenging task of recognizing the object regions from unseen categories with only one annotated example as supervision. In this paper, we propose a simple yet effective Similarity Guidance network to tackle the One-shot (SG-One) segmentation problem. We aim at predicting the segmentation mask of a query image with the reference to one densely labeled support image of the same category. To obtain the robust representative feature of the support image, we firstly adopt a masked average pooling strategy for producing the guidance features by only taking the pixels belonging to the support image into account. We then leverage the cosine similarity to build the relationship between the guidance features and features of pixels from the query image. In this way, the possibilities embedded in the produced similarity maps can be adapted to guide the process of segmenting objects. Furthermore, our SG-One is a unified framework which can efficiently process both support and query images within one network and be learned in an end-to-end manner. We conduct extensive experiments on Pascal VOC 2012. In particular, our SGOne achieves the mIoU score of 46.3%, surpassing the baseline methods.
A general learning system based on neuron bursting and tonic firing
Authors: Hin Wai Lui
Date: 2018-10-22 00:00:00
Tasks: Hippocampus

This paper proposes a framework for the biological learning mechanism as a
general learning system. The proposal is as follows. The bursting and tonic
modes of firing patterns found in many neuron types in the brain correspond to
two separate modes of information processing, with one mode resulting in
awareness, and another mode being subliminal. In such a coding scheme, a neuron
in bursting state codes for the highest level of perceptual abstraction
representing a pattern of sensory stimuli, or volitional abstraction
representing a pattern of muscle contraction sequences. Within the 50-250 ms
minimum integration time of experience, the bursting neurons form synchrony
ensembles to allow for binding of related percepts. The degree which different
bursting neurons can be merged into the same synchrony ensemble depends on the
underlying cortical connections that represent the degree of perceptual
similarity. These synchrony ensembles compete for selective attention to remain
active. The dominant synchrony ensemble triggers episodic memory recall in the
hippocampus, while forming new episodic memory with current sensory stimuli,
resulting in a stream of thoughts. Neuromodulation modulates both top-down
selection of synchrony ensembles, and memory formation. Episodic memory stored
in the hippocampus is transferred to semantic and procedural memory in the
cortex during rapid eye movement sleep, by updating cortical neuron synaptic
weights with spike timing dependent plasticity. With the update of synaptic
weights, new neurons become bursting while previous bursting neurons become
tonic, allowing bursting neurons to move up to a higher level of perceptual
abstraction. Finally, the proposed learning mechanism is compared with the
back-propagation algorithm used in deep neural networks, and a proposal of how
the credit assignment problem can be addressed by the current proposal is
presented.
Sparsemax and Relaxed Wasserstein for Topic Sparsity
Authors: Tianyi Lin, Zhiyue Hu, Xin Guo
Date: 2018-10-22 00:00:00

Topic sparsity refers to the observation that individual documents usually
focus on several salient topics instead of covering a wide variety of topics,
and a real topic adopts a narrow range of terms instead of a wide coverage of
the vocabulary. Understanding this topic sparsity is especially important for
analyzing user-generated web content and social media, which are featured in
the form of extremely short posts and discussions. As topic sparsity of
individual documents in online social media increases, so does the difficulty
of analyzing the online text sources using traditional methods.
  In this paper, we propose two novel neural models by providing sparse
posterior distributions over topics based on the Gaussian sparsemax
construction, enabling efficient training by stochastic backpropagation. We
construct an inference network conditioned on the input data and infer the
variational distribution with the relaxed Wasserstein (RW) divergence. Unlike
existing works based on Gaussian softmax construction and Kullback-Leibler (KL)
divergence, our approaches can identify latent topic sparsity with training
stability, predictive performance, and topic coherence. Experiments on
different genres of large text corpora have demonstrated the effectiveness of
our models as they outperform both probabilistic and neural methods.
Our Practice Of Using Machine Learning To Recognize Species By Voice
Authors: Siddhardha Balemarthy, Atul Sajjanhar, James Xi Zheng
Date: 2018-10-22 00:00:00
Tasks: BIG-bench Machine Learning

As the technology is advancing, audio recognition in machine learning is
improved as well. Research in audio recognition has traditionally focused on
speech. Living creatures (especially the small ones) are part of the whole
ecosystem, monitoring as well as maintaining them are important tasks. Species
such as animals and birds are tending to change their activities as well as
their habitats due to the adverse effects on the environment or due to other
natural or man-made calamities. For those in far deserted areas, we will not
have any idea about their existence until we can continuously monitor them.
Continuous monitoring will take a lot of hard work and labor. If there is no
continuous monitoring, then there might be instances where endangered species
may encounter dangerous situations. The best way to monitor those species are
through audio recognition. Classifying sound can be a difficult task even for
humans. Powerful audio signals and their processing techniques make it possible
to detect audio of various species. There might be many ways wherein audio
recognition can be done. We can train machines either by pre-recorded audio
files or by recording them live and detecting them. The audio of species can be
detected by removing all the background noise and echoes. Smallest sound is
considered as a syllable. Extracting various syllables is the process we are
focusing on which is known as audio recognition in terms of Machine Learning
(ML).
Atrial fibrosis quantification based on maximum likelihood estimator of multivariate images
Authors: Fuping Wu, Lei LI, Guang Yang, Tom Wong, Raad Mohiaddin, David Firmin, Jennifer Keegan, Lingchao Xu, Xiahai Zhuang
Date: 2018-10-22 00:00:00
Tasks: Segmentation, Texture Classification

We present a fully-automated segmentation and quantification of the left
atrial (LA) fibrosis and scars combining two cardiac MRIs, one is the target
late gadolinium-enhanced (LGE) image, and the other is an anatomical MRI from
the same acquisition session. We formulate the joint distribution of images
using a multivariate mixture model (MvMM), and employ the maximum likelihood
estimator (MLE) for texture classification of the images simultaneously. The
MvMM can also embed transformations assigned to the images to correct the
misregistration. The iterated conditional mode algorithm is adopted for
optimization. This method first extracts the anatomical shape of the LA, and
then estimates a prior probability map. It projects the resulting segmentation
onto the LA surface, for quantification and analysis of scarring. We applied
the proposed method to 36 clinical data sets and obtained promising results
(Accuracy: $0.809\pm .150$, Dice: $0.556\pm.187$). We compared the method with
the conventional algorithms and showed an evidently and statistically better
performance ($p<0.03$).
Labeling Gaps Between Words: Recognizing Overlapping Mentions with Mention Separators
Authors: Aldrian Obaja Muis, Wei Lu
Date: 2018-10-22 00:00:00
Proceeding: EMNLP 2017 9

In this paper, we propose a new model that is capable of recognizing
overlapping mentions. We introduce a novel notion of mention separators that
can be effectively used to capture how mentions overlap with one another. On
top of a novel multigraph representation that we introduce, we show that
efficient and exact inference can still be performed. We present some
theoretical analysis on the differences between our model and a recently
proposed model for recognizing overlapping mentions, and discuss the possible
implications of the differences. Through extensive empirical analysis on
standard datasets, we demonstrate the effectiveness of our approach.
Learning from the Kernel and the Range Space
Authors: Kar-Ann Toh
Date: 2018-10-22 00:00:00

In this article, a novel approach to learning a complex function which can be
written as the system of linear equations is introduced. This learning is
grounded upon the observation that solving the system of linear equations by a
manipulation in the kernel and the range space boils down to an estimation
based on the least squares error approximation. The learning approach is
applied to learn a deep feedforward network with full weight connections. The
numerical experiments on network learning of synthetic and benchmark data not
only show feasibility of the proposed learning approach but also provide
insights into the mechanism of data representation.
Where is this? Video geolocation based on neural network features
Authors: Salvador Medina, Zhuyun Dai, Yingkai Gao
Date: 2018-10-22 00:00:00
Tasks: Image Retrieval, Retrieval

In this work we propose a method that geolocates videos within a delimited
widespread area based solely on the frames visual content. Our proposed method
tackles video-geolocation through traditional image retrieval techniques
considering Google Street View as the reference point. To achieve this goal we
use the deep learning features obtained from NetVLAD to represent images, since
through this feature vectors the similarity is their L2 norm. In this paper, we
propose a family of voting-based methods to aggregate frame-wise geolocation
results which boost the video geolocation result. The best aggregation found
through our experiments considers both NetVLAD and SIFT similarity, as well as
the geolocation density of the most similar results. To test our proposed
method, we gathered a new video dataset from Pittsburgh Downtown area to
benefit and stimulate more work in this area. Our system achieved a precision
of 90% while geolocating videos within a range of 150 meters or two blocks away
from the original position.
VIENA2: A Driving Anticipation Dataset
Authors: Mohammad Sadegh Aliakbarian, Fatemeh Sadat Saleh, Mathieu Salzmann, Basura Fernando, Lars Petersson, Lars Andersson
Date: 2018-10-22 00:00:00
Tasks: Action Anticipation

Action anticipation is critical in scenarios where one needs to react before
the action is finalized. This is, for instance, the case in automated driving,
where a car needs to, e.g., avoid hitting pedestrians and respect traffic
lights. While solutions have been proposed to tackle subsets of the driving
anticipation tasks, by making use of diverse, task-specific sensors, there is
no single dataset or framework that addresses them all in a consistent manner.
In this paper, we therefore introduce a new, large-scale dataset, called
VIENA2, covering 5 generic driving scenarios, with a total of 25 distinct
action classes. It contains more than 15K full HD, 5s long videos acquired in
various driving conditions, weathers, daytimes and environments, complemented
with a common and realistic set of sensor measurements. This amounts to more
than 2.25M frames, each annotated with an action label, corresponding to 600
samples per action class. We discuss our data acquisition strategy and the
statistics of our dataset, and benchmark state-of-the-art action anticipation
techniques, including a new multi-modal LSTM architecture with an effective
loss function for action anticipation in driving scenarios.
Patient Subtyping with Disease Progression and Irregular Observation Trajectories
Authors: Nikhil Galagali, Minnan Xu-Wilson
Date: 2018-10-21 00:00:00
Tasks: Decompensation

Patient subtyping based on temporal observations can lead to significantly
nuanced subtyping that acknowledges the dynamic characteristics of diseases.
Existing methods for subtyping trajectories treat the evolution of clinical
observations as a homogeneous process or employ data available at regular
intervals. In reality, diseases may have transient underlying states and a
state-dependent observation pattern. In our paper, we present an approach to
subtype irregular patient data while acknowledging the underlying progression
of disease states. Our approach consists of two components: a probabilistic
model to determine the likelihood of a patient's observation trajectory and a
mixture model to measure similarity between asynchronous patient trajectories.
We demonstrate our model by discovering subtypes of progression to hemodynamic
instability (requiring cardiovascular intervention) in a patient cohort from a
multi-institution ICU dataset. We find three primary patterns: two of which
show classic signs of decompensation (rising heart rate with dropping blood
pressure), with one of these showing a faster course of decompensation than the
other. The third pattern has transient period of low heart rate and blood
pressure. We also show that our model results in a 13% reduction in average
cross-entropy error compared to a model with no state progression when
forecasting vital signs.
Depth with Nonlinearity Creates No Bad Local Minima in ResNets
Authors: Kenji Kawaguchi, Yoshua Bengio
Date: 2018-10-21 00:00:00
Tasks: BIG-bench Machine Learning, Open-Ended Question Answering

In this paper, we prove that depth with nonlinearity creates no bad local minima in a type of arbitrarily deep ResNets with arbitrary nonlinear activation functions, in the sense that the values of all local minima are no worse than the global minimum value of corresponding classical machine-learning models, and are guaranteed to further improve via residual representations. As a result, this paper provides an affirmative answer to an open question stated in a paper in the conference on Neural Information Processing Systems 2018. This paper advances the optimization theory of deep learning only for ResNets and not for other network architectures.
Soft Concept Analysis
Authors: Robert E. Kent
Date: 2018-10-21 00:00:00
Tasks: Philosophy

In this chapter we discuss soft concept analysis, a study which identifies an
enriched notion of "conceptual scale" as developed in formal concept analysis
with an enriched notion of "linguistic variable" as discussed in fuzzy logic.
The identification "enriched conceptual scale" = "enriched linguistic variable"
was made in a previous paper (Enriched interpretation, Robert E. Kent). In this
chapter we offer further arguments for the importance of this identification by
discussing the philosophy, spirit, and practical application of conceptual
scaling to the discovery, conceptual analysis, interpretation, and
categorization of networked information resources. We argue that a linguistic
variable, which has been defined at just the right generalization of valuated
categories, provides a natural definition for the process of soft conceptual
scaling. This enrichment using valuated categories models the relation of
indiscernability, a notion of central importance in rough set theory. At a more
fundamental level for soft concept analysis, it also models the derivation of
formal concepts, a process of central importance in formal concept analysis.
Soft concept analysis is synonymous with enriched concept analysis. From one
viewpoint, the study of soft concept analysis that is initiated here extends
formal concept analysis to soft computational structures. From another
viewpoint, soft concept analysis provides a natural foundation for soft
computation by unifying and explaining notions from soft computation in terms
of suitably generalized notions from formal concept analysis, rough set theory
and fuzzy set theory.
Challenge AI Mind: A Crowd System for Proactive AI Testing
Authors: Siwei Fu, Anbang Xu, Xiaotong Liu, Huimin Zhou, Rama Akkiraju
Date: 2018-10-21 00:00:00
Tasks: BIG-bench Machine Learning

Artificial Intelligence (AI) has burrowed into our lives in various aspects;
however, without appropriate testing, deployed AI systems are often being
criticized to fail in critical and embarrassing cases. Existing testing
approaches mainly depend on fixed and pre-defined datasets, providing a limited
testing coverage. In this paper, we propose the concept of proactive testing to
dynamically generate testing data and evaluate the performance of AI systems.
We further introduce Challenge.AI, a new crowd system that features the
integration of crowdsourcing and machine learning techniques in the process of
error generation, error validation, error categorization, and error analysis.
We present experiences and insights into a participatory design with AI
developers. The evaluation shows that the crowd workflow is more effective with
the help of machine learning techniques. AI developers found that our system
can help them discover unknown errors made by the AI models, and engage in the
process of proactive testing.