pwc-archive/papers-with-abstracts huggingface dataset:
BattRAE: Bidimensional Attention-Based Recursive Autoencoders for Learning Bilingual Phrase Embeddings
Authors: Biao Zhang, Deyi Xiong, Jinsong Su
Date: 2016-05-25 00:00:00
Tasks: Semantic Similarity, Semantic Textual Similarity

In this paper, we propose a bidimensional attention based recursive
autoencoder (BattRAE) to integrate clues and sourcetarget interactions at
multiple levels of granularity into bilingual phrase representations. We employ
recursive autoencoders to generate tree structures of phrases with embeddings
at different levels of granularity (e.g., words, sub-phrases and phrases). Over
these embeddings on the source and target side, we introduce a bidimensional
attention network to learn their interactions encoded in a bidimensional
attention matrix, from which we extract two soft attention weight distributions
simultaneously. These weight distributions enable BattRAE to generate
compositive phrase representations via convolution. Based on the learned phrase
representations, we further use a bilinear neural model, trained via a
max-margin method, to measure bilingual semantic similarity. To evaluate the
effectiveness of BattRAE, we incorporate this semantic similarity as an
additional feature into a state-of-the-art SMT system. Extensive experiments on
NIST Chinese-English test sets show that our model achieves a substantial
improvement of up to 1.63 BLEU points on average over the baseline.
RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation
Authors: Guosheng Lin, Anton Milan, Chunhua Shen, Ian Reid
Date: 2016-11-20 00:00:00
Proceeding: CVPR 2017 7
Tasks: 3D Absolute Human Pose Estimation, Semantic Segmentation, Vocal Bursts Intensity Prediction

Recently, very deep convolutional neural networks (CNNs) have shown
outstanding performance in object recognition and have also been the first
choice for dense classification problems such as semantic segmentation.
However, repeated subsampling operations like pooling or convolution striding
in deep CNNs lead to a significant decrease in the initial image resolution.
Here, we present RefineNet, a generic multi-path refinement network that
explicitly exploits all the information available along the down-sampling
process to enable high-resolution prediction using long-range residual
connections. In this way, the deeper layers that capture high-level semantic
features can be directly refined using fine-grained features from earlier
convolutions. The individual components of RefineNet employ residual
connections following the identity mapping mindset, which allows for effective
end-to-end training. Further, we introduce chained residual pooling, which
captures rich background context in an efficient manner. We carry out
comprehensive experiments and set new state-of-the-art results on seven public
datasets. In particular, we achieve an intersection-over-union score of 83.4 on
the challenging PASCAL VOC 2012 dataset, which is the best reported result to
date.
A Distance Function for Comparing Straight-Edge Geometric Figures
Authors: Apoorva Honnegowda Roopa, Shrisha Rao
Date: 2016-11-25 00:00:00

This paper defines a distance function that measures the dissimilarity
between planar geometric figures formed with straight lines. This function can
in turn be used in partial matching of different geometric figures. For a given
pair of geometric figures that are graphically isomorphic, one function
measures the angular dissimilarity and another function measures the edge
length disproportionality. The distance function is then defined as the convex
sum of these two functions. The novelty of the presented function is that it
satisfies all properties of a distance function and the computation of the same
is done by projecting appropriate features to a cartesian plane. To compute the
deviation from the angular similarity property, the Euclidean distance between
the given angular pairs and the corresponding points on the $y=x$ line is
measured. Further while computing the deviation from the edge length
proportionality property, the best fit line, for the set of edge lengths, which
passes through the origin is found, and the Euclidean distance between the
given edge length pairs and the corresponding point on a $y=mx$ line is
calculated. Iterative Proportional Fitting Procedure (IPFP) is used to find
this best fit line. We demonstrate the behavior of the defined function for
some sample pairs of figures.
An Overview on Data Representation Learning: From Traditional Feature Learning to Recent Deep Learning
Authors: Guoqiang Zhong, Li-Na Wang, Junyu Dong
Date: 2016-11-25 00:00:00
Tasks: General Classification, image-classification, Image Classification, object-detection, Object Detection, Representation Learning, speech-recognition, Speech Recognition

Since about 100 years ago, to learn the intrinsic structure of data, many
representation learning approaches have been proposed, including both linear
ones and nonlinear ones, supervised ones and unsupervised ones. Particularly,
deep architectures are widely applied for representation learning in recent
years, and have delivered top results in many tasks, such as image
classification, object detection and speech recognition. In this paper, we
review the development of data representation learning methods. Specifically,
we investigate both traditional feature learning algorithms and
state-of-the-art deep learning models. The history of data representation
learning is introduced, while available resources (e.g. online course, tutorial
and book information) and toolboxes are provided. Finally, we conclude this
paper with remarks and some interesting research directions on data
representation learning.
Training and Evaluating Multimodal Word Embeddings with Large-scale Web Annotated Images
Authors: Junhua Mao, Jiajing Xu, Yushi Jing, Alan Yuille
Date: 2016-11-24 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Image Retrieval, Sentence, Word Embeddings

In this paper, we focus on training and evaluating effective word embeddings
with both text and visual information. More specifically, we introduce a
large-scale dataset with 300 million sentences describing over 40 million
images crawled and downloaded from publicly available Pins (i.e. an image with
sentence descriptions uploaded by users) on Pinterest. This dataset is more
than 200 times larger than MS COCO, the standard large-scale image dataset with
sentence descriptions. In addition, we construct an evaluation dataset to
directly assess the effectiveness of word embeddings in terms of finding
semantically similar or related words and phrases. The word/phrase pairs in
this evaluation dataset are collected from the click data with millions of
users in an image search system, thus contain rich semantic relationships.
Based on these datasets, we propose and compare several Recurrent Neural
Networks (RNNs) based multimodal (text and image) models. Experiments show that
our model benefits from incorporating the visual information into the word
embeddings, and a weight sharing strategy is crucial for learning such
multimodal embeddings. The project page is:
http://www.stat.ucla.edu/~junhua.mao/multimodal_embedding.html
A Review on Algorithms for Constraint-based Causal Discovery
Authors: Kui Yu, Jiuyong Li, Lin Liu
Date: 2016-11-12 00:00:00
Tasks: Causal Discovery

Causal discovery studies the problem of mining causal relationships between
variables from data, which is of primary interest in science. During the past
decades, significant amount of progresses have been made toward this
fundamental data mining paradigm. Recent years, as the availability of abundant
large-sized and complex observational data, the constrain-based approaches have
gradually attracted a lot of interest and have been widely applied to many
diverse real-world problems due to the fast running speed and easy generalizing
to the problem of causal insufficiency. In this paper, we aim to review the
constraint-based causal discovery algorithms. Firstly, we discuss the learning
paradigm of the constraint-based approaches. Secondly and primarily, the
state-of-the-art constraint-based casual inference algorithms are surveyed with
the detailed analysis. Thirdly, several related open-source software packages
and benchmark data repositories are briefly summarized. As a conclusion, some
open problems in constraint-based causal discovery are outlined for future
research.
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems
Authors: Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
Date: 2016-11-24 00:00:00
Tasks: BIG-bench Machine Learning, Image Captioning

We study the problem of troubleshooting machine learning systems that rely on
analytical pipelines of distinct components. Understanding and fixing errors
that arise in such integrative systems is difficult as failures can occur at
multiple points in the execution workflow. Moreover, errors can propagate,
become amplified or be suppressed, making blame assignment difficult. We
propose a human-in-the-loop methodology which leverages human intellect for
troubleshooting system failures. The approach simulates potential component
fixes through human computation tasks and measures the expected improvements in
the holistic behavior of the system. The method provides guidance to designers
about how they can best improve the system. We demonstrate the effectiveness of
the approach on an automated image captioning system that has been pressed into
real-world use.
Learning Python Code Suggestion with a Sparse Pointer Network
Authors: Avishkar Bhoopchand, Tim Rocktäschel, Earl Barr, Sebastian Riedel
Date: 2016-11-24 00:00:00
Tasks: Language Modeling, Language Modelling

To enhance developer productivity, all modern integrated development
environments (IDEs) include code suggestion functionality that proposes likely
next tokens at the cursor. While current IDEs work well for statically-typed
languages, their reliance on type annotations means that they do not provide
the same level of support for dynamic programming languages as for
statically-typed languages. Moreover, suggestion engines in modern IDEs do not
propose expressions or multi-statement idiomatic code. Recent work has shown
that language models can improve code suggestion systems by learning from
software repositories. This paper introduces a neural language model with a
sparse pointer network aimed at capturing very long-range dependencies. We
release a large-scale code suggestion corpus of 41M lines of Python code
crawled from GitHub. On this corpus, we found standard neural language models
to perform well at suggesting local phenomena, but struggle to refer to
identifiers that are introduced many tokens in the past. By augmenting a neural
language model with a pointer network specialized in referring to predefined
classes of identifiers, we obtain a much lower perplexity and a 5 percentage
points increase in accuracy for code suggestion compared to an LSTM baseline.
In fact, this increase in code suggestion accuracy is due to a 13 times more
accurate prediction of identifiers. Furthermore, a qualitative analysis shows
this model indeed captures interesting long-range dependencies, like referring
to a class member defined over 60 tokens in the past.
Interpreting Finite Automata for Sequential Data
Authors: Christian Albert Hammerschmidt, Sicco Verwer, Qin Lin, Radu State
Date: 2016-11-21 00:00:00
Tasks: Clustering, General Classification

Automaton models are often seen as interpretable models. Interpretability
itself is not well defined: it remains unclear what interpretability means
without first explicitly specifying objectives or desired attributes. In this
paper, we identify the key properties used to interpret automata and propose a
modification of a state-merging approach to learn variants of finite state
automata. We apply the approach to problems beyond typical grammar inference
tasks. Additionally, we cover several use-cases for prediction, classification,
and clustering on sequential data in both supervised and unsupervised scenarios
to show how the identified key properties are applicable in a wide range of
contexts.
Confidence-Weighted Bipartite Ranking
Authors: Majdi Khalid, Indrakshi Ray, Hamidreza Chitsaz
Date: 2016-07-04 00:00:00

Bipartite ranking is a fundamental machine learning and data mining problem.
It commonly concerns the maximization of the AUC metric. Recently, a number of
studies have proposed online bipartite ranking algorithms to learn from massive
streams of class-imbalanced data. These methods suggest both linear and
kernel-based bipartite ranking algorithms based on first and second-order
online learning. Unlike kernelized ranker, linear ranker is more scalable
learning algorithm. The existing linear online bipartite ranking algorithms
lack either handling non-separable data or constructing adaptive large margin.
These limitations yield unreliable bipartite ranking performance. In this work,
we propose a linear online confidence-weighted bipartite ranking algorithm
(CBR) that adopts soft confidence-weighted learning. The proposed algorithm
leverages the same properties of soft confidence-weighted learning in a
framework for bipartite ranking. We also develop a diagonal variation of the
proposed confidence-weighted bipartite ranking algorithm to deal with
high-dimensional data by maintaining only the diagonal elements of the
covariance matrix. We empirically evaluate the effectiveness of the proposed
algorithms on several benchmark and high-dimensional datasets. The experimental
results validate the reliability of the proposed algorithms. The results also
show that our algorithms outperform or are at least comparable to the competing
online AUC maximization methods.
Concept Stability for Constructing Taxonomies of Web-site Users
Authors: Sergei O. Kuznetsov, Dmitry I. Ignatov
Date: 2009-05-09 00:00:00

Owners of a web-site are often interested in analysis of groups of users of
their site. Information on these groups can help optimizing the structure and
contents of the site. In this paper we use an approach based on formal concepts
for constructing taxonomies of user groups. For decreasing the huge amount of
concepts that arise in applications, we employ stability index of a concept,
which describes how a group given by a concept extent differs from other such
groups. We analyze resulting taxonomies of user groups for three target
websites.
InstanceCut: from Edges to Instances with MultiCut
Authors: Alexander Kirillov, Evgeny Levinkov, Bjoern Andres, Bogdan Savchynskyy, Carsten Rother
Date: 2016-11-24 00:00:00
Proceeding: CVPR 2017 7
Tasks: Edge Detection, Segmentation, Semantic Segmentation

This work addresses the task of instance-aware semantic segmentation. Our key
motivation is to design a simple method with a new modelling-paradigm, which
therefore has a different trade-off between advantages and disadvantages
compared to known approaches. Our approach, we term InstanceCut, represents the
problem by two output modalities: (i) an instance-agnostic semantic
segmentation and (ii) all instance-boundaries. The former is computed from a
standard convolutional neural network for semantic segmentation, and the latter
is derived from a new instance-aware edge detection model. To reason globally
about the optimal partitioning of an image into instances, we combine these two
modalities into a novel MultiCut formulation. We evaluate our approach on the
challenging CityScapes dataset. Despite the conceptual simplicity of our
approach, we achieve the best result among all published methods, and perform
particularly well for rare object classes.
Weakly Supervised Cascaded Convolutional Networks
Authors: Ali Diba, Vivek Sharma, Ali Pazandeh, Hamed Pirsiavash, Luc van Gool
Date: 2016-11-24 00:00:00
Proceeding: CVPR 2017 7
Tasks: Multiple Instance Learning, Object, object-detection, Object Detection, Semantic Segmentation, Weakly Supervised Object Detection

Object detection is a challenging task in visual understanding domain, and
even more so if the supervision is to be weak. Recently, few efforts to handle
the task without expensive human annotations is established by promising deep
neural network. A new architecture of cascaded networks is proposed to learn a
convolutional neural network (CNN) under such conditions. We introduce two such
architectures, with either two cascade stages or three which are trained in an
end-to-end pipeline. The first stage of both architectures extracts best
candidate of class specific region proposals by training a fully convolutional
network. In the case of the three stage architecture, the middle stage provides
object segmentation, using the output of the activation maps of first stage.
The final stage of both architectures is a part of a convolutional neural
network that performs multiple instance learning on proposals extracted in the
previous stage(s). Our experiments on the PASCAL VOC 2007, 2010, 2012 and large
scale object datasets, ILSVRC 2013, 2014 datasets show improvements in the
areas of weakly-supervised object detection, classification and localization.
Proceedings of the Workshop on Brain Analysis using COnnectivity Networks - BACON 2016
Authors: Sarah Parisot, Jonathan Passerat-Palmbach, Markus D. Schirmer, Boris Gutman
Date: 2016-11-10 00:00:00

Understanding brain connectivity in a network-theoretic context has shown
much promise in recent years. This type of analysis identifies brain
organisational principles, bringing a new perspective to neuroscience. At the
same time, large public databases of connectomic data are now available.
However, connectome analysis is still an emerging field and there is a crucial
need for robust computational methods to fully unravelits potential. This
workshop provides a platform to discuss the development of new analytic
techniques; methods for evaluating and validating commonly used approaches; as
well as the effects of variations in pre-processing steps.
The Inverse Bagging Algorithm: Anomaly Detection by Inverse Bootstrap Aggregating
Authors: Pietro Vischia, Tommaso Dorigo
Date: 2016-11-24 00:00:00
Tasks: Anomaly Detection

For data sets populated by a very well modeled process and by another process
of unknown probability density function (PDF), a desired feature when
manipulating the fraction of the unknown process (either for enhancing it or
suppressing it) consists in avoiding to modify the kinematic distributions of
the well modeled one. A bootstrap technique is used to identify sub-samples
rich in the well modeled process, and classify each event according to the
frequency of it being part of such sub-samples. Comparisons with general MVA
algorithms will be shown, as well as a study of the asymptotic properties of
the method, making use of a public domain data set that models a typical search
for new physics as performed at hadronic colliders such as the Large Hadron
Collider (LHC).
Fast Orthonormal Sparsifying Transforms Based on Householder Reflectors
Authors: Cristian Rusu, Nuria Gonzalez-Prelcic, Robert Heath
Date: 2016-11-24 00:00:00
Tasks: Dictionary Learning

Dictionary learning is the task of determining a data-dependent transform
that yields a sparse representation of some observed data. The dictionary
learning problem is non-convex, and usually solved via computationally complex
iterative algorithms. Furthermore, the resulting transforms obtained generally
lack structure that permits their fast application to data. To address this
issue, this paper develops a framework for learning orthonormal dictionaries
which are built from products of a few Householder reflectors. Two algorithms
are proposed to learn the reflector coefficients: one that considers a
sequential update of the reflectors and one with a simultaneous update of all
reflectors that imposes an additional internal orthogonal constraint. The
proposed methods have low computational complexity and are shown to converge to
local minimum points which can be described in terms of the spectral properties
of the matrices involved. The resulting dictionaries balance between the
computational complexity and the quality of the sparse representations by
controlling the number of Householder reflectors in their product. Simulations
of the proposed algorithms are shown in the image processing setting where
well-known fast transforms are available for comparisons. The proposed
algorithms have favorable reconstruction error and the advantage of a fast
implementation relative to the classical, unstructured, dictionaries.
Interferences in match kernels
Authors: Naila Murray, Hervé Jégou, Florent Perronnin, Andrew Zisserman
Date: 2016-11-24 00:00:00
Tasks: Image Retrieval, Retrieval

We consider the design of an image representation that embeds and aggregates
a set of local descriptors into a single vector. Popular representations of
this kind include the bag-of-visual-words, the Fisher vector and the VLAD. When
two such image representations are compared with the dot-product, the
image-to-image similarity can be interpreted as a match kernel. In match
kernels, one has to deal with interference, i.e. with the fact that even if two
descriptors are unrelated, their matching score may contribute to the overall
similarity.
  We formalise this problem and propose two related solutions, both aimed at
equalising the individual contributions of the local descriptors in the final
representation. These methods modify the aggregation stage by including a set
of per-descriptor weights. They differ by the objective function that is
optimised to compute those weights. The first is a "democratisation" strategy
that aims at equalising the relative importance of each descriptor in the set
comparison metric. The second one involves equalising the match of a single
descriptor to the aggregated vector.
  These concurrent methods give a substantial performance boost over the state
of the art in image search with short or mid-size vectors, as demonstrated by
our experiments on standard public image retrieval benchmarks.
Interpreting the Predictions of Complex ML Models by Layer-wise Relevance Propagation
Authors: Wojciech Samek, Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Klaus-Robert Müller
Date: 2016-11-24 00:00:00
Tasks: General Classification, image-classification, Image Classification, speech-recognition, Speech Recognition

Complex nonlinear models such as deep neural network (DNNs) have become an
important tool for image classification, speech recognition, natural language
processing, and many other fields of application. These models however lack
transparency due to their complex nonlinear structure and to the complex data
distributions to which they typically apply. As a result, it is difficult to
fully characterize what makes these models reach a particular decision for a
given input. This lack of transparency can be a drawback, especially in the
context of sensitive applications such as medical analysis or security. In this
short paper, we summarize a recent technique introduced by Bach et al. [1] that
explains predictions by decomposing the classification decision of DNN models
in terms of input variables.