pwc-archive/papers-with-abstracts huggingface dataset:
The Great Time Series Classification Bake Off: An Experimental Evaluation of Recently Proposed Algorithms. Extended Version
Authors: Anthony Bagnall, Aaron Bostrom, James Large, Jason Lines
Date: 2016-02-04 00:00:00
Tasks: General Classification, Time Series, Time Series Analysis, Time Series Classification

In the last five years there have been a large number of new time series
classification algorithms proposed in the literature. These algorithms have
been evaluated on subsets of the 47 data sets in the University of California,
Riverside time series classification archive. The archive has recently been
expanded to 85 data sets, over half of which have been donated by researchers
at the University of East Anglia. Aspects of previous evaluations have made
comparisons between algorithms difficult. For example, several different
programming languages have been used, experiments involved a single train/test
split and some used normalised data whilst others did not. The relaunch of the
archive provides a timely opportunity to thoroughly evaluate algorithms on a
larger number of datasets. We have implemented 18 recently proposed algorithms
in a common Java framework and compared them against two standard benchmark
classifiers (and each other) by performing 100 resampling experiments on each
of the 85 datasets. We use these results to test several hypotheses relating to
whether the algorithms are significantly more accurate than the benchmarks and
each other. Our results indicate that only 9 of these algorithms are
significantly more accurate than both benchmarks and that one classifier, the
Collective of Transformation Ensembles, is significantly more accurate than all
of the others. All of our experiments and results are reproducible: we release
all of our code, results and experimental details and we hope these experiments
form the basis for more rigorous testing of new algorithms in the future.
Complex Networks of Words in Fables
Authors: Yurij Holovatch, Vasyl Palchykov
Date: 2016-02-04 00:00:00

In this chapter we give an overview of the application of complex network
theory to quantify some properties of language. Our study is based on two
fables in Ukrainian, Mykyta the Fox and Abu-Kasym's slippers. It consists of
two parts: the analysis of frequency-rank distributions of words and the
application of complex-network theory. The first part shows that the text sizes
are sufficiently large to observe statistical properties. This supports their
selection for the analysis of typical properties of the language networks in
the second part of the chapter. In describing language as a complex network,
while words are usually associated with nodes, there is more variability in the
choice of links and different representations result in different networks.
Here, we examine a number of such representations of the language network and
perform a comparative analysis of their characteristics. Our results suggest
that, irrespective of link representation, the Ukrainian language network used
in the selected fables is a strongly correlated, scale-free, small world. We
discuss how such empirical approaches may help form a useful basis for a
theoretical description of language evolution and how they may be used in
analyses of other textual narratives.
Unsupervised High-level Feature Learning by Ensemble Projection for Semi-supervised Image Classification and Image Clustering
Authors: Dengxin Dai, Luc van Gool
Date: 2016-02-02 00:00:00
Tasks: Classification, Clustering, Ensemble Learning, General Classification, image-classification, Image Classification, Image Clustering, Image Retrieval, Retrieval, Semi-Supervised Image Classification

This paper investigates the problem of image classification with limited or
no annotations, but abundant unlabeled data. The setting exists in many tasks
such as semi-supervised image classification, image clustering, and image
retrieval. Unlike previous methods, which develop or learn sophisticated
regularizers for classifiers, our method learns a new image representation by
exploiting the distribution patterns of all available data for the task at
hand. Particularly, a rich set of visual prototypes are sampled from all
available data, and are taken as surrogate classes to train discriminative
classifiers; images are projected via the classifiers; the projected values,
similarities to the prototypes, are stacked to build the new feature vector.
The training set is noisy. Hence, in the spirit of ensemble learning we create
a set of such training sets which are all diverse, leading to diverse
classifiers. The method is dubbed Ensemble Projection (EP). EP captures not
only the characteristics of individual images, but also the relationships among
images. It is conceptually simple and computationally efficient, yet effective
and flexible. Experiments on eight standard datasets show that: (1) EP
outperforms previous methods for semi-supervised image classification; (2) EP
produces promising results for self-taught image classification, where
unlabeled samples are a random collection of images rather than being from the
same distribution as the labeled ones; and (3) EP improves over the original
features for image clustering. The code of the method is available on the
project page.
A semi-automatic computer-aided method for surgical template design
Authors: Xiaojun Chen, Lu Xu, Yue Yang, Jan Egger
Date: 2016-02-04 00:00:00
Tasks: Segmentation

This paper presents a generalized integrated framework of semi-automatic
surgical template design. Several algorithms were implemented including the
mesh segmentation, offset surface generation, collision detection, ruled
surface generation, etc., and a special software named TemDesigner was
developed. With a simple user interface, a customized template can be semi-
automatically designed according to the preoperative plan. Firstly, mesh
segmentation with signed scalar of vertex is utilized to partition the inner
surface from the input surface mesh based on the indicated point loop. Then,
the offset surface of the inner surface is obtained through contouring the
distance field of the inner surface, and segmented to generate the outer
surface. Ruled surface is employed to connect inner and outer surfaces.
Finally, drilling tubes are generated according to the preoperative plan
through collision detection and merging. It has been applied to the template
design for various kinds of surgeries, including oral implantology, cervical
pedicle screw insertion, iliosacral screw insertion and osteotomy,
demonstrating the efficiency, functionality and generality of our method.
Self-Transfer Learning for Fully Weakly Supervised Object Localization
Authors: Sangheum Hwang, Hyo-Eun Kim
Date: 2016-02-04 00:00:00
Tasks: Medical Image Analysis, Object, Object Localization, Transfer Learning, Weakly-supervised Learning, Weakly-Supervised Object Localization

Recent advances of deep learning have achieved remarkable performances in
various challenging computer vision tasks. Especially in object localization,
deep convolutional neural networks outperform traditional approaches based on
extraction of data/task-driven features instead of hand-crafted features.
Although location information of region-of-interests (ROIs) gives good prior
for object localization, it requires heavy annotation efforts from human
resources. Thus a weakly supervised framework for object localization is
introduced. The term "weakly" means that this framework only uses image-level
labeled datasets to train a network. With the help of transfer learning which
adopts weight parameters of a pre-trained network, the weakly supervised
learning framework for object localization performs well because the
pre-trained network already has well-trained class-specific features. However,
those approaches cannot be used for some applications which do not have
pre-trained networks or well-localized large scale images. Medical image
analysis is a representative among those applications because it is impossible
to obtain such pre-trained networks. In this work, we present a "fully" weakly
supervised framework for object localization ("semi"-weakly is the counterpart
which uses pre-trained filters for weakly supervised localization) named as
self-transfer learning (STL). It jointly optimizes both classification and
localization networks simultaneously. By controlling a supervision level of the
localization network, STL helps the localization network focus on correct ROIs
without any types of priors. We evaluate the proposed STL framework using two
medical image datasets, chest X-rays and mammograms, and achieve signiticantly
better localization performance compared to previous weakly supervised
approaches.
How Important is Weight Symmetry in Backpropagation?
Authors: Qianli Liao, Joel Z. Leibo, Tomaso Poggio
Date: 2015-10-17 00:00:00
Tasks: Handwritten Digit Recognition, Image Classification

Gradient backpropagation (BP) requires symmetric feedforward and feedback
connections -- the same weights must be used for forward and backward passes.
This "weight transport problem" (Grossberg 1987) is thought to be one of the
main reasons to doubt BP's biologically plausibility. Using 15 different
classification datasets, we systematically investigate to what extent BP really
depends on weight symmetry. In a study that turned out to be surprisingly
similar in spirit to Lillicrap et al.'s demonstration (Lillicrap et al. 2014)
but orthogonal in its results, our experiments indicate that: (1) the
magnitudes of feedback weights do not matter to performance (2) the signs of
feedback weights do matter -- the more concordant signs between feedforward and
their corresponding feedback connections, the better (3) with feedback weights
having random magnitudes and 100% concordant signs, we were able to achieve the
same or even better performance than SGD. (4) some
normalizations/stabilizations are indispensable for such asymmetric BP to work,
namely Batch Normalization (BN) (Ioffe and Szegedy 2015) and/or a "Batch
Manhattan" (BM) update rule.
Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering
Authors: Ruining He, Julian McAuley
Date: 2016-02-04 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Building a successful recommender system depends on understanding both the
dimensions of people's preferences as well as their dynamics. In certain
domains, such as fashion, modeling such preferences can be incredibly
difficult, due to the need to simultaneously model the visual appearance of
products as well as their evolution over time. The subtle semantics and
non-linear dynamics of fashion evolution raise unique challenges especially
considering the sparsity and large scale of the underlying datasets. In this
paper we build novel models for the One-Class Collaborative Filtering setting,
where our goal is to estimate users' fashion-aware personalized ranking
functions based on their past feedback. To uncover the complex and evolving
visual factors that people consider when evaluating products, our method
combines high-level visual features extracted from a deep convolutional neural
network, users' past feedback, as well as evolving trends within the community.
Experimentally we evaluate our method on two large real-world datasets from
Amazon.com, where we show it to outperform state-of-the-art personalized
ranking measures, and also use it to visualize the high-level fashion trends
across the 11-year span of our dataset.
Long-term Planning by Short-term Prediction
Authors: Shai Shalev-Shwartz, Nir Ben-Zrihem, Aviad Cohen, Amnon Shashua
Date: 2016-02-04 00:00:00
Tasks: Autonomous Driving, Prediction

We consider planning problems, that often arise in autonomous driving
applications, in which an agent should decide on immediate actions so as to
optimize a long term objective. For example, when a car tries to merge in a
roundabout it should decide on an immediate acceleration/braking command, while
the long term effect of the command is the success/failure of the merge. Such
problems are characterized by continuous state and action spaces, and by
interaction with multiple agents, whose behavior can be adversarial. We argue
that dual versions of the MDP framework (that depend on the value function and
the $Q$ function) are problematic for autonomous driving applications due to
the non Markovian of the natural state space representation, and due to the
continuous state and action spaces. We propose to tackle the planning task by
decomposing the problem into two phases: First, we apply supervised learning
for predicting the near future based on the present. We require that the
predictor will be differentiable with respect to the representation of the
present. Second, we model a full trajectory of the agent using a recurrent
neural network, where unexplained factors are modeled as (additive) input
nodes. This allows us to solve the long-term planning problem using supervised
learning techniques and direct optimization over the recurrent neural network.
Our approach enables us to learn robust policies by incorporating adversarial
elements to the environment.
A Factorized Recurrent Neural Network based architecture for medium to large vocabulary Language Modelling
Authors: Anantharaman Palacode Narayana Iyer
Date: 2016-02-04 00:00:00
Tasks: Language Modelling

Statistical language models are central to many applications that use
semantics. Recurrent Neural Networks (RNN) are known to produce state of the
art results for language modelling, outperforming their traditional n-gram
counterparts in many cases. To generate a probability distribution across a
vocabulary, these models require a softmax output layer that linearly increases
in size with the size of the vocabulary. Large vocabularies need a
commensurately large softmax layer and training them on typical laptops/PCs
requires significant time and machine resources. In this paper we present a new
technique for implementing RNN based large vocabulary language models that
substantially speeds up computation while optimally using the limited memory
resources. Our technique, while building on the notion of factorizing the
output layer by having multiple output layers, improves on the earlier work by
substantially optimizing on the individual output layer size and also
eliminating the need for a multistep prediction process.
BilBOWA: Fast Bilingual Distributed Representations without Word Alignments
Authors: Stephan Gouws, Yoshua Bengio, Greg Corrado
Date: 2014-10-09 00:00:00
Tasks: Cross-Lingual Document Classification, Document Classification, General Classification, Sentence, Translation

We introduce BilBOWA (Bilingual Bag-of-Words without Alignments), a simple
and computationally-efficient model for learning bilingual distributed
representations of words which can scale to large monolingual datasets and does
not require word-aligned parallel training data. Instead it trains directly on
monolingual data and extracts a bilingual signal from a smaller set of raw-text
sentence-aligned data. This is achieved using a novel sampled bag-of-words
cross-lingual objective, which is used to regularize two noise-contrastive
language models for efficient cross-lingual feature learning. We show that
bilingual embeddings learned using the proposed model outperform
state-of-the-art methods on a cross-lingual document classification task as
well as a lexical translation task on WMT11 data.
A Grassmannian Graph Approach to Affine Invariant Feature Matching
Authors: Mark Moyou, John Corring, Adrian Peter, Anand Rangarajan
Date: 2016-01-28 00:00:00

In this work, we present a novel and practical approach to address one of the
longstanding problems in computer vision: 2D and 3D affine invariant feature
matching. Our Grassmannian Graph (GrassGraph) framework employs a two stage
procedure that is capable of robustly recovering correspondences between two
unorganized, affinely related feature (point) sets. The first stage maps the
feature sets to an affine invariant Grassmannian representation, where the
features are mapped into the same subspace. It turns out that coordinate
representations extracted from the Grassmannian differ by an arbitrary
orthonormal matrix. In the second stage, by approximating the Laplace-Beltrami
operator (LBO) on these coordinates, this extra orthonormal factor is
nullified, providing true affine-invariant coordinates which we then utilize to
recover correspondences via simple nearest neighbor relations. The resulting
GrassGraph algorithm is empirically shown to work well in non-ideal scenarios
with noise, outliers, and occlusions. Our validation benchmarks use an
unprecedented 440,000+ experimental trials performed on 2D and 3D datasets,
with a variety of parameter settings and competing methods. State-of-the-art
performance in the majority of these extensive evaluations confirm the utility
of our method.
An ensemble diversity approach to supervised binary hashing
Authors: Miguel Á. Carreira-Perpiñán, Ramin Raziperchikolaei
Date: 2016-02-04 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Diversity, Image Retrieval, Information Retrieval, Retrieval

Binary hashing is a well-known approach for fast approximate nearest-neighbor
search in information retrieval. Much work has focused on affinity-based
objective functions involving the hash functions or binary codes. These
objective functions encode neighborhood information between data points and are
often inspired by manifold learning algorithms. They ensure that the hash
functions differ from each other through constraints or penalty terms that
encourage codes to be orthogonal or dissimilar across bits, but this couples
the binary variables and complicates the already difficult optimization. We
propose a much simpler approach: we train each hash function (or bit)
independently from each other, but introduce diversity among them using
techniques from classifier ensembles. Surprisingly, we find that not only is
this faster and trivially parallelizable, but it also improves over the more
complex, coupled objective function, and achieves state-of-the-art precision
and recall in experiments with image retrieval.
Breaking Symmetries in Graph Search with Canonizing Sets
Authors: Avraham Itzhakov, Michael Codish
Date: 2015-11-25 00:00:00

There are many complex combinatorial problems which involve searching for an
undirected graph satisfying given constraints. Such problems are often highly
challenging because of the large number of isomorphic representations of their
solutions. This paper introduces effective and compact, complete symmetry
breaking constraints for small graph search. Enumerating with these symmetry
breaks generates all and only non-isomorphic solutions. For small search
problems, with up to $10$ vertices, we compute instance independent symmetry
breaking constraints. For small search problems with a larger number of
vertices we demonstrate the computation of instance dependent constraints which
are complete. We illustrate the application of complete symmetry breaking
constraints to extend two known sequences from the OEIS related to graph
enumeration. We also demonstrate the application of a generalization of our
approach to fully-interchangeable matrix search problems.
Fundamental Limits in Multi-image Alignment
Authors: Cecilia Aguerrebere, Mauricio Delbracio, Alberto Bartesaghi, Guillermo Sapiro
Date: 2016-02-04 00:00:00
Tasks: Image Registration

The performance of multi-image alignment, bringing different images into one
coordinate system, is critical in many applications with varied signal-to-noise
ratio (SNR) conditions. A great amount of effort is being invested into
developing methods to solve this problem. Several important questions thus
arise, including: Which are the fundamental limits in multi-image alignment
performance? Does having access to more images improve the alignment?
Theoretical bounds provide a fundamental benchmark to compare methods and can
help establish whether improvements can be made. In this work, we tackle the
problem of finding the performance limits in image registration when multiple
shifted and noisy observations are available. We derive and analyze the
Cram\'er-Rao and Ziv-Zakai lower bounds under different statistical models for
the underlying image. The accuracy of the derived bounds is experimentally
assessed through a comparison to the maximum likelihood estimator. We show the
existence of different behavior zones depending on the difficulty level of the
problem, given by the SNR conditions of the input images. We find that
increasing the number of images is only useful below a certain SNR threshold,
above which the pairwise MLE estimation proves to be optimal. The analysis we
present here brings further insight into the fundamental limitations of the
multi-image alignment problem.
Bearing fault diagnosis based on spectrum images of vibration signals
Authors: Wei Li, Mingquan Qiu, Zhencai Zhu, Bo Wu, Gongbo Zhou
Date: 2015-11-08 00:00:00
Tasks: Fault Diagnosis, General Classification

Bearing fault diagnosis has been a challenge in the monitoring activities of
rotating machinery, and it's receiving more and more attention. The
conventional fault diagnosis methods usually extract features from the
waveforms or spectrums of vibration signals in order to realize fault
classification. In this paper, a novel feature in the form of images is
presented, namely the spectrum images of vibration signals. The spectrum images
are simply obtained by doing fast Fourier transformation. Such images are
processed with two-dimensional principal component analysis (2DPCA) to reduce
the dimensions, and then a minimum distance method is applied to classify the
faults of bearings. The effectiveness of the proposed method is verified with
experimental data.
The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems
Authors: Ryan Lowe, Nissan Pow, Iulian Serban, Joelle Pineau
Date: 2015-06-30 00:00:00
Proceeding: WS 2015 9
Tasks: Answer Selection, Conversational Response Selection, dialog state tracking

This paper introduces the Ubuntu Dialogue Corpus, a dataset containing almost
1 million multi-turn dialogues, with a total of over 7 million utterances and
100 million words. This provides a unique resource for research into building
dialogue managers based on neural language models that can make use of large
amounts of unlabeled data. The dataset has both the multi-turn property of
conversations in the Dialog State Tracking Challenge datasets, and the
unstructured nature of interactions from microblog services such as Twitter. We
also describe two neural learning architectures suitable for analyzing this
dataset, and provide benchmark performance on the task of selecting the best
next response.
Exploiting Local Structures with the Kronecker Layer in Convolutional Networks
Authors: Shuchang Zhou, Jia-Nan Wu, Yuxin Wu, Xinyu Zhou
Date: 2015-12-31 00:00:00
Tasks: Scene Text Recognition

In this paper, we propose and study a technique to reduce the number of
parameters and computation time in convolutional neural networks. We use
Kronecker product to exploit the local structures within convolution and
fully-connected layers, by replacing the large weight matrices by combinations
of multiple Kronecker products of smaller matrices. Just as the Kronecker
product is a generalization of the outer product from vectors to matrices, our
method is a generalization of the low rank approximation method for convolution
neural networks. We also introduce combinations of different shapes of
Kronecker product to increase modeling capacity. Experiments on SVHN, scene
text recognition and ImageNet dataset demonstrate that we can achieve $3.3
\times$ speedup or $3.6 \times$ parameter reduction with less than 1\% drop in
accuracy, showing the effectiveness and efficiency of our method. Moreover, the
computation efficiency of Kronecker layer makes using larger feature map
possible, which in turn enables us to outperform the previous state-of-the-art
on both SVHN(digit recognition) and CASIA-HWDB (handwritten Chinese character
recognition) datasets.