pwc-archive/papers-with-abstracts huggingface dataset:
Scalable Semi-supervised Learning with Graph-based Kernel Machine
Authors: Trung Le, Khanh Nguyen, Van Nguyen, Vu Nguyen, Dinh Phung
Date: 2016-06-22 00:00:00
Tasks: BIG-bench Machine Learning

Acquiring labels are often costly, whereas unlabeled data are usually easy to
obtain in modern machine learning applications. Semi-supervised learning
provides a principled machine learning framework to address such situations,
and has been applied successfully in many real-word applications and
industries. Nonetheless, most of existing semi-supervised learning methods
encounter two serious limitations when applied to modern and large-scale
datasets: computational burden and memory usage demand. To this end, we present
in this paper the Graph-based semi-supervised Kernel Machine (GKM), a method
that leverages the generalization ability of kernel-based method with the
geometrical and distributive information formulated through a spectral graph
induced from data for semi-supervised learning purpose. Our proposed GKM can be
solved directly in the primal form using the Stochastic Gradient Descent method
with the ideal convergence rate $O(\frac{1}{T})$. Besides, our formulation is
suitable for a wide spectrum of important loss functions in the literature of
machine learning (e.g., Hinge, smooth Hinge, Logistic, L1, and
{\epsilon}-insensitive) and smoothness functions (i.e., $l_p(t) = |t|^p$ with
$p\ge1$). We further show that the well-known Laplacian Support Vector Machine
is a special case of our formulation. We validate our proposed method on
several benchmark datasets to demonstrate that GKM is appropriate for the
large-scale datasets since it is optimal in memory usage and yields superior
classification accuracy whilst simultaneously achieving a significant
computation speed-up in comparison with the state-of-the-art baselines.
The Evolution of Neural Network-Based Chart Patterns: A Preliminary Study
Authors: Myoung Hoon Ha, Byung-Ro Moon
Date: 2017-04-06 00:00:00

A neural network-based chart pattern represents adaptive parametric features,
including non-linear transformations, and a template that can be applied in the
feature space. The search of neural network-based chart patterns has been
unexplored despite its potential expressiveness. In this paper, we formulate a
general chart pattern search problem to enable cross-representational
quantitative comparison of various search schemes. We suggest a HyperNEAT
framework applying state-of-the-art deep neural network techniques to find
attractive neural network-based chart patterns; These techniques enable a fast
evaluation and search of robust patterns, as well as bringing a performance
gain. The proposed framework successfully found attractive patterns on the
Korean stock market. We compared newly found patterns with those found by
different search schemes, showing the proposed approach has potential.
Joint 2D-3D-Semantic Data for Indoor Scene Understanding
Authors: Iro Armeni, Sasha Sax, Amir R. Zamir, Silvio Savarese
Date: 2017-02-03 00:00:00
Tasks: Scene Understanding

We present a dataset of large-scale indoor spaces that provides a variety of
mutually registered modalities from 2D, 2.5D and 3D domains, with
instance-level semantic and geometric annotations. The dataset covers over
6,000m2 and contains over 70,000 RGB images, along with the corresponding
depths, surface normals, semantic annotations, global XYZ images (all in forms
of both regular and 360{\deg} equirectangular images) as well as camera
information. It also includes registered raw and semantically annotated 3D
meshes and point clouds. The dataset enables development of joint and
cross-modal learning models and potentially unsupervised approaches utilizing
the regularities present in large-scale indoor spaces. The dataset is available
here: http://3Dsemantics.stanford.edu/
Joint Detection and Identification Feature Learning for Person Search
Authors: Tong Xiao, Shuang Li, Bochao Wang, Liang Lin, Xiaogang Wang
Date: 2016-04-07 00:00:00
Proceeding: CVPR 2017 7
Tasks: Pedestrian Detection, Person Re-Identification, Person Search

Existing person re-identification benchmarks and methods mainly focus on
matching cropped pedestrian images between queries and candidates. However, it
is different from real-world scenarios where the annotations of pedestrian
bounding boxes are unavailable and the target person needs to be searched from
a gallery of whole scene images. To close the gap, we propose a new deep
learning framework for person search. Instead of breaking it down into two
separate tasks---pedestrian detection and person re-identification, we jointly
handle both aspects in a single convolutional neural network. An Online
Instance Matching (OIM) loss function is proposed to train the network
effectively, which is scalable to datasets with numerous identities. To
validate our approach, we collect and annotate a large-scale benchmark dataset
for person search. It contains 18,184 images, 8,432 identities, and 96,143
pedestrian bounding boxes. Experiments show that our framework outperforms
other separate approaches, and the proposed OIM loss function converges much
faster and better than the conventional Softmax loss.
Learning Features of Music from Scratch
Authors: John Thickstun, Zaid Harchaoui, Sham Kakade
Date: 2016-11-29 00:00:00
Tasks: BIG-bench Machine Learning, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Music Transcription

This paper introduces a new large-scale music dataset, MusicNet, to serve as
a source of supervision and evaluation of machine learning methods for music
research. MusicNet consists of hundreds of freely-licensed classical music
recordings by 10 composers, written for 11 instruments, together with
instrument/note annotations resulting in over 1 million temporal labels on 34
hours of chamber music performances under various studio and microphone
conditions.
  The paper defines a multi-label classification task to predict notes in
musical recordings, along with an evaluation protocol, and benchmarks several
machine learning architectures for this task: i) learning from spectrogram
features; ii) end-to-end learning with a neural net; iii) end-to-end learning
with a convolutional neural net. These experiments show that end-to-end models
trained for note prediction learn frequency selective filters as a low-level
representation of audio.
The Relative Performance of Ensemble Methods with Deep Convolutional Neural Networks for Image Classification
Authors: Cheng Ju, Aurélien Bibaut, Mark J. Van Der Laan
Date: 2017-04-05 00:00:00
Tasks: General Classification, image-classification, Image Classification, Machine Translation, Semantic Segmentation, Translation

Artificial neural networks have been successfully applied to a variety of
machine learning tasks, including image recognition, semantic segmentation, and
machine translation. However, few studies fully investigated ensembles of
artificial neural networks. In this work, we investigated multiple widely used
ensemble methods, including unweighted averaging, majority voting, the Bayes
Optimal Classifier, and the (discrete) Super Learner, for image recognition
tasks, with deep neural networks as candidate algorithms. We designed several
experiments, with the candidate algorithms being the same network structure
with different model checkpoints within a single training process, networks
with same structure but trained multiple times stochastically, and networks
with different structure. In addition, we further studied the over-confidence
phenomenon of the neural networks, as well as its impact on the ensemble
methods. Across all of our experiments, the Super Learner achieved best
performance among all the ensemble methods in this study.
Greed is Good: Near-Optimal Submodular Maximization via Greedy Optimization
Authors: Moran Feldman, Christopher Harshaw, Amin Karbasi
Date: 2017-04-05 00:00:00
Tasks: Movie Recommendation

It is known that greedy methods perform well for maximizing monotone
submodular functions. At the same time, such methods perform poorly in the face
of non-monotonicity. In this paper, we show - arguably, surprisingly - that
invoking the classical greedy algorithm $O(\sqrt{k})$-times leads to the
(currently) fastest deterministic algorithm, called Repeated Greedy, for
maximizing a general submodular function subject to $k$-independent system
constraints. Repeated Greedy achieves $(1 + O(1/\sqrt{k}))k$ approximation
using $O(nr\sqrt{k})$ function evaluations (here, $n$ and $r$ denote the size
of the ground set and the maximum size of a feasible solution, respectively).
We then show that by a careful sampling procedure, we can run the greedy
algorithm only once and obtain the (currently) fastest randomized algorithm,
called Sample Greedy, for maximizing a submodular function subject to
$k$-extendible system constraints (a subclass of $k$-independent system
constrains). Sample Greedy achieves $(k + 3)$-approximation with only $O(nr/k)$
function evaluations. Finally, we derive an almost matching lower bound, and
show that no polynomial time algorithm can have an approximation ratio smaller
than $ k + 1/2 - \varepsilon$. To further support our theoretical results, we
compare the performance of Repeated Greedy and Sample Greedy with prior art in
a concrete application (movie recommendation). We consistently observe that
while Sample Greedy achieves practically the same utility as the best baseline,
it performs at least two orders of magnitude faster.
Identifiability of an X-rank decomposition of polynomial maps
Authors: Pierre Comon, Yang Qi, Konstantin Usevich
Date: 2016-03-04 00:00:00
Tasks: BIG-bench Machine Learning

In this paper, we study a polynomial decomposition model that arises in
problems of system identification, signal processing and machine learning. We
show that this decomposition is a special case of the X-rank decomposition ---
a powerful novel concept in algebraic geometry that generalizes the tensor CP
decomposition. We prove new results on generic/maximal rank and on
identifiability of a particular polynomial decomposition model. In the paper,
we try to make results and basic tools accessible for general audience
(assuming no knowledge of algebraic geometry or its prerequisites).
A novel method for automatic localization of joint area on knee plain radiographs
Authors: Aleksei Tiulpin, Jérôme Thevenot, Esa Rahtu, Simo Saarakkala
Date: 2017-01-31 00:00:00

Osteoarthritis (OA) is a common musculoskeletal condition typically diagnosed
from radiographic assessment after clinical examination. However, a visual
evaluation made by a practitioner suffers from subjectivity and is highly
dependent on the experience. Computer-aided diagnostics (CAD) could improve the
objectivity of knee radiographic examination. The first essential step of knee
OA CAD is to automatically localize the joint area. However, according to the
literature this task itself remains challenging. The aim of this study was to
develop novel and computationally efficient method to tackle the issue. Here,
three different datasets of knee radiographs were used (n = 473/93/77) to
validate the overall performance of the method. Our pipeline consists of two
parts: anatomically-based joint area proposal and their evaluation using
Histogram of Oriented Gradients and the pre-trained Support Vector Machine
classifier scores. The obtained results for the used datasets show the mean
intersection over the union equal to: 0.84, 0.79 and 0.78. Using a high-end
computer, the method allows to automatically annotate conventional knee
radiographs within 14-16ms and high resolution ones within 170ms. Our results
demonstrate that the developed method is suitable for large-scale analyses.
Nonnegative/binary matrix factorization with a D-Wave quantum annealer
Authors: Daniel O'Malley, Velimir V. Vesselinov, Boian S. Alexandrov, Ludmil B. Alexandrov
Date: 2017-04-05 00:00:00
Tasks: BIG-bench Machine Learning

D-Wave quantum annealers represent a novel computational architecture and
have attracted significant interest, but have been used for few real-world
computations. Machine learning has been identified as an area where quantum
annealing may be useful. Here, we show that the D-Wave 2X can be effectively
used as part of an unsupervised machine learning method. This method can be
used to analyze large datasets. The D-Wave only limits the number of features
that can be extracted from the dataset. We apply this method to learn the
features from a set of facial images.
Rhetorical relations for information retrieval
Authors: Christina Lioma, Birger Larsen, Wei Lu
Date: 2017-04-05 00:00:00
Tasks: Information Retrieval, Language Modeling, Language Modelling, Retrieval

Typically, every part in most coherent text has some plausible reason for its
presence, some function that it performs to the overall semantics of the text.
Rhetorical relations, e.g. contrast, cause, explanation, describe how the parts
of a text are linked to each other. Knowledge about this socalled discourse
structure has been applied successfully to several natural language processing
tasks. This work studies the use of rhetorical relations for Information
Retrieval (IR): Is there a correlation between certain rhetorical relations and
retrieval performance? Can knowledge about a document's rhetorical relations be
useful to IR? We present a language model modification that considers
rhetorical relations when estimating the relevance of a document to a query.
Empirical evaluation of different versions of our model on TREC settings shows
that certain rhetorical relations can benefit retrieval effectiveness notably
(> 10% in mean average precision over a state-of-the-art baseline).
Best Practices for Applying Deep Learning to Novel Applications
Authors: Leslie N. Smith
Date: 2017-04-05 00:00:00
Tasks: Deep Learning

This report is targeted to groups who are subject matter experts in their
application but deep learning novices. It contains practical advice for those
interested in testing the use of deep neural networks on applications that are
novel for deep learning. We suggest making your project more manageable by
dividing it into phases. For each phase this report contains numerous
recommendations and insights to assist novice practitioners.
Comment on "Biologically inspired protection of deep networks from adversarial attacks"
Authors: Wieland Brendel, Matthias Bethge
Date: 2017-04-05 00:00:00

A recent paper suggests that Deep Neural Networks can be protected from
gradient-based adversarial perturbations by driving the network activations
into a highly saturated regime. Here we analyse such saturated networks and
show that the attacks fail due to numerical limitations in the gradient
computations. A simple stabilisation of the gradient estimates enables
successful and efficient attacks. Thus, it has yet to be shown that the
robustness observed in highly saturated networks is not simply due to numerical
limitations.
MIT at SemEval-2017 Task 10: Relation Extraction with Convolutional Neural Networks
Authors: Ji Young Lee, Franck Dernoncourt, Peter Szolovits
Date: 2017-04-05 00:00:00
Proceeding: SEMEVAL 2017 8
Tasks: Articles, Relation, Relation Extraction

Over 50 million scholarly articles have been published: they constitute a
unique repository of knowledge. In particular, one may infer from them
relations between scientific concepts, such as synonyms and hyponyms.
Artificial neural networks have been recently explored for relation extraction.
In this work, we continue this line of work and present a system based on a
convolutional neural network to extract relations. Our model ranked first in
the SemEval-2017 task 10 (ScienceIE) for relation extraction in scientific
articles (subtask C).
Generating Descriptions with Grounded and Co-Referenced People
Authors: Anna Rohrbach, Marcus Rohrbach, Siyu Tang, Seong Joon Oh, Bernt Schiele
Date: 2017-04-05 00:00:00
Proceeding: CVPR 2017 7

Learning how to generate descriptions of images or videos received major
interest both in the Computer Vision and Natural Language Processing
communities. While a few works have proposed to learn a grounding during the
generation process in an unsupervised way (via an attention mechanism), it
remains unclear how good the quality of the grounding is and whether it
benefits the description quality. In this work we propose a movie description
model which learns to generate description and jointly ground (localize) the
mentioned characters as well as do visual co-reference resolution between pairs
of consecutive sentences/clips. We also propose to use weak localization
supervision through character mentions provided in movie descriptions to learn
the character grounding. At training time, we first learn how to localize
characters by relating their visual appearance to mentions in the descriptions
via a semi-supervised approach. We then provide this (noisy) supervision into
our description model which greatly improves its performance. Our proposed
description model improves over prior work w.r.t. generated description quality
and additionally provides grounding and local co-reference resolution. We
evaluate it on the MPII Movie Description dataset using automatic and human
evaluation measures and using our newly collected grounding and co-reference
data for characters.
Isotropic reconstruction of 3D fluorescence microscopy images using convolutional neural networks
Authors: Martin Weigert, Loic Royer, Florian Jug, Gene Myers
Date: 2017-04-05 00:00:00
Tasks: Super-Resolution

Fluorescence microscopy images usually show severe anisotropy in axial versus
lateral resolution. This hampers downstream processing, i.e. the automatic
extraction of quantitative biological data. While deconvolution methods and
other techniques to address this problem exist, they are either time consuming
to apply or limited in their ability to remove anisotropy. We propose a method
to recover isotropic resolution from readily acquired anisotropic data. We
achieve this using a convolutional neural network that is trained end-to-end
from the same anisotropic body of data we later apply the network to. The
network effectively learns to restore the full isotropic resolution by
restoring the image under a trained, sample specific image prior. We apply our
method to $3$ synthetic and $3$ real datasets and show that our results improve
on results from deconvolution and state-of-the-art super-resolution techniques.
Finally, we demonstrate that a standard 3D segmentation pipeline performs on
the output of our network with comparable accuracy as on the full isotropic
data.
Weakly Supervised Dense Video Captioning
Authors: Zhiqiang Shen, Jianguo Li, Zhou Su, Minjun Li, Yurong Chen, Yu-Gang Jiang, xiangyang xue
Date: 2017-04-05 00:00:00
Proceeding: CVPR 2017 7
Tasks: Dense Video Captioning, Language Modeling, Language Modelling, Multi-Label Learning, Sentence, Video Captioning

This paper focuses on a novel and challenging vision task, dense video
captioning, which aims to automatically describe a video clip with multiple
informative and diverse caption sentences. The proposed method is trained
without explicit annotation of fine-grained sentence to video region-sequence
correspondence, but is only based on weak video-level sentence annotations. It
differs from existing video captioning systems in three technical aspects.
First, we propose lexical fully convolutional neural networks (Lexical-FCN)
with weakly supervised multi-instance multi-label learning to weakly link video
regions with lexical labels. Second, we introduce a novel submodular
maximization scheme to generate multiple informative and diverse
region-sequences based on the Lexical-FCN outputs. A winner-takes-all scheme is
adopted to weakly associate sentences to region-sequences in the training
phase. Third, a sequence-to-sequence learning based language model is trained
with the weakly supervised information obtained through the association
process. We show that the proposed method can not only produce informative and
diverse dense captions, but also outperform state-of-the-art single video
captioning methods by a large margin.
Automated Generation of Multilingual Clusters for the Evaluation of Distributed Representations
Authors: Philip Blair, Yuval Merhav, Joel Barry
Date: 2016-11-04 00:00:00
Tasks: Outlier Detection, Sentiment Analysis, Word Embeddings

We propose a language-agnostic way of automatically generating sets of
semantically similar clusters of entities along with sets of "outlier"
elements, which may then be used to perform an intrinsic evaluation of word
embeddings in the outlier detection task. We used our methodology to create a
gold-standard dataset, which we call WikiSem500, and evaluated multiple
state-of-the-art embeddings. The results show a correlation between performance
on this dataset and performance on sentiment analysis.
Comparison Based Nearest Neighbor Search
Authors: Siavash Haghiri, Debarghya Ghoshdastidar, Ulrike Von Luxburg
Date: 2017-04-05 00:00:00
Tasks: Triplet

We consider machine learning in a comparison-based setting where we are given
a set of points in a metric space, but we have no access to the actual
distances between the points. Instead, we can only ask an oracle whether the
distance between two points $i$ and $j$ is smaller than the distance between
the points $i$ and $k$. We are concerned with data structures and algorithms to
find nearest neighbors based on such comparisons. We focus on a simple yet
effective algorithm that recursively splits the space by first selecting two
random pivot points and then assigning all other points to the closer of the
two (comparison tree). We prove that if the metric space satisfies certain
expansion conditions, then with high probability the height of the comparison
tree is logarithmic in the number of points, leading to efficient search
performance. We also provide an upper bound for the failure probability to
return the true nearest neighbor. Experiments show that the comparison tree is
competitive with algorithms that have access to the actual distance values, and
needs less triplet comparisons than other competitors.
A pre-semantics for counterfactual conditionals and similar logics
Authors: Karl Schlechta
Date: 2016-12-06 00:00:00
Tasks: counterfactual

The elegant Stalnaker/Lewis semantics for counterfactual conditonals works
with distances between models. But human beings certainly have no tables of
models and distances in their head. We begin here an investigation using a more
realistic picture, based on findings in neuroscience. We call it a
pre-semantics, as its meaning is not a description of the world, but of the
brain, whose structure is (partly) determined by the world it reasons about. In
the final section, we reconsider the components, and postulate that there are
no atomic pictures, we can always look inside.
Bayesian Inference of Log Determinants
Authors: Jack Fitzsimons, Kurt Cutajar, Michael Osborne, Stephen Roberts, Maurizio Filippone
Date: 2017-04-05 00:00:00
Tasks: Bayesian Inference, Gaussian Processes, Point Processes

The log-determinant of a kernel matrix appears in a variety of machine
learning problems, ranging from determinantal point processes and generalized
Markov random fields, through to the training of Gaussian processes. Exact
calculation of this term is often intractable when the size of the kernel
matrix exceeds a few thousand. In the spirit of probabilistic numerics, we
reinterpret the problem of computing the log-determinant as a Bayesian
inference problem. In particular, we combine prior knowledge in the form of
bounds from matrix theory and evidence derived from stochastic trace estimation
to obtain probabilistic estimates for the log-determinant and its associated
uncertainty within a given computational budget. Beyond its novelty and
theoretic appeal, the performance of our proposal is competitive with
state-of-the-art approaches to approximating the log-determinant, while also
quantifying the uncertainty due to budget-constrained evidence.