pwc-archive/papers-with-abstracts huggingface dataset:
A Robust Multi-Batch L-BFGS Method for Machine Learning
Authors: Albert S. Berahas, Martin Takáč
Date: 2017-07-26 00:00:00
Tasks: BIG-bench Machine Learning, Binary Classification, Distributed Computing

This paper describes an implementation of the L-BFGS method designed to deal with two adversarial situations. The first occurs in distributed computing environments where some of the computational nodes devoted to the evaluation of the function and gradient are unable to return results on time. A similar challenge occurs in a multi-batch approach in which the data points used to compute function and gradients are purposely changed at each iteration to accelerate the learning process. Difficulties arise because L-BFGS employs gradient differences to update the Hessian approximations, and when these gradients are computed using different data points the updating process can be unstable. This paper shows how to perform stable quasi-Newton updating in the multi-batch setting, studies the convergence properties for both convex and nonconvex functions, and illustrates the behavior of the algorithm in a distributed computing platform on binary classification logistic regression and neural network training problems that arise in machine learning.
Strawman: an Ensemble of Deep Bag-of-Ngrams for Sentiment Analysis
Authors: Kyunghyun Cho
Date: 2017-07-26 00:00:00
Proceeding: WS 2017 9
Tasks: Sentence, Sentiment Analysis

This paper describes a builder entry, named "strawman", to the sentence-level
sentiment analysis task of the "Build It, Break It" shared task of the First
Workshop on Building Linguistically Generalizable NLP Systems. The goal of a
builder is to provide an automated sentiment analyzer that would serve as a
target for breakers whose goal is to find pairs of minimally-differing
sentences that break the analyzer.
Measuring Thematic Fit with Distributional Feature Overlap
Authors: Enrico Santus, Emmanuele Chersoni, Alessandro Lenci, Philippe Blache
Date: 2017-07-19 00:00:00
Proceeding: EMNLP 2017 9

In this paper, we introduce a new distributional method for modeling
predicate-argument thematic fit judgments. We use a syntax-based DSM to build a
prototypical representation of verb-specific roles: for every verb, we extract
the most salient second order contexts for each of its roles (i.e. the most
salient dimensions of typical role fillers), and then we compute thematic fit
as a weighted overlap between the top features of candidate fillers and role
prototypes. Our experiments show that our method consistently outperforms a
baseline re-implementing a state-of-the-art system, and achieves better or
comparable results to those reported in the literature for the other
unsupervised systems. Moreover, it provides an explicit representation of the
features characterizing verb-specific semantic roles.
Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image
Authors: Rui Zhu, Hamed Kiani Galoogahi, Chaoyang Wang, Simon Lucey
Date: 2017-07-15 00:00:00

An emerging problem in computer vision is the reconstruction of 3D shape and
pose of an object from a single image. Hitherto, the problem has been addressed
through the application of canonical deep learning methods to regress from the
image directly to the 3D shape and pose labels. These approaches, however, are
problematic from two perspectives. First, they are minimizing the error between
3D shapes and pose labels - with little thought about the nature of this label
error when reprojecting the shape back onto the image. Second, they rely on the
onerous and ill-posed task of hand labeling natural images with respect to 3D
shape and pose. In this paper we define the new task of pose-aware shape
reconstruction from a single image, and we advocate that cheaper 2D annotations
of objects silhouettes in natural images can be utilized. We design
architectures of pose-aware shape reconstruction which re-project the predicted
shape back on to the image using the predicted pose. Our evaluation on several
object categories demonstrates the superiority of our method for predicting
pose-aware 3D shapes from natural images.
A Guided Spatial Transformer Network for Histology Cell Differentiation
Authors: Marc Aubreville, Maximilian Krappmann, Christof Bertram, Robert Klopfleisch, Andreas Maier
Date: 2017-07-26 00:00:00
Tasks: Diagnostic, General Classification

Identification and counting of cells and mitotic figures is a standard task
in diagnostic histopathology. Due to the large overall cell count on
histological slides and the potential sparse prevalence of some relevant cell
types or mitotic figures, retrieving annotation data for sufficient statistics
is a tedious task and prone to a significant error in assessment. Automatic
classification and segmentation is a classic task in digital pathology, yet it
is not solved to a sufficient degree.
  We present a novel approach for cell and mitotic figure classification, based
on a deep convolutional network with an incorporated Spatial Transformer
Network. The network was trained on a novel data set with ten thousand mitotic
figures, about ten times more than previous data sets. The algorithm is able to
derive the cell class (mitotic tumor cells, non-mitotic tumor cells and
granulocytes) and their position within an image. The mean accuracy of the
algorithm in a five-fold cross-validation is 91.45%.
  In our view, the approach is a promising step into the direction of a more
objective and accurate, semi-automatized mitosis counting supporting the
pathologist.
Sobolev Training for Neural Networks
Authors: Wojciech Marian Czarnecki, Simon Osindero, Max Jaderberg, Grzegorz Świrszcz, Razvan Pascanu
Date: 2017-06-15 00:00:00
Proceeding: NeurIPS 2017 12

At the heart of deep learning we aim to use neural networks as function
approximators - training them to produce outputs from inputs in emulation of a
ground truth function or data creation process. In many cases we only have
access to input-output pairs from the ground truth, however it is becoming more
common to have access to derivatives of the target output with respect to the
input - for example when the ground truth function is itself a neural network
such as in network compression or distillation. Generally these target
derivatives are not computed, or are ignored. This paper introduces Sobolev
Training for neural networks, which is a method for incorporating these target
derivatives in addition the to target values while training. By optimising
neural networks to not only approximate the function's outputs but also the
function's derivatives we encode additional information about the target
function within the parameters of the neural network. Thereby we can improve
the quality of our predictors, as well as the data-efficiency and
generalization capabilities of our learned function approximation. We provide
theoretical justifications for such an approach as well as examples of
empirical evidence on three distinct domains: regression on classical
optimisation datasets, distilling policies of an agent playing Atari, and on
large-scale applications of synthetic gradients. In all three domains the use
of Sobolev Training, employing target derivatives in addition to target values,
results in models with higher accuracy and stronger generalisation.
Making the best of data derived from a daily practice in clinical legal medicine for research and practice - the example of Spe3dLab
Authors: Vincent Laugier, Eric Stindel, Alcibiade Lichterowicz, Séverine Ansart, Thomas Lefèvre
Date: 2017-07-26 00:00:00
Tasks: Specificity

Forensic science suffers from a lack of studies with high-quality design,
such as randomized controlled trials (RCT). Evidence in forensic science may be
of insufficient quality, which is a major concern. Results from RCT are
criticized for providing artificial results that are not useful in real life
and unfit for individualized prescription. Various sources of collected data
(e.g. data collected in routine practice) could be exploited for distinct
goals. Obstacles remain before such data can be practically accessed and used,
including technical issues. We present an easy-to-use software dedicated to
innovative data analyses for practitioners and researchers. We provide 2
examples in forensics. Spe3dLab has been developed by 3 French teams: a
bioinformatics laboratory (LaTIM), a private partner (Tekliko) and a department
of forensic medicine (Jean Verdier Hospital). It was designed to be open
source, relying on documented and maintained libraries, query-oriented and
capable of handling the entire data process from capture to export of best
predictive models for their integration in information systems. Spe3dLab was
used for 2 specific forensics applications: i) the search for multiple causal
factors and ii) the best predictive model of the functional impairment (total
incapacity to work, TIW) of assault survivors. 2,892 patients were included
over a 6-month period. Time to evaluation was the only direct cause identified
for TIW, and victim category was an indirect cause. The specificity and
sensitivity of the predictive model were 99.9% and 90%, respectively. Spe3dLab
is a quick and efficient tool for accessing observational, routinely collected
data and performing innovative analyses. Analyses can be exported for
validation and routine use by practitioners, e.g., for computer-aided
evaluation of complex problems. It can provide a fully integrated solution for
individualized medicine.
Dynamic Clustering Algorithms via Small-Variance Analysis of Markov Chain Mixture Models
Authors: Trevor Campbell, Brian Kulis, Jonathan How
Date: 2017-07-26 00:00:00
Tasks: Clustering

Bayesian nonparametrics are a class of probabilistic models in which the
model size is inferred from data. A recently developed methodology in this
field is small-variance asymptotic analysis, a mathematical technique for
deriving learning algorithms that capture much of the flexibility of Bayesian
nonparametric inference algorithms, but are simpler to implement and less
computationally expensive. Past work on small-variance analysis of Bayesian
nonparametric inference algorithms has exclusively considered batch models
trained on a single, static dataset, which are incapable of capturing time
evolution in the latent structure of the data. This work presents a
small-variance analysis of the maximum a posteriori filtering problem for a
temporally varying mixture model with a Markov dependence structure, which
captures temporally evolving clusters within a dataset. Two clustering
algorithms result from the analysis: D-Means, an iterative clustering algorithm
for linearly separable, spherical clusters; and SD-Means, a spectral clustering
algorithm derived from a kernelized, relaxed version of the clustering problem.
Empirical results from experiments demonstrate the advantages of using D-Means
and SD-Means over contemporary clustering algorithms, in terms of both
computational cost and clustering accuracy.
Robust Kronecker-Decomposable Component Analysis for Low-Rank Modeling
Authors: Mehdi Bahri, Yannis Panagakis, Stefanos Zafeiriou
Date: 2017-03-22 00:00:00
Proceeding: ICCV 2017 10
Tasks: Denoising, Dictionary Learning, Image Denoising

Dictionary learning and component analysis are part of one of the most
well-studied and active research fields, at the intersection of signal and
image processing, computer vision, and statistical machine learning. In
dictionary learning, the current methods of choice are arguably K-SVD and its
variants, which learn a dictionary (i.e., a decomposition) for sparse coding
via Singular Value Decomposition. In robust component analysis, leading methods
derive from Principal Component Pursuit (PCP), which recovers a low-rank matrix
from sparse corruptions of unknown magnitude and support. However, K-SVD is
sensitive to the presence of noise and outliers in the training set.
Additionally, PCP does not provide a dictionary that respects the structure of
the data (e.g., images), and requires expensive SVD computations when solved by
convex relaxation. In this paper, we introduce a new robust decomposition of
images by combining ideas from sparse dictionary learning and PCP. We propose a
novel Kronecker-decomposable component analysis which is robust to gross
corruption, can be used for low-rank modeling, and leverages separability to
solve significantly smaller problems. We design an efficient learning algorithm
by drawing links with a restricted form of tensor factorization. The
effectiveness of the proposed approach is demonstrated on real-world
applications, namely background subtraction and image denoising, by performing
a thorough comparison with the current state of the art.
Implicit Entity Linking in Tweets
Authors: Sujan Perera, Pablo N. Mendes, Adarsh Alex, Amit Sheth, Krishnaprasad Thirunarayan
Date: 2017-07-26 00:00:00
Tasks: Entity Linking, Natural Language Understanding

Over the years, Twitter has become one of the largest communication platforms
providing key data to various applications such as brand monitoring, trend
detection, among others. Entity linking is one of the major tasks in natural
language understanding from tweets and it associates entity mentions in text to
corresponding entries in knowledge bases in order to provide unambiguous
interpretation and additional con- text. State-of-the-art techniques have
focused on linking explicitly mentioned entities in tweets with reasonable
success. However, we argue that in addition to explicit mentions i.e. The movie
Gravity was more ex- pensive than the mars orbiter mission entities (movie
Gravity) can also be mentioned implicitly i.e. This new space movie is crazy.
you must watch it!. This paper introduces the problem of implicit entity
linking in tweets. We propose an approach that models the entities by
exploiting their factual and contextual knowledge. We demonstrate how to use
these models to perform implicit entity linking on a ground truth dataset with
397 tweets from two domains, namely, Movie and Book. Specifically, we show: 1)
the importance of linking implicit entities and its value addition to the
standard entity linking task, and 2) the importance of exploiting contextual
knowledge associated with an entity for linking their implicit mentions. We
also make the ground truth dataset publicly available to foster the research in
this new research area.
Men Are from Mars, Women Are from Venus: Evaluation and Modelling of Verbal Associations
Authors: Ekaterina Vylomova, Andrei Shcherbakov, Yuriy Philippovich, Galina Cherkasova
Date: 2017-07-26 00:00:00

We present a quantitative analysis of human word association pairs and study
the types of relations presented in the associations. We put our main focus on
the correlation between response types and respondent characteristics such as
occupation and gender by contrasting syntagmatic and paradigmatic associations.
Finally, we propose a personalised distributed word association model and show
the importance of incorporating demographic factors into the models commonly
used in natural language processing.
Flow Navigation by Smart Microswimmers via Reinforcement Learning
Authors: Simona Colabrese, Kristian Gustavsson, Antonio Celani, Luca Biferale
Date: 2017-01-30 00:00:00
Tasks: Navigate, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Smart active particles can acquire some limited knowledge of the fluid
environment from simple mechanical cues and exert a control on their preferred
steering direction. Their goal is to learn the best way to navigate by
exploiting the underlying flow whenever possible. As an example, we focus our
attention on smart gravitactic swimmers. These are active particles whose task
is to reach the highest altitude within some time horizon, given the
constraints enforced by fluid mechanics. By means of numerical experiments, we
show that swimmers indeed learn nearly optimal strategies just by experience. A
reinforcement learning algorithm allows particles to learn effective strategies
even in difficult situations when, in the absence of control, they would end up
being trapped by flow structures. These strategies are highly nontrivial and
cannot be easily guessed in advance. This Letter illustrates the potential of
reinforcement learning algorithms to model adaptive behavior in complex flows
and paves the way towards the engineering of smart microswimmers that solve
difficult navigation problems.
Trespassing the Boundaries: Labeling Temporal Bounds for Object Interactions in Egocentric Video
Authors: Davide Moltisanti, Michael Wray, Walterio Mayol-Cuevas, Dima Damen
Date: 2017-03-27 00:00:00
Proceeding: ICCV 2017 10
Tasks: Object

Manual annotations of temporal bounds for object interactions (i.e. start and
end times) are typical training input to recognition, localization and
detection algorithms. For three publicly available egocentric datasets, we
uncover inconsistencies in ground truth temporal bounds within and across
annotators and datasets. We systematically assess the robustness of
state-of-the-art approaches to changes in labeled temporal bounds, for object
interaction recognition. As boundaries are trespassed, a drop of up to 10% is
observed for both Improved Dense Trajectories and Two-Stream Convolutional
Neural Network.
  We demonstrate that such disagreement stems from a limited understanding of
the distinct phases of an action, and propose annotating based on the Rubicon
Boundaries, inspired by a similarly named cognitive model, for consistent
temporal bounds of object interactions. Evaluated on a public dataset, we
report a 4% increase in overall accuracy, and an increase in accuracy for 55%
of classes when Rubicon Boundaries are used for temporal annotations.
Context-Independent Polyphonic Piano Onset Transcription with an Infinite Training Dataset
Authors: Samuel Li
Date: 2017-07-26 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, Disentanglement

Many of the recent approaches to polyphonic piano note onset transcription
require training a machine learning model on a large piano database. However,
such approaches are limited by dataset availability; additional training data
is difficult to produce, and proposed systems often perform poorly on novel
recording conditions. We propose a method to quickly synthesize arbitrary
quantities of training data, avoiding the need for curating large datasets.
Various aspects of piano note dynamics - including nonlinearity of note
signatures with velocity, different articulations, temporal clustering of
onsets, and nonlinear note partial interference - are modeled to match the
characteristics of real pianos. Our method also avoids the disentanglement
problem, a recently noted issue affecting machine-learning based approaches. We
train a feed-forward neural network with two hidden layers on our generated
training data and achieve both good transcription performance on the large MAPS
piano dataset and excellent generalization qualities.
Using Program Induction to Interpret Transition System Dynamics
Authors: Svetlin Penkov, Subramanian Ramamoorthy
Date: 2017-07-26 00:00:00
Tasks: Program induction

Explaining and reasoning about processes which underlie observed black-box
phenomena enables the discovery of causal mechanisms, derivation of suitable
abstract representations and the formulation of more robust predictions. We
propose to learn high level functional programs in order to represent abstract
models which capture the invariant structure in the observed data. We introduce
the $\pi$-machine (program-induction machine) -- an architecture able to induce
interpretable LISP-like programs from observed data traces. We propose an
optimisation procedure for program learning based on backpropagation, gradient
descent and A* search. We apply the proposed method to two problems: system
identification of dynamical systems and explaining the behaviour of a DQN
agent. Our results show that the $\pi$-machine can efficiently induce
interpretable programs from individual data traces.
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose
Authors: Georgios Pavlakos, Xiaowei Zhou, Konstantinos G. Derpanis, Kostas Daniilidis
Date: 2016-11-23 00:00:00
Proceeding: CVPR 2017 7
Tasks: 3D Human Pose Estimation, Pose Estimation

This paper addresses the challenge of 3D human pose estimation from a single
color image. Despite the general success of the end-to-end learning paradigm,
top performing approaches employ a two-step solution consisting of a
Convolutional Network (ConvNet) for 2D joint localization and a subsequent
optimization step to recover 3D pose. In this paper, we identify the
representation of 3D pose as a critical issue with current ConvNet approaches
and make two important contributions towards validating the value of end-to-end
learning for this task. First, we propose a fine discretization of the 3D space
around the subject and train a ConvNet to predict per voxel likelihoods for
each joint. This creates a natural representation for 3D pose and greatly
improves performance over the direct regression of joint coordinates. Second,
to further improve upon initial estimates, we employ a coarse-to-fine
prediction scheme. This step addresses the large dimensionality increase and
enables iterative refinement and repeated processing of the image features. The
proposed approach outperforms all state-of-the-art methods on standard
benchmarks achieving a relative error reduction greater than 30% on average.
Additionally, we investigate using our volumetric representation in a related
architecture which is suboptimal compared to our end-to-end approach, but is of
practical interest, since it enables training when no image with corresponding
3D groundtruth is available, and allows us to present compelling results for
in-the-wild images.
A Harmony Search Based Wrapper Feature Selection Method for Holistic Bangla word Recognition
Authors: Supratim Das, Pawan Kumar Singh, Showmik Bhowmik, Ram Sarkar, Mita Nasipuri
Date: 2017-07-26 00:00:00
Tasks: Dimensionality Reduction, Evolutionary Algorithms, feature selection, General Classification

A lot of search approaches have been explored for the selection of features
in pattern classification domain in order to discover significant subset of the
features which produces better accuracy. In this paper, we introduced a Harmony
Search (HS) algorithm based feature selection method for feature dimensionality
reduction in handwritten Bangla word recognition problem. This algorithm has
been implemented to reduce the feature dimensionality of a technique described
in one of our previous papers by S. Bhowmik et al.[1]. In the said paper, a set
of 65 elliptical features were computed for handwritten Bangla word recognition
purpose and a recognition accuracy of 81.37% was achieved using Multi Layer
Perceptron (MLP) classifier. In the present work, a subset containing 48
features (approximately 75% of said feature vector) has been selected by HS
based wrapper feature selection method which produces an accuracy rate of
90.29%. Reasonable outcomes also validates that the introduced algorithm
utilizes optimal number of features while showing higher classification
accuracies when compared to two standard evolutionary algorithms like Genetic
Algorithm (GA), Particle Swarm Optimization (PSO) and statistical feature
dimensionality reduction technique like Principal Component Analysis (PCA).
This confirms the suitability of HS algorithm to the holistic handwritten word
recognition problem.