pwc-archive/papers-with-abstracts huggingface dataset:
Unbiased Learning-to-Rank with Biased Feedback
Authors: Thorsten Joachims, Adith Swaminathan, Tobias Schnabel
Date: 2016-08-16 00:00:00
Tasks: counterfactual, Counterfactual Inference, Learning-To-Rank, Retrieval

Implicit feedback (e.g., clicks, dwell times, etc.) is an abundant source of
data in human-interactive systems. While implicit feedback has many advantages
(e.g., it is inexpensive to collect, user centric, and timely), its inherent
biases are a key obstacle to its effective use. For example, position bias in
search rankings strongly influences how many clicks a result receives, so that
directly using click data as a training signal in Learning-to-Rank (LTR)
methods yields sub-optimal results. To overcome this bias problem, we present a
counterfactual inference framework that provides the theoretical basis for
unbiased LTR via Empirical Risk Minimization despite biased data. Using this
framework, we derive a Propensity-Weighted Ranking SVM for discriminative
learning from implicit feedback, where click models take the role of the
propensity estimator. In contrast to most conventional approaches to de-bias
the data using click models, this allows training of ranking functions even in
settings where queries do not repeat. Beyond the theoretical support, we show
empirically that the proposed learning method is highly effective in dealing
with biases, that it is robust to noise and propensity model misspecification,
and that it scales efficiently. We also demonstrate the real-world
applicability of our approach on an operational search engine, where it
substantially improves retrieval performance.
Learning to Track at 100 FPS with Deep Regression Networks
Authors: David Held, Sebastian Thrun, Silvio Savarese
Date: 2016-04-06 00:00:00
Tasks: regression

Machine learning techniques are often used in computer vision due to their
ability to leverage large amounts of training data to improve performance.
Unfortunately, most generic object trackers are still trained from scratch
online and do not benefit from the large number of videos that are readily
available for offline training. We propose a method for offline training of
neural networks that can track novel objects at test-time at 100 fps. Our
tracker is significantly faster than previous methods that use neural networks
for tracking, which are typically very slow to run and not practical for
real-time applications. Our tracker uses a simple feed-forward network with no
online training required. The tracker learns a generic relationship between
object motion and appearance and can be used to track novel objects that do not
appear in the training set. We test our network on a standard tracking
benchmark to demonstrate our tracker's state-of-the-art performance. Further,
our performance improves as we add more videos to our offline training set. To
the best of our knowledge, our tracker is the first neural-network tracker that
learns to track generic objects at 100 fps.
Fast, Small and Exact: Infinite-order Language Modelling with Compressed Suffix Trees
Authors: Ehsan Shareghi, Matthias Petri, Gholamreza Haffari, Trevor Cohn
Date: 2016-08-16 00:00:00
Proceeding: TACL 2016 1
Tasks: Language Modeling, Language Modelling

Efficient methods for storing and querying are critical for scaling
high-order n-gram language models to large corpora. We propose a language model
based on compressed suffix trees, a representation that is highly compact and
can be easily held in memory, while supporting queries needed in computing
language model probabilities on-the-fly. We present several optimisations which
improve query runtimes up to 2500x, despite only incurring a modest increase in
construction time and memory usage. For large corpora and high Markov orders,
our method is highly competitive with the state-of-the-art KenLM package. It
imposes much lower memory requirements, often by orders of magnitude, and has
runtimes that are either similar (for training) or comparable (for querying).
Uniform Transformation of Non-Separable Probability Distributions
Authors: Eric Kee
Date: 2016-08-16 00:00:00

A theoretical framework is developed to describe the transformation that
distributes probability density functions uniformly over space. In one
dimension, the cumulative distribution can be used, but does not generalize to
higher dimensions, or non-separable distributions. A potential function is
shown to link probability density functions to their transformation, and to
generalize the cumulative. A numerical method is developed to compute the
potential, and examples are shown in two dimensions.
Natural Language Processing using Hadoop and KOSHIK
Authors: Emre Erturk, Hong Shi
Date: 2016-08-15 00:00:00
Tasks: Translation

Natural language processing, as a data analytics related technology, is used
widely in many research areas such as artificial intelligence, human language
processing, and translation. At present, due to explosive growth of data, there
are many challenges for natural language processing. Hadoop is one of the
platforms that can process the large amount of data required for natural
language processing. KOSHIK is one of the natural language processing
architectures, and utilizes Hadoop and contains language processing components
such as Stanford CoreNLP and OpenNLP. This study describes how to build a
KOSHIK platform with the relevant tools, and provides the steps to analyze wiki
data. Finally, it evaluates and discusses the advantages and disadvantages of
the KOSHIK architecture, and gives recommendations on improving the processing
performance.
TerpreT: A Probabilistic Programming Language for Program Induction
Authors: Alexander L. Gaunt, Marc Brockschmidt, Rishabh Singh, Nate Kushman, Pushmeet Kohli, Jonathan Taylor, Daniel Tarlow
Date: 2016-08-15 00:00:00
Tasks: BIG-bench Machine Learning, Probabilistic Programming, Program induction, Program Synthesis

We study machine learning formulations of inductive program synthesis; given
input-output examples, we try to synthesize source code that maps inputs to
corresponding outputs. Our aims are to develop new machine learning approaches
based on neural networks and graphical models, and to understand the
capabilities of machine learning techniques relative to traditional
alternatives, such as those based on constraint solving from the programming
languages community.
  Our key contribution is the proposal of TerpreT, a domain-specific language
for expressing program synthesis problems. TerpreT is similar to a
probabilistic programming language: a model is composed of a specification of a
program representation (declarations of random variables) and an interpreter
describing how programs map inputs to outputs (a model connecting unknowns to
observations). The inference task is to observe a set of input-output examples
and infer the underlying program. TerpreT has two main benefits. First, it
enables rapid exploration of a range of domains, program representations, and
interpreter models. Second, it separates the model specification from the
inference algorithm, allowing like-to-like comparisons between different
approaches to inference. From a single TerpreT specification we automatically
perform inference using four different back-ends. These are based on gradient
descent, linear program (LP) relaxations for graphical models, discrete
satisfiability solving, and the Sketch program synthesis system.
  We illustrate the value of TerpreT by developing several interpreter models
and performing an empirical comparison between alternative inference
algorithms. Our key empirical finding is that constraint solvers dominate the
gradient descent and LP-based formulations. We conclude with suggestions for
the machine learning community to make progress on program synthesis.
Play and Learn: Using Video Games to Train Computer Vision Models
Authors: Alireza Shafaei, James J. Little, Mark Schmidt
Date: 2016-08-05 00:00:00
Tasks: Depth Estimation, Domain Adaptation, image-classification, Image Classification, Image Segmentation, Semantic Segmentation

Video games are a compelling source of annotated data as they can readily
provide fine-grained groundtruth for diverse tasks. However, it is not clear
whether the synthetically generated data has enough resemblance to the
real-world images to improve the performance of computer vision models in
practice. We present experiments assessing the effectiveness on real-world data
of systems trained on synthetic RGB images that are extracted from a video
game. We collected over 60000 synthetic samples from a modern video game with
similar conditions to the real-world CamVid and Cityscapes datasets. We provide
several experiments to demonstrate that the synthetically generated RGB images
can be used to improve the performance of deep neural networks on both image
segmentation and depth estimation. These results show that a convolutional
network trained on synthetic data achieves a similar test error to a network
that is trained on real-world data for dense image classification. Furthermore,
the synthetically generated RGB images can provide similar or better results
compared to the real-world datasets if a simple domain adaptation technique is
applied. Our results suggest that collaboration with game developers for an
accessible interface to gather data is potentially a fruitful direction for
future work in computer vision.
Multi-way Monte Carlo Method for Linear Systems
Authors: Tao Wu, David F. Gleich
Date: 2016-08-15 00:00:00

We study the Monte Carlo method for solving a linear system of the form $x =
H x + b$. A sufficient condition for the method to work is $\| H \| < 1$, which
greatly limits the usability of this method. We improve this condition by
proposing a new multi-way Markov random walk, which is a generalization of the
standard Markov random walk. Under our new framework we prove that the
necessary and sufficient condition for our method to work is the spectral
radius $\rho(H^{+}) < 1$, which is a weaker requirement than $\| H \| < 1$. In
addition to solving more problems, our new method can work faster than the
standard algorithm. In numerical experiments on both synthetic and real world
matrices, we demonstrate the effectiveness of our new method.
CNN-LTE: a Class of 1-X Pooling Convolutional Neural Networks on Label Tree Embeddings for Audio Scene Recognition
Authors: Huy Phan, Lars Hertel, Marco Maass, Philipp Koch, Alfred Mertins
Date: 2016-07-08 00:00:00
Tasks: Scene Recognition

We describe in this report our audio scene recognition system submitted to
the DCASE 2016 challenge. Firstly, given the label set of the scenes, a label
tree is automatically constructed. This category taxonomy is then used in the
feature extraction step in which an audio scene instance is represented by a
label tree embedding image. Different convolutional neural networks, which are
tailored for the task at hand, are finally learned on top of the image features
for scene recognition. Our system reaches an overall recognition accuracy of
81.2% and 83.3% and outperforms the DCASE 2016 baseline with absolute
improvements of 8.7% and 6.1% on the development and test data, respectively.
CaR-FOREST: Joint Classification-Regression Decision Forests for Overlapping Audio Event Detection
Authors: Huy Phan, Lars Hertel, Marco Maass, Philipp Koch, Alfred Mertins
Date: 2016-07-08 00:00:00
Tasks: Event Detection, General Classification, regression

This report describes our submissions to Task2 and Task3 of the DCASE 2016
challenge. The systems aim at dealing with the detection of overlapping audio
events in continuous streams, where the detectors are based on random decision
forests. The proposed forests are jointly trained for classification and
regression simultaneously. Initially, the training is classification-oriented
to encourage the trees to select discriminative features from overlapping
mixtures to separate positive audio segments from the negative ones. The
regression phase is then carried out to let the positive audio segments vote
for the event onsets and offsets, and therefore model the temporal structure of
audio events. One random decision forest is specifically trained for each event
category of interest. Experimental results on the development data show that
our systems significantly outperform the baseline on the Task2 evaluation while
they are inferior to the baseline in the Task3 evaluation.
Depth2Action: Exploring Embedded Depth for Large-Scale Action Recognition
Authors: Yi Zhu, Shawn Newsam
Date: 2016-08-15 00:00:00
Tasks: Action Recognition, Temporal Action Localization

This paper performs the first investigation into depth for large-scale human
action recognition in video where the depth cues are estimated from the videos
themselves. We develop a new framework called depth2action and experiment
thoroughly into how best to incorporate the depth information. We introduce
spatio-temporal depth normalization (STDN) to enforce temporal consistency in
our estimated depth sequences. We also propose modified depth motion maps
(MDMM) to capture the subtle temporal changes in depth. These two components
significantly improve the action recognition performance. We evaluate our
depth2action framework on three large-scale action recognition video
benchmarks. Our model achieves state-of-the-art performance when combined with
appearance and motion information thus demonstrating that depth2action is
indeed complementary to existing approaches.
Consistency constraints for overlapping data clustering
Authors: Jared Culbertson, Dan P. Guralnik, Jakob Hansen, Peter F. Stiller
Date: 2016-08-15 00:00:00
Tasks: Clustering

We examine overlapping clustering schemes with functorial constraints, in the
spirit of Carlsson--Memoli. This avoids issues arising from the chaining
required by partition-based methods. Our principal result shows that any
clustering functor is naturally constrained to refine single-linkage clusters
and be refined by maximal-linkage clusters. We work in the context of metric
spaces with non-expansive maps, which is appropriate for modeling data
processing which does not increase information content.
On the Online Frank-Wolfe Algorithms for Convex and Non-convex Optimizations
Authors: Jean Lafond, Hoi-To Wai, Eric Moulines
Date: 2015-10-05 00:00:00

In this paper, the online variants of the classical Frank-Wolfe algorithm are
considered. We consider minimizing the regret with a stochastic cost. The
online algorithms only require simple iterative updates and a non-adaptive step
size rule, in contrast to the hybrid schemes commonly considered in the
literature. Several new results are derived for convex and non-convex losses.
With a strongly convex stochastic cost and when the optimal solution lies in
the interior of the constraint set or the constraint set is a polytope, the
regret bound and anytime optimality are shown to be ${\cal O}( \log^3 T / T )$
and ${\cal O}( \log^2 T / T)$, respectively, where $T$ is the number of rounds
played. These results are based on an improved analysis on the stochastic
Frank-Wolfe algorithms. Moreover, the online algorithms are shown to converge
even when the loss is non-convex, i.e., the algorithms find a stationary point
to the time-varying/stochastic loss at a rate of ${\cal O}(\sqrt{1/T})$.
Numerical experiments on realistic data sets are presented to support our
theoretical claims.
Informal Physical Reasoning Processes
Authors: Kurt Ammon
Date: 2016-08-15 00:00:00

A fundamental question is whether Turing machines can model all reasoning
processes. We introduce an existence principle stating that the perception of
the physical existence of any Turing program can serve as a physical causation
for the application of any Turing-computable function to this Turing program.
The existence principle overcomes the limitation of the outputs of Turing
machines to lists, that is, recursively enumerable sets. The principle is
illustrated by productive partial functions for productive sets such as the set
of the Goedel numbers of the Turing-computable total functions. The existence
principle and productive functions imply the existence of physical systems
whose reasoning processes cannot be modeled by Turing machines. These systems
are called creative. Creative systems can prove the undecidable formula in
Goedel's theorem in another formal system which is constructed at a later point
in time. A hypothesis about creative systems, which is based on computer
experiments, is introduced.
Transitive Hashing Network for Heterogeneous Multimedia Retrieval
Authors: Zhangjie Cao, Mingsheng Long, Qiang Yang
Date: 2016-08-15 00:00:00
Tasks: Retrieval

Hashing has been widely applied to large-scale multimedia retrieval due to
the storage and retrieval efficiency. Cross-modal hashing enables efficient
retrieval from database of one modality in response to a query of another
modality. Existing work on cross-modal hashing assumes heterogeneous
relationship across modalities for hash function learning. In this paper, we
relax the strong assumption by only requiring such heterogeneous relationship
in an auxiliary dataset different from the query/database domain. We craft a
hybrid deep architecture to simultaneously learn the cross-modal correlation
from the auxiliary dataset, and align the dataset distributions between the
auxiliary dataset and the query/database domain, which generates transitive
hash codes for heterogeneous multimedia retrieval. Extensive experiments
exhibit that the proposed approach yields state of the art multimedia retrieval
performance on public datasets, i.e. NUS-WIDE, ImageNet-YahooQA.
Robust Volume Minimization-Based Matrix Factorization for Remote Sensing and Document Clustering
Authors: Xiao Fu, Kejun Huang, Bo Yang, Wing-Kin Ma, Nicholas D. Sidiropoulos
Date: 2016-08-15 00:00:00
Tasks: Clustering

This paper considers \emph{volume minimization} (VolMin)-based structured
matrix factorization (SMF). VolMin is a factorization criterion that decomposes
a given data matrix into a basis matrix times a structured coefficient matrix
via finding the minimum-volume simplex that encloses all the columns of the
data matrix. Recent work showed that VolMin guarantees the identifiability of
the factor matrices under mild conditions that are realistic in a wide variety
of applications. This paper focuses on both theoretical and practical aspects
of VolMin. On the theory side, exact equivalence of two independently developed
sufficient conditions for VolMin identifiability is proven here, thereby
providing a more comprehensive understanding of this aspect of VolMin. On the
algorithm side, computational complexity and sensitivity to outliers are two
key challenges associated with real-world applications of VolMin. These are
addressed here via a new VolMin algorithm that handles volume regularization in
a computationally simple way, and automatically detects and {iteratively
downweights} outliers, simultaneously. Simulations and real-data experiments
using a remotely sensed hyperspectral image and the Reuters document corpus are
employed to showcase the effectiveness of the proposed algorithm.
Visual place recognition using landmark distribution descriptors
Authors: Pilailuck Panphattarasap, Andrew Calway
Date: 2016-08-15 00:00:00
Tasks: Visual Place Recognition

Recent work by Suenderhauf et al. [1] demonstrated improved visual place
recognition using proposal regions coupled with features from convolutional
neural networks (CNN) to match landmarks between views. In this work we extend
the approach by introducing descriptors built from landmark features which also
encode the spatial distribution of the landmarks within a view. Matching
descriptors then enforces consistency of the relative positions of landmarks
between views. This has a significant impact on performance. For example, in
experiments on 10 image-pair datasets, each consisting of 200 urban locations
with significant differences in viewing positions and conditions, we recorded
average precision of around 70% (at 100% recall), compared with 58% obtained
using whole image CNN features and 50% for the method in [1].
Towards Analytics Aware Ontology Based Access to Static and Streaming Data (Extended Version)
Authors: Evgeny Kharlamov, Yannis Kotidis, Theofilos Mailis, Christian Neuenstadt, Charalampos Nikolaou, Özgür Özcep, Christoforos Svingos, Dmitriy Zheleznyakov, Sebastian Brandt, Ian Horrocks, Yannis Ioannidis, Steffen Lamparter, Ralf Möller
Date: 2016-07-18 00:00:00

Real-time analytics that requires integration and aggregation of
heterogeneous and distributed streaming and static data is a typical task in
many industrial scenarios such as diagnostics of turbines in Siemens. OBDA
approach has a great potential to facilitate such tasks; however, it has a
number of limitations in dealing with analytics that restrict its use in
important industrial applications. Based on our experience with Siemens, we
argue that in order to overcome those limitations OBDA should be extended and
become analytics, source, and cost aware. In this work we propose such an
extension. In particular, we propose an ontology, mapping, and query language
for OBDA, where aggregate and other analytical functions are first class
citizens. Moreover, we develop query optimisation techniques that allow to
efficiently process analytical tasks over static and streaming data. We
implement our approach in a system and evaluate our system with Siemens turbine
data.
Generating Synthetic Data for Text Recognition
Authors: Praveen Krishnan, C. V. Jawahar
Date: 2016-08-15 00:00:00
Tasks: Data Augmentation, Image Generation

Generating synthetic images is an art which emulates the natural process of
image generation in a closest possible manner. In this work, we exploit such a
framework for data generation in handwritten domain. We render synthetic data
using open source fonts and incorporate data augmentation schemes. As part of
this work, we release 9M synthetic handwritten word image corpus which could be
useful for training deep network architectures and advancing the performance in
handwritten word spotting and recognition tasks.
Natural Language Generation enhances human decision-making with uncertain information
Authors: Dimitra Gkatzia, Oliver Lemon, Verena Rieser
Date: 2016-06-10 00:00:00
Proceeding: ACL 2016 8
Tasks: Decision Making, Decision Making Under Uncertainty, Text Generation

Decision-making is often dependent on uncertain data, e.g. data associated
with confidence scores or probabilities. We present a comparison of different
information presentations for uncertain data and, for the first time, measure
their effects on human decision-making. We show that the use of Natural
Language Generation (NLG) improves decision-making under uncertainty, compared
to state-of-the-art graphical-based representation methods. In a task-based
study with 442 adults, we found that presentations using NLG lead to 24% better
decision-making on average than the graphical presentations, and to 44% better
decision-making when NLG is combined with graphics. We also show that women
achieve significantly better results when presented with NLG output (an 87%
increase on average compared to graphical presentations).