pwc-archive/papers-with-abstracts huggingface dataset:
An Extended Frank-Wolfe Method with "In-Face" Directions, and its Application to Low-Rank Matrix Completion
Authors: Robert M. Freund, Paul Grigas, Rahul Mazumder
Date: 2015-11-06 00:00:00
Tasks: Low-Rank Matrix Completion, Matrix Completion

Motivated principally by the low-rank matrix completion problem, we present
an extension of the Frank-Wolfe method that is designed to induce near-optimal
solutions on low-dimensional faces of the feasible region. This is accomplished
by a new approach to generating ``in-face" directions at each iteration, as
well as through new choice rules for selecting between in-face and ``regular"
Frank-Wolfe steps. Our framework for generating in-face directions generalizes
the notion of away-steps introduced by Wolfe. In particular, the in-face
directions always keep the next iterate within the minimal face containing the
current iterate. We present computational guarantees for the new method that
trade off efficiency in computing near-optimal solutions with upper bounds on
the dimension of minimal faces of iterates. We apply the new method to the
matrix completion problem, where low-dimensional faces correspond to low-rank
matrices. We present computational results that demonstrate the effectiveness
of our methodological approach at producing nearly-optimal solutions of very
low rank. On both artificial and real datasets, we demonstrate significant
speed-ups in computing very low-rank nearly-optimal solutions as compared to
either the Frank-Wolfe method or its traditional away-step variant.
Evaluating Protein-protein Interaction Predictors with a Novel 3-Dimensional Metric
Authors: Haohan Wang, Madhavi K. Ganapathiraju
Date: 2015-11-06 00:00:00

In order for the predicted interactions to be directly adopted by biologists,
the ma- chine learning predictions have to be of high precision, regardless of
recall. This aspect cannot be evaluated or numerically represented well by
traditional metrics like accuracy, ROC, or precision-recall curve. In this
work, we start from the alignment in sensitivity of ROC and recall of
precision-recall curve, and propose an evaluation metric focusing on the
ability of a model to be adopted by biologists. This metric evaluates the
ability of a machine learning algorithm to predict only new interactions,
meanwhile, it eliminates the influence of test dataset. In the experiment of
evaluating different classifiers with a same data set and evaluating the same
predictor with different datasets, our new metric fulfills the evaluation task
of our interest while two widely recognized metrics, ROC and precision-recall
curve fail the tasks for different reasons.
Optimal Non-Asymptotic Lower Bound on the Minimax Regret of Learning with Expert Advice
Authors: Francesco Orabona, David Pal
Date: 2015-11-06 00:00:00

We prove non-asymptotic lower bounds on the expectation of the maximum of $d$
independent Gaussian variables and the expectation of the maximum of $d$
independent symmetric random walks. Both lower bounds recover the optimal
leading constant in the limit. A simple application of the lower bound for
random walks is an (asymptotically optimal) non-asymptotic lower bound on the
minimax regret of online learning with expert advice.
Submodular Hamming Metrics
Authors: Jennifer Gillenwater, Rishabh Iyer, Bethany Lusch, Rahul Kidambi, Jeff Bilmes
Date: 2015-11-06 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Clustering

We show that there is a largely unexplored class of functions (positive
polymatroids) that can define proper discrete metrics over pairs of binary
vectors and that are fairly tractable to optimize over. By exploiting
submodularity, we are able to give hardness results and approximation
algorithms for optimizing over such metrics. Additionally, we demonstrate
empirically the effectiveness of these metrics and associated algorithms on
both a metric minimization task (a form of clustering) and also a metric
maximization task (generating diverse k-best lists).
Pooling the Convolutional Layers in Deep ConvNets for Action Recognition
Authors: Shichao Zhao, Yanbin Liu, Yahong Han, Richang Hong
Date: 2015-11-06 00:00:00
Tasks: Action Recognition, image-classification, Image Classification, Temporal Action Localization

Deep ConvNets have shown its good performance in image classification tasks.
However it still remains as a problem in deep video representation for action
recognition. The problem comes from two aspects: on one hand, current video
ConvNets are relatively shallow compared with image ConvNets, which limits its
capability of capturing the complex video action information; on the other
hand, temporal information of videos is not properly utilized to pool and
encode the video sequences. Towards these issues, in this paper, we utilize two
state-of-the-art ConvNets, i.e., the very deep spatial net (VGGNet) and the
temporal net from Two-Stream ConvNets, for action representation. The
convolutional layers and the proposed new layer, called frame-diff layer, are
extracted and pooled with two temporal pooling strategy: Trajectory pooling and
line pooling. The pooled local descriptors are then encoded with VLAD to form
the video representations. In order to verify the effectiveness of the proposed
framework, we conduct experiments on UCF101 and HMDB51 datasets. It achieves
the accuracy of 93.78\% on UCF101 which is the state-of-the-art and the
accuracy of 65.62\% on HMDB51 which is comparable to the state-of-the-art.
Introducing SKYSET - a Quintuple Approach for Improving Instructions
Authors: Kerry Fultz, Seth Filip
Date: 2015-11-06 00:00:00
Tasks: Translation

A new approach called SKYSET (Synthetic Knowledge Yield Social Entities
Translation) is proposed to validate completeness and to reduce ambiguity from
written instructional documentation. SKYSET utilizes a quintuple set of
standardized categories, which differs from traditional approaches that
typically use triples. The SKYSET System defines the categories required to
form a standard template for representing information that is portable across
different domains. It provides a standardized framework that enables sentences
from written instructions to be translated into sets of category typed entities
on a table or database. The SKYSET entities contain conceptual units or phrases
that represent information from the original source documentation. SKYSET
enables information concatenation where multiple documents from different
domains can be translated and combined into a single common filterable and
searchable table of entities.
Hierarchical Coupled Geometry Analysis for Neuronal Structure and Activity Pattern Discovery
Authors: Gal Mishne, Ronen Talmon, Ron Meir, Jackie Schiller, Uri Dubin, Ronald R. Coifman
Date: 2015-11-06 00:00:00

In the wake of recent advances in experimental methods in neuroscience, the
ability to record in-vivo neuronal activity from awake animals has become
feasible. The availability of such rich and detailed physiological measurements
calls for the development of advanced data analysis tools, as commonly used
techniques do not suffice to capture the spatio-temporal network complexity. In
this paper, we propose a new hierarchical coupled geometry analysis, which
exploits the hidden connectivity structures between neurons and the dynamic
patterns at multiple time-scales. Our approach gives rise to the joint
organization of neurons and dynamic patterns in data-driven hierarchical data
structures. These structures provide local to global data representations, from
local partitioning of the data in flexible trees through a new multiscale
metric to a global manifold embedding. The application of our techniques to
in-vivo neuronal recordings demonstrate the capability of extracting neuronal
activity patterns and identifying temporal trends, associated with particular
behavioral events and manipulations introduced in the experiments.
Texture Synthesis Using Convolutional Neural Networks
Authors: Leon A. Gatys, Alexander S. Ecker, Matthias Bethge
Date: 2015-05-27 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Object, Object Recognition, Texture Synthesis

Here we introduce a new model of natural textures based on the feature spaces
of convolutional neural networks optimised for object recognition. Samples from
the model are of high perceptual quality demonstrating the generative power of
neural networks trained in a purely discriminative fashion. Within the model,
textures are represented by the correlations between feature maps in several
layers of the network. We show that across layers the texture representations
increasingly capture the statistical properties of natural images while making
object information more and more explicit. The model provides a new tool to
generate stimuli for neuroscience and might offer insights into the deep
representations learned by convolutional neural networks.
ALOJA: A Framework for Benchmarking and Predictive Analytics in Big Data Deployments
Authors: Josep Ll. Berral, Nicolas Poggi, David Carrera, Aaron Call, Rob Reinauer, Daron Green
Date: 2015-11-06 00:00:00
Tasks: Anomaly Detection, Benchmarking

This article presents the ALOJA project and its analytics tools, which
leverages machine learning to interpret Big Data benchmark performance data and
tuning. ALOJA is part of a long-term collaboration between BSC and Microsoft to
automate the characterization of cost-effectiveness on Big Data deployments,
currently focusing on Hadoop. Hadoop presents a complex run-time environment,
where costs and performance depend on a large number of configuration choices.
The ALOJA project has created an open, vendor-neutral repository, featuring
over 40,000 Hadoop job executions and their performance details. The repository
is accompanied by a test-bed and tools to deploy and evaluate the
cost-effectiveness of different hardware configurations, parameters and Cloud
services. Despite early success within ALOJA, a comprehensive study requires
automation of modeling procedures to allow an analysis of large and
resource-constrained search spaces. The predictive analytics extension,
ALOJA-ML, provides an automated system allowing knowledge discovery by modeling
environments from observed executions. The resulting models can forecast
execution behaviors, predicting execution times for new configurations and
hardware choices. That also enables model-based anomaly detection or efficient
benchmark guidance by prioritizing executions. In addition, the community can
benefit from ALOJA data-sets and framework to improve the design and deployment
of Big Data applications.
ALOJA-ML: A Framework for Automating Characterization and Knowledge Discovery in Hadoop Deployments
Authors: Josep Ll. Berral, Nicolas Poggi, David Carrera, Aaron Call, Rob Reinauer, Daron Green
Date: 2015-11-06 00:00:00
Tasks: Anomaly Detection, Benchmarking, BIG-bench Machine Learning

This article presents ALOJA-Machine Learning (ALOJA-ML) an extension to the
ALOJA project that uses machine learning techniques to interpret Hadoop
benchmark performance data and performance tuning; here we detail the approach,
efficacy of the model and initial results. Hadoop presents a complex execution
environment, where costs and performance depends on a large number of software
(SW) configurations and on multiple hardware (HW) deployment choices. These
results are accompanied by a test bed and tools to deploy and evaluate the
cost-effectiveness of the different hardware configurations, parameter tunings,
and Cloud services. Despite early success within ALOJA from expert-guided
benchmarking, it became clear that a genuinely comprehensive study requires
automation of modeling procedures to allow a systematic analysis of large and
resource-constrained search spaces. ALOJA-ML provides such an automated system
allowing knowledge discovery by modeling Hadoop executions from observed
benchmarks across a broad set of configuration parameters. The resulting
performance models can be used to forecast execution behavior of various
workloads; they allow 'a-priori' prediction of the execution times for new
configurations and HW choices and they offer a route to model-based anomaly
detection. In addition, these models can guide the benchmarking exploration
efficiently, by automatically prioritizing candidate future benchmark tests.
Insights from ALOJA-ML's models can be used to reduce the operational time on
clusters, speed-up the data acquisition and knowledge discovery process, and
importantly, reduce running costs. In addition to learning from the methodology
presented in this work, the community can benefit in general from ALOJA
data-sets, framework, and derived insights to improve the design and deployment
of Big Data applications.
Towards a Better Understanding of Predict and Count Models
Authors: S. Sathiya Keerthi, Tobias Schnabel, Rajiv Khanna
Date: 2015-11-06 00:00:00
Tasks: L2 Regularization

In a recent paper, Levy and Goldberg pointed out an interesting connection
between prediction-based word embedding models and count models based on
pointwise mutual information. Under certain conditions, they showed that both
models end up optimizing equivalent objective functions. This paper explores
this connection in more detail and lays out the factors leading to differences
between these models. We find that the most relevant differences from an
optimization perspective are (i) predict models work in a low dimensional space
where embedding vectors can interact heavily; (ii) since predict models have
fewer parameters, they are less prone to overfitting.
  Motivated by the insight of our analysis, we show how count models can be
regularized in a principled manner and provide closed-form solutions for L1 and
L2 regularization. Finally, we propose a new embedding model with a convex
objective and the additional benefit of being intelligible.
Facial Expression Recognition Using Sparse Gaussian Conditional Random Field
Authors: Mohammadamin Abbasnejad, Mohammad Ali Masnadi-Shirazi
Date: 2015-11-06 00:00:00
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER)

The analysis of expression and facial Action Units (AUs) detection are very
important tasks in fields of computer vision and Human Computer Interaction
(HCI) due to the wide range of applications in human life. Many works has been
done during the past few years which has their own advantages and
disadvantages. In this work we present a new model based on Gaussian
Conditional Random Field. We solve our objective problem using ADMM and we show
how well the proposed model works. We train and test our work on two facial
expression datasets, CK+ and RU-FACS. Experimental evaluation shows that our
proposed approach outperform state of the art expression recognition.
Next Generation Multicuts for Semi-Planar Graphs
Authors: Julian Yarkony
Date: 2015-11-06 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

We study the problem of multicut segmentation. We introduce modified versions
of the Semi-PlanarCC based on bounding Lagrange multipliers. We apply our work
to natural image segmentation.
Neutralized Empirical Risk Minimization with Generalization Neutrality Bound
Authors: Kazuto Fukuchi, Jun Sakuma
Date: 2015-11-06 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, General Classification

Currently, machine learning plays an important role in the lives and
individual activities of numerous people. Accordingly, it has become necessary
to design machine learning algorithms to ensure that discrimination, biased
views, or unfair treatment do not result from decision making or predictions
made via machine learning. In this work, we introduce a novel empirical risk
minimization (ERM) framework for supervised learning, neutralized ERM (NERM)
that ensures that any classifiers obtained can be guaranteed to be neutral with
respect to a viewpoint hypothesis. More specifically, given a viewpoint
hypothesis, NERM works to find a target hypothesis that minimizes the empirical
risk while simultaneously identifying a target hypothesis that is neutral to
the viewpoint hypothesis. Within the NERM framework, we derive a theoretical
bound on empirical and generalization neutrality risks. Furthermore, as a
realization of NERM with linear classification, we derive a max-margin
algorithm, neutral support vector machine (SVM). Experimental results show that
our neutral SVM shows improved classification performance in real datasets
without sacrificing the neutrality guarantee.
Multi-lingual Geoparsing based on Machine Translation
Authors: Xu Chen, Han Zhang, Judith Gelernter
Date: 2015-11-06 00:00:00
Tasks: Machine Translation, Translation

Our method for multi-lingual geoparsing uses monolingual tools and resources
along with machine translation and alignment to return location words in many
languages. Not only does our method save the time and cost of developing
geoparsers for each language separately, but also it allows the possibility of
a wide range of language capabilities within a single interface. We evaluated
our method in our LanguageBridge prototype on location named entities using
newswire, broadcast news and telephone conversations in English, Arabic and
Chinese data from the Linguistic Data Consortium (LDC). Our results for
geoparsing Chinese and Arabic text using our multi-lingual geoparsing method
are comparable to our results for geoparsing English text with our English
tools. Furthermore, experiments using our machine translation approach results
in accuracy comparable to results from the same data that was translated
manually.
Combinatorial Bandits Revisited
Authors: Richard Combes, M. Sadegh Talebi, Alexandre Proutiere, Marc Lelarge
Date: 2015-02-11 00:00:00
Proceeding: NeurIPS 2015 12

This paper investigates stochastic and adversarial combinatorial multi-armed
bandit problems. In the stochastic setting under semi-bandit feedback, we
derive a problem-specific regret lower bound, and discuss its scaling with the
dimension of the decision space. We propose ESCB, an algorithm that efficiently
exploits the structure of the problem and provide a finite-time analysis of its
regret. ESCB has better performance guarantees than existing algorithms, and
significantly outperforms these algorithms in practice. In the adversarial
setting under bandit feedback, we propose \textsc{CombEXP}, an algorithm with
the same regret scaling as state-of-the-art algorithms, but with lower
computational complexity for some combinatorial problems.
An Action Language for Multi-Agent Domains: Foundations
Authors: Chitta Baral, Gregory Gelfond, Enrico Pontelli, Tran Cao Son
Date: 2015-11-06 00:00:00

In multi-agent domains (MADs), an agent's action may not just change the world and the agent's knowledge and beliefs about the world, but also may change other agents' knowledge and beliefs about the world and their knowledge and beliefs about other agents' knowledge and beliefs about the world. The goals of an agent in a multi-agent world may involve manipulating the knowledge and beliefs of other agents' and again, not just their knowledge/belief about the world, but also their knowledge about other agents' knowledge about the world. Our goal is to present an action language (mA+) that has the necessary features to address the above aspects in representing and RAC in MADs. mA+ allows the representation of and reasoning about different types of actions that an agent can perform in a domain where many other agents might be present -- such as world-altering actions, sensing actions, and announcement/communication actions. It also allows the specification of agents' dynamic awareness of action occurrences which has future implications on what agents' know about the world and other agents' knowledge about the world. mA+ considers three different types of awareness: full-, partial- awareness, and complete oblivion of an action occurrence and its effects. This keeps the language simple, yet powerful enough to address a large variety of knowledge manipulation scenarios in MADs. The semantics of mA+ relies on the notion of state, which is described by a pointed Kripke model and is used to encode the agent's knowledge and the real state of the world. It is defined by a transition function that maps pairs of actions and states into sets of states. We illustrate properties of the action theories, including properties that guarantee finiteness of the set of initial states and their practical implementability. Finally, we relate mA+ to other related formalisms that contribute to RAC in MADs.
Stop Wasting My Gradients: Practical SVRG
Authors: Reza Babanezhad, Mohamed Osama Ahmed, Alim Virani, Mark Schmidt, Jakub Konečný, Scott Sallinen
Date: 2015-11-05 00:00:00

We present and analyze several strategies for improving the performance of
stochastic variance-reduced gradient (SVRG) methods. We first show that the
convergence rate of these methods can be preserved under a decreasing sequence
of errors in the control variate, and use this to derive variants of SVRG that
use growing-batch strategies to reduce the number of gradient calculations
required in the early iterations. We further (i) show how to exploit support
vectors to reduce the number of gradient computations in the later iterations,
(ii) prove that the commonly-used regularized SVRG iteration is justified and
improves the convergence rate, (iii) consider alternate mini-batch selection
strategies, and (iv) consider the generalization error of the method.
Rethinking LDA: moment matching for discrete ICA
Authors: Anastasia Podosinnikova, Francis Bach, Simon Lacoste-Julien
Date: 2015-07-07 00:00:00
Proceeding: NeurIPS 2015 12

We consider moment matching techniques for estimation in Latent Dirichlet
Allocation (LDA). By drawing explicit links between LDA and discrete versions
of independent component analysis (ICA), we first derive a new set of
cumulant-based tensors, with an improved sample complexity. Moreover, we reuse
standard ICA techniques such as joint diagonalization of tensors to improve
over existing methods based on the tensor power method. In an extensive set of
experiments on both synthetic and real datasets, we show that our new
combination of tensors and orthogonal joint diagonalization techniques
outperforms existing moment matching methods.
Thoughts on Massively Scalable Gaussian Processes
Authors: Andrew Gordon Wilson, Christoph Dann, Hannes Nickisch
Date: 2015-11-05 00:00:00
Tasks: Gaussian Processes

We introduce a framework and early results for massively scalable Gaussian
processes (MSGP), significantly extending the KISS-GP approach of Wilson and
Nickisch (2015). The MSGP framework enables the use of Gaussian processes (GPs)
on billions of datapoints, without requiring distributed inference, or severe
assumptions. In particular, MSGP reduces the standard $O(n^3)$ complexity of GP
learning and inference to $O(n)$, and the standard $O(n^2)$ complexity per test
point prediction to $O(1)$. MSGP involves 1) decomposing covariance matrices as
Kronecker products of Toeplitz matrices approximated by circulant matrices.
This multi-level circulant approximation allows one to unify the orthogonal
computational benefits of fast Kronecker and Toeplitz approaches, and is
significantly faster than either approach in isolation; 2) local kernel
interpolation and inducing points to allow for arbitrarily located data inputs,
and $O(1)$ test time predictions; 3) exploiting block-Toeplitz Toeplitz-block
structure (BTTB), which enables fast inference and learning when
multidimensional Kronecker structure is not present; and 4) projections of the
input space to flexibly model correlated inputs and high dimensional data. The
ability to handle many ($m \approx n$) inducing points allows for near-exact
accuracy and large scale kernel learning.