pwc-archive/papers-with-abstracts huggingface dataset:
C-mix: a high dimensional mixture model for censored durations, with applications to genetic data
Authors: Simon Bussy, Agathe Guilloux, Stéphane Gaïffas, Anne-Sophie Jannot
Date: 2016-10-24 00:00:00

We introduce a mixture model for censored durations (C-mix), and develop
maximum likelihood inference for the joint estimation of the time distributions
and latent regression parameters of the model. We consider a high-dimensional
setting, with datasets containing a large number of biomedical covariates. We
therefore penalize the negative log-likelihood by the Elastic-Net, which leads
to a sparse parameterization of the model. Inference is achieved using an
efficient Quasi-Newton Expectation Maximization (QNEM) algorithm, for which we
provide convergence properties. We then propose a score by assessing the
patients risk of early adverse event. The statistical performance of the method
is examined on an extensive Monte Carlo simulation study, and finally
illustrated on three genetic datasets with high-dimensional covariates. We show
that our approach outperforms the state-of-the-art, namely both the CURE and
Cox proportional hazards models for this task, both in terms of C-index and
AUC(t).
The model of an anomaly detector for HiLumi LHC magnets based on Recurrent Neural Networks and adaptive quantization
Authors: Maciej Wielgosz, Matej Mertik, Andrzej Skoczeń, Ernesto De Matteis
Date: 2017-09-28 00:00:00
Tasks: Anomaly Detection, Quantization

This paper focuses on an examination of an applicability of Recurrent Neural
Network models for detecting anomalous behavior of the CERN superconducting
magnets. In order to conduct the experiments, the authors designed and
implemented an adaptive signal quantization algorithm and a custom GRU-based
detector and developed a method for the detector parameters selection. Three
different datasets were used for testing the detector. Two artificially
generated datasets were used to assess the raw performance of the system
whereas the 231 MB dataset composed of the signals acquired from HiLumi magnets
was intended for real-life experiments and model training. Several different
setups of the developed anomaly detection system were evaluated and compared
with state-of-the-art OC-SVM reference model operating on the same data. The
OC-SVM model was equipped with a rich set of feature extractors accounting for
a range of the input signal properties. It was determined in the course of the
experiments that the detector, along with its supporting design methodology,
reaches F1 equal or very close to 1 for almost all test sets. Due to the
profile of the data, the best_length setup of the detector turned out to
perform the best among all five tested configuration schemes of the detection
system. The quantization parameters have the biggest impact on the overall
performance of the detector with the best values of input/output grid equal to
16 and 8, respectively. The proposed solution of the detection significantly
outperformed OC-SVM-based detector in most of the cases, with much more stable
performance across all the datasets.
Expectation maximization transfer learning and its application for bionic hand prostheses
Authors: Benjamin Paaßen, Alexander Schulz, Janne Hahne, Barbara Hammer
Date: 2017-11-25 00:00:00
Tasks: BIG-bench Machine Learning, Quantization, Transfer Learning

Machine learning models in practical settings are typically confronted with
changes to the distribution of the incoming data. Such changes can severely
affect the model performance, leading for example to misclassifications of
data. This is particularly apparent in the domain of bionic hand prostheses,
where machine learning models promise faster and more intuitive user
interfaces, but are hindered by their lack of robustness to everyday
disturbances, such as electrode shifts. One way to address changes in the data
distribution is transfer learning, that is, to transfer the disturbed data to a
space where the original model is applicable again. In this contribution, we
propose a novel expectation maximization algorithm to learn linear
transformations that maximize the likelihood of disturbed data after the
transformation. We also show that this approach generalizes to discriminative
models, in particular learning vector quantization models. In our evaluation on
data from the bionic prostheses domain we demonstrate that our approach can
learn a transformation which improves classification accuracy significantly and
outperforms all tested baselines, if few data or few classes are available in
the target domain.
Binarsity: a penalization for one-hot encoded features in linear supervised learning
Authors: Mokhtar Z. Alaya, Simon Bussy, Stéphane Gaïffas, Agathe Guilloux
Date: 2017-03-24 00:00:00

This paper deals with the problem of large-scale linear supervised learning
in settings where a large number of continuous features are available. We
propose to combine the well-known trick of one-hot encoding of continuous
features with a new penalization called \emph{binarsity}. In each group of
binary features coming from the one-hot encoding of a single raw continuous
feature, this penalization uses total-variation regularization together with an
extra linear constraint. This induces two interesting properties on the model
weights of the one-hot encoded features: they are piecewise constant, and are
eventually block sparse. Non-asymptotic oracle inequalities for generalized
linear models are proposed. Moreover, under a sparse additive model assumption,
we prove that our procedure matches the state-of-the-art in this setting.
Numerical experiments illustrate the good performances of our approach on
several datasets. It is also noteworthy that our method has a numerical
complexity comparable to standard $\ell_1$ penalization.
Learning 3D Human Pose from Structure and Motion
Authors: Rishabh Dabral, Anurag Mundhada, Uday Kusupati, Safeer Afaque, Abhishek Sharma, Arjun Jain
Date: 2017-11-25 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Human Pose Estimation, Monocular 3D Human Pose Estimation, Pose Estimation, Weakly-supervised Learning

3D human pose estimation from a single image is a challenging problem,
especially for in-the-wild settings due to the lack of 3D annotated data. We
propose two anatomically inspired loss functions and use them with a
weakly-supervised learning framework to jointly learn from large-scale
in-the-wild 2D and indoor/synthetic 3D data. We also present a simple temporal
network that exploits temporal and structural cues present in predicted pose
sequences to temporally harmonize the pose estimations. We carefully analyze
the proposed contributions through loss surface visualizations and sensitivity
analysis to facilitate deeper understanding of their working mechanism. Our
complete pipeline improves the state-of-the-art by 11.8% and 12% on Human3.6M
and MPI-INF-3DHP, respectively, and runs at 30 FPS on a commodity graphics
card.
On the Relations of Correlation Filter Based Trackers and Struck
Authors: Jinqiao Wang, Ming Tang, Linyu Zheng, Jiayi Feng
Date: 2017-11-25 00:00:00
Tasks: Relation

In recent years, two types of trackers, namely correlation filter based
tracker (CF tracker) and structured output tracker (Struck), have exhibited the
state-of-the-art performance. However, there seems to be lack of analytic work
on their relations in the computer vision community. In this paper, we
investigate two state-of-the-art CF trackers, i.e., spatial regularization
discriminative correlation filter (SRDCF) and correlation filter with limited
boundaries (CFLB), and Struck, and reveal their relations. Specifically, after
extending the CFLB to its multiple channel version we prove the relation
between SRDCF and CFLB on the condition that the spatial regularization factor
of SRDCF is replaced by the masking matrix of CFLB. We also prove the
asymptotical approximate relation between SRDCF and Struck on the conditions
that the spatial regularization factor of SRDCF is replaced by an indicator
function of object bounding box, the weights of SRDCF in its loss item are
replaced by those of Struck, the linear kernel is employed by Struck, and the
search region tends to infinity. Extensive experiments on public benchmarks
OTB50 and OTB100 are conducted to verify our theoretical results. Moreover, we
explain how detailed differences among SRDCF, CFLB, and Struck would give rise
to slightly different performances on visual sequences
BENCHIP: Benchmarking Intelligence Processors
Authors: Jinhua Tao, Zidong Du, Qi Guo, Huiying Lan, Lei Zhang, Shengyuan Zhou, Lingjie Xu, Cong Liu, Haifeng Liu, Shan Tang, Allen Rush, Willian Chen, Shaoli Liu, Yunji Chen, Tianshi Chen
Date: 2017-10-23 00:00:00
Tasks: Benchmarking, Diversity

The increasing attention on deep learning has tremendously spurred the design
of intelligence processing hardware. The variety of emerging intelligence
processors requires standard benchmarks for fair comparison and system
optimization (in both software and hardware). However, existing benchmarks are
unsuitable for benchmarking intelligence processors due to their non-diversity
and nonrepresentativeness. Also, the lack of a standard benchmarking
methodology further exacerbates this problem. In this paper, we propose
BENCHIP, a benchmark suite and benchmarking methodology for intelligence
processors. The benchmark suite in BENCHIP consists of two sets of benchmarks:
microbenchmarks and macrobenchmarks. The microbenchmarks consist of
single-layer networks. They are mainly designed for bottleneck analysis and
system optimization. The macrobenchmarks contain state-of-the-art industrial
networks, so as to offer a realistic comparison of different platforms. We also
propose a standard benchmarking methodology built upon an industrial software
stack and evaluation metrics that comprehensively reflect the various
characteristics of the evaluated intelligence processors. BENCHIP is utilized
for evaluating various hardware platforms, including CPUs, GPUs, and
accelerators. BENCHIP will be open-sourced soon.
Malaria Likelihood Prediction By Effectively Surveying Households Using Deep Reinforcement Learning
Authors: Pranav Rajpurkar, Vinaya Polamreddi, Anusha Balakrishnan
Date: 2017-11-25 00:00:00
Tasks: Deep Reinforcement Learning, Holdout Set, Prediction, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Survey

We build a deep reinforcement learning (RL) agent that can predict the
likelihood of an individual testing positive for malaria by asking questions
about their household. The RL agent learns to determine which survey question
to ask next and when to stop to make a prediction about their likelihood of
malaria based on their responses hitherto. The agent incurs a small penalty for
each question asked, and a large reward/penalty for making the correct/wrong
prediction; it thus has to learn to balance the length of the survey with the
accuracy of its final predictions. Our RL agent is a Deep Q-network that learns
a policy directly from the responses to the questions, with an action defined
for each possible survey question and for each possible prediction class. We
focus on Kenya, where malaria is a massive health burden, and train the RL
agent on a dataset of 6481 households from the Kenya Malaria Indicator Survey
2015. To investigate the importance of having survey questions be adaptive to
responses, we compare our RL agent to a supervised learning (SL) baseline that
fixes its set of survey questions a priori. We evaluate on prediction accuracy
and on the number of survey questions asked on a holdout set and find that the
RL agent is able to predict with 80% accuracy, using only 2.5 questions on
average. In addition, the RL agent learns to survey adaptively to responses and
is able to match the SL baseline in prediction accuracy while significantly
reducing survey length.
The Incredible Shrinking Neural Network: New Perspectives on Learning Representations Through The Lens of Pruning
Authors: Aditya Sharma, Nikolas Wolfe, Bhiksha Raj
Date: 2017-01-16 00:00:00
Tasks: Network Pruning

How much can pruning algorithms teach us about the fundamentals of learning
representations in neural networks? And how much can these fundamentals help
while devising new pruning techniques? A lot, it turns out. Neural network
pruning has become a topic of great interest in recent years, and many
different techniques have been proposed to address this problem. The decision
of what to prune and when to prune necessarily forces us to confront our
assumptions about how neural networks actually learn to represent patterns in
data. In this work, we set out to test several long-held hypotheses about
neural network learning representations, approaches to pruning and the
relevance of one in the context of the other. To accomplish this, we argue in
favor of pruning whole neurons as opposed to the traditional method of pruning
weights from optimally trained networks. We first review the historical
literature, point out some common assumptions it makes, and propose methods to
demonstrate the inherent flaws in these assumptions. We then propose our novel
approach to pruning and set about analyzing the quality of the decisions it
makes. Our analysis led us to question the validity of many widely-held
assumptions behind pruning algorithms and the trade-offs we often make in the
interest of reducing computational complexity. We discovered that there is a
straightforward way, however expensive, to serially prune 40-70% of the neurons
in a trained network with minimal effect on the learning representation and
without any re-training. It is to be noted here that the motivation behind this
work is not to propose an algorithm that would outperform all existing methods,
but to shed light on what some inherent flaws in these methods can teach us
about learning representations and how this can lead us to superior pruning
techniques.
Stacked Kernel Network
Authors: Shuai Zhang, Jian-Xin Li, Pengtao Xie, Yingchun Zhang, Minglai Shao, Haoyi Zhou, Mengyi Yan
Date: 2017-11-25 00:00:00

Kernel methods are powerful tools to capture nonlinear patterns behind data.
They implicitly learn high (even infinite) dimensional nonlinear features in
the Reproducing Kernel Hilbert Space (RKHS) while making the computation
tractable by leveraging the kernel trick. Classic kernel methods learn a single
layer of nonlinear features, whose representational power may be limited.
Motivated by recent success of deep neural networks (DNNs) that learn
multi-layer hierarchical representations, we propose a Stacked Kernel Network
(SKN) that learns a hierarchy of RKHS-based nonlinear features. SKN interleaves
several layers of nonlinear transformations (from a linear space to a RKHS) and
linear transformations (from a RKHS to a linear space). Similar to DNNs, a SKN
is composed of multiple layers of hidden units, but each parameterized by a
RKHS function rather than a finite-dimensional vector. We propose three ways to
represent the RKHS functions in SKN: (1)nonparametric representation,
(2)parametric representation and (3)random Fourier feature representation.
Furthermore, we expand SKN into CNN architecture called Stacked Kernel
Convolutional Network (SKCN). SKCN learning a hierarchy of RKHS-based nonlinear
features by convolutional operation with each filter also parameterized by a
RKHS function rather than a finite-dimensional matrix in CNN, which is suitable
for image inputs. Experiments on various datasets demonstrate the effectiveness
of SKN and SKCN, which outperform the competitive methods.
Complex Structure Leads to Overfitting: A Structure Regularization Decoding Method for Natural Language Processing
Authors: Xu Sun, Weiwei Sun, Shuming Ma, Xuancheng Ren, Yi Zhang, Wenjie Li, Houfeng Wang
Date: 2017-11-25 00:00:00
Tasks: Structured Prediction

Recent systems on structured prediction focus on increasing the level of
structural dependencies within the model. However, our study suggests that
complex structures entail high overfitting risks. To control the
structure-based overfitting, we propose to conduct structure regularization
decoding (SR decoding). The decoding of the complex structure model is
regularized by the additionally trained simple structure model. We
theoretically analyze the quantitative relations between the structural
complexity and the overfitting risk. The analysis shows that complex structure
models are prone to the structure-based overfitting. Empirical evaluations show
that the proposed method improves the performance of the complex structure
models by reducing the structure-based overfitting. On the sequence labeling
tasks, the proposed method substantially improves the performance of the
complex neural network models. The maximum F1 error rate reduction is 36.4% for
the third-order model. The proposed method also works for the parsing task. The
maximum UAS improvement is 5.5% for the tri-sibling model. The results are
competitive with or better than the state-of-the-art results.
Feature Selection Facilitates Learning Mixtures of Discrete Product Distributions
Authors: Vincent Zhao, Steven W. Zucker
Date: 2017-11-25 00:00:00
Tasks: feature selection

Feature selection can facilitate the learning of mixtures of discrete random
variables as they arise, e.g. in crowdsourcing tasks. Intuitively, not all
workers are equally reliable but, if the less reliable ones could be
eliminated, then learning should be more robust. By analogy with Gaussian
mixture models, we seek a low-order statistical approach, and here introduce an
algorithm based on the (pairwise) mutual information. This induces an order
over workers that is well structured for the `one coin' model. More generally,
it is justified by a goodness-of-fit measure and is validated empirically.
Improvement in real data sets can be substantial.
Multiple Instance Curriculum Learning for Weakly Supervised Object Detection
Authors: Siyang Li, Xiangxin Zhu, Qin Huang, Hao Xu, C. -C. Jay Kuo
Date: 2017-11-25 00:00:00
Tasks: Multiple Instance Learning, Object, object-detection, Object Detection, Segmentation, Semantic Segmentation, Weakly Supervised Object Detection

When supervising an object detector with weakly labeled data, most existing
approaches are prone to trapping in the discriminative object parts, e.g.,
finding the face of a cat instead of the full body, due to lacking the
supervision on the extent of full objects. To address this challenge, we
incorporate object segmentation into the detector training, which guides the
model to correctly localize the full objects. We propose the multiple instance
curriculum learning (MICL) method, which injects curriculum learning (CL) into
the multiple instance learning (MIL) framework. The MICL method starts by
automatically picking the easy training examples, where the extent of the
segmentation masks agree with detection bounding boxes. The training set is
gradually expanded to include harder examples to train strong detectors that
handle complex images. The proposed MICL method with segmentation in the loop
outperforms the state-of-the-art weakly supervised object detectors by a
substantial margin on the PASCAL VOC datasets.
D numbers theory based game-theoretic framework in adversarial decision making under fuzzy environment
Authors: Xinyang Deng, Wen Jiang
Date: 2017-11-25 00:00:00
Tasks: Decision Making, Skeleton Based Action Recognition

Adversarial decision making is a particular type of decision making problem
where the gain a decision maker obtains as a result of his decisions is
affected by the actions taken by others. Representation of alternatives'
evaluations and methods to find the optimal alternative are two important
aspects in the adversarial decision making. The aim of this study is to develop
a general framework for solving the adversarial decision making issue under
uncertain environment. By combining fuzzy set theory, game theory and D numbers
theory (DNT), a DNT based game-theoretic framework for adversarial decision
making under fuzzy environment is presented. Within the proposed framework or
model, fuzzy set theory is used to model the uncertain evaluations of decision
makers to alternatives, the non-exclusiveness among fuzzy evaluations are taken
into consideration by using DNT, and the conflict of interests among decision
makers is considered in a two-person non-constant sum game theory perspective.
An illustrative application is given to demonstrate the effectiveness of the
proposed model. This work, on one hand, has developed an effective framework
for adversarial decision making under fuzzy environment; One the other hand, it
has further improved the basis of DNT as a generalization of Dempster-Shafer
theory for uncertainty reasoning.
A Unified Approach to Interpreting Model Predictions
Authors: Scott Lundberg, Su-In Lee
Date: 2017-05-22 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Feature Importance, Image Attribution, Interpretability Techniques for Deep Learning, Interpretable Machine Learning, model

Understanding why a model makes a certain prediction can be as crucial as the
prediction's accuracy in many applications. However, the highest accuracy for
large modern datasets is often achieved by complex models that even experts
struggle to interpret, such as ensemble or deep learning models, creating a
tension between accuracy and interpretability. In response, various methods
have recently been proposed to help users interpret the predictions of complex
models, but it is often unclear how these methods are related and when one
method is preferable over another. To address this problem, we present a
unified framework for interpreting predictions, SHAP (SHapley Additive
exPlanations). SHAP assigns each feature an importance value for a particular
prediction. Its novel components include: (1) the identification of a new class
of additive feature importance measures, and (2) theoretical results showing
there is a unique solution in this class with a set of desirable properties.
The new class unifies six existing methods, notable because several recent
methods in the class lack the proposed desirable properties. Based on insights
from this unification, we present new methods that show improved computational
performance and/or better consistency with human intuition than previous
approaches.
Selling to a No-Regret Buyer
Authors: Mark Braverman, Jieming Mao, Jon Schneider, S. Matthew Weinberg
Date: 2017-11-25 00:00:00

We consider the problem of a single seller repeatedly selling a single item
to a single buyer (specifically, the buyer has a value drawn fresh from known
distribution $D$ in every round). Prior work assumes that the buyer is fully
rational and will perfectly reason about how their bids today affect the
seller's decisions tomorrow. In this work we initiate a different direction:
the buyer simply runs a no-regret learning algorithm over possible bids. We
provide a fairly complete characterization of optimal auctions for the seller
in this domain. Specifically:
  - If the buyer bids according to EXP3 (or any "mean-based" learning
algorithm), then the seller can extract expected revenue arbitrarily close to
the expected welfare. This auction is independent of the buyer's valuation $D$,
but somewhat unnatural as it is sometimes in the buyer's interest to overbid. -
There exists a learning algorithm $\mathcal{A}$ such that if the buyer bids
according to $\mathcal{A}$ then the optimal strategy for the seller is simply
to post the Myerson reserve for $D$ every round. - If the buyer bids according
to EXP3 (or any "mean-based" learning algorithm), but the seller is restricted
to "natural" auction formats where overbidding is dominated (e.g. Generalized
First-Price or Generalized Second-Price), then the optimal strategy for the
seller is a pay-your-bid format with decreasing reserves over time. Moreover,
the seller's optimal achievable revenue is characterized by a linear program,
and can be unboundedly better than the best truthful auction yet simultaneously
unboundedly worse than the expected welfare.