pwc-archive/papers-with-abstracts huggingface dataset:
Feature importance scores and lossless feature pruning using Banzhaf power indices
Authors: Bogdan Kulynych, Carmela Troncoso
Date: 2017-11-14 00:00:00
Tasks: Feature Importance, General Classification

Understanding the influence of features in machine learning is crucial to
interpreting models and selecting the best features for classification. In this
work we propose the use of principles from coalitional game theory to reason
about importance of features. In particular, we propose the use of the Banzhaf
power index as a measure of influence of features on the outcome of a
classifier. We show that features having Banzhaf power index of zero can be
losslessly pruned without damage to classifier accuracy. Computing the power
indices does not require having access to data samples. However, if samples are
available, the indices can be empirically estimated. We compute Banzhaf power
indices for a neural network classifier on real-life data, and compare the
results with gradient-based feature saliency, and coefficients of a logistic
regression model with $L_1$ regularization.
Simulated Annealing Algorithm for Graph Coloring
Authors: Alper Kose, Berke Aral Sonmez, Metin Balaban
Date: 2017-12-03 00:00:00

The goal of this Random Walks project is to code and experiment the Markov
Chain Monte Carlo (MCMC) method for the problem of graph coloring. In this
report, we present the plots of cost function \(\mathbf{H}\) by varying the
parameters like \(\mathbf{q}\) (Number of colors that can be used in coloring)
and \(\mathbf{c}\) (Average node degree). The results are obtained by using
simulated annealing scheme, where the temperature (inverse of
\(\mathbf{\beta}\)) parameter in the MCMC is lowered progressively.
TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering
Authors: Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, Gunhee Kim
Date: 2017-04-14 00:00:00
Proceeding: CVPR 2017 7
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA), Zero-Shot Video Question Answer

Vision and language understanding has emerged as a subject undergoing intense
study in Artificial Intelligence. Among many tasks in this line of research,
visual question answering (VQA) has been one of the most successful ones, where
the goal is to learn a model that understands visual content at region-level
details and finds their associations with pairs of questions and answers in the
natural language form. Despite the rapid progress in the past few years, most
existing work in VQA have focused primarily on images. In this paper, we focus
on extending VQA to the video domain and contribute to the literature in three
important ways. First, we propose three new tasks designed specifically for
video VQA, which require spatio-temporal reasoning from videos to answer
questions correctly. Next, we introduce a new large-scale dataset for video VQA
named TGIF-QA that extends existing VQA work with our new tasks. Finally, we
propose a dual-LSTM based approach with both spatial and temporal attention,
and show its effectiveness over conventional VQA techniques through empirical
evaluations.
Structured Deep Hashing with Convolutional Neural Networks for Fast Person Re-identification
Authors: Lin Wu, Yang Wang
Date: 2017-02-14 00:00:00
Tasks: Deep Hashing, Person Re-Identification

Given a pedestrian image as a query, the purpose of person re-identification
is to identify the correct match from a large collection of gallery images
depicting the same person captured by disjoint camera views. The critical
challenge is how to construct a robust yet discriminative feature
representation to capture the compounded variations in pedestrian appearance.
To this end, deep learning methods have been proposed to extract hierarchical
features against extreme variability of appearance. However, existing methods
in this category generally neglect the efficiency in the matching stage whereas
the searching speed of a re-identification system is crucial in real-world
applications. In this paper, we present a novel deep hashing framework with
Convolutional Neural Networks (CNNs) for fast person re-identification.
Technically, we simultaneously learn both CNN features and hash functions/codes
to get robust yet discriminative features and similarity-preserving hash codes.
Thereby, person re-identification can be resolved by efficiently computing and
ranking the Hamming distances between images. A structured loss function
defined over positive pairs and hard negatives is proposed to formulate a novel
optimization problem so that fast convergence and more stable optimized
solution can be obtained. Extensive experiments on two benchmarks CUHK03
\cite{FPNN} and Market-1501 \cite{Market1501} show that the proposed deep
architecture is efficacy over state-of-the-arts.
Machine learning methods for histopathological image analysis
Authors: Daisuke Komura, Shumpei Ishikawa
Date: 2017-09-04 00:00:00
Tasks: BIG-bench Machine Learning

Abundant accumulation of digital histopathological images has led to the
increased demand for their analysis, such as computer-aided diagnosis using
machine learning techniques. However, digital pathological images and related
tasks have some issues to be considered. In this mini-review, we introduce the
application of digital pathological image analysis using machine learning
algorithms, address some problems specific to such analysis, and propose
possible solutions.
Context-Aware Generative Adversarial Privacy
Authors: Chong Huang, Peter Kairouz, Xiao Chen, Lalitha Sankar, Ram Rajagopal
Date: 2017-10-26 00:00:00

Preserving the utility of published datasets while simultaneously providing
provable privacy guarantees is a well-known challenge. On the one hand,
context-free privacy solutions, such as differential privacy, provide strong
privacy guarantees, but often lead to a significant reduction in utility. On
the other hand, context-aware privacy solutions, such as information theoretic
privacy, achieve an improved privacy-utility tradeoff, but assume that the data
holder has access to dataset statistics. We circumvent these limitations by
introducing a novel context-aware privacy framework called generative
adversarial privacy (GAP). GAP leverages recent advancements in generative
adversarial networks (GANs) to allow the data holder to learn privatization
schemes from the dataset itself. Under GAP, learning the privacy mechanism is
formulated as a constrained minimax game between two players: a privatizer that
sanitizes the dataset in a way that limits the risk of inference attacks on the
individuals' private variables, and an adversary that tries to infer the
private variables from the sanitized dataset. To evaluate GAP's performance, we
investigate two simple (yet canonical) statistical dataset models: (a) the
binary data model, and (b) the binary Gaussian mixture model. For both models,
we derive game-theoretically optimal minimax privacy mechanisms, and show that
the privacy mechanisms learned from data (in a generative adversarial fashion)
match the theoretically optimal ones. This demonstrates that our framework can
be easily applied in practice, even in the absence of dataset statistics.
Revisiting Graph Construction for Fast Image Segmentation
Authors: Zizhao Zhang, Fuyong Xing, Hanzi Wang, Yan Yan, Ying Huang, Xiaoshuang Shi, Lin Yang
Date: 2017-02-18 00:00:00
Tasks: Clustering, graph construction, graph partitioning, Image Segmentation, Segmentation, Semantic Segmentation

In this paper, we propose a simple but effective method for fast image
segmentation. We re-examine the locality-preserving character of spectral
clustering by constructing a graph over image regions with both global and
local connections. Our novel approach to build graph connections relies on two
key observations: 1) local region pairs that co-occur frequently will have a
high probability to reside on a common object; 2) spatially distant regions in
a common object often exhibit similar visual saliency, which implies their
neighborship in a manifold. We present a novel energy function to efficiently
conduct graph partitioning. Based on multiple high quality partitions, we show
that the generated eigenvector histogram based representation can automatically
drive effective unary potentials for a hierarchical random field model to
produce multi-class segmentation. Sufficient experiments, on the BSDS500
benchmark, large-scale PASCAL VOC and COCO datasets, demonstrate the
competitive segmentation accuracy and significantly improved efficiency of our
proposed method compared with other state of the arts.
Towards Robust Neural Networks via Random Self-ensemble
Authors: Xuanqing Liu, Minhao Cheng, huan zhang, Cho-Jui Hsieh
Date: 2017-12-02 00:00:00
Proceeding: ECCV 2018 9

Recent studies have revealed the vulnerability of deep neural networks: A
small adversarial perturbation that is imperceptible to human can easily make a
well-trained deep neural network misclassify. This makes it unsafe to apply
neural networks in security-critical applications. In this paper, we propose a
new defense algorithm called Random Self-Ensemble (RSE) by combining two
important concepts: {\bf randomness} and {\bf ensemble}. To protect a targeted
model, RSE adds random noise layers to the neural network to prevent the strong
gradient-based attacks, and ensembles the prediction over random noises to
stabilize the performance. We show that our algorithm is equivalent to ensemble
an infinite number of noisy models $f_\epsilon$ without any additional memory
overhead, and the proposed training procedure based on noisy stochastic
gradient descent can ensure the ensemble model has a good predictive
capability. Our algorithm significantly outperforms previous defense techniques
on real data sets. For instance, on CIFAR-10 with VGG network (which has 92\%
accuracy without any attack), under the strong C\&W attack within a certain
distortion tolerance, the accuracy of unprotected model drops to less than
10\%, the best previous defense technique has $48\%$ accuracy, while our method
still has $86\%$ prediction accuracy under the same level of attack. Finally,
our method is simple and easy to integrate into any neural network.
LightNN: Filling the Gap between Conventional Deep Neural Networks and Binarized Networks
Authors: Ruizhou Ding, Zeye Liu, Rongye Shi, Diana Marculescu, R. D. Blanton
Date: 2017-12-02 00:00:00

Application-specific integrated circuit (ASIC) implementations for Deep
Neural Networks (DNNs) have been adopted in many systems because of their
higher classification speed. However, although they may be characterized by
better accuracy, larger DNNs require significant energy and area, thereby
limiting their wide adoption. The energy consumption of DNNs is driven by both
memory accesses and computation. Binarized Neural Networks (BNNs), as a
trade-off between accuracy and energy consumption, can achieve great energy
reduction, and have good accuracy for large DNNs due to its regularization
effect. However, BNNs show poor accuracy when a smaller DNN configuration is
adopted. In this paper, we propose a new DNN model, LightNN, which replaces the
multiplications to one shift or a constrained number of shifts and adds. For a
fixed DNN configuration, LightNNs have better accuracy at a slight energy
increase than BNNs, yet are more energy efficient with only slightly less
accuracy than conventional DNNs. Therefore, LightNNs provide more options for
hardware designers to make trade-offs between accuracy and energy. Moreover,
for large DNN configurations, LightNNs have a regularization effect, making
them better in accuracy than conventional DNNs. These conclusions are verified
by experiment using the MNIST and CIFAR-10 datasets for different DNN
configurations.
DeepRadiologyNet: Radiologist Level Pathology Detection in CT Head Images
Authors: Jameson Merkow, Robert Lufkin, Kim Nguyen, Stefano Soatto, Zhuowen Tu, Andrea Vedaldi
Date: 2017-11-26 00:00:00

We describe a system to automatically filter clinically significant findings
from computerized tomography (CT) head scans, operating at performance levels
exceeding that of practicing radiologists. Our system, named DeepRadiologyNet,
builds on top of deep convolutional neural networks (CNNs) trained using
approximately 3.5 million CT head images gathered from over 24,000 studies
taken from January 1, 2015 to August 31, 2015 and January 1, 2016 to April 30
2016 in over 80 clinical sites. For our initial system, we identified 30
phenomenological traits to be recognized in the CT scans. To test the system,
we designed a clinical trial using over 4.8 million CT head images (29,925
studies), completely disjoint from the training and validation set, interpreted
by 35 US Board Certified radiologists with specialized CT head experience. We
measured clinically significant error rates to ascertain whether the
performance of DeepRadiologyNet was comparable to or better than that of US
Board Certified radiologists. DeepRadiologyNet achieved a clinically
significant miss rate of 0.0367% on automatically selected high-confidence
studies. Thus, DeepRadiologyNet enables significant reduction in the workload
of human radiologists by automatically filtering studies and reporting on the
high-confidence ones at an operating point well below the literal error rate
for US Board Certified radiologists, estimated at 0.82%.
An Asymptotically Optimal Algorithm for Communicating Multiplayer Multi-Armed Bandit Problems
Authors: Noyan Evirgen, Alper Kose, Hakan Gokcesu
Date: 2017-12-02 00:00:00

We consider a decentralized stochastic multi-armed bandit problem with
multiple players. Each player aims to maximize his/her own reward by pulling an
arm. The arms give rewards based on i.i.d. stochastic Bernoulli distributions.
Players are not aware about the probability distributions of the arms. At the
end of each turn, the players inform their neighbors about the arm he/she
pulled and the reward he/she got. Neighbors of players are determined according
to an Erd{\H{o}}s-R{\'e}nyi graph with connectivity $\alpha$. This graph is
reproduced in the beginning of every turn with the same connectivity. When more
than one player choose the same arm in a turn, we assume that only one of the
players who is randomly chosen gets the reward where the others get nothing. We
first start by assuming players are not aware of the collision model and offer
an asymptotically optimal algorithm for $\alpha = 1$ case. Then, we extend our
prior work and offer an asymptotically optimal algorithm for any connectivity
but zero, assuming players aware of the collision model. We also study the
effect of $\alpha$, the degree of communication between players, empirically on
the cumulative regret by comparing them with traditional multi-armed bandit
algorithms.
Representation and Reinforcement Learning for Personalized Glycemic Control in Septic Patients
Authors: Wei-Hung Weng, Mingwu Gao, Ze He, Susu Yan, Peter Szolovits
Date: 2017-12-02 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Glycemic control is essential for critical care. However, it is a challenging
task because there has been no study on personalized optimal strategies for
glycemic control. This work aims to learn personalized optimal glycemic
trajectories for severely ill septic patients by learning data-driven policies
to identify optimal targeted blood glucose levels as a reference for
clinicians. We encoded patient states using a sparse autoencoder and adopted a
reinforcement learning paradigm using policy iteration to learn the optimal
policy from data. We also estimated the expected return following the policy
learned from the recorded glycemic trajectories, which yielded a function
indicating the relationship between real blood glucose values and 90-day
mortality rates. This suggests that the learned optimal policy could reduce the
patients' estimated 90-day mortality rate by 6.3%, from 31% to 24.7%. The
result demonstrates that reinforcement learning with appropriate patient state
encoding can potentially provide optimal glycemic trajectories and allow
clinicians to design a personalized strategy for glycemic control in septic
patients.
The Achievement of Higher Flexibility in Multiple Choice-based Tests Using Image Classification Techniques
Authors: Mahmoud Afifi, Khaled F. Hussain
Date: 2017-11-02 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, image-classification, Image Classification, Image Registration, Multiple-choice

In spite of the high accuracy of the existing optical mark reading (OMR)
systems and devices, a few restrictions remain existent. In this work, we aim
to reduce the restrictions of multiple choice questions (MCQ) within tests. We
use an image registration technique to extract the answer boxes from answer
sheets. Unlike other systems that rely on simple image processing steps to
recognize the extracted answer boxes, we address the problem from another
perspective by training a machine learning classifier to recognize the class of
each answer box (i.e., confirmed, crossed out, or blank answer). This gives us
the ability to deal with a variety of shading and mark patterns, and
distinguish between chosen (i.e., confirmed) and canceled answers (i.e.,
crossed out). All existing machine learning techniques require a large number
of examples in order to train a model for classification, therefore we present
a dataset including six real MCQ assessments with different answer sheet
templates. We evaluate two strategies of classification: a straight-forward
approach and a two-stage classifier approach. We test two handcrafted feature
methods and a convolutional neural network. In the end, we present an
easy-to-use graphical user interface of the proposed system. Compared with
existing OMR systems, the proposed system has the least constraints and
achieves a high accuracy. We believe that the presented work will further
direct the development of OMR systems towards reducing the restrictions of the
MCQ tests.
From knowledge-based to data-driven modeling of fuzzy rule-based systems: A critical reflection
Authors: Eyke Hüllermeier
Date: 2017-12-02 00:00:00

This paper briefly elaborates on a development in (applied) fuzzy logic that
has taken place in the last couple of decades, namely, the complementation or
even replacement of the traditional knowledge-based approach to fuzzy
rule-based systems design by a data-driven one. It is argued that the classical
rule-based modeling paradigm is actually more amenable to the knowledge-based
approach, for which it has originally been conceived, while being less apt to
data-driven model design. An important reason that prevents fuzzy (rule-based)
systems from being leveraged in large-scale applications is the flat structure
of rule bases, along with the local nature of fuzzy rules and their limited
ability to express complex dependencies between variables. This motivates
alternative approaches to fuzzy systems modeling, in which functional
dependencies can be represented more flexibly and more compactly in terms of
hierarchical structures.
Short-term Mortality Prediction for Elderly Patients Using Medicare Claims Data
Authors: Maggie Makar, Marzyeh Ghassemi, David Cutler, Ziad Obermeyer
Date: 2017-12-02 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Mortality Prediction, Prediction

Risk prediction is central to both clinical medicine and public health. While
many machine learning models have been developed to predict mortality, they are
rarely applied in the clinical literature, where classification tasks typically
rely on logistic regression. One reason for this is that existing machine
learning models often seek to optimize predictions by incorporating features
that are not present in the databases readily available to providers and policy
makers, limiting generalizability and implementation. Here we tested a number
of machine learning classifiers for prediction of six-month mortality in a
population of elderly Medicare beneficiaries, using an administrative claims
database of the kind available to the majority of health care payers and
providers. We show that machine learning classifiers substantially outperform
current widely-used methods of risk prediction but only when used with an
improved feature set incorporating insights from clinical medicine, developed
for this study. Our work has applications to supporting patient and provider
decision making at the end of life, as well as population health-oriented
efforts to identify patients at high risk of poor outcomes.
Learning Sparse Adversarial Dictionaries For Multi-Class Audio Classification
Authors: Vaisakh Shaj, Puranjoy Bhattacharya
Date: 2017-12-02 00:00:00
Tasks: Audio Classification, Audio Denoising, Classification, Denoising, Dictionary Learning, General Classification, Speech Enhancement

Audio events are quite often overlapping in nature, and more prone to noise
than visual signals. There has been increasing evidence for the superior
performance of representations learned using sparse dictionaries for
applications like audio denoising and speech enhancement. This paper
concentrates on modifying the traditional reconstructive dictionary learning
algorithms, by incorporating a discriminative term into the objective function
in order to learn class-specific adversarial dictionaries that are good at
representing samples of their own class at the same time poor at representing
samples belonging to any other class. We quantitatively demonstrate the
effectiveness of our learned dictionaries as a stand-alone solution for both
binary as well as multi-class audio classification problems.
Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks
Authors: Urs Köster, Tristan J. Webb, Xin Wang, Marcel Nassar, Arjun K. Bansal, William H. Constable, Oğuz H. Elibol, Scott Gray, Stewart Hall, Luke Hornof, Amir Khosrowshahi, Carey Kloss, Ruby J. Pai, Naveen Rao
Date: 2017-11-06 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Generative Adversarial Network

Deep neural networks are commonly developed and trained in 32-bit floating
point format. Significant gains in performance and energy efficiency could be
realized by training and inference in numerical formats optimized for deep
learning. Despite advances in limited precision inference in recent years,
training of neural networks in low bit-width remains a challenging problem.
Here we present the Flexpoint data format, aiming at a complete replacement of
32-bit floating point format training and inference, designed to support modern
deep network topologies without modifications. Flexpoint tensors have a shared
exponent that is dynamically adjusted to minimize overflows and maximize
available dynamic range. We validate Flexpoint by training AlexNet, a deep
residual network and a generative adversarial network, using a simulator
implemented with the neon deep learning framework. We demonstrate that 16-bit
Flexpoint closely matches 32-bit floating point in training all three models,
without any need for tuning of model hyperparameters. Our results suggest
Flexpoint as a promising numerical format for future hardware for training and
inference.