pwc-archive/papers-with-abstracts huggingface dataset:
ICADx: Interpretable computer aided diagnosis of breast masses
Authors: Seong Tae Kim, Hakmin Lee, Hak Gu Kim, Yong Man Ro
Date: 2018-05-23 00:00:00
Tasks: Deep Learning, Diagnostic, Generative Adversarial Network, Medical Image Analysis

In this study, a novel computer aided diagnosis (CADx) framework is devised
to investigate interpretability for classifying breast masses. Recently, a deep
learning technology has been successfully applied to medical image analysis
including CADx. Existing deep learning based CADx approaches, however, have a
limitation in explaining the diagnostic decision. In real clinical practice,
clinical decisions could be made with reasonable explanation. So current deep
learning approaches in CADx are limited in real world deployment. In this
paper, we investigate interpretability in CADx with the proposed interpretable
CADx (ICADx) framework. The proposed framework is devised with a generative
adversarial network, which consists of interpretable diagnosis network and
synthetic lesion generative network to learn the relationship between
malignancy and a standardized description (BI-RADS). The lesion generative
network and the interpretable diagnosis network compete in an adversarial
learning so that the two networks are improved. The effectiveness of the
proposed method was validated on public mammogram database. Experimental
results showed that the proposed ICADx framework could provide the
interpretability of mass as well as mass classification. It was mainly
attributed to the fact that the proposed method was effectively trained to find
the relationship between malignancy and interpretations via the adversarial
learning. These results imply that the proposed ICADx framework could be a
promising approach to develop the CADx system.
A Brand-level Ranking System with the Customized Attention-GRU Model
Authors: Yu Zhu, Junxiong Zhu, Jie Hou, Yongliang Li, Beidou Wang, Ziyu Guan, Deng Cai
Date: 2018-05-23 00:00:00
Tasks: Feature Engineering

In e-commerce websites like Taobao, brand is playing a more important role in
influencing users' decision of click/purchase, partly because users are now
attaching more importance to the quality of products and brand is an indicator
of quality. However, existing ranking systems are not specifically designed to
satisfy this kind of demand. Some design tricks may partially alleviate this
problem, but still cannot provide satisfactory results or may create additional
interaction cost. In this paper, we design the first brand-level ranking system
to address this problem. The key challenge of this system is how to
sufficiently exploit users' rich behavior in e-commerce websites to rank the
brands. In our solution, we firstly conduct the feature engineering
specifically tailored for the personalized brand ranking problem and then rank
the brands by an adapted Attention-GRU model containing three important
modifications. Note that our proposed modifications can also apply to many
other machine learning models on various tasks. We conduct a series of
experiments to evaluate the effectiveness of our proposed ranking model and
test the response to the brand-level ranking system from real users on a
large-scale e-commerce platform, i.e. Taobao.
Learning discrete Bayesian networks in polynomial time and sample complexity
Authors: Adarsh Barik, Jean Honorio
Date: 2018-03-12 00:00:00

In this paper, we study the problem of structure learning for Bayesian
networks in which nodes take discrete values. The problem is NP-hard in general
but we show that under certain conditions we can recover the true structure of
a Bayesian network with sufficient number of samples. We develop a mathematical
model which does not assume any specific conditional probability distributions
for the nodes. We use a primal-dual witness construction to prove that, under
some technical conditions on the interaction between node pairs, we can do
exact recovery of the parents and children of a node by performing group
l_12-regularized multivariate regression. Thus, we recover the true Bayesian
network structure. If degree of a node is bounded then the sample complexity of
our proposed approach grows logarithmically with respect to the number of nodes
in the Bayesian network. Furthermore, our method runs in polynomial time.
Semi-Supervised Learning with GANs: Revisiting Manifold Regularization
Authors: Bruno Lecouat, Chuan-Sheng Foo, Houssam Zenati, Vijay R. Chandrasekhar
Date: 2018-05-23 00:00:00

GANS are powerful generative models that are able to model the manifold of
natural images. We leverage this property to perform manifold regularization by
approximating the Laplacian norm using a Monte Carlo approximation that is
easily computed with the GAN. When incorporated into the feature-matching GAN
of Improved GAN, we achieve state-of-the-art results for GAN-based
semi-supervised learning on the CIFAR-10 dataset, with a method that is
significantly easier to implement than competing methods.
Hypergraph Spectral Clustering in the Weighted Stochastic Block Model
Authors: Kwangjun Ahn, Kangwook Lee, Changho Suh
Date: 2018-05-23 00:00:00
Tasks: Clustering, Stochastic Block Model

Spectral clustering is a celebrated algorithm that partitions objects based
on pairwise similarity information. While this approach has been successfully
applied to a variety of domains, it comes with limitations. The reason is that
there are many other applications in which only \emph{multi}-way similarity
measures are available. This motivates us to explore the multi-way measurement
setting. In this work, we develop two algorithms intended for such setting:
Hypergraph Spectral Clustering (HSC) and Hypergraph Spectral Clustering with
Local Refinement (HSCLR). Our main contribution lies in performance analysis of
the poly-time algorithms under a random hypergraph model, which we name the
weighted stochastic block model, in which objects and multi-way measures are
modeled as nodes and weights of hyperedges, respectively. Denoting by $n$ the
number of nodes, our analysis reveals the following: (1) HSC outputs a
partition which is better than a random guess if the sum of edge weights (to be
explained later) is $\Omega(n)$; (2) HSC outputs a partition which coincides
with the hidden partition except for a vanishing fraction of nodes if the sum
of edge weights is $\omega(n)$; and (3) HSCLR exactly recovers the hidden
partition if the sum of edge weights is on the order of $n \log n$. Our results
improve upon the state of the arts recently established under the model and
they firstly settle the order-wise optimal results for the binary edge weight
case. Moreover, we show that our results lead to efficient sketching algorithms
for subspace clustering, a computer vision application. Lastly, we show that
HSCLR achieves the information-theoretic limits for a special yet practically
relevant model, thereby showing no computational barrier for the case.
Global Model Interpretation via Recursive Partitioning
Authors: Chengliang Yang, Anand Rangarajan, Sanjay Ranka
Date: 2018-02-11 00:00:00
Tasks: BIG-bench Machine Learning, model

In this work, we propose a simple but effective method to interpret black-box
machine learning models globally. That is, we use a compact binary tree, the
interpretation tree, to explicitly represent the most important decision rules
that are implicitly contained in the black-box machine learning models. This
tree is learned from the contribution matrix which consists of the
contributions of input variables to predicted scores for each single
prediction. To generate the interpretation tree, a unified process recursively
partitions the input variable space by maximizing the difference in the average
contribution of the split variable between the divided spaces. We demonstrate
the effectiveness of our method in diagnosing machine learning models on
multiple tasks. Also, it is useful for new knowledge discovery as such insights
are not easily identifiable when only looking at single predictions. In
general, our work makes it easier and more efficient for human beings to
understand machine learning models.
Dictionary Learning by Dynamical Neural Networks
Authors: Tsung-Han Lin, Ping Tak Peter Tang
Date: 2018-05-23 00:00:00
Tasks: Contrastive Learning, Dictionary Learning

A dynamical neural network consists of a set of interconnected neurons that
interact over time continuously. It can exhibit computational properties in the
sense that the dynamical system's evolution and/or limit points in the
associated state space can correspond to numerical solutions to certain
mathematical optimization or learning problems. Such a computational system is
particularly attractive in that it can be mapped to a massively parallel
computer architecture for power and throughput efficiency, especially if each
neuron can rely solely on local information (i.e., local memory). Deriving
gradients from the dynamical network's various states while conforming to this
last constraint, however, is challenging. We show that by combining ideas of
top-down feedback and contrastive learning, a dynamical network for solving the
l1-minimizing dictionary learning problem can be constructed, and the true
gradients for learning are provably computable by individual neurons. Using
spiking neurons to construct our dynamical network, we present a learning
process, its rigorous mathematical analysis, and numerical results on several
dictionary learning problems.
Minimax Distribution Estimation in Wasserstein Distance
Authors: Shashank Singh, Barnabás Póczos
Date: 2018-02-24 00:00:00
Tasks: BIG-bench Machine Learning

The Wasserstein metric is an important measure of distance between probability distributions, with applications in machine learning, statistics, probability theory, and data analysis. This paper provides upper and lower bounds on statistical minimax rates for the problem of estimating a probability distribution under Wasserstein loss, using only metric properties, such as covering and packing numbers, of the sample space, and weak moment assumptions on the probability distributions.
Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow
Authors: Pengcheng Yin, Bowen Deng, Edgar Chen, Bogdan Vasilescu, Graham Neubig
Date: 2018-05-23 00:00:00
Tasks: Code Summarization, Retrieval, Source Code Summarization

For tasks like code synthesis from natural language, code retrieval, and code
summarization, data-driven models have shown great promise. However, creating
these models require parallel data between natural language (NL) and code with
fine-grained alignments. Stack Overflow (SO) is a promising source to create
such a data set: the questions are diverse and most of them have corresponding
answers with high-quality code snippets. However, existing heuristic methods
(e.g., pairing the title of a post with the code in the accepted answer) are
limited both in their coverage and the correctness of the NL-code pairs
obtained. In this paper, we propose a novel method to mine high-quality aligned
data from SO using two sets of features: hand-crafted features considering the
structure of the extracted snippets, and correspondence features obtained by
training a probabilistic model to capture the correlation between NL and code
using neural networks. These features are fed into a classifier that determines
the quality of mined NL-code pairs. Experiments using Python and Java as test
beds show that the proposed method greatly expands coverage and accuracy over
existing mining methods, even when using only a small number of labeled
examples. Further, we find that reasonable results are achieved even when
training the classifier on one language and testing on another, showing promise
for scaling NL-code mining to a wide variety of programming languages beyond
those for which we are able to annotate data.
Scalable Coordinated Exploration in Concurrent Reinforcement Learning
Authors: Maria Dimakopoulou, Ian Osband, Benjamin Van Roy
Date: 2018-05-23 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We consider a team of reinforcement learning agents that concurrently operate
in a common environment, and we develop an approach to efficient coordinated
exploration that is suitable for problems of practical scale. Our approach
builds on seed sampling (Dimakopoulou and Van Roy, 2018) and randomized value
function learning (Osband et al., 2016). We demonstrate that, for simple
tabular contexts, the approach is competitive with previously proposed tabular
model learning methods (Dimakopoulou and Van Roy, 2018). With a
higher-dimensional problem and a neural network value function representation,
the approach learns quickly with far fewer agents than alternative exploration
schemes.
Building Extraction at Scale using Convolutional Neural Network: Mapping of the United States
Authors: Hsiuhan Lexie Yang, Jiangye Yuan, Dalton Lunga, Melanie Laverdiere, Amy Rose, Budhendra Bhaduri
Date: 2018-05-23 00:00:00

Establishing up-to-date large scale building maps is essential to understand
urban dynamics, such as estimating population, urban planning and many other
applications. Although many computer vision tasks has been successfully carried
out with deep convolutional neural networks, there is a growing need to
understand their large scale impact on building mapping with remote sensing
imagery. Taking advantage of the scalability of CNNs and using only few areas
with the abundance of building footprints, for the first time we conduct a
comparative analysis of four state-of-the-art CNNs for extracting building
footprints across the entire continental United States. The four CNN
architectures namely: branch-out CNN, fully convolutional neural network (FCN),
conditional random field as recurrent neural network (CRFasRNN), and SegNet,
support semantic pixel-wise labeling and focus on capturing textural
information at multi-scale. We use 1-meter resolution aerial images from
National Agriculture Imagery Program (NAIP) as the test-bed, and compare the
extraction results across the four methods. In addition, we propose to combine
signed-distance labels with SegNet, the preferred CNN architecture identified
by our extensive evaluations, to advance building extraction results to
instance level. We further demonstrate the usefulness of fusing additional near
IR information into the building extraction framework. Large scale experimental
evaluations are conducted and reported using metrics that include: precision,
recall rate, intersection over union, and the number of buildings extracted.
With the improved CNN model and no requirement of further post-processing, we
have generated building maps for the United States. The quality of extracted
buildings and processing time demonstrated the proposed CNN-based framework
fits the need of building extraction at scale.
Shared Autonomy via Deep Reinforcement Learning
Authors: Siddharth Reddy, Anca D. Dragan, Sergey Levine
Date: 2018-02-06 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In shared autonomy, user input is combined with semi-autonomous control to
achieve a common goal. The goal is often unknown ex-ante, so prior work enables
agents to infer the goal from user input and assist with the task. Such methods
tend to assume some combination of knowledge of the dynamics of the
environment, the user's policy given their goal, and the set of possible goals
the user might target, which limits their application to real-world scenarios.
We propose a deep reinforcement learning framework for model-free shared
autonomy that lifts these assumptions. We use human-in-the-loop reinforcement
learning with neural network function approximation to learn an end-to-end
mapping from environmental observation and user input to agent action values,
with task reward as the only form of supervision. This approach poses the
challenge of following user commands closely enough to provide the user with
real-time action feedback and thereby ensure high-quality user input, but also
deviating from the user's actions when they are suboptimal. We balance these
two needs by discarding actions whose values fall below some threshold, then
selecting the remaining action closest to the user's input. Controlled studies
with users (n = 12) and synthetic pilots playing a video game, and a pilot
study with users (n = 4) flying a real quadrotor, demonstrate the ability of
our algorithm to assist users with real-time control tasks in which the agent
cannot directly access the user's private information through observations, but
receives a reward signal and user input that both depend on the user's intent.
The agent learns to assist the user without access to this private information,
implicitly inferring it from the user's input. This paper is a proof of concept
that illustrates the potential for deep reinforcement learning to enable
flexible and practical assistive systems.
Image Matching: An Application-oriented Benchmark
Authors: Jia-Wang Bian, Le Zhang, Yun Liu, Wen-Yan Lin, Ming-Ming Cheng, Ian D. Reid
Date: 2017-09-12 00:00:00
Tasks: Attribute, Benchmarking, Diversity

Image matching approaches have been widely used in computer vision
applications in which the image-level matching performance of matchers is
critical. However, it has not been well investigated by previous works which
place more emphases on evaluating local features. To this end, we present a
uniform benchmark with novel evaluation metrics and a large-scale dataset for
evaluating the overall performance of image matching methods. The proposed
metrics are application-oriented as they emphasize application requirements for
matchers. The dataset contains two portions for benchmarking video frame
matching and unordered image matching separately, where each portion consists
of real-world image sequences and each sequence has a specific attribute.
Subsequently, we carry out a comprehensive performance evaluation of different
state-of-the-art methods and conduct in-depth analyses regarding various
aspects such as application requirements, matching types, and data diversity.
Moreover, we shed light on how to choose appropriate approaches for different
applications based on empirical results and analyses. Conclusions in this
benchmark can be used as general guidelines to design practical matching
systems and also advocate potential future research directions in this field.
Approximate Random Dropout
Authors: Zhuoran Song, Ru Wang, Dongyu Ru, Hongru Huang, Zhenghao Peng, Jing Ke, Xiaoyao Liang, Li Jiang
Date: 2018-05-23 00:00:00

The training phases of Deep neural network~(DNN) consumes enormous processing
time and energy. Compression techniques utilizing the sparsity of DNNs can
effectively accelerate the inference phase of DNNs. However, it can be hardly
used in the training phase because the training phase involves dense
matrix-multiplication using General Purpose Computation on Graphics Processors
(GPGPU), which endorse regular and structural data layout. In this paper, we
propose the Approximate Random Dropout that replaces the conventional random
dropout of neurons and synapses with a regular and predefined patterns to
eliminate the unnecessary computation and data access. To compensate the
potential performance loss we develop a SGD-based Search Algorithm to produce
the distribution of dropout patterns. We prove our approach is statistically
equivalent to the previous dropout method. Experiments results on MLP and LSTM
using well-known benchmarks show that the proposed Approximate Random Dropout
can reduce the training time by $20\%$-$77\%$ ($19\%$-$60\%$) when dropout rate
is $0.3$-$0.7$ on MLP (LSTM) with marginal accuracy drop.
Cost-Aware Learning for Improved Identifiability with Multiple Experiments
Authors: Longyun Guo, Jean Honorio, John Morgan
Date: 2018-02-12 00:00:00

We analyze the sample complexity of learning from multiple experiments where the experimenter has a total budget for obtaining samples. In this problem, the learner should choose a hypothesis that performs well with respect to multiple experiments, and their related data distributions. Each collected sample is associated with a cost which depends on the particular experiments. In our setup, a learner performs $m$ experiments, while incurring a total cost $C$. We first show that learning from multiple experiments allows to improve identifiability. Additionally, by using a Rademacher complexity approach, we show that the gap between the training and generalization error is $O(C^{-1/2})$. We also provide some examples for linear prediction, two-layer neural networks and kernel methods.
Large-Scale Neuromorphic Spiking Array Processors: A quest to mimic the brain
Authors: Chetan Singh Thakur, Jamal Molin, Gert Cauwenberghs, Giacomo Indiveri, Kundan Kumar, Ning Qiao, Johannes Schemmel, Runchun Wang, Elisabetta Chicca, Jennifer Olson Hasler, Jae-sun Seo, Shimeng Yu, Yu Cao, André van Schaik, Ralph Etienne-Cummings
Date: 2018-05-23 00:00:00

Neuromorphic engineering (NE) encompasses a diverse range of approaches to
information processing that are inspired by neurobiological systems, and this
feature distinguishes neuromorphic systems from conventional computing systems.
The brain has evolved over billions of years to solve difficult engineering
problems by using efficient, parallel, low-power computation. The goal of NE is
to design systems capable of brain-like computation. Numerous large-scale
neuromorphic projects have emerged recently. This interdisciplinary field was
listed among the top 10 technology breakthroughs of 2014 by the MIT Technology
Review and among the top 10 emerging technologies of 2015 by the World Economic
Forum. NE has two-way goals: one, a scientific goal to understand the
computational properties of biological neural systems by using models
implemented in integrated circuits (ICs); second, an engineering goal to
exploit the known properties of biological systems to design and implement
efficient devices for engineering applications. Building hardware neural
emulators can be extremely useful for simulating large-scale neural models to
explain how intelligent behavior arises in the brain. The principle advantages
of neuromorphic emulators are that they are highly energy efficient, parallel
and distributed, and require a small silicon area. Thus, compared to
conventional CPUs, these neuromorphic emulators are beneficial in many
engineering applications such as for the porting of deep learning algorithms
for various recognitions tasks. In this review article, we describe some of the
most significant neuromorphic spiking emulators, compare the different
architectures and approaches used by them, illustrate their advantages and
drawbacks, and highlight the capabilities that each can deliver to neural
modelers.
Analysis of Thompson Sampling for Graphical Bandits Without the Graphs
Authors: Fang Liu, Zizhan Zheng, Ness Shroff
Date: 2018-05-23 00:00:00
Tasks: Thompson Sampling

We study multi-armed bandit problems with graph feedback, in which the
decision maker is allowed to observe the neighboring actions of the chosen
action, in a setting where the graph may vary over time and is never fully
revealed to the decision maker. We show that when the feedback graphs are
undirected, the original Thompson Sampling achieves the optimal (within
logarithmic factors) regret $\tilde{O}\left(\sqrt{\beta_0(G)T}\right)$ over
time horizon $T$, where $\beta_0(G)$ is the average independence number of the
latent graphs. To the best of our knowledge, this is the first result showing
that the original Thompson Sampling is optimal for graphical bandits in the
undirected setting. A slightly weaker regret bound of Thompson Sampling in the
directed setting is also presented. To fill this gap, we propose a variant of
Thompson Sampling, that attains the optimal regret in the directed setting
within a logarithmic factor. Both algorithms can be implemented efficiently and
do not require the knowledge of the feedback graphs at any time.