pwc-archive/papers-with-abstracts huggingface dataset:
Fast Single-Class Classification and the Principle of Logit Separation
Authors: Gil Keren, Sivan Sabato, Björn Schuller
Date: 2017-05-29 00:00:00
Tasks: Binary Classification, Classification, General Classification, Image Retrieval

We consider neural network training, in applications in which there are many
possible classes, but at test-time, the task is a binary classification task of
determining whether the given example belongs to a specific class, where the
class of interest can be different each time the classifier is applied. For
instance, this is the case for real-time image search. We define the Single
Logit Classification (SLC) task: training the network so that at test-time, it
would be possible to accurately identify whether the example belongs to a given
class in a computationally efficient manner, based only on the output logit for
this class. We propose a natural principle, the Principle of Logit Separation,
as a guideline for choosing and designing losses suitable for the SLC. We show
that the cross-entropy loss function is not aligned with the Principle of Logit
Separation. In contrast, there are known loss functions, as well as novel batch
loss functions that we propose, which are aligned with this principle. In
total, we study seven loss functions. Our experiments show that indeed in
almost all cases, losses that are aligned with the Principle of Logit
Separation obtain at least 20% relative accuracy improvement in the SLC task
compared to losses that are not aligned with it, and sometimes considerably
more. Furthermore, we show that fast SLC does not cause any drop in binary
classification accuracy, compared to standard classification in which all
logits are computed, and yields a speedup which grows with the number of
classes. For instance, we demonstrate a 10x speedup when the number of classes
is 400,000. Tensorflow code for optimizing the new batch losses is publicly
available at https://github.com/cruvadom/Logit Separation.
SMR: Medical Knowledge Graph Embedding for Safe Medicine Recommendation
Authors: Fang Gong, Meng Wang, Haofen Wang, Sen Wang, Mengyue Liu
Date: 2017-10-16 00:00:00
Tasks: Graph Embedding, Knowledge Graph Embedding, Knowledge Graphs, Link Prediction, Recommendation Systems

Most of the existing medicine recommendation systems that are mainly based on electronic medical records (EMRs) are significantly assisting doctors to make better clinical decisions benefiting both patients and caregivers. Even though the growth of EMRs is at a lighting fast speed in the era of big data, content limitations in EMRs restrain the existed recommendation systems to reflect relevant medical facts, such as drug-drug interactions. Many medical knowledge graphs that contain drug-related information, such as DrugBank, may give hope for the recommendation systems. However, the direct use of these knowledge graphs in the systems suffers from robustness caused by the incompleteness of the graphs. To address these challenges, we stand on recent advances in graph embedding learning techniques and propose a novel framework, called Safe Medicine Recommendation (SMR), in this paper. Specifically, SMR first constructs a high-quality heterogeneous graph by bridging EMRs (MIMIC-III) and medical knowledge graphs (ICD-9 ontology and DrugBank). Then, SMR jointly embeds diseases, medicines, patients, and their corresponding relations into a shared lower dimensional space. Finally, SMR uses the embeddings to decompose the medicine recommendation into a link prediction process while considering the patient's diagnoses and adverse drug reactions. To our best knowledge, SMR is the first to learn embeddings of a patient-disease-medicine graph for medicine recommendation in the world. Extensive experiments on real datasets are conducted to evaluate the effectiveness of proposed framework.
Biologically Inspired Feedforward Supervised Learning for Deep Self-Organizing Map Networks
Authors: Takashi Shinozaki
Date: 2017-10-26 00:00:00
Tasks: General Classification

In this study, we propose a novel deep neural network and its supervised
learning method that uses a feedforward supervisory signal. The method is
inspired by the human visual system and performs human-like association-based
learning without any backward error propagation. The feedforward supervisory
signal that produces the correct result is preceded by the target signal and
associates its confirmed label with the classification result of the target
signal. It effectively uses a large amount of information from the feedforward
signal, and forms a continuous and rich learning representation. The method is
validated using visual recognition tasks on the MNIST handwritten dataset.
Big Data Classification Using Augmented Decision Trees
Authors: Rajiv Sambasivan, Sourish Das
Date: 2017-10-26 00:00:00
Tasks: Classification, General Classification

We present an algorithm for classification tasks on big data. Experiments
conducted as part of this study indicate that the algorithm can be as accurate
as ensemble methods such as random forests or gradient boosted trees. Unlike
ensemble methods, the models produced by the algorithm can be easily
interpreted. The algorithm is based on a divide and conquer strategy and
consists of two steps. The first step consists of using a decision tree to
segment the large dataset. By construction, decision trees attempt to create
homogeneous class distributions in their leaf nodes. However, non-homogeneous
leaf nodes are usually produced. The second step of the algorithm consists of
using a suitable classifier to determine the class labels for the
non-homogeneous leaf nodes. The decision tree segment provides a coarse segment
profile while the leaf level classifier can provide information about the
attributes that affect the label within a segment.
Duality-free Methods for Stochastic Composition Optimization
Authors: Liu Liu, Ji Liu, DaCheng Tao
Date: 2017-10-26 00:00:00
Tasks: Reinforcement Learning

We consider the composition optimization with two expected-value functions in
the form of $\frac{1}{n}\sum\nolimits_{i = 1}^n F_i(\frac{1}{m}\sum\nolimits_{j
= 1}^m G_j(x))+R(x)$, { which formulates many important problems in statistical
learning and machine learning such as solving Bellman equations in
reinforcement learning and nonlinear embedding}. Full Gradient or classical
stochastic gradient descent based optimization algorithms are unsuitable or
computationally expensive to solve this problem due to the inner expectation
$\frac{1}{m}\sum\nolimits_{j = 1}^m G_j(x)$. We propose a duality-free based
stochastic composition method that combines variance reduction methods to
address the stochastic composition problem. We apply SVRG and SAGA based
methods to estimate the inner function, and duality-free method to estimate the
outer function. We prove the linear convergence rate not only for the convex
composition problem, but also for the case that the individual outer functions
are non-convex while the objective function is strongly-convex. We also provide
the results of experiments that show the effectiveness of our proposed methods.
Rethinking generalization requires revisiting old ideas: statistical mechanics approaches and complex learning behavior
Authors: Charles H. Martin, Michael W. Mahoney
Date: 2017-10-26 00:00:00
Proceeding: ICLR 2018 1

We describe an approach to understand the peculiar and counterintuitive
generalization properties of deep neural networks. The approach involves going
beyond worst-case theoretical capacity control frameworks that have been
popular in machine learning in recent years to revisit old ideas in the
statistical mechanics of neural networks. Within this approach, we present a
prototypical Very Simple Deep Learning (VSDL) model, whose behavior is
controlled by two control parameters, one describing an effective amount of
data, or load, on the network (that decreases when noise is added to the
input), and one with an effective temperature interpretation (that increases
when algorithms are early stopped). Using this model, we describe how a very
simple application of ideas from the statistical mechanics theory of
generalization provides a strong qualitative description of recently-observed
empirical results regarding the inability of deep neural networks not to
overfit training data, discontinuous learning and sharp transitions in the
generalization properties of learning algorithms, etc.
Artifact reduction for separable non-local means
Authors: Sanjay Ghosh, Kunal. N. Chaudhury
Date: 2017-10-26 00:00:00
Tasks: Denoising

It was recently demonstrated [J. Electron. Imaging, 25(2), 2016] that one can
perform fast non-local means (NLM) denoising of one-dimensional signals using a
method called lifting. The cost of lifting is independent of the patch length,
which dramatically reduces the run-time for large patches. Unfortunately, it is
difficult to directly extend lifting for non-local means denoising of images.
To bypass this, the authors proposed a separable approximation in which the
image rows and columns are filtered using lifting. The overall algorithm is
significantly faster than NLM, and the results are comparable in terms of PSNR.
However, the separable processing often produces vertical and horizontal
stripes in the image. This problem was previously addressed by using a
bilateral filter-based post-smoothing, which was effective in removing some of
the stripes. In this letter, we demonstrate that stripes can be mitigated in
the first place simply by involving the neighboring rows (or columns) in the
filtering. In other words, we use a two-dimensional search (similar to NLM),
while still using one-dimensional patches (as in the previous proposal). The
novelty is in the observation that one can use lifting for performing
two-dimensional searches. The proposed approach produces artifact-free images,
whose quality and PSNR are comparable to NLM, while being significantly faster.
A Generative Model for Volume Rendering
Authors: Matthew Berger, Jixian Li, Joshua A. Levine
Date: 2017-10-26 00:00:00
Tasks: Generative Adversarial Network, model

We present a technique to synthesize and analyze volume-rendered images using generative models. We use the Generative Adversarial Network (GAN) framework to compute a model from a large collection of volume renderings, conditioned on (1) viewpoint and (2) transfer functions for opacity and color. Our approach facilitates tasks for volume analysis that are challenging to achieve using existing rendering techniques such as ray casting or texture-based methods. We show how to guide the user in transfer function editing by quantifying expected change in the output image. Additionally, the generative model transforms transfer functions into a view-invariant latent space specifically designed to synthesize volume-rendered images. We use this space directly for rendering, enabling the user to explore the space of volume-rendered images. As our model is independent of the choice of volume rendering process, we show how to analyze volume-rendered images produced by direct and global illumination lighting, for a variety of volume datasets.
The Complete Extensions do not form a Complete Semilattice
Authors: Anthony P. Young
Date: 2017-10-15 00:00:00
Tasks: Abstract Argumentation, Form

In his seminal paper that inaugurated abstract argumentation, Dung proved
that the set of complete extensions forms a complete semilattice with respect
to set inclusion. In this note we demonstrate that this proof is incorrect with
counterexamples. We then trace the error in the proof and explain why it arose.
We then examine the implications for the grounded extension.
  [Reason for withdrawal continued] Page 4, Example 2 is not a counterexample
to Dung 1995 Theorem 25(3). It was believed to be a counter-example because the
author misunderstood ``glb'' to be set-theoretic intersection. But in this
case, ``glb'' is defined to be other than set-theoretic intersection such that
Theorem 25(3) is true.
  The author was motivated to fully understand the lattice-theoretic claims of
Dung 1995 in writing this note and was not aware that this issue is probably
folklore; the author bears full responsibility for this error.
Rotational Unit of Memory
Authors: Rumen Dangovski, Li Jing, Marin Soljacic
Date: 2017-10-26 00:00:00
Proceeding: ICLR 2018 1
Tasks: Language Modeling, Language Modelling, Machine Translation, Memorization, Question Answering, speech-recognition, Speech Recognition

The concepts of unitary evolution matrices and associative memory have
boosted the field of Recurrent Neural Networks (RNN) to state-of-the-art
performance in a variety of sequential tasks. However, RNN still have a limited
capacity to manipulate long-term memory. To bypass this weakness the most
successful applications of RNN use external techniques such as attention
mechanisms. In this paper we propose a novel RNN model that unifies the
state-of-the-art approaches: Rotational Unit of Memory (RUM). The core of RUM
is its rotational operation, which is, naturally, a unitary matrix, providing
architectures with the power to learn long-term dependencies by overcoming the
vanishing and exploding gradients problem. Moreover, the rotational unit also
serves as associative memory. We evaluate our model on synthetic memorization,
question answering and language modeling tasks. RUM learns the Copying Memory
task completely and improves the state-of-the-art result in the Recall task.
RUM's performance in the bAbI Question Answering task is comparable to that of
models with attention mechanism. We also improve the state-of-the-art result to
1.189 bits-per-character (BPC) loss in the Character Level Penn Treebank (PTB)
task, which is to signify the applications of RUM to real-world sequential
data. The universality of our construction, at the core of RNN, establishes RUM
as a promising approach to language modeling, speech recognition and machine
translation.
Laplacian Prior Variational Automatic Relevance Determination for Transmission Tomography
Authors: Jingwei Lu, David G. Politte, Joseph A. O'Sullivan
Date: 2017-10-26 00:00:00

In the classic sparsity-driven problems, the fundamental L-1 penalty method
has been shown to have good performance in reconstructing signals for a wide
range of problems. However this performance relies on a good choice of penalty
weight which is often found from empirical experiments. We propose an algorithm
called the Laplacian variational automatic relevance determination (Lap-VARD)
that takes this penalty weight as a parameter of a prior Laplace distribution.
Optimization of this parameter using an automatic relevance determination
framework results in a balance between the sparsity and accuracy of signal
reconstruction. Our algorithm is implemented in a transmission tomography model
with sparsity constraint in wavelet domain.
Soft Methodology for Cost-and-error Sensitive Classification
Authors: Te-Kang Jan, Da-Wei Wang, Chi-Hung Lin, Hsuan-Tien Lin
Date: 2017-10-26 00:00:00
Tasks: Classification, General Classification

Many real-world data mining applications need varying cost for different
types of classification errors and thus call for cost-sensitive classification
algorithms. Existing algorithms for cost-sensitive classification are
successful in terms of minimizing the cost, but can result in a high error rate
as the trade-off. The high error rate holds back the practical use of those
algorithms. In this paper, we propose a novel cost-sensitive classification
methodology that takes both the cost and the error rate into account. The
methodology, called soft cost-sensitive classification, is established from a
multicriteria optimization problem of the cost and the error rate, and can be
viewed as regularizing cost-sensitive classification with the error rate. The
simple methodology allows immediate improvements of existing cost-sensitive
classification algorithms. Experiments on the benchmark and the real-world data
sets show that our proposed methodology indeed achieves lower test error rates
and similar (sometimes lower) test costs than existing cost-sensitive
classification algorithms. We also demonstrate that the methodology can be
extended for considering the weighted error rate instead of the original error
rate. This extension is useful for tackling unbalanced classification problems.
Reparameterizing the Birkhoff Polytope for Variational Permutation Inference
Authors: Scott W. Linderman, Gonzalo E. Mena, Hal Cooper, Liam Paninski, John P. Cunningham
Date: 2017-10-26 00:00:00
Tasks: Bayesian Inference, Combinatorial Optimization, Variational Inference

Many matching, tracking, sorting, and ranking problems require probabilistic
reasoning about possible permutations, a set that grows factorially with
dimension. Combinatorial optimization algorithms may enable efficient point
estimation, but fully Bayesian inference poses a severe challenge in this
high-dimensional, discrete space. To surmount this challenge, we start with the
usual step of relaxing a discrete set (here, of permutation matrices) to its
convex hull, which here is the Birkhoff polytope: the set of all
doubly-stochastic matrices. We then introduce two novel transformations: first,
an invertible and differentiable stick-breaking procedure that maps
unconstrained space to the Birkhoff polytope; second, a map that rounds points
toward the vertices of the polytope. Both transformations include a temperature
parameter that, in the limit, concentrates the densities on permutation
matrices. We then exploit these transformations and reparameterization
gradients to introduce variational inference over permutation matrices, and we
demonstrate its utility in a series of experiments.
Knowledge Projection for Deep Neural Networks
Authors: Zhi Zhang, Guanghan Ning, Zhihai He
Date: 2017-10-26 00:00:00

While deeper and wider neural networks are actively pushing the performance
limits of various computer vision and machine learning tasks, they often
require large sets of labeled data for effective training and suffer from
extremely high computational complexity. In this paper, we will develop a new
framework for training deep neural networks on datasets with limited labeled
samples using cross-network knowledge projection which is able to improve the
network performance while reducing the overall computational complexity
significantly. Specifically, a large pre-trained teacher network is used to
observe samples from the training data. A projection matrix is learned to
project this teacher-level knowledge and its visual representations from an
intermediate layer of the teacher network to an intermediate layer of a thinner
and faster student network to guide and regulate its training process. Both the
intermediate layers from the teacher network and the injection layers from the
student network are adaptively selected during training by evaluating a joint
loss function in an iterative manner. This knowledge projection framework
allows us to use crucial knowledge learned by large networks to guide the
training of thinner student networks, avoiding over-fitting, achieving better
network performance, and significantly reducing the complexity. Extensive
experimental results on benchmark datasets have demonstrated that our proposed
knowledge projection approach outperforms existing methods, improving accuracy
by up to 4% while reducing network complexity by 4 to 10 times, which is very
attractive for practical applications of deep neural networks.
Hierarchical compositional feature learning
Authors: Miguel Lázaro-Gredilla, Yi Liu, D. Scott Phoenix, Dileep George
Date: 2016-11-07 00:00:00

We introduce the hierarchical compositional network (HCN), a directed
generative model able to discover and disentangle, without supervision, the
building blocks of a set of binary images. The building blocks are binary
features defined hierarchically as a composition of some of the features in the
layer immediately below, arranged in a particular manner. At a high level, HCN
is similar to a sigmoid belief network with pooling. Inference and learning in
HCN are very challenging and existing variational approximations do not work
satisfactorily. A main contribution of this work is to show that both can be
addressed using max-product message passing (MPMP) with a particular schedule
(no EM required). Also, using MPMP as an inference engine for HCN makes new
tasks simple: adding supervision information, classifying images, or performing
inpainting all correspond to clamping some variables of the model to their
known values and running MPMP on the rest. When used for classification, fast
inference with HCN has exactly the same functional form as a convolutional
neural network (CNN) with linear activations and binary weights. However, HCN's
features are qualitatively very different.
Complete 3D Scene Parsing from an RGBD Image
Authors: Chuhang Zou, Ruiqi Guo, Zhizhong Li, Derek Hoiem
Date: 2017-10-25 00:00:00
Tasks: Diversity, Retrieval, Scene Parsing, Visual Reasoning

One major goal of vision is to infer physical models of objects, surfaces,
and their layout from sensors. In this paper, we aim to interpret indoor scenes
from one RGBD image. Our representation encodes the layout of orthogonal walls
and the extent of objects, modeled with CAD-like 3D shapes. We parse both the
visible and occluded portions of the scene and all observable objects,
producing a complete 3D parse. Such a scene interpretation is useful for
robotics and visual reasoning, but difficult to produce due to the well-known
challenge of segmentation, the high degree of occlusion, and the diversity of
objects in indoor scenes. We take a data-driven approach, generating sets of
potential object regions, matching to regions in training images, and
transferring and aligning associated 3D models while encouraging fit to
observations and spatial consistency. We use support inference to aid
interpretation and propose a retrieval scheme that uses convolutional neural
networks (CNNs) to classify regions and retrieve objects with similar shapes.
We demonstrate the performance of our method on our newly annotated NYUd v2
dataset with detailed 3D shapes.
Dynamic Evaluation of Neural Sequence Models
Authors: Ben Krause, Emmanuel Kahembwe, Iain Murray, Steve Renals
Date: 2017-09-21 00:00:00
Proceeding: ICML 2018 7
Tasks: Language Modelling

We present methodology for using dynamic evaluation to improve neural
sequence models. Models are adapted to recent history via a gradient descent
based mechanism, causing them to assign higher probabilities to re-occurring
sequential patterns. Dynamic evaluation outperforms existing adaptation
approaches in our comparisons. Dynamic evaluation improves the state-of-the-art
word-level perplexities on the Penn Treebank and WikiText-2 datasets to 51.1
and 44.3 respectively, and the state-of-the-art character-level cross-entropies
on the text8 and Hutter Prize datasets to 1.19 bits/char and 1.08 bits/char
respectively.