pwc-archive/papers-with-abstracts huggingface dataset:
Sparsity-based Defense against Adversarial Attacks on Linear Classifiers
Authors: Zhinus Marzi, Soorya Gopalakrishnan, Upamanyu Madhow, Ramtin Pedarsani
Date: 2018-01-15 00:00:00

Deep neural networks represent the state of the art in machine learning in a
growing number of fields, including vision, speech and natural language
processing. However, recent work raises important questions about the
robustness of such architectures, by showing that it is possible to induce
classification errors through tiny, almost imperceptible, perturbations.
Vulnerability to such "adversarial attacks", or "adversarial examples", has
been conjectured to be due to the excessive linearity of deep networks. In this
paper, we study this phenomenon in the setting of a linear classifier, and show
that it is possible to exploit sparsity in natural data to combat
$\ell_{\infty}$-bounded adversarial perturbations. Specifically, we demonstrate
the efficacy of a sparsifying front end via an ensemble averaged analysis, and
experimental results for the MNIST handwritten digit database. To the best of
our knowledge, this is the first work to show that sparsity provides a
theoretically rigorous framework for defense against adversarial attacks.
Convergent Actor-Critic Algorithms Under Off-Policy Training and Function Approximation
Authors: Hamid Reza Maei
Date: 2018-02-21 00:00:00
Tasks: Reinforcement Learning

We present the first class of policy-gradient algorithms that work with both
state-value and policy function-approximation, and are guaranteed to converge
under off-policy training. Our solution targets problems in reinforcement
learning where the action representation adds to the-curse-of-dimensionality;
that is, with continuous or large action sets, thus making it infeasible to
estimate state-action value functions (Q functions). Using state-value
functions helps to lift the curse and as a result naturally turn our
policy-gradient solution into classical Actor-Critic architecture whose Actor
uses state-value function for the update. Our algorithms, Gradient Actor-Critic
and Emphatic Actor-Critic, are derived based on the exact gradient of averaged
state-value function objective and thus are guaranteed to converge to its
optimal solution, while maintaining all the desirable properties of classical
Actor-Critic methods with no additional hyper-parameters. To our knowledge,
this is the first time that convergent off-policy learning methods have been
extended to classical Actor-Critic methods with function approximation.
A Radically New Theory of how the Brain Represents and Computes with Probabilities
Authors: Gerard Rinkus
Date: 2017-01-26 00:00:00

The brain is believed to implement probabilistic reasoning and to represent
information via population, or distributed, coding. Most previous
population-based probabilistic (PPC) theories share several basic properties:
1) continuous-valued neurons; 2) fully(densely)-distributed codes, i.e.,
all(most) units participate in every code; 3) graded synapses; 4) rate coding;
5) units have innate unimodal tuning functions (TFs); 6) intrinsically noisy
units; and 7) noise/correlation is considered harmful. We present a radically
different theory that assumes: 1) binary units; 2) only a small subset of
units, i.e., a sparse distributed representation (SDR) (cell assembly),
comprises any individual code; 3) binary synapses; 4) signaling formally
requires only single (i.e., first) spikes; 5) units initially have completely
flat TFs (all weights zero); 6) units are far less intrinsically noisy than
traditionally thought; rather 7) noise is a resource generated/used to cause
similar inputs to map to similar codes, controlling a tradeoff between storage
capacity and embedding the input space statistics in the pattern of
intersections over stored codes, epiphenomenally determining correlation
patterns across neurons. The theory, Sparsey, was introduced 20+ years ago as a
canonical cortical circuit/algorithm model achieving efficient sequence
learning/recognition, but not elaborated as an alternative to PPC theories.
Here, we show that: a) the active SDR simultaneously represents both the most
similar/likely input and the entire (coarsely-ranked) similarity
likelihood/distribution over all stored inputs (hypotheses); and b) given an
input, the SDR code selection algorithm, which underlies both learning and
inference, updates both the most likely hypothesis and the entire likelihood
distribution (cf. belief update) with a number of steps that remains constant
as the number of stored items increases.
Robust Locally-Linear Controllable Embedding
Authors: Ershad Banijamali, Rui Shu, Mohammad Ghavamzadeh, Hung Bui, Ali Ghodsi
Date: 2017-10-15 00:00:00

Embed-to-control (E2C) is a model for solving high-dimensional optimal
control problems by combining variational auto-encoders with locally-optimal
controllers. However, the E2C model suffers from two major drawbacks: 1) its
objective function does not correspond to the likelihood of the data sequence
and 2) the variational encoder used for embedding typically has large
variational approximation error, especially when there is noise in the system
dynamics. In this paper, we present a new model for learning robust
locally-linear controllable embedding (RCE). Our model directly estimates the
predictive conditional density of the future observation given the current one,
while introducing the bottleneck between the current and future observations.
Although the bottleneck provides a natural embedding candidate for control, our
RCE model introduces additional specific structures in the generative graphical
model so that the model dynamics can be robustly linearized. We also propose a
principled variational approximation of the embedding posterior that takes the
future observation into account, and thus, makes the variational approximation
more robust against the noise. Experimental results show that RCE outperforms
the E2C model, and does so significantly when the underlying dynamics is noisy.
CoVeR: Learning Covariate-Specific Vector Representations with Tensor Decompositions
Authors: Kevin Tian, Teng Zhang, James Zou
Date: 2018-02-21 00:00:00
Proceeding: ICML 2018 7
Tasks: Natural Questions, Tensor Decomposition

Word embedding is a useful approach to capture co-occurrence structures in
large text corpora. However, in addition to the text data itself, we often have
additional covariates associated with individual corpus documents---e.g. the
demographic of the author, time and venue of publication---and we would like
the embedding to naturally capture this information. We propose CoVeR, a new
tensor decomposition model for vector embeddings with covariates. CoVeR jointly
learns a \emph{base} embedding for all the words as well as a weighted diagonal
matrix to model how each covariate affects the base embedding. To obtain author
or venue-specific embedding, for example, we can then simply multiply the base
embedding by the associated transformation matrix. The main advantages of our
approach are data efficiency and interpretability of the covariate
transformation. Our experiments demonstrate that our joint model learns
substantially better covariate-specific embeddings compared to the standard
approach of learning a separate embedding for each covariate using only the
relevant subset of data, as well as other related methods. Furthermore, CoVeR
encourages the embeddings to be "topic-aligned" in that the dimensions have
specific independent meanings. This allows our covariate-specific embeddings to
be compared by topic, enabling downstream differential analysis. We empirically
evaluate the benefits of our algorithm on datasets, and demonstrate how it can
be used to address many natural questions about covariate effects.
  Accompanying code to this paper can be found at
http://github.com/kjtian/CoVeR.
Learning to Gather without Communication
Authors: El Mahdi El Mhamdi, Rachid Guerraoui, Alexandre Maurer, Vladislav Tempez
Date: 2018-02-21 00:00:00
Tasks: Multi-agent Reinforcement Learning, Position, Reinforcement Learning

A standard belief on emerging collective behavior is that it emerges from
simple individual rules. Most of the mathematical research on such collective
behavior starts from imperative individual rules, like always go to the center.
But how could an (optimal) individual rule emerge during a short period within
the group lifetime, especially if communication is not available. We argue that
such rules can actually emerge in a group in a short span of time via
collective (multi-agent) reinforcement learning, i.e learning via rewards and
punishments. We consider the gathering problem: several agents (social animals,
swarming robots...) must gather around a same position, which is not determined
in advance. They must do so without communication on their planned decision,
just by looking at the position of other agents. We present the first
experimental evidence that a gathering behavior can be learned without
communication in a partially observable environment. The learned behavior has
the same properties as a self-stabilizing distributed algorithm, as processes
can gather from any initial state (and thus tolerate any transient failure).
Besides, we show that it is possible to tolerate the brutal loss of up to 90\%
of agents without significant impact on the behavior.
BridgeNets: Student-Teacher Transfer Learning Based on Recursive Neural Networks and its Application to Distant Speech Recognition
Authors: Jaeyoung Kim, Mostafa El-Khamy, Jungwon Lee
Date: 2017-10-27 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Denoising, Distant Speech Recognition, speech-recognition, Speech Recognition, Transfer Learning

Despite the remarkable progress achieved on automatic speech recognition,
recognizing far-field speeches mixed with various noise sources is still a
challenging task. In this paper, we introduce novel student-teacher transfer
learning, BridgeNet which can provide a solution to improve distant speech
recognition. There are two key features in BridgeNet. First, BridgeNet extends
traditional student-teacher frameworks by providing multiple hints from a
teacher network. Hints are not limited to the soft labels from a teacher
network. Teacher's intermediate feature representations can better guide a
student network to learn how to denoise or dereverberate noisy input. Second,
the proposed recursive architecture in the BridgeNet can iteratively improve
denoising and recognition performance. The experimental results of BridgeNet
showed significant improvements in tackling the distant speech recognition
problem, where it achieved up to 13.24% relative WER reductions on AMI corpus
compared to a baseline neural network without teacher's hints.
Manipulating and Measuring Model Interpretability
Authors: Forough Poursabzi-Sangdeh, Daniel G. Goldstein, Jake M. Hofman, Jennifer Wortman Vaughan, Hanna Wallach
Date: 2018-02-21 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Interpretable Machine Learning, model

With machine learning models being increasingly used to aid decision making even in high-stakes domains, there has been a growing interest in developing interpretable models. Although many supposedly interpretable models have been proposed, there have been relatively few experimental studies investigating whether these models achieve their intended effects, such as making people more closely follow a model's predictions when it is beneficial for them to do so or enabling them to detect when a model has made a mistake. We present a sequence of pre-registered experiments (N=3,800) in which we showed participants functionally identical models that varied only in two factors commonly thought to make machine learning models more or less interpretable: the number of features and the transparency of the model (i.e., whether the model internals are clear or black box). Predictably, participants who saw a clear model with few features could better simulate the model's predictions. However, we did not find that participants more closely followed its predictions. Furthermore, showing participants a clear model meant that they were less able to detect and correct for the model's sizable mistakes, seemingly due to information overload. These counterintuitive findings emphasize the importance of testing over intuition when developing interpretable models.
Low complexity convolutional neural network for vessel segmentation in portable retinal diagnostic devices
Authors: M. Hajabdollahi, R. Esfandiarpoor, S. M. R. Soroushmehr, N. Karimi, S. Samavi, K. Najarian
Date: 2018-02-21 00:00:00
Tasks: Diagnostic, General Classification, Quantization, Retinal Vessel Segmentation, Segmentation

Retinal vessel information is helpful in retinal disease screening and
diagnosis. Retinal vessel segmentation provides useful information about
vessels and can be used by physicians during intraocular surgery and retinal
diagnostic operations. Convolutional neural networks (CNNs) are powerful tools
for classification and segmentation of medical images. Complexity of CNNs makes
it difficult to implement them in portable devices such as binocular indirect
ophthalmoscopes. In this paper a simplification approach is proposed for CNNs
based on combination of quantization and pruning. Fully connected layers are
quantized and convolutional layers are pruned to have a simple and efficient
network structure. Experiments on images of the STARE dataset show that our
simplified network is able to segment retinal vessels with acceptable accuracy
and low complexity.
Boundary-Seeking Generative Adversarial Networks
Authors: R. Devon Hjelm, Athul Paul Jacob, Tong Che, Adam Trischler, Kyunghyun Cho, Yoshua Bengio
Date: 2017-02-27 00:00:00
Tasks: Scene Understanding, Text Generation

Generative adversarial networks (GANs) are a learning framework that rely on
training a discriminator to estimate a measure of difference between a target
and generated distributions. GANs, as normally formulated, rely on the
generated samples being completely differentiable w.r.t. the generative
parameters, and thus do not work for discrete data. We introduce a method for
training GANs with discrete data that uses the estimated difference measure
from the discriminator to compute importance weights for generated samples,
thus providing a policy gradient for training the generator. The importance
weights have a strong connection to the decision boundary of the discriminator,
and we call our method boundary-seeking GANs (BGANs). We demonstrate the
effectiveness of the proposed algorithm with discrete image and character-based
natural language generation. In addition, the boundary-seeking objective
extends to continuous data, which can be used to improve stability of training,
and we demonstrate this on Celeba, Large-scale Scene Understanding (LSUN)
bedrooms, and Imagenet without conditioning.
Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networks
Authors: Yu-Gang Jiang, Zuxuan Wu, Jun Wang, xiangyang xue, Shih-Fu Chang
Date: 2015-02-25 00:00:00
Tasks: GPU

In this paper, we study the challenging problem of categorizing videos
according to high-level semantics such as the existence of a particular human
action or a complex event. Although extensive efforts have been devoted in
recent years, most existing works combined multiple video features using simple
fusion strategies and neglected the utilization of inter-class semantic
relationships. This paper proposes a novel unified framework that jointly
exploits the feature relationships and the class relationships for improved
categorization performance. Specifically, these two types of relationships are
estimated and utilized by rigorously imposing regularizations in the learning
process of a deep neural network (DNN). Such a regularized DNN (rDNN) can be
efficiently realized using a GPU-based implementation with an affordable
training cost. Through arming the DNN with better capability of harnessing both
the feature and the class relationships, the proposed rDNN is more suitable for
modeling video semantics. With extensive experimental evaluations, we show that
rDNN produces superior performance over several state-of-the-art approaches. On
the well-known Hollywood2 and Columbia Consumer Video benchmarks, we obtain
very competitive results: 66.9\% and 73.5\% respectively in terms of mean
average precision. In addition, to substantially evaluate our rDNN and
stimulate future research on large scale video categorization, we collect and
release a new benchmark dataset, called FCVID, which contains 91,223 Internet
videos and 239 manually annotated categories.
Liver Segmentation in Abdominal CT Images by Adaptive 3D Region Growing
Authors: Shima Rafiei, Nader Karimi, Behzad Mirmahboub, S. M. Reza Soroushmehr, Banafsheh Felfelian, Shadrokh Samavi, Kayvan Najarian
Date: 2018-02-21 00:00:00
Tasks: Liver Segmentation, Segmentation

Automatic liver segmentation plays an important role in computer-aided
diagnosis and treatment. Manual segmentation of organs is a difficult and
tedious task and so prone to human errors. In this paper, we propose an
adaptive 3D region growing with subject-specific conditions. For this aim we
use the intensity distribution of most probable voxels in prior map along with
location prior. We also incorporate the boundary of target organs to restrict
the region growing. In order to obtain strong edges and high contrast, we
propose an effective contrast enhancement algorithm to facilitate more accurate
segmentation. In this paper, 92.56% Dice score is achieved. We compare our
method with the method of hard thresholding on Deeds prior map and also with
the majority voting on Deeds registration with 13 organs.
Semantic Segmentation Refinement by Monte Carlo Region Growing of High Confidence Detections
Authors: Philipe A. Dias, Henry Medeiros
Date: 2018-02-21 00:00:00
Tasks: Segmentation, Semantic Segmentation, Vocal Bursts Intensity Prediction

Despite recent improvements using fully convolutional networks, in general,
the segmentation produced by most state-of-the-art semantic segmentation
methods does not show satisfactory adherence to the object boundaries. We
propose a method to refine the segmentation results generated by such deep
learning models. Our method takes as input the confidence scores generated by a
pixel-dense segmentation network and re-labels pixels with low confidence
levels. The re-labeling approach employs a region growing mechanism that
aggregates these pixels to neighboring areas with high confidence scores and
similar appearance. In order to correct the labels of pixels that were
incorrectly classified with high confidence level by the semantic segmentation
algorithm, we generate multiple region growing steps through a Monte Carlo
sampling of the seeds of the regions. Our method improves the accuracy of a
state-of-the-art fully convolutional semantic segmentation approach on the
publicly available COCO and PASCAL datasets, and it shows significantly better
results on selected sequences of the finely-annotated DAVIS dataset.
Segmentation of Bleeding Regions in Wireless Capsule Endoscopy Images an Approach for inside Capsule Video Summarization
Authors: Mohsen Hajabdollahi, Reza Esfandiarpoor, S. M. Reza Soroushmehr, Nader Karimi, Shadrokh Samavi, Kayvan Najarian
Date: 2018-02-21 00:00:00
Tasks: Video Summarization

Wireless capsule endoscopy (WCE) is an effective means of diagnosis of
gastrointestinal disorders. Detection of informative scenes by WCE could reduce
the length of transmitted videos and can help with the diagnosis. In this paper
we propose a simple and efficient method for segmentation of the bleeding
regions in WCE captured images. Suitable color channels are selected and
classified by a multi-layer perceptron (MLP) structure. The MLP structure is
quantized such that the implementation does not require multiplications. The
proposed method is tested by simulation on WCE bleeding image dataset. The
proposed structure is designed considering hardware resource constrains that
exist in WCE systems.
Reversible Image Watermarking for Health Informatics Systems Using Distortion Compensation in Wavelet Domain
Authors: Hamidreza Zarrabi, Mohsen Hajabdollahi, S. M. Reza Soroushmehr, Nader Karimi, Shadrokh Samavi, Kayvan Najarian
Date: 2018-02-21 00:00:00

Reversible image watermarking guaranties restoration of both original cover
and watermark logo from the watermarked image. Capacity and distortion of the
image under reversible watermarking are two important parameters. In this study
a reversible watermarking is investigated with focusing on increasing the
embedding capacity and reducing the distortion in medical images. Integer
wavelet transform is used for embedding where in each iteration, one watermark
bit is embedded in one transform coefficient. We devise a novel approach that
when a coefficient is modified in an iteration, the produced distortion is
compensated in the next iteration. This distortion compensation method would
result in low distortion rate. The proposed method is tested on four types of
medical images including MRI of brain, cardiac MRI, MRI of breast, and
intestinal polyp images. Using a one-level wavelet transform, maximum capacity
of 1.5 BPP is obtained. Experimental results demonstrate that the proposed
method is superior to the state-of-the-art works in terms of capacity and
distortion.
Artificial Intelligence and Legal Liability
Authors: John Kingston
Date: 2018-02-21 00:00:00

A recent issue of a popular computing journal asked which laws would apply if
a self-driving car killed a pedestrian. This paper considers the question of
legal liability for artificially intelligent computer systems. It discusses
whether criminal liability could ever apply; to whom it might apply; and, under
civil law, whether an AI program is a product that is subject to product design
legislation or a service to which the tort of negligence applies. The issue of
sales warranties is also considered. A discussion of some of the practical
limitations that AI systems are subject to is also included.
Lossless Image Compression Algorithm for Wireless Capsule Endoscopy by Content-Based Classification of Image Blocks
Authors: Atefe Rajaeefar, Ali Emami, S. M. Reza Soroushmehr, Nader Karimi, Shadrokh Samavi, Kayvan Najarian
Date: 2018-02-21 00:00:00
Tasks: General Classification, Image Compression

Recent advances in capsule endoscopy systems have introduced new methods and
capabilities. The capsule endoscopy system, by observing the entire digestive
tract, has significantly improved diagnosing gastrointestinal disorders and
diseases. The system has challenges such as the need to enhance the quality of
the transmitted images, low frame rates of transmission, and battery lifetime
that need to be addressed. One of the important parts of a capsule endoscopy
system is the image compression unit. Better compression of images increases
the frame rate and hence improves the diagnosis process. In this paper a high
precision compression algorithm with high compression ratio is proposed. In
this algorithm we use the similarity between frames to compress the data more
efficiently.