pwc-archive/papers-with-abstracts huggingface dataset:
Semi-supervised Learning with Sparse Autoencoders in Phone Classification
Authors: Akash Kumar Dhaka, Giampiero Salvi
Date: 2016-10-03 00:00:00
Tasks: Acoustic Modelling, Automatic Speech Recognition, Automatic Speech Recognition (ASR), Classification, General Classification, speech-recognition, Speech Recognition

We propose the application of a semi-supervised learning method to improve
the performance of acoustic modelling for automatic speech recognition based on
deep neural net- works. As opposed to unsupervised initialisation followed by
supervised fine tuning, our method takes advantage of both unlabelled and
labelled data simultaneously through mini- batch stochastic gradient descent.
We tested the method with varying proportions of labelled vs unlabelled
observations in frame-based phoneme classification on the TIMIT database. Our
experiments show that the method outperforms standard supervised training for
an equal amount of labelled data and provides competitive error rates compared
to state-of-the-art graph-based semi-supervised learning techniques.
Can Evolutionary Sampling Improve Bagged Ensembles?
Authors: Harsh Nisar, Bhanu Pratap Singh Rawat
Date: 2016-10-03 00:00:00
Tasks: Evolutionary Algorithms

Perturb and Combine (P&C) group of methods generate multiple versions of the
predictor by perturbing the training set or construction and then combining
them into a single predictor (Breiman, 1996b). The motive is to improve the
accuracy in unstable classification and regression methods. One of the most
well known method in this group is Bagging. Arcing or Adaptive Resampling and
Combining methods like AdaBoost are smarter variants of P&C methods. In this
extended abstract, we lay the groundwork for a new family of methods under the
P&C umbrella, known as Evolutionary Sampling (ES). We employ Evolutionary
algorithms to suggest smarter sampling in both the feature space (sub-spaces)
as well as training samples. We discuss multiple fitness functions to assess
ensembles and empirically compare our performance against randomized sampling
of training data and feature sub-spaces.
Multiscale Segmentation via Bregman Distances and Nonlinear Spectral Analysis
Authors: Leonie Zeune, Guus van Dalum, Leon W. M. M. Terstappen, S. A. van Gils, Christoph Brune
Date: 2016-04-22 00:00:00
Tasks: Medical Diagnosis, Segmentation

In biomedical imaging reliable segmentation of objects (e.g. from small cells
up to large organs) is of fundamental importance for automated medical
diagnosis. New approaches for multi-scale segmentation can considerably improve
performance in case of natural variations in intensity, size and shape. This
paper aims at segmenting objects of interest based on shape contours and
automatically finding multiple objects with different scales. The overall
strategy of this work is to combine nonlinear segmentation with scales spaces
and spectral decompositions recently introduced in literature. For this we
generalize a variational segmentation model based on total variation using
Bregman distances to construct an inverse scale space. This offers the new
model to be accomplished by a scale analysis approach based on a spectral
decomposition of the total variation. As a result we obtain a very efficient,
(nearly) parameter-free multiscale segmentation method that comes with an
adaptive regularization parameter choice. The added benefit of our method is
demonstrated by systematic synthetic tests and its usage in a new biomedical
toolbox for identifying and classifying circulating tumor cells. Due to the
nature of nonlinear diffusion underlying, the mathematical concepts in this
work offer promising extensions to nonlocal classification problems.
Should Algorithms for Random SAT and Max-SAT be Different?
Authors: Sixue Liu, Gerard de Melo
Date: 2016-10-03 00:00:00

We analyze to what extent the random SAT and Max-SAT problems differ in their
properties. Our findings suggest that for random $k$-CNF with ratio in a
certain range, Max-SAT can be solved by any SAT algorithm with subexponential
slowdown, while for formulae with ratios greater than some constant, algorithms
under the random walk framework require substantially different heuristics. In
light of these results, we propose a novel probabilistic approach for random
Max-SAT called ProMS. Experimental results illustrate that ProMS outperforms
many state-of-the-art local search solvers on random Max-SAT benchmarks.
Rain structure transfer using an exemplar rain image for synthetic rain image generation
Authors: Chang-Hwan Son, Xiao-Ping Zhang
Date: 2016-10-03 00:00:00
Tasks: Image Generation, Rain Removal

This letter proposes a simple method of transferring rain structures of a
given exemplar rain image into a target image. Given the exemplar rain image
and its corresponding masked rain image, rain patches including rain structures
are extracted randomly, and then residual rain patches are obtained by
subtracting those rain patches from their mean patches. Next, residual rain
patches are selected randomly, and then added to the given target image along a
raster scanning direction. To decrease boundary artifacts around the added
patches on the target image, minimum error boundary cuts are found using
dynamic programming, and then blending is conducted between overlapping
patches. Our experiment shows that the proposed method can generate realistic
rain images that have similar rain structures in the exemplar images. Moreover,
it is expected that the proposed method can be used for rain removal. More
specifically, natural images and synthetic rain images generated via the
proposed method can be used to learn classifiers, for example, deep neural
networks, in a supervised manner.
On the Empirical Effect of Gaussian Noise in Under-sampled MRI Reconstruction
Authors: Patrick Virtue, Michael Lustig
Date: 2016-10-03 00:00:00
Tasks: Image Reconstruction, MRI Reconstruction, Prediction

In Fourier-based medical imaging, sampling below the Nyquist rate results in
an underdetermined system, in which linear reconstructions will exhibit
artifacts. Another consequence of under-sampling is lower signal to noise ratio
(SNR) due to fewer acquired measurements. Even if an oracle provided the
information to perfectly disambiguate the underdetermined system, the
reconstructed image could still have lower image quality than a corresponding
fully sampled acquisition because of the reduced measurement time. The effects
of lower SNR and the underdetermined system are coupled during reconstruction,
making it difficult to isolate the impact of lower SNR on image quality. To
this end, we present an image quality prediction process that reconstructs
fully sampled, fully determined data with noise added to simulate the loss of
SNR induced by a given under-sampling pattern. The resulting prediction image
empirically shows the effect of noise in under-sampled image reconstruction
without any effect from an underdetermined system.
  We discuss how our image quality prediction process can simulate the
distribution of noise for a given under-sampling pattern, including variable
density sampling that produces colored noise in the measurement data. An
interesting consequence of our prediction model is that we can show that
recovery from underdetermined non-uniform sampling is equivalent to a weighted
least squares optimization that accounts for heterogeneous noise levels across
measurements.
  Through a series of experiments with synthetic and in vivo datasets, we
demonstrate the efficacy of the image quality prediction process and show that
it provides a better estimation of reconstruction image quality than the
corresponding fully-sampled reference image.
Seeing into Darkness: Scotopic Visual Recognition
Authors: Bo Chen, Pietro Perona
Date: 2016-10-03 00:00:00
Proceeding: CVPR 2017 7
Tasks: Astronomy, General Classification

Images are formed by counting how many photons traveling from a given set of
directions hit an image sensor during a given time interval. When photons are
few and far in between, the concept of `image' breaks down and it is best to
consider directly the flow of photons. Computer vision in this regime, which we
call `scotopic', is radically different from the classical image-based paradigm
in that visual computations (classification, control, search) have to take
place while the stream of photons is captured and decisions may be taken as
soon as enough information is available. The scotopic regime is important for
biomedical imaging, security, astronomy and many other fields. Here we develop
a framework that allows a machine to classify objects with as few photons as
possible, while maintaining the error rate below an acceptable threshold. A
dynamic and asymptotically optimal speed-accuracy tradeoff is a key feature of
this framework. We propose and study an algorithm to optimize the tradeoff of a
convolutional network directly from lowlight images and evaluate on simulated
images from standard datasets. Surprisingly, scotopic systems can achieve
comparable classification performance as traditional vision systems while using
less than 0.1% of the photons in a conventional image. In addition, we
demonstrate that our algorithms work even when the illuminance of the
environment is unknown and varying. Last, we outline a spiking neural network
coupled with photon-counting sensors as a power-efficient hardware realization
of scotopic algorithms.
Stealing Machine Learning Models via Prediction APIs
Authors: Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, Thomas Ristenpart
Date: 2016-09-09 00:00:00
Tasks: BIG-bench Machine Learning, Learning Theory, Model extraction, Prediction

Machine learning (ML) models may be deemed confidential due to their
sensitive training data, commercial value, or use in security applications.
Increasingly often, confidential ML models are being deployed with publicly
accessible query interfaces. ML-as-a-service ("predictive analytics") systems
are an example: Some allow users to train models on potentially sensitive data
and charge others for access on a pay-per-query basis.
  The tension between model confidentiality and public access motivates our
investigation of model extraction attacks. In such attacks, an adversary with
black-box access, but no prior knowledge of an ML model's parameters or
training data, aims to duplicate the functionality of (i.e., "steal") the
model. Unlike in classical learning theory settings, ML-as-a-service offerings
may accept partial feature vectors as inputs and include confidence values with
predictions. Given these practices, we show simple, efficient attacks that
extract target ML models with near-perfect fidelity for popular model classes
including logistic regression, neural networks, and decision trees. We
demonstrate these attacks against the online services of BigML and Amazon
Machine Learning. We further show that the natural countermeasure of omitting
confidence values from model outputs still admits potentially harmful model
extraction attacks. Our results highlight the need for careful ML model
deployment and new model extraction countermeasures.
Rain Removal via Shrinkage-Based Sparse Coding and Learned Rain Dictionary
Authors: Chang-Hwan Son, Xiao-Ping Zhang
Date: 2016-10-03 00:00:00
Tasks: Dictionary Learning, Image Restoration, Rain Removal

This paper introduces a new rain removal model based on the shrinkage of the
sparse codes for a single image. Recently, dictionary learning and sparse
coding have been widely used for image restoration problems. These methods can
also be applied to the rain removal by learning two types of rain and non-rain
dictionaries and forcing the sparse codes of the rain dictionary to be zero
vectors. However, this approach can generate unwanted edge artifacts and detail
loss in the non-rain regions. Based on this observation, a new approach for
shrinking the sparse codes is presented in this paper. To effectively shrink
the sparse codes in the rain and non-rain regions, an error map between the
input rain image and the reconstructed rain image is generated by using the
learned rain dictionary. Based on this error map, both the sparse codes of rain
and non-rain dictionaries are used jointly to represent the image structures of
objects and avoid the edge artifacts in the non-rain regions. In the rain
regions, the correlation matrix between the rain and non-rain dictionaries is
calculated. Then, the sparse codes corresponding to the highly correlated
signal-atoms in the rain and non-rain dictionaries are shrunk jointly to
improve the removal of the rain structures. The experimental results show that
the proposed shrinkage-based sparse coding can preserve image structures and
avoid the edge artifacts in the non-rain regions, and it can remove the rain
structures in the rain regions. Also, visual quality evaluation confirms that
the proposed method outperforms the conventional texture and rain removal
methods.
Much Ado About Time: Exhaustive Annotation of Temporal Data
Authors: Gunnar A. Sigurdsson, Olga Russakovsky, Ali Farhadi, Ivan Laptev, Abhinav Gupta
Date: 2016-07-25 00:00:00

Large-scale annotated datasets allow AI systems to learn from and build upon
the knowledge of the crowd. Many crowdsourcing techniques have been developed
for collecting image annotations. These techniques often implicitly rely on the
fact that a new input image takes a negligible amount of time to perceive. In
contrast, we investigate and determine the most cost-effective way of obtaining
high-quality multi-label annotations for temporal data such as videos. Watching
even a short 30-second video clip requires a significant time investment from a
crowd worker; thus, requesting multiple annotations following a single viewing
is an important cost-saving strategy. But how many questions should we ask per
video? We conclude that the optimal strategy is to ask as many questions as
possible in a HIT (up to 52 binary questions after watching a 30-second video
clip in our experiments). We demonstrate that while workers may not correctly
answer all questions, the cost-benefit analysis nevertheless favors consensus
from multiple such cheap-yet-imperfect iterations over more complex
alternatives. When compared with a one-question-per-video baseline, our method
is able to achieve a 10% improvement in recall 76.7% ours versus 66.7%
baseline) at comparable precision (83.8% ours versus 83.0% baseline) in about
half the annotation time (3.8 minutes ours compared to 7.1 minutes baseline).
We demonstrate the effectiveness of our method by collecting multi-label
annotations of 157 human activities on 1,815 videos.
Near-Infrared Coloring via a Contrast-Preserving Mapping Model
Authors: Chang-Hwan Son, Xiao-Ping Zhang
Date: 2016-10-03 00:00:00
Tasks: Denoising, Image Restoration

Near-infrared gray images captured together with corresponding visible color
images have recently proven useful for image restoration and classification.
This paper introduces a new coloring method to add colors to near-infrared gray
images based on a contrast-preserving mapping model. A naive coloring method
directly adds the colors from the visible color image to the near-infrared gray
image; however, this method results in an unrealistic image because of the
discrepancies in brightness and image structure between the captured
near-infrared gray image and the visible color image. To solve the discrepancy
problem, first we present a new contrast-preserving mapping model to create a
new near-infrared gray image with a similar appearance in the luminance plane
to the visible color image, while preserving the contrast and details of the
captured near-infrared gray image. Then based on the proposed
contrast-preserving mapping model, we develop a method to derive realistic
colors that can be added to the newly created near-infrared gray image.
Experimental results show that the proposed method can not only preserve the
local contrasts and details of the captured near-infrared gray image, but
transfers the realistic colors from the visible color image to the newly
created near-infrared gray image. Experimental results also show that the
proposed approach can be applied to near-infrared denoising.
Funneled Bayesian Optimization for Design, Tuning and Control of Autonomous Systems
Authors: Ruben Martinez-Cantin
Date: 2016-10-02 00:00:00
Tasks: Bayesian Optimization, Experimental Design, global-optimization, regression

Bayesian optimization has become a fundamental global optimization algorithm
in many problems where sample efficiency is of paramount importance. Recently,
there has been proposed a large number of new applications in fields such as
robotics, machine learning, experimental design, simulation, etc. In this
paper, we focus on several problems that appear in robotics and autonomous
systems: algorithm tuning, automatic control and intelligent design. All those
problems can be mapped to global optimization problems. However, they become
hard optimization problems. Bayesian optimization internally uses a
probabilistic surrogate model (e.g.: Gaussian process) to learn from the
process and reduce the number of samples required. In order to generalize to
unknown functions in a black-box fashion, the common assumption is that the
underlying function can be modeled with a stationary process. Nonstationary
Gaussian process regression cannot generalize easily and it typically requires
prior knowledge of the function. Some works have designed techniques to
generalize Bayesian optimization to nonstationary functions in an indirect way,
but using techniques originally designed for regression, where the objective is
to improve the quality of the surrogate model everywhere. Instead optimization
should focus on improving the surrogate model near the optimum. In this paper,
we present a novel kernel function specially designed for Bayesian
optimization, that allows nonstationary behavior of the surrogate model in an
adaptive local region. In our experiments, we found that this new kernel
results in an improved local search (exploitation), without penalizing the
global search (exploration). We provide results in well-known benchmarks and
real applications. The new method outperforms the state of the art in Bayesian
optimization both in stationary and nonstationary problems.
An Empirical Evaluation of various Deep Learning Architectures for Bi-Sequence Classification Tasks
Authors: Anirban Laha, Vikas Raykar
Date: 2016-07-17 00:00:00
Proceeding: COLING 2016 12
Tasks: Classification, Deep Learning, General Classification, Natural Language Inference, Question Answering

Several tasks in argumentation mining and debating, question-answering, and
natural language inference involve classifying a sequence in the context of
another sequence (referred as bi-sequence classification). For several single
sequence classification tasks, the current state-of-the-art approaches are
based on recurrent and convolutional neural networks. On the other hand, for
bi-sequence classification problems, there is not much understanding as to the
best deep learning architecture. In this paper, we attempt to get an
understanding of this category of problems by extensive empirical evaluation of
19 different deep learning architectures (specifically on different ways of
handling context) for various problems originating in natural language
processing like debating, textual entailment and question-answering. Following
the empirical evaluation, we offer our insights and conclusions regarding the
architectures we have considered. We also establish the first deep learning
baselines for three argumentation mining tasks.
Accelerating Deep Convolutional Networks using low-precision and sparsity
Authors: Ganesh Venkatesh, Eriko Nurvitadhi, Debbie Marr
Date: 2016-10-02 00:00:00
Tasks: General Classification

We explore techniques to significantly improve the compute efficiency and
performance of Deep Convolution Networks without impacting their accuracy. To
improve the compute efficiency, we focus on achieving high accuracy with
extremely low-precision (2-bit) weight networks, and to accelerate the
execution time, we aggressively skip operations on zero-values. We achieve the
highest reported accuracy of 76.6% Top-1/93% Top-5 on the Imagenet object
classification challenge with low-precision network\footnote{github release of
the source code coming soon} while reducing the compute requirement by ~3x
compared to a full-precision network that achieves similar accuracy.
Furthermore, to fully exploit the benefits of our low-precision networks, we
build a deep learning accelerator core, dLAC, that can achieve up to 1
TFLOP/mm^2 equivalent for single-precision floating-point operations (~2
TFLOP/mm^2 for half-precision).
Low-dose CT denoising with convolutional neural network
Authors: Hu Chen, Yi Zhang, Weihua Zhang, Peixi Liao, Ke Li, Jiliu Zhou, Ge Wang
Date: 2016-10-02 00:00:00
Tasks: Denoising

To reduce the potential radiation risk, low-dose CT has attracted much
attention. However, simply lowering the radiation dose will lead to significant
deterioration of the image quality. In this paper, we propose a noise reduction
method for low-dose CT via deep neural network without accessing original
projection data. A deep convolutional neural network is trained to transform
low-dose CT images towards normal-dose CT images, patch by patch. Visual and
quantitative evaluation demonstrates a competing performance of the proposed
method.
Stacked Autoencoders for Medical Image Search
Authors: S. Sharma, I. Umar, L. Ospina, D. Wong, H. R. Tizhoosh
Date: 2016-10-02 00:00:00
Tasks: Content-Based Image Retrieval, Image Retrieval, Medical Diagnosis, Retrieval

Medical images can be a valuable resource for reliable information to support
medical diagnosis. However, the large volume of medical images makes it
challenging to retrieve relevant information given a particular scenario. To
solve this challenge, content-based image retrieval (CBIR) attempts to
characterize images (or image regions) with invariant content information in
order to facilitate image search. This work presents a feature extraction
technique for medical images using stacked autoencoders, which encode images to
binary vectors. The technique is applied to the IRMA dataset, a collection of
14,410 x-ray images in order to demonstrate the ability of autoencoders to
retrieve similar x-rays given test queries. Using IRMA dataset as a benchmark,
it was found that stacked autoencoders gave excellent results with a retrieval
error of 376 for 1,733 test images with a compression of 74.61%.
MinMax Radon Barcodes for Medical Image Retrieval
Authors: H. R. Tizhoosh, Shujin Zhu, Hanson Lo, Varun Chaudhari, Tahmid Mehdi
Date: 2016-10-02 00:00:00
Tasks: Diagnostic, Image Retrieval, Medical Image Retrieval, Retrieval, TAG

Content-based medical image retrieval can support diagnostic decisions by
clinical experts. Examining similar images may provide clues to the expert to
remove uncertainties in his/her final diagnosis. Beyond conventional feature
descriptors, binary features in different ways have been recently proposed to
encode the image content. A recent proposal is "Radon barcodes" that employ
binarized Radon projections to tag/annotate medical images with content-based
binary vectors, called barcodes. In this paper, MinMax Radon barcodes are
introduced which are superior to "local thresholding" scheme suggested in the
literature. Using IRMA dataset with 14,410 x-ray images from 193 different
classes, the advantage of using MinMax Radon barcodes over \emph{thresholded}
Radon barcodes are demonstrated. The retrieval error for direct search drops by
more than 15\%. As well, SURF, as a well-established non-binary approach, and
BRISK, as a recent binary method are examined to compare their results with
MinMax Radon barcodes when retrieving images from IRMA dataset. The results
demonstrate that MinMax Radon barcodes are faster and more accurate when
applied on IRMA images.