pwc-archive/papers-with-abstracts huggingface dataset:
Quantization Error as a Metric for Dynamic Precision Scaling in Neural Net Training
Authors: Ian Taras, Dylan Malone Stuart
Date: 2018-01-25 00:00:00
Tasks: Quantization

Recent work has explored reduced numerical precision for parameters,
activations, and gradients during neural network training as a way to reduce
the computational cost of training (Na & Mukhopadhyay, 2016) (Courbariaux et
al., 2014). We present a novel dynamic precision scaling (DPS) scheme. Using
stochastic fixed-point rounding, a quantization-error based scaling scheme, and
dynamic bit-widths during training, we achieve 98.8% test accuracy on the MNIST
dataset using an average bit-width of just 16 bits for weights and 14 bits for
activations, compared to the standard 32-bit floating point values used in deep
learning frameworks.
JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services
Authors: Amir Erfan Eshratifar, Mohammad Saeed Abrishami, Massoud Pedram
Date: 2018-01-25 00:00:00
Tasks: Cloud Computing

Deep learning models are being deployed in many mobile intelligent applications. End-side services, such as intelligent personal assistants, autonomous cars, and smart home services often employ either simple local models on the mobile or complex remote models on the cloud. However, recent studies have shown that partitioning the DNN computations between the mobile and cloud can increase the latency and energy efficiencies. In this paper, we propose an efficient, adaptive, and practical engine, JointDNN, for collaborative computation between a mobile device and cloud for DNNs in both inference and training phase. JointDNN not only provides an energy and performance efficient method of querying DNNs for the mobile side but also benefits the cloud server by reducing the amount of its workload and communications compared to the cloud-only approach. Given the DNN architecture, we investigate the efficiency of processing some layers on the mobile device and some layers on the cloud server. We provide optimization formulations at layer granularity for forward- and backward-propagations in DNNs, which can adapt to mobile battery limitations and cloud server load constraints and quality of service. JointDNN achieves up to 18 and 32 times reductions on the latency and mobile energy consumption of querying DNNs compared to the status-quo approaches, respectively.
DeepPap: Deep Convolutional Networks for Cervical Cell Classification
Authors: Ling Zhang, Le Lu, Isabella Nogues, Ronald M. Summers, Shaoxiong Liu, Jianhua Yao
Date: 2018-01-25 00:00:00
Tasks: Classification, General Classification, Specificity

Automation-assisted cervical screening via Pap smear or liquid-based cytology
(LBC) is a highly effective cell imaging based cancer detection tool, where
cells are partitioned into "abnormal" and "normal" categories. However, the
success of most traditional classification methods relies on the presence of
accurate cell segmentations. Despite sixty years of research in this field,
accurate segmentation remains a challenge in the presence of cell clusters and
pathologies. Moreover, previous classification methods are only built upon the
extraction of hand-crafted features, such as morphology and texture. This paper
addresses these limitations by proposing a method to directly classify cervical
cells - without prior segmentation - based on deep features, using
convolutional neural networks (ConvNets). First, the ConvNet is pre-trained on
a natural image dataset. It is subsequently fine-tuned on a cervical cell
dataset consisting of adaptively re-sampled image patches coarsely centered on
the nuclei. In the testing phase, aggregation is used to average the prediction
scores of a similar set of image patches. The proposed method is evaluated on
both Pap smear and LBC datasets. Results show that our method outperforms
previous algorithms in classification accuracy (98.3%), area under the curve
(AUC) (0.99) values, and especially specificity (98.3%), when applied to the
Herlev benchmark Pap smear dataset and evaluated using five-fold
cross-validation. Similar superior performances are also achieved on the HEMLBC
(H&E stained manual LBC) dataset. Our method is promising for the development
of automation-assisted reading systems in primary cervical screening.
Accurate Weakly Supervised Deep Lesion Segmentation on CT Scans: Self-Paced 3D Mask Generation from RECIST
Authors: Jinzheng Cai, You-Bao Tang, Le Lu, Adam P. Harrison, Ke Yan, Jing Xiao, Lin Yang, Ronald M. Summers
Date: 2018-01-25 00:00:00
Tasks: Generative Adversarial Network, Lesion Segmentation, Segmentation, Super-Resolution

Volumetric lesion segmentation via medical imaging is a powerful means to
precisely assess multiple time-point lesion/tumor changes. Because manual 3D
segmentation is prohibitively time consuming and requires radiological
experience, current practices rely on an imprecise surrogate called response
evaluation criteria in solid tumors (RECIST). Despite their coarseness, RECIST
marks are commonly found in current hospital picture and archiving systems
(PACS), meaning they can provide a potentially powerful, yet extraordinarily
challenging, source of weak supervision for full 3D segmentation. Toward this
end, we introduce a convolutional neural network based weakly supervised
self-paced segmentation (WSSS) method to 1) generate the initial lesion
segmentation on the axial RECIST-slice; 2) learn the data distribution on
RECIST-slices; 3) adapt to segment the whole volume slice by slice to finally
obtain a volumetric segmentation. In addition, we explore how super-resolution
images (2~5 times beyond the physical CT imaging), generated from a proposed
stacked generative adversarial network, can aid the WSSS performance. We employ
the DeepLesion dataset, a comprehensive CT-image lesion dataset of 32,735
PACS-bookmarked findings, which include lesions, tumors, and lymph nodes of
varying sizes, categories, body regions and surrounding contexts. These are
drawn from 10,594 studies of 4,459 patients. We also validate on a lymph-node
dataset, where 3D ground truth masks are available for all images. For the
DeepLesion dataset, we report mean Dice coefficients of 93% on RECIST-slices
and 76% in 3D lesion volumes. We further validate using a subjective user
study, where an experienced radiologist accepted our WSSS-generated lesion
segmentation results with a high probability of 92.4%.
Development of ICA and IVA Algorithms with Application to Medical Image Analysis
Authors: Zois Boukouvalas
Date: 2018-01-25 00:00:00
Tasks: Medical Image Analysis

Independent component analysis (ICA) is a widely used BSS method that can
uniquely achieve source recovery, subject to only scaling and permutation
ambiguities, through the assumption of statistical independence on the part of
the latent sources. Independent vector analysis (IVA) extends the applicability
of ICA by jointly decomposing multiple datasets through the exploitation of the
dependencies across datasets. Though both ICA and IVA algorithms cast in the
maximum likelihood (ML) framework enable the use of all available statistical
information in reality, they often deviate from their theoretical optimality
properties due to improper estimation of the probability density function
(PDF). This motivates the development of flexible ICA and IVA algorithms that
closely adhere to the underlying statistical description of the data. Although
it is attractive minimize the assumptions, important prior information about
the data, such as sparsity, is usually available. If incorporated into the ICA
model, use of this additional information can relax the independence
assumption, resulting in an improvement in the overall separation performance.
Therefore, the development of a unified mathematical framework that can take
into account both statistical independence and sparsity is of great interest.
In this work, we first introduce a flexible ICA algorithm that uses an
effective PDF estimator to accurately capture the underlying statistical
properties of the data. We then discuss several techniques to accurately
estimate the parameters of the multivariate generalized Gaussian distribution,
and how to integrate them into the IVA model. Finally, we provide a
mathematical framework that enables direct control over the influence of
statistical independence and sparsity, and use this framework to develop an
effective ICA algorithm that can jointly exploit these two forms of diversity.
Deep LOGISMOS: Deep Learning Graph-based 3D Segmentation of Pancreatic Tumors on CT scans
Authors: Zhihui Guo, Ling Zhang, Le Lu, Mohammadhadi Bagheri, Ronald M. Summers, Milan Sonka, Jianhua Yao
Date: 2018-01-25 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation, Tumor Segmentation, UNET Segmentation

This paper reports Deep LOGISMOS approach to 3D tumor segmentation by
incorporating boundary information derived from deep contextual learning to
LOGISMOS - layered optimal graph image segmentation of multiple objects and
surfaces. Accurate and reliable tumor segmentation is essential to tumor growth
analysis and treatment selection. A fully convolutional network (FCN), UNet, is
first trained using three adjacent 2D patches centered at the tumor, providing
contextual UNet segmentation and probability map for each 2D patch. The UNet
segmentation is then refined by Gaussian Mixture Model (GMM) and morphological
operations. The refined UNet segmentation is used to provide the initial shape
boundary to build a segmentation graph. The cost for each node of the graph is
determined by the UNet probability maps. Finally, a max-flow algorithm is
employed to find the globally optimal solution thus obtaining the final
segmentation. For evaluation, we applied the method to pancreatic tumor
segmentation on a dataset of 51 CT scans, among which 30 scans were used for
training and 21 for testing. With Deep LOGISMOS, DICE Similarity Coefficient
(DSC) and Relative Volume Difference (RVD) reached 83.2+-7.8% and 18.6+-17.4%
respectively, both are significantly improved (p<0.05) compared with contextual
UNet and/or LOGISMOS alone.
Quantum-Assisted Learning of Hardware-Embedded Probabilistic Graphical Models
Authors: Marcello Benedetti, John Realpe-Gómez, Rupak Biswas, Alejandro Perdomo-Ortiz
Date: 2016-09-08 00:00:00
Tasks: Benchmarking, BIG-bench Machine Learning, Probabilistic Programming

Mainstream machine-learning techniques such as deep learning and
probabilistic programming rely heavily on sampling from generally intractable
probability distributions. There is increasing interest in the potential
advantages of using quantum computing technologies as sampling engines to speed
up these tasks or to make them more effective. However, some pressing
challenges in state-of-the-art quantum annealers have to be overcome before we
can assess their actual performance. The sparse connectivity, resulting from
the local interaction between quantum bits in physical hardware
implementations, is considered the most severe limitation to the quality of
constructing powerful generative unsupervised machine-learning models. Here we
use embedding techniques to add redundancy to data sets, allowing us to
increase the modeling capacity of quantum annealers. We illustrate our findings
by training hardware-embedded graphical models on a binarized data set of
handwritten digits and two synthetic data sets in experiments with up to 940
quantum bits. Our model can be trained in quantum hardware without full
knowledge of the effective parameters specifying the corresponding quantum
Gibbs-like distribution; therefore, this approach avoids the need to infer the
effective temperature at each iteration, speeding up learning; it also
mitigates the effect of noise in the control parameters, making it robust to
deviations from the reference Gibbs distribution. Our approach demonstrates the
feasibility of using quantum annealers for implementing generative models, and
it provides a suitable framework for benchmarking these quantum technologies on
machine-learning-related tasks.
Optimal Cooperative Inference
Authors: Scott Cheng-Hsin Yang, Yue Yu, Arash Givchi, Pei Wang, Wai Keen Vong, Patrick Shafto
Date: 2017-05-24 00:00:00
Tasks: BIG-bench Machine Learning

Cooperative transmission of data fosters rapid accumulation of knowledge by
efficiently combining experiences across learners. Although well studied in
human learning and increasingly in machine learning, we lack formal frameworks
through which we may reason about the benefits and limitations of cooperative
inference. We present such a framework. We introduce novel indices for
measuring the effectiveness of probabilistic and cooperative information
transmission. We relate our indices to the well-known Teaching Dimension in
deterministic settings. We prove conditions under which optimal cooperative
inference can be achieved, including a representation theorem that constrains
the form of inductive biases for learners optimized for cooperative inference.
We conclude by demonstrating how these principles may inform the design of
machine learning algorithms and discuss implications for human and machine
learning.
Effective Building Block Design for Deep Convolutional Neural Networks using Search
Authors: Jayanta K. Dutta, Jiayi Liu, Unmesh Kurup, Mohak Shah
Date: 2018-01-25 00:00:00

Deep learning has shown promising results on many machine learning tasks but
DL models are often complex networks with large number of neurons and layers,
and recently, complex layer structures known as building blocks. Finding the
best deep model requires a combination of finding both the right architecture
and the correct set of parameters appropriate for that architecture. In
addition, this complexity (in terms of layer types, number of neurons, and
number of layers) also present problems with generalization since larger
networks are easier to overfit to the data. In this paper, we propose a search
framework for finding effective architectural building blocks for convolutional
neural networks (CNN). Our approach is much faster at finding models that are
close to state-of-the-art in performance. In addition, the models discovered by
our approach are also smaller than models discovered by similar techniques. We
achieve these twin advantages by designing our search space in such a way that
it searches over a reduced set of state-of-the-art building blocks for CNNs
including residual block, inception block, inception-residual block, ResNeXt
block and many others. We apply this technique to generate models for multiple
image datasets and show that these models achieve performance comparable to
state-of-the-art (and even surpassing the state-of-the-art in one case). We
also show that learned models are transferable between datasets.
Etymo: A New Discovery Engine for AI Research
Authors: Weijian Zhang, Jonathan Deakin, Nicholas J. Higham, Shuaiqiang Wang
Date: 2018-01-25 00:00:00
Tasks: Navigate

We present Etymo (https://etymo.io), a discovery engine to facilitate
artificial intelligence (AI) research and development. It aims to help readers
navigate a large number of AI-related papers published every week by using a
novel form of search that finds relevant papers and displays related papers in
a graphical interface. Etymo constructs and maintains an adaptive
similarity-based network of research papers as an all-purpose knowledge graph
for ranking, recommendation, and visualisation. The network is constantly
evolving and can learn from user feedback to adjust itself.
Deep Learning for End-to-End Automatic Target Recognition from Synthetic Aperture Radar Imagery
Authors: Hidetoshi Furukawa
Date: 2018-01-25 00:00:00
Tasks: General Classification

The standard architecture of synthetic aperture radar (SAR) automatic target
recognition (ATR) consists of three stages: detection, discrimination, and
classification. In recent years, convolutional neural networks (CNNs) for SAR
ATR have been proposed, but most of them classify target classes from a target
chip extracted from SAR imagery, as a classification for the third stage of SAR
ATR. In this report, we propose a novel CNN for end-to-end ATR from SAR
imagery. The CNN named verification support network (VersNet) performs all
three stages of SAR ATR end-to-end. VersNet inputs a SAR image of arbitrary
sizes with multiple classes and multiple targets, and outputs a SAR ATR image
representing the position, class, and pose of each detected target. This report
describes the evaluation results of VersNet which trained to output scores of
all 12 classes: 10 target classes, a target front class, and a background
class, for each pixel using the moving and stationary target acquisition and
recognition (MSTAR) public dataset.
Countering Adversarial Images using Input Transformations
Authors: Chuan Guo, Mayank Rana, Moustapha Cisse, Laurens van der Maaten
Date: 2017-10-31 00:00:00
Proceeding: ICLR 2018 1
Tasks: Adversarial Defense, General Classification, image-classification, Image Classification

This paper investigates strategies that defend against adversarial-example
attacks on image-classification systems by transforming the inputs before
feeding them to the system. Specifically, we study applying image
transformations such as bit-depth reduction, JPEG compression, total variance
minimization, and image quilting before feeding the image to a convolutional
network classifier. Our experiments on ImageNet show that total variance
minimization and image quilting are very effective defenses in practice, in
particular, when the network is trained on transformed images. The strength of
those defenses lies in their non-differentiable nature and their inherent
randomness, which makes it difficult for an adversary to circumvent the
defenses. Our best defense eliminates 60% of strong gray-box and 90% of strong
black-box attacks by a variety of major attack methods
HappyDB: A Corpus of 100,000 Crowdsourced Happy Moments
Authors: Akari Asai, Sara Evensen, Behzad Golshan, Alon Halevy, Vivian Li, Andrei Lopatenko, Daniela Stepanov, Yoshihiko Suhara, Wang-Chiew Tan, Yinzhan Xu
Date: 2018-01-23 00:00:00
Proceeding: LREC 2018 5
Tasks: Art Analysis

The science of happiness is an area of positive psychology concerned with
understanding what behaviors make people happy in a sustainable fashion.
Recently, there has been interest in developing technologies that help
incorporate the findings of the science of happiness into users' daily lives by
steering them towards behaviors that increase happiness. With the goal of
building technology that can understand how people express their happy moments
in text, we crowd-sourced HappyDB, a corpus of 100,000 happy moments that we
make publicly available. This paper describes HappyDB and its properties, and
outlines several important NLP problems that can be studied with the help of
the corpus. We also apply several state-of-the-art analysis techniques to
analyze HappyDB. Our results demonstrate the need for deeper NLP techniques to
be developed which makes HappyDB an exciting resource for follow-on research.
Unmixing urban hyperspectral imagery with a Gaussian mixture model on endmember variability
Authors: Yuan Zhou, Erin B. Wetherley, Paul D. Gader
Date: 2018-01-25 00:00:00

In this paper, we model a pixel as a linear combination of endmembers sampled
from probability distributions of Gaussian mixture models (GMM). The parameters
of the GMM distributions are estimated using spectral libraries. Abundances are
estimated based on the distribution parameters. The advantage of this algorithm
is that the model size grows very slowly as a function of the library size. To
validate this method, we used data collected by the AVIRIS sensor over the
Santa Barbara region: two 16 m spatial resolution and two 4 m spatial
resolution images. 64 validated regions of interest (ROI) (180 m by 180 m) were
used to assess estimate accuracy. Ground truth was obtained using 1 m images
leading to the following 6 classes: turfgrass, non-photosynthetic vegetation
(NPV), paved, roof, soil, and tree. Spectral libraries were built by manually
identifying and extracting pure spectra from both resolution images, resulting
in 3,287 spectra at 16 m and 15,426 spectra at 4 m. We then unmixed ROIs of
each resolution using the following unmixing algorithms: the set-based
algorithms MESMA and AAM, and the distribution-based algorithms GMM, NCM, and
BCM. The original libraries were used for the distribution-based algorithms
whereas set-based methods required a sophisticated reduction method, resulting
in reduced libraries of 61 spectra at 16 m and 95 spectra at 4 m. The results
show that GMM performs best among the distribution-based methods, producing
comparable accuracy to MESMA, and may be more robust across datasets.
Novel Methods for Enhancing the Performance of Genetic Algorithms
Authors: Esra'a O Alkafaween
Date: 2018-01-09 00:00:00

In this thesis we propose new methods for crossover operator namely: cut on
worst gene (COWGC), cut on worst L+R gene (COWLRGC) and Collision Crossovers.
And also we propose several types of mutation operator such as: worst gene with
random gene mutation (WGWRGM) , worst LR gene with random gene mutation
(WLRGWRGM), worst gene with worst gene mutation (WGWWGM), worst gene with
nearest neighbour mutation (WGWNNM), worst gene with the worst around the
nearest neighbour mutation (WGWWNNM), worst gene inserted beside nearest
neighbour mutation (WGIBNNM), random gene inserted beside nearest neighbour
mutation (RGIBNNM), Swap worst gene locally mutation (SWGLM), Insert best
random gene before worst gene mutation (IBRGBWGM) and Insert best random gene
before random gene mutation (IBRGBRGM). In addition to proposing four selection
strategies, namely: select any crossover (SAC), select any mutation (SAM),
select best crossover (SBC) and select best mutation (SBM). The first two are
based on selection of the best crossover and mutation operator respectively,
and the other two strategies randomly select any operator. So we investigate
the use of more than one crossover/mutation operator (based on the proposed
strategies) to enhance the performance of genetic algorithms. Our experiments,
conducted on several Travelling Salesman Problems (TSP), show the superiority
of some of the proposed methods in crossover and mutation over some of the
well-known crossover and mutation operators described in the literature. In
addition, using any of the four strategies (SAC, SAM, SBC and SBM), found to be
better than using one crossover/mutation operator in general, because those
allow the GA to avoid local optima, or the so-called premature convergence.
Keywords: GAs, Collision crossover, Multi crossovers, Multi mutations, TSP.
Self-Learning to Detect and Segment Cysts in Lung CT Images without Manual Annotation
Authors: Ling Zhang, Vissagan Gopalakrishnan, Le Lu, Ronald M. Summers, Joel Moss, Jianhua Yao
Date: 2018-01-25 00:00:00
Tasks: Image Segmentation, Lesion Detection, Medical Image Analysis, Medical Image Segmentation, Segmentation, Self-Learning, Semantic Segmentation, Weakly-supervised Learning

Image segmentation is a fundamental problem in medical image analysis. In
recent years, deep neural networks achieve impressive performances on many
medical image segmentation tasks by supervised learning on large manually
annotated data. However, expert annotations on big medical datasets are
tedious, expensive or sometimes unavailable. Weakly supervised learning could
reduce the effort for annotation but still required certain amounts of
expertise. Recently, deep learning shows a potential to produce more accurate
predictions than the original erroneous labels. Inspired by this, we introduce
a very weakly supervised learning method, for cystic lesion detection and
segmentation in lung CT images, without any manual annotation. Our method works
in a self-learning manner, where segmentation generated in previous steps
(first by unsupervised segmentation then by neural networks) is used as ground
truth for the next level of network learning. Experiments on a cystic lung
lesion dataset show that the deep learning could perform better than the
initial unsupervised annotation, and progressively improve itself after
self-learning.