pwc-archive/papers-with-abstracts huggingface dataset:
Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inference
Authors: Jeffrey L. McKinstry, Steven K. Esser, Rathinakumar Appuswamy, Deepika Bablani, John V. Arthur, Izzet B. Yildiz, Dharmendra S. Modha
Date: 2018-09-11 00:00:00
Proceeding: ICLR 2019 5
Tasks: General Classification, Quantization

To realize the promise of ubiquitous embedded deep network inference, it is
essential to seek limits of energy and area efficiency. To this end,
low-precision networks offer tremendous promise because both energy and area
scale down quadratically with the reduction in precision. Here we demonstrate
ResNet-18, -34, -50, -152, Inception-v3, Densenet-161, and VGG-16bn networks on
the ImageNet classification benchmark that, at 8-bit precision exceed the
accuracy of the full-precision baseline networks after one epoch of finetuning,
thereby leveraging the availability of pretrained models. We also demonstrate
ResNet-18, -34, -50, -152, Densenet-161, and VGG-16bn 4-bit models that match
the accuracy of the full-precision baseline networks -- the highest scores to
date. Surprisingly, the weights of the low-precision networks are very close
(in cosine similarity) to the weights of the corresponding baseline networks,
making training from scratch unnecessary.
  We find that gradient noise due to quantization during training increases
with reduced precision, and seek ways to overcome this noise. The number of
iterations required by SGD to achieve a given training error is related to the
square of (a) the distance of the initial solution from the final plus (b) the
maximum variance of the gradient estimates. Therefore, we (a) reduce solution
distance by starting with pretrained fp32 precision baseline networks and
fine-tuning, and (b) combat gradient noise introduced by quantization by
training longer and reducing learning rates. Sensitivity analysis indicates
that these simple techniques, coupled with proper activation function range
calibration to take full advantage of the limited precision, are sufficient to
discover low-precision networks, if they exist, close to fp32 precision
baseline networks. The results herein provide evidence that 4-bits suffice for
classification.
Layerwise Perturbation-Based Adversarial Training for Hard Drive Health Degree Prediction
Authors: Jian-Guo Zhang, Ji Wang, Lifang He, Zhao Li, Philip S. Yu
Date: 2018-09-11 00:00:00
Tasks: Anomaly Detection, Cloud Computing

With the development of cloud computing and big data, the reliability of data
storage systems becomes increasingly important. Previous researchers have shown
that machine learning algorithms based on SMART attributes are effective
methods to predict hard drive failures. In this paper, we use SMART attributes
to predict hard drive health degrees which are helpful for taking different
fault tolerant actions in advance. Given the highly imbalanced SMART datasets,
it is a nontrivial work to predict the health degree precisely. The proposed
model would encounter overfitting and biased fitting problems if it is trained
by the traditional methods. In order to resolve this problem, we propose two
strategies to better utilize imbalanced data and improve performance. Firstly,
we design a layerwise perturbation-based adversarial training method which can
add perturbations to any layers of a neural network to improve the
generalization of the network. Secondly, we extend the training method to the
semi-supervised settings. Then, it is possible to utilize unlabeled data that
have a potential of failure to further improve the performance of the model.
Our extensive experiments on two real-world hard drive datasets demonstrate the
superiority of the proposed schemes for both supervised and semi-supervised
classification. The model trained by the proposed method can correctly predict
the hard drive health status 5 and 15 days in advance. Finally, we verify the
generality of the proposed training method in other similar anomaly detection
tasks where the dataset is imbalanced. The results argue that the proposed
methods are applicable to other domains.
Fourier-Domain Optimization for Image Processing
Authors: Majed El Helou, Frederike Dümbgen, Radhakrishna Achanta, Sabine Süsstrunk
Date: 2018-09-11 00:00:00
Tasks: Deblurring, Image Matting, Reflection Removal

Image optimization problems encompass many applications such as spectral
fusion, deblurring, deconvolution, dehazing, matting, reflection removal and
image interpolation, among others. With current image sizes in the order of
megabytes, it is extremely expensive to run conventional algorithms such as
gradient descent, making them unfavorable especially when closed-form solutions
can be derived and computed efficiently. This paper explains in detail the
framework for solving convex image optimization and deconvolution in the
Fourier domain. We begin by explaining the mathematical background and
motivating why the presented setups can be transformed and solved very
efficiently in the Fourier domain. We also show how to practically use these
solutions, by providing the corresponding implementations. The explanations are
aimed at a broad audience with minimal knowledge of convolution and image
optimization. The eager reader can jump to Section 3 for a footprint of how to
solve and implement a sample optimization function, and Section 5 for the more
complex cases.
Deep Micro-Dictionary Learning and Coding Network
Authors: Hao Tang, Heng Wei, Wei Xiao, Wei Wang, Dan Xu, Yan Yan, Nicu Sebe
Date: 2018-09-11 00:00:00
Tasks: Dictionary Learning

In this paper, we propose a novel Deep Micro-Dictionary Learning and Coding
Network (DDLCN). DDLCN has most of the standard deep learning layers (pooling,
fully, connected, input/output, etc.) but the main difference is that the
fundamental convolutional layers are replaced by novel compound dictionary
learning and coding layers. The dictionary learning layer learns an
over-complete dictionary for the input training data. At the deep coding layer,
a locality constraint is added to guarantee that the activated dictionary bases
are close to each other. Next, the activated dictionary atoms are assembled
together and passed to the next compound dictionary learning and coding layers.
In this way, the activated atoms in the first layer can be represented by the
deeper atoms in the second dictionary. Intuitively, the second dictionary is
designed to learn the fine-grained components which are shared among the input
dictionary atoms. In this way, a more informative and discriminative low-level
representation of the dictionary atoms can be obtained. We empirically compare
the proposed DDLCN with several dictionary learning methods and deep learning
architectures. The experimental results on four popular benchmark datasets
demonstrate that the proposed DDLCN achieves competitive results compared with
state-of-the-art approaches.
Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
Authors: Liang-Chieh Chen, Maxwell D. Collins, Yukun Zhu, George Papandreou, Barret Zoph, Florian Schroff, Hartwig Adam, Jonathon Shlens
Date: 2018-09-11 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: image-classification, Image Classification, Image Segmentation, Meta-Learning, Open-Ended Question Answering, Scene Parsing, Segmentation, Semantic Segmentation, Street Scene Parsing

The design of neural network architectures is an important component for
achieving state-of-the-art performance with machine learning systems across a
broad array of tasks. Much work has endeavored to design and build
architectures automatically through clever construction of a search space
paired with simple learning algorithms. Recent progress has demonstrated that
such meta-learning methods may exceed scalable human-invented architectures on
image classification tasks. An open question is the degree to which such
methods may generalize to new domains. In this work we explore the construction
of meta-learning techniques for dense image prediction focused on the tasks of
scene parsing, person-part segmentation, and semantic image segmentation.
Constructing viable search spaces in this domain is challenging because of the
multi-scale representation of visual information and the necessity to operate
on high resolution imagery. Based on a survey of techniques in dense image
prediction, we construct a recursive search space and demonstrate that even
with efficient random search, we can identify architectures that outperform
human-invented architectures and achieve state-of-the-art performance on three
dense prediction tasks including 82.7\% on Cityscapes (street scene parsing),
71.3\% on PASCAL-Person-Part (person-part segmentation), and 87.9\% on PASCAL
VOC 2012 (semantic image segmentation). Additionally, the resulting
architecture is more computationally efficient, requiring half the parameters
and half the computational cost as previous state of the art systems.
Iterative Segmentation from Limited Training Data: Applications to Congenital Heart Disease
Authors: Danielle F. Pace, Adrian V. Dalca, Tom Brosch, Tal Geva, Andrew J. Powell, Jürgen Weese, Mehdi H. Moghari, Polina Golland
Date: 2018-09-11 00:00:00
Tasks: Segmentation

We propose a new iterative segmentation model which can be accurately learned
from a small dataset. A common approach is to train a model to directly segment
an image, requiring a large collection of manually annotated images to capture
the anatomical variability in a cohort. In contrast, we develop a segmentation
model that recursively evolves a segmentation in several steps, and implement
it as a recurrent neural network. We learn model parameters by optimizing the
interme- diate steps of the evolution in addition to the final segmentation. To
this end, we train our segmentation propagation model by presenting incom-
plete and/or inaccurate input segmentations paired with a recommended next
step. Our work aims to alleviate challenges in segmenting heart structures from
cardiac MRI for patients with congenital heart disease (CHD), which encompasses
a range of morphological deformations and topological changes. We demonstrate
the advantages of this approach on a dataset of 20 images from CHD patients,
learning a model that accurately segments individual heart chambers and great
vessels. Com- pared to direct segmentation, the iterative method yields more
accurate segmentation for patients with the most severe CHD malformations.
What can linguistics and deep learning contribute to each other?
Authors: Tal Linzen
Date: 2018-09-11 00:00:00
Tasks: Deep Learning, Language Acquisition, Sentence

Joe Pater's target article calls for greater interaction between neural
network research and linguistics. I expand on this call and show how such
interaction can benefit both fields. Linguists can contribute to research on
neural networks for language technologies by clearly delineating the linguistic
capabilities that can be expected of such systems, and by constructing
controlled experimental paradigms that can determine whether those desiderata
have been met. In the other direction, neural networks can benefit the
scientific study of language by providing infrastructure for modeling human
sentence processing and for evaluating the necessity of particular innate
constraints on language acquisition.
Phaseless Subspace Tracking
Authors: Seyedehsara Nayer, Namrata Vaswani
Date: 2018-09-11 00:00:00
Tasks: Retrieval

This work takes the first steps towards solving the "phaseless subspace
tracking" (PST) problem. PST involves recovering a time sequence of signals (or
images) from phaseless linear projections of each signal under the following
structural assumption: the signal sequence is generated from a much lower
dimensional subspace (than the signal dimension) and this subspace can change
over time, albeit gradually. It can be simply understood as a dynamic
(time-varying subspace) extension of the low-rank phase retrieval problem
studied in recent work.
Leabra7: a Python package for modeling recurrent, biologically-realistic neural networks
Authors: C. Daniel Greenidge, Noam Miller, Kenneth A. Norman
Date: 2018-09-11 00:00:00
Tasks: BIG-bench Machine Learning

Emergent is a software package that uses the AdEx neural dynamics model and
LEABRA learning algorithm to simulate and train arbitrary recurrent neural
network architectures in a biologically-realistic manner. We present Leabra7, a
complementary Python library that implements these same algorithms. Leabra7 is
developed and distributed using modern software development principles, and
integrates tightly with Python's scientific stack. We demonstrate recurrent
Leabra7 networks using traditional pattern-association tasks and a standard
machine learning task, classifying the IRIS dataset.
Adversarial Propagation and Zero-Shot Cross-Lingual Transfer of Word Vector Specialization
Authors: Edoardo Maria Ponti, Ivan Vulić, Goran Glavaš, Nikola Mrkšić, Anna Korhonen
Date: 2018-09-11 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Cross-Lingual Transfer, dialog state tracking, Lexical Simplification, Word Similarity, Zero-Shot Cross-Lingual Transfer

Semantic specialization is the process of fine-tuning pre-trained
distributional word vectors using external lexical knowledge (e.g., WordNet) to
accentuate a particular semantic relation in the specialized vector space.
While post-processing specialization methods are applicable to arbitrary
distributional vectors, they are limited to updating only the vectors of words
occurring in external lexicons (i.e., seen words), leaving the vectors of all
other words unchanged. We propose a novel approach to specializing the full
distributional vocabulary. Our adversarial post-specialization method
propagates the external lexical knowledge to the full distributional space. We
exploit words seen in the resources as training examples for learning a global
specialization function. This function is learned by combining a standard
L2-distance loss with an adversarial loss: the adversarial component produces
more realistic output vectors. We show the effectiveness and robustness of the
proposed method across three languages and on three tasks: word similarity,
dialog state tracking, and lexical simplification. We report consistent
improvements over distributional word vectors and vectors specialized by other
state-of-the-art specialization frameworks. Finally, we also propose a
cross-lingual transfer method for zero-shot specialization which successfully
specializes a full target distributional space without any lexical knowledge in
the target language and without any bilingual data.
Heated-Up Softmax Embedding
Authors: Xu Zhang, Felix Xinnan Yu, Svebor Karaman, Wei zhang, Shih-Fu Chang
Date: 2018-09-11 00:00:00
Proceeding: ICLR 2019 5
Tasks: Metric Learning

Metric learning aims at learning a distance which is consistent with the
semantic meaning of the samples. The problem is generally solved by learning an
embedding for each sample such that the embeddings of samples of the same
category are compact while the embeddings of samples of different categories
are spread-out in the feature space. We study the features extracted from the
second last layer of a deep neural network based classifier trained with the
cross entropy loss on top of the softmax layer. We show that training
classifiers with different temperature values of softmax function leads to
features with different levels of compactness. Leveraging these insights, we
propose a "heating-up" strategy to train a classifier with increasing
temperatures, leading the corresponding embeddings to achieve state-of-the-art
performance on a variety of metric learning benchmarks.
Intensity and Rescale Invariant Copy Move Forgery Detection Techniques
Authors: Tejas K, Swathi C, Rajesh Kumar M
Date: 2018-09-11 00:00:00

In this contemporary world digital media such as videos and images behave as
an active medium to carry valuable information across the globe on all fronts.
However there are several techniques evolved to tamper the image which has made
their authenticity untrustworthy. CopyMove Forgery CMF is one of the most
common forgeries present in an image where a cluster of pixels are duplicated
in the same image with potential postprocessing techniques. Various
state-of-art techniques are developed in the recent years which are effective
in detecting passive image forgery. However most methods do fail when the
copied image is rescaled or added with certain intensity before being pasted
due to de-synchronization of pixels in the searching process. To tackle this
problem the paper proposes distinct novel algorithms which recognize a unique
approach of using Hus invariant moments and Discreet Cosine Transformations DCT
to attain the desired rescale invariant and intensity invariant CMF detection
techniques respectively. The experiments conducted quantitatively and
qualitatively demonstrate the effectiveness of the algorithm.
End-to-end Image Captioning Exploits Multimodal Distributional Similarity
Authors: Pranava Madhyastha, Josiah Wang, Lucia Specia
Date: 2018-09-11 00:00:00
Tasks: Image Captioning, Text Generation

We hypothesize that end-to-end neural image captioning systems work seemingly
well because they exploit and learn `distributional similarity' in a multimodal
feature space by mapping a test image to similar training images in this space
and generating a caption from the same space. To validate our hypothesis, we
focus on the `image' side of image captioning, and vary the input image
representation but keep the RNN text generation component of a CNN-RNN model
constant. Our analysis indicates that image captioning models (i) are capable
of separating structure from noisy input representations; (ii) suffer virtually
no significant performance loss when a high dimensional representation is
compressed to a lower dimensional space; (iii) cluster images with similar
visual and linguistic information together. Our findings indicate that our
distributional similarity hypothesis holds. We conclude that regardless of the
image representation used image captioning systems seem to match images and
generate captions in a learned joint image-text semantic subspace.
JigsawNet: Shredded Image Reassembly using Convolutional Neural Network and Loop-based Composition
Authors: Canyu Le, Xin Li
Date: 2018-09-11 00:00:00

This paper proposes a novel algorithm to reassemble an arbitrarily shredded
image to its original status. Existing reassembly pipelines commonly consist of
a local matching stage and a global compositions stage. In the local stage, a
key challenge in fragment reassembly is to reliably compute and identify
correct pairwise matching, for which most existing algorithms use handcrafted
features, and hence, cannot reliably handle complicated puzzles. We build a
deep convolutional neural network to detect the compatibility of a pairwise
stitching, and use it to prune computed pairwise matches. To improve the
network efficiency and accuracy, we transfer the calculation of CNN to the
stitching region and apply a boost training strategy. In the global composition
stage, we modify the commonly adopted greedy edge selection strategies to two
new loop closure based searching algorithms. Extensive experiments show that
our algorithm significantly outperforms existing methods on solving various
puzzles, especially those challenging ones with many fragment pieces.
Randomized Wagering Mechanisms
Authors: Yiling Chen, Yang Liu, Juntao Wang
Date: 2018-09-11 00:00:00
Tasks: Learning with noisy labels

Wagering mechanisms are one-shot betting mechanisms that elicit agents' predictions of an event. For deterministic wagering mechanisms, an existing impossibility result has shown incompatibility of some desirable theoretical properties. In particular, Pareto optimality (no profitable side bet before allocation) can not be achieved together with weak incentive compatibility, weak budget balance and individual rationality. In this paper, we expand the design space of wagering mechanisms to allow randomization and ask whether there are randomized wagering mechanisms that can achieve all previously considered desirable properties, including Pareto optimality. We answer this question positively with two classes of randomized wagering mechanisms: i) one simple randomized lottery-type implementation of existing deterministic wagering mechanisms, and ii) another family of simple and randomized wagering mechanisms which we call surrogate wagering mechanisms, which are robust to noisy ground truth. This family of mechanisms builds on the idea of learning with noisy labels (Natarajan et al. 2013) as well as a recent extension of this idea to the information elicitation without verification setting (Liu and Chen 2018). We show that a broad family of randomized wagering mechanisms satisfy all desirable theoretical properties.
Simultaneous Localization and Layout Model Selection in Manhattan Worlds
Authors: Armon Shariati, Bernd Pfrommer, Camillo J. Taylor
Date: 2018-09-11 00:00:00
Tasks: Model Selection, Simultaneous Localization and Mapping

In this paper, we will demonstrate how Manhattan structure can be exploited
to transform the Simultaneous Localization and Mapping (SLAM) problem, which is
typically solved by a nonlinear optimization over feature positions, into a
model selection problem solved by a convex optimization over higher order
layout structures, namely walls, floors, and ceilings. Furthermore, we show how
our novel formulation leads to an optimization procedure that automatically
performs data association and loop closure and which ultimately produces the
simplest model of the environment that is consistent with the available
measurements. We verify our method on real world data sets collected with
various sensing modalities.
Magnetically Guided Capsule Endoscopy
Authors: Thomas Kruezer
Date: 2018-09-11 00:00:00

The following research undertakes a historical review of this technology with
specific highlighting of its advancement in medical diagnostics as well as the
therapeutic functionality of wireless capsule endoscopy. Without restriction to
the gastrointestinal tract alone, the review will additionally investigate the
developments in the technology of micro-robots guided through the magnetic
power and are capable of navigating through multiple forms of air and fluid
filled lumina as well as cavities within the body. All these capabilities are
of use in the utilization of minimally invasive medicine.
On learning an interpreted language with recurrent models
Authors: Denis Paperno
Date: 2018-09-11 00:00:00
Proceeding: WS 2018 11

Can recurrent neural nets, inspired by human sequential data processing, learn to understand language? We construct simplified datasets reflecting core properties of natural language as modeled in formal syntax and semantics: recursive syntactic structure and compositionality. We find LSTM and GRU networks to generalise to compositional interpretation well, but only in the most favorable learning settings, with a well-paced curriculum, extensive training data, and left-to-right (but not right-to-left) composition.