pwc-archive/papers-with-abstracts huggingface dataset:
In-Place Activated BatchNorm for Memory-Optimized Training of DNNs
Authors: Samuel Rota Bulò, Lorenzo Porzi, Peter Kontschieder
Date: 2017-12-07 00:00:00
Proceeding: CVPR 2018 6
Tasks: image-classification, Image Classification, Semantic Segmentation

In this work we present In-Place Activated Batch Normalization (InPlace-ABN)
- a novel approach to drastically reduce the training memory footprint of
modern deep neural networks in a computationally efficient way. Our solution
substitutes the conventionally used succession of BatchNorm + Activation layers
with a single plugin layer, hence avoiding invasive framework surgery while
providing straightforward applicability for existing deep learning frameworks.
We obtain memory savings of up to 50% by dropping intermediate results and by
recovering required information during the backward pass through the inversion
of stored forward results, with only minor increase (0.8-2%) in computation
time. Also, we demonstrate how frequently used checkpointing approaches can be
made computationally as efficient as InPlace-ABN. In our experiments on image
classification, we demonstrate on-par results on ImageNet-1k with
state-of-the-art approaches. On the memory-demanding task of semantic
segmentation, we report results for COCO-Stuff, Cityscapes and Mapillary
Vistas, obtaining new state-of-the-art results on the latter without additional
training data but in a single-scale and -model scenario. Code can be found at
https://github.com/mapillary/inplace_abn .
On Quadratic Penalties in Elastic Weight Consolidation
Authors: Ferenc Huszár
Date: 2017-12-11 00:00:00

Elastic weight consolidation (EWC, Kirkpatrick et al, 2017) is a novel
algorithm designed to safeguard against catastrophic forgetting in neural
networks. EWC can be seen as an approximation to Laplace propagation (Eskin et
al, 2004), and this view is consistent with the motivation given by Kirkpatrick
et al (2017). In this note, I present an extended derivation that covers the
case when there are more than two tasks. I show that the quadratic penalties in
EWC are inconsistent with this derivation and might lead to double-counting
data from earlier tasks.
Tight Semi-Nonnegative Matrix Factorization
Authors: David W. Dreisigmeyer
Date: 2017-09-13 00:00:00
Tasks: Information Retrieval, Retrieval

The nonnegative matrix factorization is a widely used, flexible matrix
decomposition, finding applications in biology, image and signal processing and
information retrieval, among other areas. Here we present a related matrix
factorization. A multi-objective optimization problem finds conical
combinations of templates that approximate a given data matrix. The templates
are chosen so that as far as possible only the initial data set can be
represented this way. However, the templates are not required to be nonnegative
nor convex combinations of the original data.
Unsupervised Feature Learning for Audio Analysis
Authors: Matthias Meyer, Jan Beutel, Lothar Thiele
Date: 2017-12-11 00:00:00
Tasks: Clustering

Identifying acoustic events from a continuously streaming audio source is of
interest for many applications including environmental monitoring for basic
research. In this scenario neither different event classes are known nor what
distinguishes one class from another. Therefore, an unsupervised feature
learning method for exploration of audio data is presented in this paper. It
incorporates the two following novel contributions: First, an audio frame
predictor based on a Convolutional LSTM autoencoder is demonstrated, which is
used for unsupervised feature extraction. Second, a training method for
autoencoders is presented, which leads to distinct features by amplifying event
similarities. In comparison to standard approaches, the features extracted from
the audio frame predictor trained with the novel approach show 13 % better
results when used with a classifier and 36 % better results when used for
clustering.
Error Correction for Dense Semantic Image Labeling
Authors: Yu-Hui Huang, Xu Jia, Stamatios Georgoulis, Tinne Tuytelaars, Luc van Gool
Date: 2017-12-11 00:00:00
Tasks: Face Parsing, Semantic Segmentation

Pixelwise semantic image labeling is an important, yet challenging, task with
many applications. Typical approaches to tackle this problem involve either the
training of deep networks on vast amounts of images to directly infer the
labels or the use of probabilistic graphical models to jointly model the
dependencies of the input (i.e. images) and output (i.e. labels). Yet, the
former approaches do not capture the structure of the output labels, which is
crucial for the performance of dense labeling, and the latter rely on carefully
hand-designed priors that require costly parameter tuning via optimization
techniques, which in turn leads to long inference times. To alleviate these
restrictions, we explore how to arrive at dense semantic pixel labels given
both the input image and an initial estimate of the output labels. We propose a
parallel architecture that: 1) exploits the context information through a
LabelPropagation network to propagate correct labels from nearby pixels to
improve the object boundaries, 2) uses a LabelReplacement network to directly
replace possibly erroneous, initial labels with new ones, and 3) combines the
different intermediate results via a Fusion network to obtain the final
per-pixel label. We experimentally validate our approach on two different
datasets for the semantic segmentation and face parsing tasks respectively,
where we show improvements over the state-of-the-art. We also provide both a
quantitative and qualitative analysis of the generated results.
Exploring Approximations for Floating-Point Arithmetic using UppSAT
Authors: Aleksandar Zeljic, Peter Backeman, Christoph M. Wintersteiger, Philipp Ruemmer
Date: 2017-11-24 00:00:00

We consider the problem of solving floating-point constraints obtained from
software verification. We present UppSAT --- a new implementation of a
systematic approximation refinement framework [ZWR17] as an abstract SMT
solver. Provided with an approximation and a decision procedure (implemented in
an off-the-shelf SMT solver), UppSAT yields an approximating SMT solver.
Additionally, UppSAT includes a library of predefined approximation components
which can be combined and extended to define new encodings, orderings and
solving strategies. We propose that UppSAT can be used as a sandbox for easy
and flexible exploration of new approximations. To substantiate this, we
explore several approximations of floating-point arithmetic. Approximations can
be viewed as a composition of an encoding into a target theory, a precision
ordering, and a number of strategies for model reconstruction and precision (or
approximation) refinement. We present encodings of floating-point arithmetic
into reduced precision floating-point arithmetic, real-arithmetic, and
fixed-point arithmetic (encoded in the theory of bit-vectors). In an
experimental evaluation, we compare the advantages and disadvantages of
approximating solvers obtained by combining various encodings and decision
procedures (based on existing state-of-the-art SMT solvers for floating-point,
real, and bit-vector arithmetic).
Identifying the Mislabeled Training Samples of ECG Signals using Machine Learning
Authors: Yaoguang Li, Wei Cui, Cong Wang
Date: 2017-12-11 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification

The classification accuracy of electrocardiogram signal is often affected by
diverse factors in which mislabeled training samples issue is one of the most
influential problems. In order to mitigate this negative effect, the method of
cross validation is introduced to identify the mislabeled samples. The method
utilizes the cooperative advantages of different classifiers to act as a filter
for the training samples. The filter removes the mislabeled training samples
and retains the correctly labeled ones with the help of 10-fold cross
validation. Consequently, a new training set is provided to the final
classifiers to acquire higher classification accuracies. Finally, we
numerically show the effectiveness of the proposed method with the MIT-BIH
arrhythmia database.
A Heuristic Search Algorithm Using the Stability of Learning Algorithms in Certain Scenarios as the Fitness Function: An Artificial General Intelligence Engineering Approach
Authors: Zengkun Li
Date: 2017-12-08 00:00:00
Tasks: Heuristic Search, Meta-Learning

This paper presents a non-manual design engineering method based on heuristic
search algorithm to search for candidate agents in the solution space which
formed by artificial intelligence agents modeled on the base of
bionics.Compared with the artificial design method represented by meta-learning
and the bionics method represented by the neural architecture chip,this method
is more feasible for realizing artificial general intelligence,and it has a
much better interaction with cognitive neuroscience;at the same time,the
engineering method is based on the theoretical hypothesis that the final
learning algorithm is stable in certain scenarios,and has generalization
ability in various scenarios.The paper discusses the theory preliminarily and
proposes the possible correlation between the theory and the fixed-point
theorem in the field of mathematics.Limited by the author's knowledge
level,this correlation is proposed only as a kind of conjecture.
Learning Surrogate Models of Document Image Quality Metrics for Automated Document Image Processing
Authors: Prashant Singh, Ekta Vats, Anders Hast
Date: 2017-12-11 00:00:00
Tasks: Binarization, Hyperparameter Optimization

Computation of document image quality metrics often depends upon the
availability of a ground truth image corresponding to the document. This limits
the applicability of quality metrics in applications such as hyperparameter
optimization of image processing algorithms that operate on-the-fly on unseen
documents. This work proposes the use of surrogate models to learn the behavior
of a given document quality metric on existing datasets where ground truth
images are available. The trained surrogate model can later be used to predict
the metric value on previously unseen document images without requiring access
to ground truth images. The surrogate model is empirically evaluated on the
Document Image Binarization Competition (DIBCO) and the Handwritten Document
Image Binarization Competition (H-DIBCO) datasets.
Can We Teach Computers to Understand Art? Domain Adaptation for Enhancing Deep Networks Capacity to De-Abstract Art
Authors: Mihai Badea, Corneliu Florea, Laura Florea, Constantin Vertan
Date: 2017-12-11 00:00:00
Tasks: Domain Adaptation, Style Transfer

Humans comprehend a natural scene at a single glance; painters and other
visual artists, through their abstract representations, stressed this capacity
to the limit. The performance of computer vision solutions matched that of
humans in many problems of visual recognition. In this paper we address the
problem of recognizing the genre (subject) in digitized paintings using
Convolutional Neural Networks (CNN) as part of the more general dealing with
abstract and/or artistic representation of scenes. Initially we establish the
state of the art performance by training a CNN from scratch. In the next level
of evaluation, we identify aspects that hinder the CNNs' recognition, such as
artistic abstraction. Further, we test various domain adaptation methods that
could enhance the subject recognition capabilities of the CNNs. The evaluation
is performed on a database of 80,000 annotated digitized paintings, which is
tentatively extended with artistic photographs, either original or stylized, in
order to emulate artistic representations. Surprisingly, the most efficient
domain adaptation is not the neural style transfer. Finally, the paper provides
an experiment-based assessment of the abstraction level that CNNs are able to
achieve.
Cogniculture: Towards a Better Human-Machine Co-evolution
Authors: Rakesh R Pimplikar, Kushal Mukherjee, Gyana Parija, Harit Vishwakarma, Ramasuri Narayanam, Sarthak Ahuja, Rohith D Vallam, Ritwik Chaudhuri, Joydeep Mondal
Date: 2017-12-11 00:00:00

Research in Artificial Intelligence is breaking technology barriers every
day. New algorithms and high performance computing are making things possible
which we could only have imagined earlier. Though the enhancements in AI are
making life easier for human beings day by day, there is constant fear that AI
based systems will pose a threat to humanity. People in AI community have
diverse set of opinions regarding the pros and cons of AI mimicking human
behavior. Instead of worrying about AI advancements, we propose a novel idea of
cognitive agents, including both human and machines, living together in a
complex adaptive ecosystem, collaborating on human computation for producing
essential social goods while promoting sustenance, survival and evolution of
the agents' life cycle. We highlight several research challenges and technology
barriers in achieving this goal. We propose a governance mechanism around this
ecosystem to ensure ethical behaviors of all cognitive agents. Along with a
novel set of use-cases of Cogniculture, we discuss the road map ahead for this
journey.
A Flow Model of Neural Networks
Authors: Zhen Li, Zuoqiang Shi
Date: 2017-08-21 00:00:00
Tasks: model

Based on a natural connection between ResNet and transport equation or its
characteristic equation, we propose a continuous flow model for both ResNet and
plain net. Through this continuous model, a ResNet can be explicitly
constructed as a refinement of a plain net. The flow model provides an
alternative perspective to understand phenomena in deep neural networks, such
as why it is necessary and sufficient to use 2-layer blocks in ResNets, why
deeper is better, and why ResNets are even deeper, and so on. It also opens a
gate to bring in more tools from the huge area of differential equations.
The Effectiveness of Data Augmentation for Detection of Gastrointestinal Diseases from Endoscopical Images
Authors: Andrea Asperti, Claudio Mastronardo
Date: 2017-12-11 00:00:00
Tasks: Data Augmentation, General Classification

The lack, due to privacy concerns, of large public databases of medical
pathologies is a well-known and major problem, substantially hindering the
application of deep learning techniques in this field. In this article, we
investigate the possibility to supply to the deficiency in the number of data
by means of data augmentation techniques, working on the recent Kvasir dataset
of endoscopical images of gastrointestinal diseases. The dataset comprises
4,000 colored images labeled and verified by medical endoscopists, covering a
few common pathologies at different anatomical landmarks: Z-line, pylorus and
cecum. We show how the application of data augmentation techniques allows to
achieve sensible improvements of the classification with respect to previous
approaches, both in terms of precision and recall.
FHEDN: A based on context modeling Feature Hierarchy Encoder-Decoder Network for face detection
Authors: Zexun Zhou, Zhongshi He, Ziyu Chen, Yuanyuan Jia, HaiYan Wang, Jinglong Du, Dingding Chen
Date: 2017-12-11 00:00:00
Tasks: Decoder, Face Detection, Face Recognition

Because of affected by weather conditions, camera pose and range, etc.
Objects are usually small, blur, occluded and diverse pose in the images
gathered from outdoor surveillance cameras or access control system. It is
challenging and important to detect faces precisely for face recognition system
in the field of public security. In this paper, we design a based on context
modeling structure named Feature Hierarchy Encoder-Decoder Network for face
detection(FHEDN), which can detect small, blur and occluded face with hierarchy
by hierarchy from the end to the beginning likes encoder-decoder in a single
network. The proposed network is consist of multiple context modeling and
prediction modules, which are in order to detect small, blur, occluded and
diverse pose faces. In addition, we analyse the influence of distribution of
training set, scale of default box and receipt field size to detection
performance in implement stage. Demonstrated by experiments, Our network
achieves promising performance on WIDER FACE and FDDB benchmarks.
Reactive Multi-Context Systems: Heterogeneous Reasoning in Dynamic Environments
Authors: Gerhard Brewka, Stefan Ellmauthaler, Ricardo Gonçalves, Matthias Knorr, João Leite, Jörg Pührer
Date: 2016-09-12 00:00:00

Managed multi-context systems (mMCSs) allow for the integration of
heterogeneous knowledge sources in a modular and very general way. They were,
however, mainly designed for static scenarios and are therefore not well-suited
for dynamic environments in which continuous reasoning over such heterogeneous
knowledge with constantly arriving streams of data is necessary. In this paper,
we introduce reactive multi-context systems (rMCSs), a framework for reactive
reasoning in the presence of heterogeneous knowledge sources and data streams.
We show that rMCSs are indeed well-suited for this purpose by illustrating how
several typical problems arising in the context of stream reasoning can be
handled using them, by showing how inconsistencies possibly occurring in the
integration of multiple knowledge sources can be handled, and by arguing that
the potential non-determinism of rMCSs can be avoided if needed using an
alternative, more skeptical well-founded semantics instead with beneficial
computational properties. We also investigate the computational complexity of
various reasoning problems related to rMCSs. Finally, we discuss related work,
and show that rMCSs do not only generalize mMCSs to dynamic settings, but also
capture/extend relevant approaches w.r.t. dynamics in knowledge representation
and stream reasoning.
Deep Expander Networks: Efficient Deep Networks from Graph Theory
Authors: Ameya Prabhu, Girish Varma, Anoop Namboodiri
Date: 2017-11-23 00:00:00
Proceeding: ECCV 2018 9

Efficient CNN designs like ResNets and DenseNet were proposed to improve
accuracy vs efficiency trade-offs. They essentially increased the connectivity,
allowing efficient information flow across layers. Inspired by these
techniques, we propose to model connections between filters of a CNN using
graphs which are simultaneously sparse and well connected. Sparsity results in
efficiency while well connectedness can preserve the expressive power of the
CNNs. We use a well-studied class of graphs from theoretical computer science
that satisfies these properties known as Expander graphs. Expander graphs are
used to model connections between filters in CNNs to design networks called
X-Nets. We present two guarantees on the connectivity of X-Nets: Each node
influences every node in a layer in logarithmic steps, and the number of paths
between two sets of nodes is proportional to the product of their sizes. We
also propose efficient training and inference algorithms, making it possible to
train deeper and wider X-Nets effectively.
  Expander based models give a 4% improvement in accuracy on MobileNet over
grouped convolutions, a popular technique, which has the same sparsity but
worse connectivity. X-Nets give better performance trade-offs than the original
ResNet and DenseNet-BC architectures. We achieve model sizes comparable to
state-of-the-art pruning techniques using our simple architecture design,
without any pruning. We hope that this work motivates other approaches to
utilize results from graph theory to develop efficient network architectures.
Finite Sample Analyses for TD(0) with Function Approximation
Authors: Gal Dalal, Balázs Szörényi, Gugan Thoppe, Shie Mannor
Date: 2017-04-04 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

TD(0) is one of the most commonly used algorithms in reinforcement learning.
Despite this, there is no existing finite sample analysis for TD(0) with
function approximation, even for the linear case. Our work is the first to
provide such results. Existing convergence rates for Temporal Difference (TD)
methods apply only to somewhat modified versions, e.g., projected variants or
ones where stepsizes depend on unknown problem parameters. Our analyses obviate
these artificial alterations by exploiting strong properties of TD(0). We
provide convergence rates both in expectation and with high-probability. The
two are obtained via different approaches that use relatively unknown, recently
developed stochastic approximation techniques.
READ-BAD: A New Dataset and Evaluation Scheme for Baseline Detection in Archival Documents
Authors: Tobias Grüning, Roger Labahn, Markus Diem, Florian Kleber, Stefan Fiel
Date: 2017-05-09 00:00:00
Tasks: Binarization, Keyword Spotting, Line Detection

Text line detection is crucial for any application associated with Automatic
Text Recognition or Keyword Spotting. Modern algorithms perform good on
well-established datasets since they either comprise clean data or
simple/homogeneous page layouts. We have collected and annotated 2036 archival
document images from different locations and time periods. The dataset contains
varying page layouts and degradations that challenge text line segmentation
methods. Well established text line segmentation evaluation schemes such as the
Detection Rate or Recognition Accuracy demand for binarized data that is
annotated on a pixel level. Producing ground truth by these means is laborious
and not needed to determine a method's quality. In this paper we propose a new
evaluation scheme that is based on baselines. The proposed scheme has no need
for binarization and it can handle skewed as well as rotated text lines. The
ICDAR 2017 Competition on Baseline Detection and the ICDAR 2017 Competition on
Layout Analysis for Challenging Medieval Manuscripts used this evaluation
scheme. Finally, we present results achieved by a recently published text line
detection algorithm.
Scale Up Event Extraction Learning via Automatic Training Data Generation
Authors: Ying Zeng, Yansong Feng, Rong Ma, Zheng Wang, Rui Yan, Chongde Shi, Dongyan Zhao
Date: 2017-12-11 00:00:00
Tasks: Articles, Event Extraction

The task of event extraction has long been investigated in a supervised
learning paradigm, which is bound by the number and the quality of the training
instances. Existing training data must be manually generated through a
combination of expert domain knowledge and extensive human involvement.
However, due to drastic efforts required in annotating text, the resultant
datasets are usually small, which severally affects the quality of the learned
model, making it hard to generalize. Our work develops an automatic approach
for generating training data for event extraction. Our approach allows us to
scale up event extraction training instances from thousands to hundreds of
thousands, and it does this at a much lower cost than a manual approach. We
achieve this by employing distant supervision to automatically create event
annotations from unlabelled text using existing structured knowledge bases or
tables.We then develop a neural network model with post inference to transfer
the knowledge extracted from structured knowledge bases to automatically
annotate typed events with corresponding arguments in text.We evaluate our
approach by using the knowledge extracted from Freebase to label texts from
Wikipedia articles. Experimental results show that our approach can generate a
large number of high quality training instances. We show that this large volume
of training data not only leads to a better event extractor, but also allows us
to detect multiple typed events.