pwc-archive/papers-with-abstracts huggingface dataset:
f-CNN$^{\text{x}}$: A Toolflow for Mapping Multi-CNN Applications on FPGAs
Authors: Stylianos I. Venieris, Christos-Savvas Bouganis
Date: 2018-05-25 00:00:00
Tasks: GPU, Scheduling

The predictive power of Convolutional Neural Networks (CNNs) has been an integral factor for emerging latency-sensitive applications, such as autonomous drones and vehicles. Such systems employ multiple CNNs, each one trained for a particular task. The efficient mapping of multiple CNNs on a single FPGA device is a challenging task as the allocation of compute resources and external memory bandwidth needs to be optimised at design time. This paper proposes f-CNN$^{\text{x}}$, an automated toolflow for the optimised mapping of multiple CNNs on FPGAs, comprising a novel multi-CNN hardware architecture together with an automated design space exploration method that considers the user-specified performance requirements for each model to allocate compute resources and generate a synthesisable accelerator. Moreover, f-CNN$^{\text{x}}$ employs a novel scheduling algorithm that alleviates the limitations of the memory bandwidth contention between CNNs and sustains the high utilisation of the architecture. Experimental evaluation shows that f-CNN$^{\text{x}}$'s designs outperform contention-unaware FPGA mappings by up to 50% and deliver up to 6.8x higher performance-per-Watt over highly optimised GPU designs for multi-CNN systems.
A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocols
Authors: Neerav Karani, Krishna Chaitanya, Christian Baumgartner, Ender Konukoglu
Date: 2018-05-25 00:00:00
Tasks: Lifelong learning, Segmentation

Convolutional neural networks (CNNs) have shown promising results on several
segmentation tasks in magnetic resonance (MR) images. However, the accuracy of
CNNs may degrade severely when segmenting images acquired with different
scanners and/or protocols as compared to the training data, thus limiting their
practical utility. We address this shortcoming in a lifelong multi-domain
learning setting by treating images acquired with different scanners or
protocols as samples from different, but related domains. Our solution is a
single CNN with shared convolutional filters and domain-specific batch
normalization layers, which can be tuned to new domains with only a few
($\approx$ 4) labelled images. Importantly, this is achieved while retaining
performance on the older domains whose training data may no longer be
available. We evaluate the method for brain structure segmentation in MR
images. Results demonstrate that the proposed method largely closes the gap to
the benchmark, which is training a dedicated CNN for each scanner.
Destructiveness of Lexicographic Parsimony Pressure and Alleviation by a Concatenation Crossover in Genetic Programming
Authors: Timo Kötzing, J. A. Gregor Lagodzinski, Johannes Lengler, Anna Melnichenko
Date: 2018-05-25 00:00:00

For theoretical analyses there are two specifics distinguishing GP from many
other areas of evolutionary computation. First, the variable size
representations, in particular yielding a possible bloat (i.e. the growth of
individuals with redundant parts). Second, the role and realization of
crossover, which is particularly central in GP due to the tree-based
representation. Whereas some theoretical work on GP has studied the effects of
bloat, crossover had a surprisingly little share in this work. We analyze a
simple crossover operator in combination with local search, where a preference
for small solutions minimizes bloat (lexicographic parsimony pressure); the
resulting algorithm is denoted Concatenation Crossover GP. For this purpose
three variants of the well-studied MAJORITY test function with large plateaus
are considered. We show that the Concatenation Crossover GP can efficiently
optimize these test functions, while local search cannot be efficient for all
three variants independent of employing bloat control.
Futuristic Classification with Dynamic Reference Frame Strategy
Authors: Kumarjit Pathak, Jitin Kapila, Aasheesh Barvey
Date: 2018-05-25 00:00:00
Tasks: Classification, General Classification, Prediction

Classification is one of the widely used analytical techniques in data
science domain across different business to associate a pattern which
contribute to the occurrence of certain event which is predicted with some
likelihood. This Paper address a lacuna of creating some time window before the
prediction actually happen to enable organizations some space to act on the
prediction. There are some really good state of the art machine learning
techniques to optimally identify the possible churners in either customer base
or employee base, similarly for fault prediction too if the prediction does not
come with some buffer time to act on the fault it is very difficult to provide
a seamless experience to the user. New concept of reference frame creation is
introduced to solve this problem in this paper
Context-Aware Neural Machine Translation Learns Anaphora Resolution
Authors: Elena Voita, Pavel Serdyukov, Rico Sennrich, Ivan Titov
Date: 2018-05-25 00:00:00
Proceeding: ACL 2018 7
Tasks: Machine Translation, Translation

Standard machine translation systems process sentences in isolation and hence
ignore extra-sentential information, even though extended context can both
prevent mistakes in ambiguous cases and improve translation coherence. We
introduce a context-aware neural machine translation model designed in such way
that the flow of information from the extended context to the translation model
can be controlled and analyzed. We experiment with an English-Russian subtitles
dataset, and observe that much of what is captured by our model deals with
improving pronoun translation. We measure correspondences between induced
attention distributions and coreference relations and observe that the model
implicitly captures anaphora. It is consistent with gains for sentences where
pronouns need to be gendered in translation. Beside improvements in anaphoric
cases, the model also improves in overall BLEU, both over its context-agnostic
version (+0.7) and over simple concatenation of the context and source
sentences (+0.6).
Identifiability of Kronecker-structured Dictionaries for Tensor Data
Authors: Zahra Shakeri, Anand D. Sarwate, Waheed U. Bajwa
Date: 2017-12-10 00:00:00

This paper derives sufficient conditions for local recovery of coordinate
dictionaries comprising a Kronecker-structured dictionary that is used for
representing $K$th-order tensor data. Tensor observations are assumed to be
generated from a Kronecker-structured dictionary multiplied by sparse
coefficient tensors that follow the separable sparsity model. This work
provides sufficient conditions on the underlying coordinate dictionaries,
coefficient and noise distributions, and number of samples that guarantee
recovery of the individual coordinate dictionaries up to a specified error, as
a local minimum of the objective function, with high probability. In
particular, the sample complexity to recover $K$ coordinate dictionaries with
dimensions $m_k \times p_k$ up to estimation error $\varepsilon_k$ is shown to
be $\max_{k \in [K]}\mathcal{O}(m_kp_k^3\varepsilon_k^{-2})$.
Think Visually: Question Answering through Virtual Imagery
Authors: Ankit Goyal, Jian Wang, Jia Deng
Date: 2018-05-25 00:00:00
Proceeding: ACL 2018 7
Tasks: Question Answering, Visual Commonsense Reasoning

In this paper, we study the problem of geometric reasoning in the context of
question-answering. We introduce Dynamic Spatial Memory Network (DSMN), a new
deep network architecture designed for answering questions that admit latent
visual representations. DSMN learns to generate and reason over such
representations. Further, we propose two synthetic benchmarks, FloorPlanQA and
ShapeIntersection, to evaluate the geometric reasoning capability of QA
systems. Experimental results validate the effectiveness of our proposed DSMN
for visual thinking tasks.
Information-Propogation-Enhanced Neural Machine Translation by Relation Model
Authors: Wen Zhang, Jiawei Hu, Yang Feng, Qun Liu
Date: 2017-09-06 00:00:00
Tasks: Decoder, Machine Translation, NMT, Relation, Relation Network, Sentence, Translation

Even though sequence-to-sequence neural machine translation (NMT) model have
achieved state-of-art performance in the recent fewer years, but it is widely
concerned that the recurrent neural network (RNN) units are very hard to
capture the long-distance state information, which means RNN can hardly find
the feature with long term dependency as the sequence becomes longer.
Similarly, convolutional neural network (CNN) is introduced into NMT for
speeding recently, however, CNN focus on capturing the local feature of the
sequence; To relieve this issue, we incorporate a relation network into the
standard encoder-decoder framework to enhance information-propogation in neural
network, ensuring that the information of the source sentence can flow into the
decoder adequately. Experiments show that proposed framework outperforms the
statistical MT model and the state-of-art NMT model significantly on two data
sets with different scales.
Refining Source Representations with Relation Networks for Neural Machine Translation
Authors: Wen Zhang, Jiawei Hu, Yang Feng, Qun Liu
Date: 2017-09-12 00:00:00
Tasks: Decoder, Machine Translation, NMT, Relation, Translation

Although neural machine translation (NMT) with the encoder-decoder framework
has achieved great success in recent times, it still suffers from some
drawbacks: RNNs tend to forget old information which is often useful and the
encoder only operates through words without considering word relationship. To
solve these problems, we introduce a relation networks (RN) into NMT to refine
the encoding representations of the source. In our method, the RN first
augments the representation of each source word with its neighbors and reasons
all the possible pairwise relations between them. Then the source
representations and all the relations are fed to the attention module and the
decoder together, keeping the main encoder-decoder architecture unchanged.
Experiments on two Chinese-to-English data sets in different scales both show
that our method can outperform the competitive baselines significantly.
Refining Source Representations with Relation Networks for Neural Machine Translation
Authors: Wen Zhang, Jiawei Hu, Yang Feng, Qun Liu
Date: 2018-05-25 00:00:00
Proceeding: COLING 2018 8
Tasks: Decoder, Machine Translation, Memorization, Relation, Translation

Although neural machine translation with the encoder-decoder framework has
achieved great success recently, it still suffers drawbacks of forgetting
distant information, which is an inherent disadvantage of recurrent neural
network structure, and disregarding relationship between source words during
encoding step. Whereas in practice, the former information and relationship are
often useful in current step. We target on solving these problems and thus
introduce relation networks to learn better representations of the source. The
relation networks are able to facilitate memorization capability of recurrent
neural network via associating source words with each other, this would also
help retain their relationships. Then the source representations and all the
relations are fed into the attention component together while decoding, with
the main encoder-decoder framework unchanged. Experiments on several datasets
show that our method can improve the translation performance significantly over
the conventional encoder-decoder model and even outperform the approach
involving supervised syntactic knowledge.
Mapping Images to Scene Graphs with Permutation-Invariant Structured Prediction
Authors: Roei Herzig, Moshiko Raboh, Gal Chechik, Jonathan Berant, Amir Globerson
Date: 2018-02-15 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Scene Graph Classification, Scene Graph Generation, Structured Prediction

Machine understanding of complex images is a key goal of artificial
intelligence. One challenge underlying this task is that visual scenes contain
multiple inter-related objects, and that global context plays an important role
in interpreting the scene. A natural modeling framework for capturing such
effects is structured prediction, which optimizes over complex labels, while
modeling within-label interactions. However, it is unclear what principles
should guide the design of a structured prediction model that utilizes the
power of deep learning components. Here we propose a design principle for such
architectures that follows from a natural requirement of permutation
invariance. We prove a necessary and sufficient characterization for
architectures that follow this invariance, and discuss its implication on model
design. Finally, we show that the resulting model achieves new state of the art
results on the Visual Genome scene graph labeling benchmark, outperforming all
recent approaches.
YEDDA: A Lightweight Collaborative Text Span Annotation Tool
Authors: Jie Yang, Yue Zhang, Linwei Li, Xingxuan Li
Date: 2017-11-10 00:00:00
Proceeding: ACL 2018 7
Tasks: text annotation

In this paper, we introduce \textsc{Yedda}, a lightweight but efficient and
comprehensive open-source tool for text span annotation. \textsc{Yedda}
provides a systematic solution for text span annotation, ranging from
collaborative user annotation to administrator evaluation and analysis. It
overcomes the low efficiency of traditional text annotation tools by annotating
entities through both command line and shortcut keys, which are configurable
with custom labels. \textsc{Yedda} also gives intelligent recommendations by
learning the up-to-date annotated text. An administrator client is developed to
evaluate annotation quality of multiple annotators and generate detailed
comparison report for each annotator pair. Experiments show that the proposed
system can reduce the annotation time by half compared with existing annotation
tools. And the annotation time can be further compressed by 16.47\% through
intelligent recommendation.
ELFI: Engine for Likelihood-Free Inference
Authors: Jarno Lintusaari, Henri Vuollekoski, Antti Kangasrääsiö, Kusti Skytén, Marko Järvenpää, Pekka Marttinen, Michael U. Gutmann, Aki Vehtari, Jukka Corander, Samuel Kaski
Date: 2017-08-02 00:00:00
Tasks: Bayesian Optimization

Engine for Likelihood-Free Inference (ELFI) is a Python software library for
performing likelihood-free inference (LFI). ELFI provides a convenient syntax
for arranging components in LFI, such as priors, simulators, summaries or
distances, to a network called ELFI graph. The components can be implemented in
a wide variety of languages. The stand-alone ELFI graph can be used with any of
the available inference methods without modifications. A central method
implemented in ELFI is Bayesian Optimization for Likelihood-Free Inference
(BOLFI), which has recently been shown to accelerate likelihood-free inference
up to several orders of magnitude by surrogate-modelling the distance. ELFI
also has an inbuilt support for output data storing for reuse and analysis, and
supports parallelization of computation from multiple cores up to a cluster
environment. ELFI is designed to be extensible and provides interfaces for
widening its functionality. This makes the adding of new inference methods to
ELFI straightforward and automatically compatible with the inbuilt features.
Pooling is neither necessary nor sufficient for appropriate deformation stability in CNNs
Authors: Avraham Ruderman, Neil C. Rabinowitz, Ari S. Morcos, Daniel Zoran
Date: 2018-04-12 00:00:00
Proceeding: ICLR 2019 5
Tasks: General Classification, image-classification, Image Classification, Inductive Bias

Many of our core assumptions about how neural networks operate remain
empirically untested. One common assumption is that convolutional neural
networks need to be stable to small translations and deformations to solve
image recognition tasks. For many years, this stability was baked into CNN
architectures by incorporating interleaved pooling layers. Recently, however,
interleaved pooling has largely been abandoned. This raises a number of
questions: Are our intuitions about deformation stability right at all? Is it
important? Is pooling necessary for deformation invariance? If not, how is
deformation invariance achieved in its absence? In this work, we rigorously
test these questions, and find that deformation stability in convolutional
networks is more nuanced than it first appears: (1) Deformation invariance is
not a binary property, but rather that different tasks require different
degrees of deformation stability at different layers. (2) Deformation stability
is not a fixed property of a network and is heavily adjusted over the course of
training, largely through the smoothness of the convolutional filters. (3)
Interleaved pooling layers are neither necessary nor sufficient for achieving
the optimal form of deformation stability for natural image classification. (4)
Pooling confers too much deformation stability for image classification at
initialization, and during training, networks have to learn to counteract this
inductive bias. Together, these findings provide new insights into the role of
interleaved pooling and deformation invariance in CNNs, and demonstrate the
importance of rigorous empirical testing of even our most basic assumptions
about the working of neural networks.
The Reconstruction Approach: From Interpolation to Regression
Authors: Shifeng Xiong
Date: 2018-05-25 00:00:00
Tasks: Experimental Design, regression

This paper introduces an interpolation-based method, called the reconstruction approach, for nonparametric regression. Based on the fact that interpolation usually has negligible errors compared to statistical estimation, the reconstruction approach uses an interpolator to parameterize the regression function with its values at finite knots, and then estimates these values by (regularized) least squares. Some popular methods including kernel ridge regression can be viewed as its special cases. It is shown that, the reconstruction idea not only provides different angles to look into existing methods, but also produces new effective experimental design and estimation methods for nonparametric models. In particular, for some methods of complexity O(n3), where n is the sample size, this approach provides effective surrogates with much less computational burden. This point makes it very suitable for large datasets.
UnibucKernel Reloaded: First Place in Arabic Dialect Identification for the Second Year in a Row
Authors: Andrei M. Butnaru, Radu Tudor Ionescu
Date: 2018-05-13 00:00:00
Proceeding: COLING 2018 8
Tasks: Dialect Identification

We present a machine learning approach that ranked on the first place in the
Arabic Dialect Identification (ADI) Closed Shared Tasks of the 2018 VarDial
Evaluation Campaign. The proposed approach combines several kernels using
multiple kernel learning. While most of our kernels are based on character
p-grams (also known as n-grams) extracted from speech or phonetic transcripts,
we also use a kernel based on dialectal embeddings generated from audio
recordings by the organizers. In the learning stage, we independently employ
Kernel Discriminant Analysis (KDA) and Kernel Ridge Regression (KRR).
Preliminary experiments indicate that KRR provides better classification
results. Our approach is shallow and simple, but the empirical results obtained
in the 2018 ADI Closed Shared Task prove that it achieves the best performance.
Furthermore, our top macro-F1 score (58.92%) is significantly better than the
second best score (57.59%) in the 2018 ADI Shared Task, according to the
statistical significance test performed by the organizers. Nevertheless, we
obtain even better post-competition results (a macro-F1 score of 62.28%) using
the audio embeddings released by the organizers after the competition. With a
very similar approach (that did not include phonetic features), we also ranked
first in the ADI Closed Shared Tasks of the 2017 VarDial Evaluation Campaign,
surpassing the second best method by 4.62%. We therefore conclude that our
multiple kernel learning method is the best approach to date for Arabic dialect
identification.
An Analysis of Scale Invariance in Object Detection - SNIP
Authors: Bharat Singh, Larry S. Davis
Date: 2017-11-22 00:00:00
Tasks: Object, object-detection, Object Detection

An analysis of different techniques for recognizing and detecting objects
under extreme scale variation is presented. Scale specific and scale invariant
design of detectors are compared by training them with different configurations
of input data. By evaluating the performance of different network architectures
for classifying small objects on ImageNet, we show that CNNs are not robust to
changes in scale. Based on this analysis, we propose to train and test
detectors on the same scales of an image-pyramid. Since small and large objects
are difficult to recognize at smaller and larger scales respectively, we
present a novel training scheme called Scale Normalization for Image Pyramids
(SNIP) which selectively back-propagates the gradients of object instances of
different sizes as a function of the image scale. On the COCO dataset, our
single model performance is 45.7% and an ensemble of 3 networks obtains an mAP
of 48.3%. We use off-the-shelf ImageNet-1000 pre-trained models and only train
with bounding box supervision. Our submission won the Best Student Entry in the
COCO 2017 challenge. Code will be made available at
\url{http://bit.ly/2yXVg4c}.
Double Quantization for Communication-Efficient Distributed Optimization
Authors: Yue Yu, Jiaxiang Wu, Longbo Huang
Date: 2018-05-25 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Distributed Optimization, Quantization

Modern distributed training of machine learning models suffers from high communication overhead for synchronizing stochastic gradients and model parameters. In this paper, to reduce the communication complexity, we propose \emph{double quantization}, a general scheme for quantizing both model parameters and gradients. Three communication-efficient algorithms are proposed under this general scheme. Specifically, (i) we propose a low-precision algorithm AsyLPG with asynchronous parallelism, (ii) we explore integrating gradient sparsification with double quantization and develop Sparse-AsyLPG, (iii) we show that double quantization can also be accelerated by momentum technique and design accelerated AsyLPG. We establish rigorous performance guarantees for the algorithms, and conduct experiments on a multi-server test-bed to demonstrate that our algorithms can effectively save transmitted bits without performance degradation.
The Logistic Network Lasso
Authors: Henrik Ambos, Nguyen Tran, Alexander Jung
Date: 2018-05-07 00:00:00
Tasks: Binary Classification, Clustering, General Classification, regression

We apply the network Lasso to solve binary classification and clustering
problems for network-structured data. To this end, we generalize ordinary
logistic regression to non-Euclidean data with an intrinsic network structure.
The resulting "logistic network Lasso" amounts to solving a non-smooth convex
regularized empirical risk minimization. The risk is measured using the
logistic loss incurred over a small set of labeled nodes. For the
regularization, we propose to use the total variation of the classifier
requiring it to conform to the underlying network structure. A scalable
implementation of the learning method is obtained using an inexact variant of
the alternating direction methods of multipliers which results in a scalable
learning algorithm