pwc-archive/papers-with-abstracts huggingface dataset:
DeeSIL: Deep-Shallow Incremental Learning
Authors: Eden Belouadah, Adrian Popescu
Date: 2018-08-20 00:00:00
Tasks: Incremental Learning, Transfer Learning

Incremental Learning (IL) is an interesting AI problem when the algorithm is
assumed to work on a budget. This is especially true when IL is modeled using a
deep learning approach, where two com- plex challenges arise due to limited
memory, which induces catastrophic forgetting and delays related to the
retraining needed in order to incorpo- rate new classes. Here we introduce
DeeSIL, an adaptation of a known transfer learning scheme that combines a fixed
deep representation used as feature extractor and learning independent shallow
classifiers to in- crease recognition capacity. This scheme tackles the two
aforementioned challenges since it works well with a limited memory budget and
each new concept can be added within a minute. Moreover, since no deep re-
training is needed when the model is incremented, DeeSIL can integrate larger
amounts of initial data that provide more transferable features. Performance is
evaluated on ImageNet LSVRC 2012 against three state of the art algorithms.
Results show that, at scale, DeeSIL performance is 23 and 33 points higher than
the best baseline when using the same and more initial data respectively.
Faster Support Vector Machines
Authors: Sebastian Schlag, Matthias Schmitt, Christian Schulz
Date: 2018-08-20 00:00:00
Tasks: General Classification

The time complexity of support vector machines (SVMs) prohibits training on huge data sets with millions of data points. Recently, multilevel approaches to train SVMs have been developed to allow for time-efficient training on huge data sets. While regular SVMs perform the entire training in one -- time consuming -- optimization step, multilevel SVMs first build a hierarchy of problems decreasing in size that resemble the original problem and then train an SVM model for each hierarchy level, benefiting from the solved models of previous levels. We present a faster multilevel support vector machine that uses a label propagation algorithm to construct the problem hierarchy. Extensive experiments indicate that our approach is up to orders of magnitude faster than the previous fastest algorithm while having comparable classification quality. For example, already one of our sequential solvers is on average a factor 15 faster than the parallel ThunderSVM algorithm, while having similar classification quality.
FusionNet and AugmentedFlowNet: Selective Proxy Ground Truth for Training on Unlabeled Images
Authors: Osama Makansi, Eddy Ilg, Thomas Brox
Date: 2018-08-20 00:00:00
Tasks: Optical Flow Estimation

Recent work has shown that convolutional neural networks (CNNs) can be used
to estimate optical flow with high quality and fast runtime. This makes them
preferable for real-world applications. However, such networks require very
large training datasets. Engineering the training data is difficult and/or
laborious. This paper shows how to augment a network trained on an existing
synthetic dataset with large amounts of additional unlabelled data. In
particular, we introduce a selection mechanism to assemble from multiple
estimates a joint optical flow field, which outperforms that of all input
methods. The latter can be used as proxy-ground-truth to train a network on
real-world data and to adapt it to specific domains of interest. Our
experimental results show that the performance of networks improves
considerably, both, in cross-domain and in domain-specific scenarios. As a
consequence, we obtain state-of-the-art results on the KITTI benchmarks.
Progressive Operational Perceptron with Memory
Authors: Dat Thanh Tran, Serkan Kiranyaz, Moncef Gabbouj, Alexandros Iosifidis
Date: 2018-08-20 00:00:00

Generalized Operational Perceptron (GOP) was proposed to generalize the linear neuron model in the traditional Multilayer Perceptron (MLP) and this model can mimic the synaptic connections of the biological neurons that have nonlinear neurochemical behaviours. Progressive Operational Perceptron (POP) is a multilayer network composing of GOPs which is formed layer-wise progressively. In this work, we propose major modifications that can accelerate as well as augment the progressive learning procedure of POP by incorporating an information-preserving, linear projection path from the input to the output layer at each progressive step. The proposed extensions can be interpreted as a mechanism that provides direct information extracted from the previously learned layers to the network, hence the term "memory". This allows the network to learn deeper architectures with better data representations. An extensive set of experiments show that the proposed modifications can surpass the learning capability of the original POPs and other related algorithms.
Learning from #Barcelona Instagram data what Locals and Tourists post about its Neighbourhoods
Authors: Raul Gomez, Lluis Gomez, Jaume Gibert, Dimosthenis Karatzas
Date: 2018-08-20 00:00:00

Massive tourism is becoming a big problem for some cities, such as Barcelona,
due to its concentration in some neighborhoods. In this work we gather
Instagram data related to Barcelona consisting on images-captions pairs and,
using the text as a supervisory signal, we learn relations between images,
words and neighborhoods. Our goal is to learn which visual elements appear in
photos when people is posting about each neighborhood. We perform a language
separate treatment of the data and show that it can be extrapolated to a
tourists and locals separate analysis, and that tourism is reflected in Social
Media at a neighborhood level. The presented pipeline allows analyzing the
differences between the images that tourists and locals associate to the
different neighborhoods. The proposed method, which can be extended to other
cities or subjects, proves that Instagram data can be used to train multi-modal
(image and text) machine learning models that are useful to analyze
publications about a city at a neighborhood level. We publish the collected
dataset, InstaBarcelona and the code used in the analysis.
Learning to Learn from Web Data through Deep Semantic Embeddings
Authors: Raul Gomez, Lluis Gomez, Jaume Gibert, Dimosthenis Karatzas
Date: 2018-08-20 00:00:00
Tasks: Image Retrieval, Retrieval

In this paper we propose to learn a multimodal image and text embedding from
Web and Social Media data, aiming to leverage the semantic knowledge learnt in
the text domain and transfer it to a visual model for semantic image retrieval.
We demonstrate that the pipeline can learn from images with associated text
without supervision and perform a thourough analysis of five different text
embeddings in three different benchmarks. We show that the embeddings learnt
with Web and Social Media data have competitive performances over supervised
methods in the text based image retrieval task, and we clearly outperform state
of the art in the MIRFlickr dataset when training in the target data. Further
we demonstrate how semantic multimodal image retrieval can be performed using
the learnt embeddings, going beyond classical instance-level retrieval
problems. Finally, we present a new dataset, InstaCities1M, composed by
Instagram images and their associated texts that can be used for fair
comparison of image-text embeddings.
Towards Anticipation of Architectural Smells using Link Prediction Techniques
Authors: J. Andrés Díaz-Pace, Antonela Tommasel, Daniela Godoy
Date: 2018-08-20 00:00:00
Tasks: Link Prediction

Software systems naturally evolve, and this evolution often brings design
problems that cause system degradation. Architectural smells are typical
symptoms of such problems, and several of these smells are related to undesired
dependencies among modules. The early detection of these smells is important
for developers, because they can plan ahead for maintenance or refactoring
efforts, thus preventing system degradation. Existing tools for identifying
architectural smells can detect the smells once they exist in the source code.
This means that their undesired dependencies are already created. In this work,
we explore a forward-looking approach that is able to infer groups of likely
module dependencies that can anticipate architectural smells in a future system
version. Our approach considers the current module structure as a network,
along with information from previous versions, and applies link prediction
techniques (from the field of social network analysis). In particular, we focus
on dependency-related smells, such as Cyclic Dependency and Hublike Dependency,
which fit well with the link prediction model. An initial evaluation with two
open-source projects shows that, under certain considerations, the predictions
of our approach are satisfactory. Furthermore, the approach can be extended to
other types of dependency-based smells or metrics.
Causal Discovery by Telling Apart Parents and Children
Authors: Alexander Marx, Jilles Vreeken
Date: 2018-08-20 00:00:00
Tasks: Causal Discovery

We consider the problem of inferring the directed, causal graph from
observational data, assuming no hidden confounders. We take an information
theoretic approach, and make three main contributions.
  First, we show how through algorithmic information theory we can obtain SCI,
a highly robust, effective and computationally efficient test for conditional
independence---and show it outperforms the state of the art when applied in
constraint-based inference methods such as stable PC.
  Second, building upon on SCI, we show how to tell apart the parents and
children of a given node based on the algorithmic Markov condition. We give the
Climb algorithm to efficiently discover the directed, causal Markov
blanket---and show it is at least as accurate as inferring the global network,
while being much more efficient.
  Last, but not least, we detail how we can use the Climb score to direct those
edges that state of the art causal discovery algorithms based on PC or GES
leave undirected---and show this improves their precision, recall and F1 scores
by up to 20%.
Navigating the Landscape for Real-time Localisation and Mapping for Robotics and Virtual and Augmented Reality
Authors: Sajad Saeedi, Bruno Bodin, Harry Wagstaff, Andy Nisbet, Luigi Nardi, John Mawer, Nicolas Melot, Oscar Palomar, Emanuele Vespa, Tom Spink, Cosmin Gorgovan, Andrew Webb, James Clarkson, Erik Tomusk, Thomas Debrunner, Kuba Kaszyk, Pablo Gonzalez-de-Aledo, Andrey Rodchenko, Graham Riley, Christos Kotselidis, Björn Franke, Michael F. P. O'Boyle, Andrew J. Davison, Paul H. J. Kelly, Mikel Luján, Steve Furber
Date: 2018-08-20 00:00:00
Tasks: Autonomous Vehicles

Visual understanding of 3D environments in real-time, at low power, is a huge
computational challenge. Often referred to as SLAM (Simultaneous Localisation
and Mapping), it is central to applications spanning domestic and industrial
robotics, autonomous vehicles, virtual and augmented reality. This paper
describes the results of a major research effort to assemble the algorithms,
architectures, tools, and systems software needed to enable delivery of SLAM,
by supporting applications specialists in selecting and configuring the
appropriate algorithm and the appropriate hardware, and compilation pathway, to
meet their performance, accuracy, and energy consumption goals. The major
contributions we present are (1) tools and methodology for systematic
quantitative evaluation of SLAM algorithms, (2) automated,
machine-learning-guided exploration of the algorithmic and implementation
design space with respect to multiple objectives, (3) end-to-end simulation
tools to enable optimisation of heterogeneous, accelerated architectures for
the specific algorithmic requirements of the various SLAM algorithmic
approaches, and (4) tools for delivering, where appropriate, accelerated,
adaptive SLAM solutions in a managed, JIT-compiled, adaptive runtime context.
A Distribution Similarity Based Regularizer for Learning Bayesian Networks
Authors: Weirui Kong, Wenyi Wang
Date: 2018-08-20 00:00:00

Probabilistic graphical models compactly represent joint distributions by
decomposing them into factors over subsets of random variables. In Bayesian
networks, the factors are conditional probability distributions. For many
problems, common information exists among those factors. Adding similarity
restrictions can be viewed as imposing prior knowledge for model
regularization. With proper restrictions, learned models usually generalize
better. In this work, we study methods that exploit such high-level
similarities to regularize the learning process and apply them to the task of
modeling the wave propagation in inhomogeneous media. We propose a novel
distribution-based penalization approach that encourages similar conditional
probability distribution rather than force the parameters to be similar
explicitly. We show in experiment that our proposed algorithm solves the
modeling wave propagation problem, which other baseline methods are not able to
solve.
The Mismatch Principle: The Generalized Lasso Under Large Model Uncertainties
Authors: Martin Genzel, Gitta Kutyniok
Date: 2018-08-20 00:00:00
Tasks: compressed sensing, Learning Theory, Variable Selection

We study the estimation capacity of the generalized Lasso, i.e., least squares minimization combined with a (convex) structural constraint. While Lasso-type estimators were originally designed for noisy linear regression problems, it has recently turned out that they are in fact robust against various types of model uncertainties and misspecifications, most notably, non-linearly distorted observation models. This work provides more theoretical evidence for this somewhat astonishing phenomenon. At the heart of our analysis stands the mismatch principle, which is a simple recipe to establish theoretical error bounds for the generalized Lasso. The associated estimation guarantees are of independent interest and are formulated in a fairly general setup, permitting arbitrary sub-Gaussian data, possibly with strongly correlated feature designs; in particular, we do not assume a specific observation model which connects the input and output variables. Although the mismatch principle is conceived based on ideas from statistical learning theory, its actual application area are (high-dimensional) estimation tasks for semi-parametric models. In this context, the benefits of the mismatch principle are demonstrated for a variety of popular problem classes, such as single-index models, generalized linear models, and variable selection. Apart from that, our findings are also relevant to recent advances in quantized and distributed compressed sensing.
PAC-learning is Undecidable
Authors: Sairaam Venkatraman, S. Balasubramanian, R. Raghunatha Sarma
Date: 2018-08-20 00:00:00
Tasks: BIG-bench Machine Learning, PAC learning

The problem of attempting to learn the mapping between data and labels is the crux of any machine learning task. It is, therefore, of interest to the machine learning community on practical as well as theoretical counts to consider the existence of a test or criterion for deciding the feasibility of attempting to learn. We investigate the existence of such a criterion in the setting of PAC-learning, basing the feasibility solely on whether the mapping to be learnt lends itself to approximation by a given class of hypothesis functions. We show that no such criterion exists, exposing a fundamental limitation in the decidability of learning. In other words, we prove that testing for PAC-learnability is undecidable in the Turing sense. We also briefly discuss some of the probable implications of this result to the current practice of machine learning.
Universal Image Manipulation Detection using Deep Siamese Convolutional Neural Network
Authors: Aniruddha Mazumdar, Jaya Singh, Yosha Singh Tomar, Prabin Kumar Bora
Date: 2018-08-20 00:00:00
Tasks: Image Forensics, Image Manipulation, Image Manipulation Detection

Detection of different types of image editing operations carried out on an
image is an important problem in image forensics. It gives the information
about the processing history of an image, and also can expose forgeries present
in an image. There have been few methods proposed to detect different types of
image editing operations in a single framework. However, all the operations
have to be known a priori in the training phase. But, in real-forensics
scenarios it may not be possible to know about the editing operations carried
out on an image. To solve this problem, we propose a novel deep learning-based
method which can differentiate between different types of image editing
operations. The proposed method classifies image patches in a pair-wise fashion
as either similarly or differently processed using a deep siamese neural
network. Once the network learns feature that can discriminate between
different image editing operations, it can differentiate between different
image editing operations not present in the training stage. The experimental
results show the efficacy of the proposed method in detecting/discriminating
different image editing operations.
Discovering Context Specific Causal Relationships
Authors: Saisai Ma, Jiuyong Li, Lin Liu, Thuc Duy Le
Date: 2018-08-20 00:00:00
Tasks: Causal Inference, Decision Making, Efficient Exploration

With the increasing need of personalised decision making, such as
personalised medicine and online recommendations, a growing attention has been
paid to the discovery of the context and heterogeneity of causal relationships.
Most existing methods, however, assume a known cause (e.g. a new drug) and
focus on identifying from data the contexts of heterogeneous effects of the
cause (e.g. patient groups with different responses to the new drug). There is
no approach to efficiently detecting directly from observational data context
specific causal relationships, i.e. discovering the causes and their contexts
simultaneously. In this paper, by taking the advantages of highly efficient
decision tree induction and the well established causal inference framework, we
propose the Tree based Context Causal rule discovery (TCC) method, for
efficient exploration of context specific causal relationships from data.
Experiments with both synthetic and real world data sets show that TCC can
effectively discover context specific causal rules from the data.
A General Framework of Multi-Armed Bandit Processes by Arm Switch Restrictions
Authors: Wenqing Bao, Xiaoqiang Cai, Xianyi Wu
Date: 2018-08-20 00:00:00

This paper proposes a general framework of multi-armed bandit (MAB) processes
by introducing a type of restrictions on the switches among arms evolving in
continuous time.
  The Gittins index process is constructed for any single arm subject to the
restrictions on switches and then the optimality of the corresponding Gittins
index rule is established. The Gittins indices defined in this paper are
consistent with the ones for MAB processes in continuous time, integer time,
semi-Markovian setting as well as general discrete time setting, so that the
new theory covers the classical models as special cases and also applies to
many other situations that have not yet been touched in the literature. While
the proof of the optimality of Gittins index policies benefits from ideas in
the existing theory of MAB processes in continuous time, new techniques are
introduced which drastically simplify the proof.
Post-Processing of Word Representations via Variance Normalization and Dynamic Embedding
Authors: Bin Wang, Fenxiao Chen, Angela Wang, C. -C. Jay Kuo
Date: 2018-08-20 00:00:00

Although embedded vector representations of words offer impressive
performance on many natural language processing (NLP) applications, the
information of ordered input sequences is lost to some extent if only
context-based samples are used in the training. For further performance
improvement, two new post-processing techniques, called post-processing via
variance normalization (PVN) and post-processing via dynamic embedding (PDE),
are proposed in this work. The PVN method normalizes the variance of principal
components of word vectors while the PDE method learns orthogonal latent
variables from ordered input sequences. The PVN and the PDE methods can be
integrated to achieve better performance. We apply these post-processing
techniques to two popular word embedding methods (i.e., word2vec and GloVe) to
yield their post-processed representations. Extensive experiments are conducted
to demonstrate the effectiveness of the proposed post-processing techniques.
Question Generation from SQL Queries Improves Neural Semantic Parsing
Authors: Daya Guo, Yibo Sun, Duyu Tang, Nan Duan, Jian Yin, Hong Chi, James Cao, Peng Chen, Ming Zhou
Date: 2018-08-20 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Question Generation, Question-Generation, Semantic Parsing

We study how to learn a semantic parser of state-of-the-art accuracy with
less supervised training data. We conduct our study on WikiSQL, the largest
hand-annotated semantic parsing dataset to date. First, we demonstrate that
question generation is an effective method that empowers us to learn a
state-of-the-art neural network based semantic parser with thirty percent of
the supervised training data. Second, we show that applying question generation
to the full supervised training data further improves the state-of-the-art
model. In addition, we observe that there is a logarithmic relationship between
the accuracy of a semantic parser and the amount of training data.
Universal Stagewise Learning for Non-Convex Problems with Convergence on Averaged Solutions
Authors: Zaiyi Chen, Zhuoning Yuan, Jin-Feng Yi, Bo-Wen Zhou, Enhong Chen, Tianbao Yang
Date: 2018-08-20 00:00:00
Proceeding: ICLR 2019 5

Although stochastic gradient descent (SGD) method and its variants (e.g.,
stochastic momentum methods, AdaGrad) are the choice of algorithms for solving
non-convex problems (especially deep learning), there still remain big gaps
between the theory and the practice with many questions unresolved. For
example, there is still a lack of theories of convergence for SGD and its
variants that use stagewise step size and return an averaged solution in
practice. In addition, theoretical insights of why adaptive step size of
AdaGrad could improve non-adaptive step size of {\sgd} is still missing for
non-convex optimization. This paper aims to address these questions and fill
the gap between theory and practice. We propose a universal stagewise
optimization framework for a broad family of {\bf non-smooth non-convex}
(namely weakly convex) problems with the following key features: (i) at each
stage any suitable stochastic convex optimization algorithms (e.g., SGD or
AdaGrad) that return an averaged solution can be employed for minimizing a
regularized convex problem; (ii) the step size is decreased in a stagewise
manner; (iii) an averaged solution is returned as the final solution that is
selected from all stagewise averaged solutions with sampling probabilities {\it
increasing} as the stage number. Our theoretical results of stagewise AdaGrad
exhibit its adaptive convergence, therefore shed insights on its faster
convergence for problems with sparse stochastic gradients than stagewise SGD.
To the best of our knowledge, these new results are the first of their kind for
addressing the unresolved issues of existing theories mentioned earlier.
Besides theoretical contributions, our empirical studies show that our
stagewise SGD and ADAGRAD improve the generalization performance of existing
variants/implementations of SGD and ADAGRAD.