pwc-archive/papers-with-abstracts huggingface dataset:
Fast Hyperparameter Optimization of Deep Neural Networks via Ensembling Multiple Surrogates
Authors: Yang Li, Jiawei Jiang, Yingxia Shao, Bin Cui
Date: 2018-11-06 00:00:00
Tasks: Bayesian Optimization, Hyperparameter Optimization

The performance of deep neural networks crucially depends on good hyperparameter configurations. Bayesian optimization is a powerful framework for optimizing the hyperparameters of DNNs. These methods need sufficient evaluation data to approximate and minimize the validation error function of hyperparameters. However, the expensive evaluation cost of DNNs leads to very few evaluation data within a limited time, which greatly reduces the efficiency of Bayesian optimization. Besides, the previous researches focus on using the complete evaluation data to conduct Bayesian optimization, and ignore the intermediate evaluation data generated by early stopping methods. To alleviate the insufficient evaluation data problem, we propose a fast hyperparameter optimization method, HOIST, that utilizes both the complete and intermediate evaluation data to accelerate the hyperparameter optimization of DNNs. Specifically, we train multiple basic surrogates to gather information from the mixed evaluation data, and then combine all basic surrogates using weighted bagging to provide an accurate ensemble surrogate. Our empirical studies show that HOIST outperforms the state-of-the-art approaches on a wide range of DNNs, including feed forward neural networks, convolutional neural networks, recurrent neural networks, and variational autoencoder.
Recurrent Skipping Networks for Entity Alignment
Authors: Lingbing Guo, Zequn Sun, Ermei Cao, Wei Hu
Date: 2018-11-06 00:00:00
Tasks: Entity Alignment

We consider the problem of learning knowledge graph (KG) embeddings for
entity alignment (EA). Current methods use the embedding models mainly focusing
on triple-level learning, which lacks the ability of capturing long-term
dependencies existing in KGs. Consequently, the embedding-based EA methods
heavily rely on the amount of prior (known) alignment, due to the identity
information in the prior alignment cannot be efficiently propagated from one KG
to another. In this paper, we propose RSN4EA (recurrent skipping networks for
EA), which leverages biased random walk sampling for generating long paths
across KGs and models the paths with a novel recurrent skipping network (RSN).
RSN integrates the conventional recurrent neural network (RNN) with residual
learning and can largely improve the convergence speed and performance with
only a few more parameters. We evaluated RSN4EA on a series of datasets
constructed from real-world KGs. Our experimental results showed that it
outperformed a number of state-of-the-art embedding-based EA methods and also
achieved comparable performance for KG completion.
Stacked Penalized Logistic Regression for Selecting Views in Multi-View Learning
Authors: Wouter van Loon, Marjolein Fokkema, Botond Szabo, Mark de Rooij
Date: 2018-11-06 00:00:00
Tasks: General Classification, MULTI-VIEW LEARNING, regression

In biomedical research, many different types of patient data can be collected, such as various types of omics data and medical imaging modalities. Applying multi-view learning to these different sources of information can increase the accuracy of medical classification models compared with single-view procedures. However, collecting biomedical data can be expensive and/or burdening for patients, so that it is important to reduce the amount of required data collection. It is therefore necessary to develop multi-view learning methods which can accurately identify those views that are most important for prediction. In recent years, several biomedical studies have used an approach known as multi-view stacking (MVS), where a model is trained on each view separately and the resulting predictions are combined through stacking. In these studies, MVS has been shown to increase classification accuracy. However, the MVS framework can also be used for selecting a subset of important views. To study the view selection potential of MVS, we develop a special case called stacked penalized logistic regression (StaPLR). Compared with existing view-selection methods, StaPLR can make use of faster optimization algorithms and is easily parallelized. We show that nonnegativity constraints on the parameters of the function which combines the views play an important role in preventing unimportant views from entering the model. We investigate the performance of StaPLR through simulations, and consider two real data examples. We compare the performance of StaPLR with an existing view selection method called the group lasso and observe that, in terms of view selection, StaPLR is often more conservative and has a consistently lower false positive rate.
Kernel Regression for Graph Signal Prediction in Presence of Sparse Noise
Authors: Arun Venkitaraman, Pascal Frossard, Saikat Chatterjee
Date: 2018-11-06 00:00:00
Tasks: regression

In presence of sparse noise we propose kernel regression for predicting
output vectors which are smooth over a given graph. Sparse noise models the
training outputs being corrupted either with missing samples or large
perturbations. The presence of sparse noise is handled using appropriate use of
$\ell_1$-norm along-with use of $\ell_2$-norm in a convex cost function. For
optimization of the cost function, we propose an iteratively reweighted
least-squares (IRLS) approach that is suitable for kernel substitution or
kernel trick due to availability of a closed form solution. Simulations using
real-world temperature data show efficacy of our proposed method, mainly for
limited-size training datasets.
An Enhanced Multi-Objective Biogeography-Based Optimization for Overlapping Community Detection in Social Networks with Node Attributes
Authors: Ali Reihanian, Mohammad-Reza Feizi-Derakhshi, Hadi S. Aghdasi
Date: 2018-11-06 00:00:00
Tasks: Community Detection

Community detection is one of the most important and interesting issues in social network analysis. In recent years, simultaneous considering of nodes' attributes and topological structures of social networks in the process of community detection has attracted the attentions of many scholars, and this consideration has been recently used in some community detection methods to increase their efficiencies and to enhance their performances in finding meaningful and relevant communities. But the problem is that most of these methods tend to find non-overlapping communities, while many real-world networks include communities that often overlap to some extent. In order to solve this problem, an evolutionary algorithm called MOBBO-OCD, which is based on multi-objective biogeography-based optimization (BBO), is proposed in this paper to automatically find overlapping communities in a social network with node attributes with synchronously considering the density of connections and the similarity of nodes' attributes in the network. In MOBBO-OCD, an extended locus-based adjacency representation called OLAR is introduced to encode and decode overlapping communities. Based on OLAR, a rank-based migration operator along with a novel two-phase mutation strategy and a new double-point crossover are used in the evolution process of MOBBO-OCD to effectively lead the population into the evolution path. In order to assess the performance of MOBBO-OCD, a new metric called alpha_SAEM is proposed in this paper, which is able to evaluate the goodness of both overlapping and non-overlapping partitions with considering the two aspects of node attributes and linkage structure. Quantitative evaluations reveal that MOBBO-OCD achieves favorable results which are quite superior to the results of 15 relevant community detection algorithms in the literature.
Fast Adaptive Bilateral Filtering
Authors: Ruturaj G. Gavaskar, Kunal. N. Chaudhury
Date: 2018-11-06 00:00:00

In the classical bilateral filter, a fixed Gaussian range kernel is used
along with a spatial kernel for edge-preserving smoothing. We consider a
generalization of this filter, the so-called adaptive bilateral filter, where
the center and width of the Gaussian range kernel is allowed to change from
pixel to pixel. Though this variant was originally proposed for sharpening and
noise removal, it can also be used for other applications such as artifact
removal and texture filtering. Similar to the bilateral filter, the brute-force
implementation of its adaptive counterpart requires intense computations. While
several fast algorithms have been proposed in the literature for bilateral
filtering, most of them work only with a fixed range kernel. In this paper, we
propose a fast algorithm for adaptive bilateral filtering, whose complexity
does not scale with the spatial filter width. This is based on the observation
that the concerned filtering can be performed purely in range space using an
appropriately defined local histogram. We show that by replacing the histogram
with a polynomial and the finite range-space sum with an integral, we can
approximate the filter using analytic functions. In particular, an efficient
algorithm is derived using the following innovations: the polynomial is fitted
by matching its moments to those of the target histogram (this is done using
fast convolutions), and the analytic functions are recursively computed using
integration-by-parts. Our algorithm can accelerate the brute-force
implementation by at least $20 \times$, without perceptible distortions in the
visual quality. We demonstrate the effectiveness of our algorithm for
sharpening, JPEG deblocking, and texture filtering.
Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoning
Authors: Vasili Ramanishka, Yi-Ting Chen, Teruhisa Misu, Kate Saenko
Date: 2018-11-06 00:00:00
Proceeding: CVPR 2018 6
Tasks: Scene Understanding

Driving Scene understanding is a key ingredient for intelligent
transportation systems. To achieve systems that can operate in a complex
physical and social environment, they need to understand and learn how humans
drive and interact with traffic scenes. We present the Honda Research Institute
Driving Dataset (HDD), a challenging dataset to enable research on learning
driver behavior in real-life environments. The dataset includes 104 hours of
real human driving in the San Francisco Bay Area collected using an
instrumented vehicle equipped with different sensors. We provide a detailed
analysis of HDD with a comparison to other driving datasets. A novel annotation
methodology is introduced to enable research on driver behavior understanding
from untrimmed data sequences. As the first step, baseline algorithms for
driver behavior detection are trained and tested to demonstrate the feasibility
of the proposed task.
Modular Materialisation of Datalog Programs
Authors: Pan Hu, Boris Motik, Ian Horrocks
Date: 2018-11-06 00:00:00

The semina\"ive algorithm can materialise all consequences of arbitrary
datalog rules, and it also forms the basis for incremental algorithms that
update a materialisation as the input facts change. Certain (combinations of)
rules, however, can be handled much more efficiently using custom algorithms.
To integrate such algorithms into a general reasoning approach that can handle
arbitrary rules, we propose a modular framework for materialisation computation
and its maintenance. We split a datalog program into modules that can be
handled using specialised algorithms, and handle the remaining rules using the
semina\"ive algorithm. We also present two algorithms for computing the
transitive and the symmetric-transitive closure of a relation that can be used
within our framework. Finally, we show empirically that our framework can
handle arbitrary datalog programs while outperforming existing approaches,
often by orders of magnitude.
MDLatLRR: A novel decomposition method for infrared and visible image fusion
Authors: Hui Li, Xiao-Jun Wu, Josef Kittler
Date: 2018-11-06 00:00:00
Tasks: Infrared And Visible Image Fusion

Image decomposition is crucial for many image processing tasks, as it allows to extract salient features from source images. A good image decomposition method could lead to a better performance, especially in image fusion tasks. We propose a multi-level image decomposition method based on latent low-rank representation(LatLRR), which is called MDLatLRR. This decomposition method is applicable to many image processing fields. In this paper, we focus on the image fusion task. We develop a novel image fusion framework based on MDLatLRR, which is used to decompose source images into detail parts(salient features) and base parts. A nuclear-norm based fusion strategy is used to fuse the detail parts, and the base parts are fused by an averaging strategy. Compared with other state-of-the-art fusion methods, the proposed algorithm exhibits better fusion performance in both subjective and objective evaluation.
Revealing Fine Structures of the Retinal Receptive Field by Deep Learning Networks
Authors: Qi Yan, Yajing Zheng, Shanshan Jia, Yichen Zhang, Zhaofei Yu, Feng Chen, Yonghong Tian, Tiejun Huang, Jian. K. Liu
Date: 2018-11-06 00:00:00
Tasks: Transfer Learning

Deep convolutional neural networks (CNNs) have demonstrated impressive performance on many visual tasks. Recently, they became useful models for the visual system in neuroscience. However, it is still not clear what are learned by CNNs in terms of neuronal circuits. When a deep CNN with many layers is used for the visual system, it is not easy to compare the structure components of CNNs with possible neuroscience underpinnings due to highly complex circuits from the retina to higher visual cortex. Here we address this issue by focusing on single retinal ganglion cells with biophysical models and recording data from animals. By training CNNs with white noise images to predict neuronal responses, we found that fine structures of the retinal receptive field can be revealed. Specifically, convolutional filters learned are resembling biological components of the retinal circuit. This suggests that a CNN learning from one single retinal cell reveals a minimal neural network carried out in this cell. Furthermore, when CNNs learned from different cells are transferred between cells, there is a diversity of transfer learning performance, which indicates that CNNs are cell-specific. Moreover, when CNNs are transferred between different types of input images, here white noise v.s. natural images, transfer learning shows a good performance, which implies that CNNs indeed capture the full computational ability of a single retinal cell for different inputs. Taken together, these results suggest that CNNs could be used to reveal structure components of neuronal circuits, and provide a powerful model for neural system identification.
Comparison of Discrete Choice Models and Artificial Neural Networks in Presence of Missing Variables
Authors: Johan Barthélemy, Morgane Dumont, Timoteo Carletti
Date: 2018-11-06 00:00:00
Tasks: Binary Classification, Classification, Discrete Choice Models, General Classification

Classification, the process of assigning a label (or class) to an observation
given its features, is a common task in many applications. Nonetheless in most
real-life applications, the labels can not be fully explained by the observed
features. Indeed there can be many factors hidden to the modellers. The
unexplained variation is then treated as some random noise which is handled
differently depending on the method retained by the practitioner. This work
focuses on two simple and widely used supervised classification algorithms:
discrete choice models and artificial neural networks in the context of binary
classification.
  Through various numerical experiments involving continuous or discrete
explanatory features, we present a comparison of the retained methods'
performance in presence of missing variables. The impact of the distribution of
the two classes in the training data is also investigated. The outcomes of
those experiments highlight the fact that artificial neural networks
outperforms the discrete choice models, except when the distribution of the
classes in the training data is highly unbalanced.
  Finally, this work provides some guidelines for choosing the right classifier
with respect to the training data.
Off-the-Shelf Unsupervised NMT
Authors: Chris Hokamp, Sebastian Ruder, John Glover
Date: 2018-11-06 00:00:00
Tasks: Machine Translation, Multi-Task Learning, NMT, Translation, Unsupervised Machine Translation

We frame unsupervised machine translation (MT) in the context of multi-task
learning (MTL), combining insights from both directions. We leverage
off-the-shelf neural MT architectures to train unsupervised MT models with no
parallel data and show that such models can achieve reasonably good
performance, competitive with models purpose-built for unsupervised MT.
Finally, we propose improvements that allow us to apply our models to
English-Turkish, a truly low-resource language pair.
SparseFool: a few pixels make a big difference
Authors: Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli, Pascal Frossard
Date: 2018-11-06 00:00:00
Proceeding: CVPR 2019 6
Tasks: image-classification, Image Classification

Deep Neural Networks have achieved extraordinary results on image classification tasks, but have been shown to be vulnerable to attacks with carefully crafted perturbations of the input data. Although most attacks usually change values of many image's pixels, it has been shown that deep networks are also vulnerable to sparse alterations of the input. However, no computationally efficient method has been proposed to compute sparse perturbations. In this paper, we exploit the low mean curvature of the decision boundary, and propose SparseFool, a geometry inspired sparse attack that controls the sparsity of the perturbations. Extensive evaluations show that our approach computes sparse perturbations very fast, and scales efficiently to high dimensional data. We further analyze the transferability and the visual effects of the perturbations, and show the existence of shared semantic information across the images and the networks. Finally, we show that adversarial training can only slightly improve the robustness against sparse additive perturbations computed with SparseFool.
Semantic bottleneck for computer vision tasks
Authors: Maxime Bucher, Stéphane Herbin, Frédéric Jurie
Date: 2018-11-06 00:00:00
Tasks: Content-Based Image Retrieval, General Classification, image-classification, Image Classification, Image Retrieval, Retrieval

This paper introduces a novel method for the representation of images that is
semantic by nature, addressing the question of computation intelligibility in
computer vision tasks. More specifically, our proposition is to introduce what
we call a semantic bottleneck in the processing pipeline, which is a crossing
point in which the representation of the image is entirely expressed with
natural language , while retaining the efficiency of numerical representations.
We show that our approach is able to generate semantic representations that
give state-of-the-art results on semantic content-based image retrieval and
also perform very well on image classification tasks. Intelligibility is
evaluated through user centered experiments for failure detection.
Weakly Supervised Scene Parsing with Point-based Distance Metric Learning
Authors: Rui Qian, Yunchao Wei, Honghui Shi, Jiachen Li, Jiaying Liu, Thomas Huang
Date: 2018-11-06 00:00:00
Tasks: Metric Learning, Scene Parsing

Semantic scene parsing is suffering from the fact that pixel-level
annotations are hard to be collected. To tackle this issue, we propose a
Point-based Distance Metric Learning (PDML) in this paper. PDML does not
require dense annotated masks and only leverages several labeled points that
are much easier to obtain to guide the training process. Concretely, we
leverage semantic relationship among the annotated points by encouraging the
feature representations of the intra- and inter-category points to keep
consistent, i.e. points within the same category should have more similar
feature representations compared to those from different categories. We
formulate such a characteristic into a simple distance metric loss, which
collaborates with the point-wise cross-entropy loss to optimize the deep neural
networks. Furthermore, to fully exploit the limited annotations, distance
metric learning is conducted across different training images instead of simply
adopting an image-dependent manner. We conduct extensive experiments on two
challenging scene parsing benchmarks of PASCAL-Context and ADE 20K to validate
the effectiveness of our PDML, and competitive mIoU scores are achieved.
CIS at TAC Cold Start 2015: Neural Networks and Coreference Resolution for Slot Filling
Authors: Heike Adel, Hinrich Schütze
Date: 2018-11-06 00:00:00
Tasks: coreference-resolution, Coreference Resolution, General Classification, slot-filling, Slot Filling

This paper describes the CIS slot filling system for the TAC Cold Start
evaluations 2015. It extends and improves the system we have built for the
evaluation last year. This paper mainly describes the changes to our last
year's system. Especially, it focuses on the coreference and classification
component. For coreference, we have performed several analysis and prepared a
resource to simplify our end-to-end system and improve its runtime. For
classification, we propose to use neural networks. We have trained
convolutional and recurrent neural networks and combined them with traditional
evaluation methods, namely patterns and support vector machines. Our runs for
the 2015 evaluation have been designed to directly assess the effect of each
network on the end-to-end performance of the system. The CIS system achieved
rank 3 of all slot filling systems participating in the task.
Kernel Exponential Family Estimation via Doubly Dual Embedding
Authors: Bo Dai, Hanjun Dai, Arthur Gretton, Le Song, Dale Schuurmans, Niao He
Date: 2018-11-06 00:00:00

We investigate penalized maximum log-likelihood estimation for exponential
family distributions whose natural parameter resides in a reproducing kernel
Hilbert space. Key to our approach is a novel technique, doubly dual embedding,
that avoids computation of the partition function. This technique also allows
the development of a flexible sampling strategy that amortizes the cost of
Monte-Carlo sampling in the inference stage. The resulting estimator can be
easily generalized to kernel conditional exponential families. We establish a
connection between kernel exponential family estimation and MMD-GANs, revealing
a new perspective for understanding GANs. Compared to the score matching based
estimators, the proposed method improves both memory and time efficiency while
enjoying stronger statistical properties, such as fully capturing smoothness in
its statistical convergence rate while the score matching estimator appears to
saturate. Finally, we show that the proposed estimator empirically outperforms
state-of-the-art
A Quasi-Newton algorithm on the orthogonal manifold for NMF with transform learning
Authors: Pierre Ablin, Dylan Fagot, Herwig Wendt, Alexandre Gramfort, Cédric Févotte
Date: 2018-11-06 00:00:00

Nonnegative matrix factorization (NMF) is a popular method for audio spectral
unmixing. While NMF is traditionally applied to off-the-shelf time-frequency
representations based on the short-time Fourier or Cosine transforms, the
ability to learn transforms from raw data attracts increasing attention.
However, this adds an important computational overhead. When assumed orthogonal
(like the Fourier or Cosine transforms), learning the transform yields a
non-convex optimization problem on the orthogonal matrix manifold. In this
paper, we derive a quasi-Newton method on the manifold using sparse
approximations of the Hessian. Experiments on synthetic and real audio data
show that the proposed algorithm out-performs state-of-the-art first-order and
coordinate-descent methods by orders of magnitude. A Python package for fast
TL-NMF is released online at https://github.com/pierreablin/tlnmf.