pwc-archive/papers-with-abstracts huggingface dataset:
Information geometry for approximate Bayesian computation
Authors: Konstantinos Spiliopoulos
Date: 2018-12-05 00:00:00

The goal of this paper is to explore the basic Approximate Bayesian Computation (ABC) algorithm via the lens of information theory. ABC is a widely used algorithm in cases where the likelihood of the data is hard to work with or intractable, but one can simulate from it. We use relative entropy ideas to analyze the behavior of the algorithm as a function of the threshold parameter and of the size of the data. Relative entropy here is data driven as it depends on the values of the observed statistics. Relative entropy also allows us to explore the effect of the distance metric and sets up a mathematical framework for sensitivity analysis allowing to find important directions which could lead to lower computational cost of the algorithm for the same level of accuracy. In addition, we also investigate the bias of the estimators for generic observables as a function of both the threshold parameters and the size of the data. Our analysis provides error bounds on performance for positive tolerances and finite sample sizes. Simulation studies complement and illustrate the theoretical results.
Learning Attraction Field Representation for Robust Line Segment Detection
Authors: Nan Xue, Song Bai, Fu-Dong Wang, Gui-Song Xia, Tianfu Wu, Liangpei Zhang
Date: 2018-12-05 00:00:00
Proceeding: CVPR 2019 6
Tasks: Line Segment Detection, Semantic Segmentation

This paper presents a region-partition based attraction field dual
representation for line segment maps, and thus poses the problem of line
segment detection (LSD) as the region coloring problem. The latter is then
addressed by learning deep convolutional neural networks (ConvNets) for
accuracy, robustness and efficiency. For a 2D line segment map, our dual
representation consists of three components: (i) A region-partition map in
which every pixel is assigned to one and only one line segment; (ii) An
attraction field map in which every pixel in a partition region is encoded by
its 2D projection vector w.r.t. the associated line segment; and (iii) A
squeeze module which squashes the attraction field to a line segment map that
almost perfectly recovers the input one. By leveraging the duality, we learn
ConvNets to compute the attraction field maps for raw in-put images, followed
by the squeeze module for LSD, in an end-to-end manner. Our method rigorously
addresses several challenges in LSD such as local ambiguity and class
imbalance. Our method also harnesses the best practices developed in ConvNets
based semantic segmentation methods such as the encoder-decoder architecture
and the a-trous convolution. In experiments, our method is tested on the
WireFrame dataset and the YorkUrban dataset with state-of-the-art performance
obtained. Especially, we advance the performance by 4.5 percents on the
WireFrame dataset. Our method is also fast with 6.6~10.4 FPS, outperforming
most of existing line segment detectors.
Inflection-Tolerant Ontology-Based Named Entity Recognition for Real-Time Applications
Authors: Christian Jilek, Markus Schröder, Rudolf Novik, Sven Schwarz, Heiko Maus, Andreas Dengel
Date: 2018-12-05 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

A growing number of applications users daily interact with have to operate in
(near) real-time: chatbots, digital companions, knowledge work support systems
-- just to name a few. To perform the services desired by the user, these
systems have to analyze user activity logs or explicit user input extremely
fast. In particular, text content (e.g. in form of text snippets) needs to be
processed in an information extraction task. Regarding the aforementioned
temporal requirements, this has to be accomplished in just a few milliseconds,
which limits the number of methods that can be applied. Practically, only very
fast methods remain, which on the other hand deliver worse results than slower
but more sophisticated Natural Language Processing (NLP) pipelines. In this
paper, we investigate and propose methods for real-time capable Named Entity
Recognition (NER). As a first improvement step we address are word variations
induced by inflection, for example present in the German language. Our approach
is ontology-based and makes use of several language information sources like
Wiktionary. We evaluated it using the German Wikipedia (about 9.4B characters),
for which the whole NER process took considerably less than an hour. Since
precision and recall are higher than with comparably fast methods, we conclude
that the quality gap between high speed methods and sophisticated NLP pipelines
can be narrowed a bit more without losing too much runtime performance.
Relative concentration bounds for the spectrum of kernel matrices
Authors: Ernesto Araya Valdivia
Date: 2018-12-05 00:00:00

In this paper we study the concentration properties for the eigenvalues of kernel matrices, which are central objects in a wide range of kernel methods and, more recently, in network analysis. We present a set of concentration inequalities tailored for each individual eigenvalue of the kernel matrix with respect to its known asymptotic limit. The inequalities presented here are of relative type, meaning that they scale with the eigenvalue in consideration, which results in convergence rates that vary across the spectrum. The rates we obtain here are faster than the typical $\O(\frac{1}{\sqrt n})$ and are often exponential, depending on regularity assumptions of Sobolev type. One key feature of our results is that they apply to non positive kernels, which is fundamental in the context of network analysis. We show how our results are well suited for the study of dot product kernels, which are related to random geometric graphs on the sphere, via the graphon formalism. We illustrate our results by applying them to a variety of dot product kernels on the sphere and to the one dimensional Gaussian kernel.
Understanding Individual Decisions of CNNs via Contrastive Backpropagation
Authors: Jindong Gu, Yinchong Yang, Volker Tresp
Date: 2018-12-05 00:00:00
Tasks: General Classification

A number of backpropagation-based approaches such as DeConvNets, vanilla Gradient Visualization and Guided Backpropagation have been proposed to better understand individual decisions of deep convolutional neural networks. The saliency maps produced by them are proven to be non-discriminative. Recently, the Layer-wise Relevance Propagation (LRP) approach was proposed to explain the classification decisions of rectifier neural networks. In this work, we evaluate the discriminativeness of the generated explanations and analyze the theoretical foundation of LRP, i.e. Deep Taylor Decomposition. The experiments and analysis conclude that the explanations generated by LRP are not class-discriminative. Based on LRP, we propose Contrastive Layer-wise Relevance Propagation (CLRP), which is capable of producing instance-specific, class-discriminative, pixel-wise explanations. In the experiments, we use the CLRP to explain the decisions and understand the difference between neurons in individual classification decisions. We also evaluate the explanations quantitatively with a Pointing Game and an ablation study. Both qualitative and quantitative evaluations show that the CLRP generates better explanations than the LRP. The code is available.
Unlabeled sample compression schemes and corner peelings for ample and maximum classes
Authors: Jérémie Chalopin, Victor Chepoi, Shay Moran, Manfred K. Warmuth
Date: 2018-12-05 00:00:00
Tasks: BIG-bench Machine Learning

We examine connections between combinatorial notions that arise in machine learning and topological notions in cubical/simplicial geometry. These connections enable to export results from geometry to machine learning. Our first main result is based on a geometric construction by Tracy Hall (2004) of a partial shelling of the cross-polytope which can not be extended. We use it to derive a maximum class of VC dimension 3 that has no corners. This refutes several previous works in machine learning from the past 11 years. In particular, it implies that all previous constructions of optimal unlabeled sample compression schemes for maximum classes are erroneous. On the positive side we present a new construction of an unlabeled sample compression scheme for maximum classes. We leave as open whether our unlabeled sample compression scheme extends to ample (a.k.a. lopsided or extremal) classes, which represent a natural and far-reaching generalization of maximum classes. Towards resolving this question, we provide a geometric characterization in terms of unique sink orientations of the 1-skeletons of associated cubical complexes.
Low-Complexity Data-Parallel Earth Mover's Distance Approximations
Authors: Kubilay Atasu, Thomas Mittelholzer
Date: 2018-12-05 00:00:00
Tasks: CPU, GPU

The Earth Mover's Distance (EMD) is a state-of-the art metric for comparing discrete probability distributions, but its high distinguishability comes at a high cost in computational complexity. Even though linear-complexity approximation algorithms have been proposed to improve its scalability, these algorithms are either limited to vector spaces with only a few dimensions or they become ineffective when the degree of overlap between the probability distributions is high. We propose novel approximation algorithms that overcome both of these limitations, yet still achieve linear time complexity. All our algorithms are data parallel, and thus, we take advantage of massively parallel computing engines, such as Graphics Processing Units (GPUs). On the popular text-based 20 Newsgroups dataset, the new algorithms are four orders of magnitude faster than a multi-threaded CPU implementation of Word Mover's Distance and match its nearest-neighbors-search accuracy. On MNIST images, the new algorithms are four orders of magnitude faster than a GPU implementation of the Sinkhorn's algorithm while offering a slightly higher nearest-neighbors-search accuracy.
Brain Segmentation from k-space with End-to-end Recurrent Attention Network
Authors: Qiaoying Huang, Xiao Chen, Dimitris Metaxas, Mariappan S. Nadar
Date: 2018-12-05 00:00:00
Tasks: Brain Image Segmentation, Brain Segmentation, Image Reconstruction, Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

The task of medical image segmentation commonly involves an image reconstruction step to convert acquired raw data to images before any analysis. However, noises, artifacts and loss of information due to the reconstruction process are almost inevitable, which compromises the final performance of segmentation. We present a novel learning framework that performs magnetic resonance brain image segmentation directly from k-space data. The end-to-end framework consists of a unique task-driven attention module that recurrently utilizes intermediate segmentation estimation to facilitate image-domain feature extraction from the raw data, thus closely bridging the reconstruction and the segmentation tasks. In addition, to address the challenge of manual labeling, we introduce a novel workflow to generate labeled training data for segmentation by exploiting imaging modality simulators and digital phantoms. Extensive experimental results show that the proposed method outperforms several state-of-the-art methods.
Estimation of multivariate asymmetric power GARCH models
Authors: Yacouba Boubacar Maïnassara, Othman Kadmiri, Bruno Saussereau
Date: 2018-12-05 00:00:00
Tasks: Time Series, Time Series Analysis

It is now widely accepted that volatility models have to incorporate the so-called leverage effect in order to to model the dynamics of daily financial returns.We suggest a new class of multivariate power transformed asymmetric models. It includes several functional forms of multivariate GARCH models which are of great interest in financial modeling and time series literature. We provide an explicit necessary and sufficient condition to establish the strict stationarity of the model. We derive the asymptotic properties of the quasi-maximum likelihood estimator of the parameters. These properties are established both when the power of the transformation is known or is unknown. The asymptotic results are illustrated by Monte Carlo experiments. An application to real financial data is also proposed.
Point-to-Pose Voting based Hand Pose Estimation using Residual Permutation Equivariant Layer
Authors: Shile Li, Dongheui Lee
Date: 2018-12-05 00:00:00
Proceeding: CVPR 2019 6
Tasks: Hand Pose Estimation, Point Cloud Segmentation, Pose Estimation

Recently, 3D input data based hand pose estimation methods have shown
state-of-the-art performance, because 3D data capture more spatial information
than the depth image. Whereas 3D voxel-based methods need a large amount of
memory, PointNet based methods need tedious preprocessing steps such as
K-nearest neighbour search for each point. In this paper, we present a novel
deep learning hand pose estimation method for an unordered point cloud. Our
method takes 1024 3D points as input and does not require additional
information. We use Permutation Equivariant Layer (PEL) as the basic element,
where a residual network version of PEL is proposed for the hand pose
estimation task. Furthermore, we propose a voting based scheme to merge
information from individual points to the final pose output. In addition to the
pose estimation task, the voting-based scheme can also provide point cloud
segmentation result without ground-truth for segmentation. We evaluate our
method on both NYU dataset and the Hands2017Challenge dataset. Our method
outperforms recent state-of-the-art methods, where our pose accuracy is
currently the best for the Hands2017Challenge dataset.
Learn to See by Events: Color Frame Synthesis from Event and RGB Cameras
Authors: Stefano Pini, Guido Borghi, Roberto Vezzani
Date: 2018-12-05 00:00:00

Event cameras are biologically-inspired sensors that gather the temporal evolution of the scene. They capture pixel-wise brightness variations and output a corresponding stream of asynchronous events. Despite having multiple advantages with respect to traditional cameras, their use is partially prevented by the limited applicability of traditional data processing and vision algorithms. To this aim, we present a framework which exploits the output stream of event cameras to synthesize RGB frames, relying on an initial or a periodic set of color key-frames and the sequence of intermediate events. Differently from existing work, we propose a deep learning-based frame synthesis method, consisting of an adversarial architecture combined with a recurrent module. Qualitative results and quantitative per-pixel, perceptual, and semantic evaluation on four public datasets confirm the quality of the synthesized images.
Stochastic Model Pruning via Weight Dropping Away and Back
Authors: Haipeng Jia, Xueshuang Xiang, Da Fan, Meiyu Huang, Changhao Sun, Yang He
Date: 2018-12-05 00:00:00
Tasks: Model Compression, Stochastic Optimization

Deep neural networks have dramatically achieved great success on a variety of challenging tasks. However, most successful DNNs have an extremely complex structure, leading to extensive research on model compression.As a significant area of progress in model compression, traditional gradual pruning approaches involve an iterative prune-retrain procedure and may suffer from two critical issues: local importance judgment, where the pruned weights are merely unimportant in the current model; and an irretrievable pruning process, where the pruned weights have no chance to come back. Addressing these two issues, this paper proposes the Drop Pruning approach, which leverages stochastic optimization in the pruning process by introducing a drop strategy at each pruning step, namely, drop away, which stochastically deletes some unimportant weights, and drop back, which stochastically recovers some pruned weights. The suitable choice of drop probabilities decreases the model size during the pruning process and helps it flow to the target sparsity. Compared to the Bayesian approaches that stochastically train a compact model for pruning, we directly aim at stochastic gradual pruning. We provide a detailed analysis showing that the drop away and drop back approaches have individual contributions. Moreover, Drop Pruning can achieve competitive compression performance and accuracy on many benchmark tasks compared with state-of-the-art weights pruning and Bayesian training approaches.
Wireless Data Acquisition for Edge Learning: Data-Importance Aware Retransmission
Authors: Dongzhu Liu, Guangxu Zhu, Jun Zhang, Kaibin Huang
Date: 2018-12-05 00:00:00

By deploying machine-learning algorithms at the network edge, edge learning
can leverage the enormous real-time data generated by billions of mobile
devices to train AI models, which enable intelligent mobile applications. In
this emerging research area, one key direction is to efficiently utilize radio
resources for wireless data acquisition to minimize the latency of executing a
learning task at an edge server. Along this direction, we consider the specific
problem of retransmission decision in each communication round to ensure both
reliability and quantity of those training data for accelerating model
convergence. To solve the problem, a new retransmission protocol called
data-importance aware automatic-repeat-request (importance ARQ) is proposed.
Unlike the classic ARQ focusing merely on reliability, importance ARQ
selectively retransmits a data sample based on its uncertainty which helps
learning and can be measured using the model under training. Underpinning the
proposed protocol is a derived elegant communication-learning relation between
two corresponding metrics, i.e., signal-to-noise ratio (SNR) and data
uncertainty. This relation facilitates the design of a simple threshold based
policy for importance ARQ. The policy is first derived based on the classic
classifier model of support vector machine (SVM), where the uncertainty of a
data sample is measured by its distance to the decision boundary. The policy is
then extended to the more complex model of convolutional neural networks (CNN)
where data uncertainty is measured by entropy. Extensive experiments have been
conducted for both the SVM and CNN using real datasets with balanced and
imbalanced distributions. Experimental results demonstrate that importance ARQ
effectively copes with channel fading and noise in wireless data acquisition to
achieve faster model convergence than the conventional channel-aware ARQ.
Dynamic Spatio-temporal Graph-based CNNs for Traffic Prediction
Authors: Ken Chen, Fei Chen, Baisheng Lai, Zhongming Jin, Yong liu, Kai Li, Long Wei, Pengfei Wang, Yandong Tang, Jianqiang Huang, Xian-Sheng Hua
Date: 2018-12-05 00:00:00
Tasks: Prediction, Traffic Prediction

Forecasting future traffic flows from previous ones is a challenging problem because of their complex and dynamic nature of spatio-temporal structures. Most existing graph-based CNNs attempt to capture the static relations while largely neglecting the dynamics underlying sequential data. In this paper, we present dynamic spatio-temporal graph-based CNNs (DST-GCNNs) by learning expressive features to represent spatio-temporal structures and predict future traffic flows from surveillance video data. In particular, DST-GCNN is a two stream network. In the flow prediction stream, we present a novel graph-based spatio-temporal convolutional layer to extract features from a graph representation of traffic flows. Then several such layers are stacked together to predict future flows over time. Meanwhile, the relations between traffic flows in the graph are often time variant as the traffic condition changes over time. To capture the graph dynamics, we use the graph prediction stream to predict the dynamic graph structures, and the predicted structures are fed into the flow prediction stream. Experiments on real datasets demonstrate that the proposed model achieves competitive performances compared with the other state-of-the-art methods.
VideoMem: Constructing, Analyzing, Predicting Short-term and Long-term Video Memorability
Authors: Romain Cohendet, Claire-Hélène Demarty, Ngoc Q. K. Duong, Martin Engilberge
Date: 2018-12-05 00:00:00
Proceeding: ICCV 2019 10
Tasks: Memorization, Prediction

Humans share a strong tendency to memorize/forget some of the visual
information they encounter. This paper focuses on providing computational
models for the prediction of the intrinsic memorability of visual content. To
address this new challenge, we introduce a large scale dataset (VideoMem)
composed of 10,000 videos annotated with memorability scores. In contrast to
previous work on image memorability -- where memorability was measured a few
minutes after memorization -- memory performance is measured twice: a few
minutes after memorization and again 24-72 hours later. Hence, the dataset
comes with short-term and long-term memorability annotations. After an in-depth
analysis of the dataset, we investigate several deep neural network based
models for the prediction of video memorability. Our best model using a ranking
loss achieves a Spearman's rank correlation of 0.494 for short-term
memorability prediction, while our proposed model with attention mechanism
provides insights of what makes a content memorable. The VideoMem dataset with
pre-extracted features is publicly available.
Summarizing Videos with Attention
Authors: Jiri Fajtl, Hajar Sadeghi Sokeh, Vasileios Argyriou, Dorothy Monekosso, Paolo Remagnino
Date: 2018-12-05 00:00:00
Tasks: Video Summarization

In this work we propose a novel method for supervised, keyshots based video
summarization by applying a conceptually simple and computationally efficient
soft, self-attention mechanism. Current state of the art methods leverage
bi-directional recurrent networks such as BiLSTM combined with attention. These
networks are complex to implement and computationally demanding compared to
fully connected networks. To that end we propose a simple, self-attention based
network for video summarization which performs the entire sequence to sequence
transformation in a single feed forward pass and single backward pass during
training. Our method sets a new state of the art results on two benchmarks
TvSum and SumMe, commonly used in this domain.
Unsupervised Feature Learning Architecture with Multi-clustering Integration RBM
Authors: Jielei Chu, Hongjun Wang, Jing Liu, Zhiguo Gong, Tianrui Li
Date: 2018-12-05 00:00:00
Tasks: Clustering

In this paper, we present a novel unsupervised feature learning architecture, which consists of a multi-clustering integration module and a variant of RBM termed multi-clustering integration RBM (MIRBM). In the multi-clustering integration module, we apply three unsupervised K-means, affinity propagation and spectral clustering algorithms to obtain three different clustering partitions (CPs) without any background knowledge or label. Then, an unanimous voting strategy is used to generate a local clustering partition (LCP). The novel MIRBM model is a core feature encoding part of the proposed unsupervised feature learning architecture. The novelty of it is that the LCP as an unsupervised guidance is integrated into one step contrastive divergence (CD1) learning to guide the distribution of the hidden layer features. For the instance in the same LCP cluster, the hidden and reconstructed hidden layer features of the MIRBM model in the proposed architecture tend to constrict together in the training process. Meanwhile, each LCP center tends to disperse from each other as much as possible in the hidden and reconstructed hidden layer during training. The experiments demonstrate that the proposed unsupervised feature learning architecture has more powerful feature representation and generalization capability than the state-of-the-art graph regularized RBM (GraphRBM) for clustering tasks in the Microsoft Research Asia Multimedia (MSRA-MM)2.0 dataset.