pwc-archive/papers-with-abstracts huggingface dataset:
A Generative Model of Textures Using Hierarchical Probabilistic Principal Component Analysis
Authors: Aiga Suzuki, Hayaru Shouno
Date: 2018-10-16 00:00:00
Tasks: Dimensionality Reduction

Modeling of textures in natural images is an important task to make a
microscopic model of natural images. Portilla and Simoncelli proposed a
generative texture model, which is based on the mechanism of visual systems in
brains, with a set of texture features and a feature matching. On the other
hand, the texture features, used in Portillas' model, have redundancy between
its components came from typical natural textures. In this paper, we propose a
contracted texture model which provides a dimension reduction for the
Portillas' feature. This model is based on a hierarchical principal components
analysis using known group structure of the feature. In the experiment, we
reveal effective dimensions to describe texture is fewer than the original
description. Moreover, we also demonstrate how well the textures can be
synthesized from the contracted texture representations.
The LORACs prior for VAEs: Letting the Trees Speak for the Data
Authors: Sharad Vikram, Matthew D. Hoffman, Matthew J. Johnson
Date: 2018-10-16 00:00:00
Tasks: Clustering

In variational autoencoders, the prior on the latent codes $z$ is often
treated as an afterthought, but the prior shapes the kind of latent
representation that the model learns. If the goal is to learn a representation
that is interpretable and useful, then the prior should reflect the ways in
which the high-level factors that describe the data vary. The "default" prior
is an isotropic normal, but if the natural factors of variation in the dataset
exhibit discrete structure or are not independent, then the isotropic-normal
prior will actually encourage learning representations that mask this
structure. To alleviate this problem, we propose using a flexible Bayesian
nonparametric hierarchical clustering prior based on the time-marginalized
coalescent (TMC). To scale learning to large datasets, we develop a new
inducing-point approximation and inference algorithm. We then apply the method
without supervision to several datasets and examine the interpretability and
practical performance of the inferred hierarchies and learned latent space.
Rotational 3D Texture Classification Using Group Equivariant CNNs
Authors: Vincent Andrearczyk, Adrien Depeursinge
Date: 2018-10-16 00:00:00
Tasks: Classification, Diversity, General Classification, Texture Classification, Translation

Convolutional Neural Networks (CNNs) traditionally encode translation
equivariance via the convolution operation. Generalization to other
transformations has recently received attraction to encode the knowledge of the
data geometry in group convolution operations. Equivariance to rotation is
particularly important for 3D image analysis due to the large diversity of
possible pattern orientations. 3D texture is a particularly important cue for
the analysis of medical images such as CT and MRI scans as it describes
different types of tissues and lesions. In this paper, we evaluate the use of
3D group equivariant CNNs accounting for the simplified group of right-angle
rotations to classify 3D synthetic textures from a publicly available dataset.
The results validate the importance of rotation equivariance in a controlled
setup and yet motivate the use of a finer coverage of orientations in order to
obtain equivariance to realistic rotations present in 3D textures.
Collaborative Deep Learning Across Multiple Data Centers
Authors: Kele Xu, Haibo Mi, Dawei Feng, Huaimin Wang, Chuan Chen, Zibin Zheng, Xu Lan
Date: 2018-10-16 00:00:00
Tasks: Deep Learning

Valuable training data is often owned by independent organizations and
located in multiple data centers. Most deep learning approaches require to
centralize the multi-datacenter data for performance purpose. In practice,
however, it is often infeasible to transfer all data to a centralized data
center due to not only bandwidth limitation but also the constraints of privacy
regulations. Model averaging is a conventional choice for data parallelized
training, but its ineffectiveness is claimed by previous studies as deep neural
networks are often non-convex. In this paper, we argue that model averaging can
be effective in the decentralized environment by using two strategies, namely,
the cyclical learning rate and the increased number of epochs for local model
training. With the two strategies, we show that model averaging can provide
competitive performance in the decentralized mode compared to the
data-centralized one. In a practical environment with multiple data centers, we
conduct extensive experiments using state-of-the-art deep network architectures
on different types of data. Results demonstrate the effectiveness and
robustness of the proposed method.
A Roadmap Towards Resilient Internet of Things for Cyber-Physical Systems
Authors: Denise Ratasich, Faiq Khalid, Florian Geissler, Radu Grosu, Muhammad Shafique, Ezio Bartocci
Date: 2018-10-16 00:00:00
Tasks: Anomaly Detection, Autonomous Vehicles

The Internet of Things (IoT) is a ubiquitous system connecting many different
devices - the things - which can be accessed from the distance. The
cyber-physical systems (CPS) monitor and control the things from the distance.
As a result, the concepts of dependability and security get deeply intertwined.
The increasing level of dynamicity, heterogeneity, and complexity adds to the
system's vulnerability, and challenges its ability to react to faults. This
paper summarizes state-of-the-art of existing work on anomaly detection,
fault-tolerance and self-healing, and adds a number of other methods applicable
to achieve resilience in an IoT. We particularly focus on non-intrusive methods
ensuring data integrity in the network. Furthermore, this paper presents the
main challenges in building a resilient IoT for CPS which is crucial in the era
of smart CPS with enhanced connectivity (an excellent example of such a system
is connected autonomous vehicles). It further summarizes our solutions,
work-in-progress and future work to this topic to enable "Trustworthy IoT for
CPS". Finally, this framework is illustrated on a selected use case: A smart
sensor infrastructure in the transport domain.
Faster Matrix Completion Using Randomized SVD
Authors: Xu Feng, Wenjian Yu, Yaohang Li
Date: 2018-10-16 00:00:00
Tasks: CPU, Image Inpainting, Matrix Completion, Recommendation Systems

Matrix completion is a widely used technique for image inpainting and
personalized recommender system, etc. In this work, we focus on accelerating
the matrix completion using faster randomized singular value decomposition
(rSVD). Firstly, two fast randomized algorithms (rSVD-PI and rSVD- BKI) are
proposed for handling sparse matrix. They make use of an eigSVD procedure and
several accelerating skills. Then, with the rSVD-BKI algorithm and a new
subspace recycling technique, we accelerate the singular value thresholding
(SVT) method in [1] to realize faster matrix completion. Experiments show that
the proposed rSVD algorithms can be 6X faster than the basic rSVD algorithm [2]
while keeping same accuracy. For image inpainting and movie-rating estimation
problems, the proposed accelerated SVT algorithm consumes 15X and 8X less CPU
time than the methods using svds and lansvd respectively, without loss of
accuracy.
Semantic Aware Attention Based Deep Object Co-segmentation
Authors: Hong Chen, Yifei HUANG, Hideki Nakayama
Date: 2018-10-16 00:00:00
Tasks: Object, Segmentation

Object co-segmentation is the task of segmenting the same objects from
multiple images. In this paper, we propose the Attention Based Object
Co-Segmentation for object co-segmentation that utilize a novel attention
mechanism in the bottleneck layer of deep neural network for the selection of
semantically related features. Furthermore, we take the benefit of attention
learner and propose an algorithm to segment multi-input images in linear time
complexity. Experiment results demonstrate that our model achieves state of the
art performance on multiple datasets, with a significant reduction of
computational time.
Sharp Analysis of Learning with Discrete Losses
Authors: Alex Nowak-Vila, Francis Bach, Alessandro Rudi
Date: 2018-10-16 00:00:00

The problem of devising learning strategies for discrete losses (e.g.,
multilabeling, ranking) is currently addressed with methods and theoretical
analyses ad-hoc for each loss. In this paper we study a least-squares framework
to systematically design learning algorithms for discrete losses, with
quantitative characterizations in terms of statistical and computational
complexity. In particular we improve existing results by providing explicit
dependence on the number of labels for a wide class of losses and faster
learning rates in conditions of low-noise. Theoretical results are complemented
with experiments on real datasets, showing the effectiveness of the proposed
general approach.
Finite-sample analysis of M-estimators using self-concordance
Authors: Dmitrii Ostrovskii, Francis Bach
Date: 2018-10-16 00:00:00

The classical asymptotic theory for parametric $M$-estimators guarantees that, in the limit of infinite sample size, the excess risk has a chi-square type distribution, even in the misspecified case. We demonstrate how self-concordance of the loss allows to characterize the critical sample size sufficient to guarantee a chi-square type in-probability bound for the excess risk. Specifically, we consider two classes of losses: (i) self-concordant losses in the classical sense of Nesterov and Nemirovski, i.e., whose third derivative is uniformly bounded with the $3/2$ power of the second derivative; (ii) pseudo self-concordant losses, for which the power is removed. These classes contain losses corresponding to several generalized linear models, including the logistic loss and pseudo-Huber losses. Our basic result under minimal assumptions bounds the critical sample size by $O(d \cdot d_{\text{eff}}),$ where $d$ the parameter dimension and $d_{\text{eff}}$ the effective dimension that accounts for model misspecification. In contrast to the existing results, we only impose local assumptions that concern the population risk minimizer $\theta_*$. Namely, we assume that the calibrated design, i.e., design scaled by the square root of the second derivative of the loss, is subgaussian at $\theta_*$. Besides, for type-ii losses we require boundedness of a certain measure of curvature of the population risk at $\theta_*$.Our improved result bounds the critical sample size from above as $O(\max\{d_{\text{eff}}, d \log d\})$ under slightly stronger assumptions. Namely, the local assumptions must hold in the neighborhood of $\theta_*$ given by the Dikin ellipsoid of the population risk. Interestingly, we find that, for logistic regression with Gaussian design, there is no actual restriction of conditions: the subgaussian parameter and curvature measure remain near-constant over the Dikin ellipsoid. Finally, we extend some of these results to $\ell_1$-penalized estimators in high dimensions.
SpiNNTools: The Execution Engine for the SpiNNaker Platform
Authors: Andrew G. D. Rowley, Christian Brenninkmeijer, Simon Davidson, Donal Fellows, Andrew Gait, David R. Lester, Luis A. Plana, Oliver Rhodes, Alan B. Stokes, Steve B. Furber
Date: 2018-10-16 00:00:00
Tasks: Retrieval

Distributed systems are becoming more common place, as computers typically
contain multiple computation processors. The SpiNNaker architecture is such a
distributed architecture, containing millions of cores connected with a unique
communication network, making it one of the largest neuromorphic computing
platforms in the world. Utilising these processors efficiently usually requires
expert knowledge of the architecture to generate executable code. This work
introduces a set of tools (SpiNNTools) that can map computational work
described as a graph in to executable code that runs on this novel machine. The
SpiNNaker architecture is highly scalable which in turn produces unique
challenges in loading data, executing the mapped problem and the retrieval of
data. In this paper we describe these challenges in detail and the solutions
implemented.
Maximizing Monotone DR-submodular Continuous Functions by Derivative-free Optimization
Authors: Yibo Zhang, Chao Qian, Ke Tang
Date: 2018-10-16 00:00:00

In this paper, we study the problem of monotone (weakly) DR-submodular
continuous maximization. While previous methods require the gradient
information of the objective function, we propose a derivative-free algorithm
LDGM for the first time. We define $\beta$ and $\alpha$ to characterize how
close a function is to continuous DR-submodulr and submodular, respectively.
Under a convex polytope constraint, we prove that LDGM can achieve a
$(1-e^{-\beta}-\epsilon)$-approximation guarantee after $O(1/\epsilon)$
iterations, which is the same as the best previous gradient-based algorithm.
Moreover, in some special cases, a variant of LDGM can achieve a
$((\alpha/2)(1-e^{-\alpha})-\epsilon)$-approximation guarantee for (weakly)
submodular functions. We also compare LDGM with the gradient-based algorithm
Frank-Wolfe under noise, and show that LDGM can be more robust. Empirical
results on budget allocation verify the effectiveness of LDGM.
Combined Static and Motion Features for Deep-Networks Based Activity Recognition in Videos
Authors: Sameera Ramasinghe, Jathushan Rajasegaran, Vinoj Jayasundara, Kanchana Ranasinghe, Ranga Rodrigo, Ajith A. Pasqual
Date: 2018-10-16 00:00:00
Tasks: Activity Recognition, Activity Recognition In Videos

Activity recognition in videos in a deep-learning setting---or
otherwise---uses both static and pre-computed motion components. The method of
combining the two components, whilst keeping the burden on the deep network
less, still remains uninvestigated. Moreover, it is not clear what the level of
contribution of individual components is, and how to control the contribution.
In this work, we use a combination of CNN-generated static features and motion
features in the form of motion tubes. We propose three schemas for combining
static and motion components: based on a variance ratio, principal components,
and Cholesky decomposition. The Cholesky decomposition based method allows the
control of contributions. The ratio given by variance analysis of static and
motion features match well with the experimental optimal ratio used in the
Cholesky decomposition based method. The resulting activity recognition system
is better or on par with existing state-of-the-art when tested with three
popular datasets. The findings also enable us to characterize a dataset with
respect to its richness in motion information.
Multi-Source Neural Machine Translation with Data Augmentation
Authors: Yuta Nishimura, Katsuhito Sudoh, Graham Neubig, Satoshi Nakamura
Date: 2018-10-16 00:00:00
Proceeding: IWSLT (EMNLP) 2018 10
Tasks: Data Augmentation, Machine Translation, NMT, Translation

Multi-source translation systems translate from multiple languages to a
single target language. By using information from these multiple sources, these
systems achieve large gains in accuracy. To train these systems, it is
necessary to have corpora with parallel text in multiple sources and the target
language. However, these corpora are rarely complete in practice due to the
difficulty of providing human translations in all of the relevant languages. In
this paper, we propose a data augmentation approach to fill such incomplete
parts using multi-source neural machine translation (NMT). In our experiments,
results varied over different language combinations but significant gains were
observed when using a source language similar to the target language.
Fast Randomized PCA for Sparse Data
Authors: Xu Feng, Yuyang Xie, Mingye Song, Wenjian Yu, Jie Tang
Date: 2018-10-16 00:00:00
Tasks: Dimensionality Reduction, Information Retrieval, Retrieval

Principal component analysis (PCA) is widely used for dimension reduction and
embedding of real data in social network analysis, information retrieval, and
natural language processing, etc. In this work we propose a fast randomized PCA
algorithm for processing large sparse data. The algorithm has similar accuracy
to the basic randomized SVD (rPCA) algorithm (Halko et al., 2011), but is
largely optimized for sparse data. It also has good flexibility to trade off
runtime against accuracy for practical usage. Experiments on real data show
that the proposed algorithm is up to 9.1X faster than the basic rPCA algorithm
without accuracy loss, and is up to 20X faster than the svds in Matlab with
little error. The algorithm computes the first 100 principal components of a
large information retrieval data with 12,869,521 persons and 323,899 keywords
in less than 400 seconds on a 24-core machine, while all conventional methods
fail due to the out-of-memory issue.
Named Entity Analysis and Extraction with Uncommon Words
Authors: Xiaoshi Zhong, Erik Cambria, Jagath C. Rajapakse
Date: 2018-10-16 00:00:00
Tasks: Entity Extraction using GAN, General Classification

Most previous research treats named entity extraction and classification as
an end-to-end task. We argue that the two sub-tasks should be addressed
separately. Entity extraction lies at the level of syntactic analysis while
entity classification lies at the level of semantic analysis. According to Noam
Chomsky's "Syntactic Structures," pp. 93-94 (Chomsky 1957), syntax is not
appealed to semantics and semantics does not affect syntax. We analyze two
benchmark datasets for the characteristics of named entities, finding that
uncommon words can distinguish named entities from common text; where uncommon
words are the words that hardly appear in common text and they are mainly the
proper nouns. Experiments validate that lexical and syntactic features achieve
state-of-the-art performance on entity extraction and that semantic features do
not further improve the extraction performance, in both of our model and the
state-of-the-art baselines. With Chomsky's view, we also explain the failure of
joint syntactic and semantic parsings in other works.
Morph: Flexible Acceleration for 3D CNN-based Video Understanding
Authors: Kartik Hegde, Rohit Agrawal, Yulun Yao, Christopher W. Fletcher
Date: 2018-10-16 00:00:00
Tasks: MORPH, Video Recognition, Video Understanding

The past several years have seen both an explosion in the use of
Convolutional Neural Networks (CNNs) and the design of accelerators to make CNN
inference practical. In the architecture community, the lion share of effort
has targeted CNN inference for image recognition. The closely related problem
of video recognition has received far less attention as an accelerator target.
This is surprising, as video recognition is more computationally intensive than
image recognition, and video traffic is predicted to be the majority of
internet traffic in the coming years.
  This paper fills the gap between algorithmic and hardware advances for video
recognition by providing a design space exploration and flexible architecture
for accelerating 3D Convolutional Neural Networks (3D CNNs) - the core kernel
in modern video understanding. When compared to (2D) CNNs used for image
recognition, efficiently accelerating 3D CNNs poses a significant engineering
challenge due to their large (and variable over time) memory footprint and
higher dimensionality.
  To address these challenges, we design a novel accelerator, called Morph,
that can adaptively support different spatial and temporal tiling strategies
depending on the needs of each layer of each target 3D CNN. We codesign a
software infrastructure alongside the Morph hardware to find good-fit
parameters to control the hardware. Evaluated on state-of-the-art 3D CNNs,
Morph achieves up to 3.4x (2.5x average) reduction in energy consumption and
improves performance/watt by up to 5.1x (4x average) compared to a baseline 3D
CNN accelerator, with an area overhead of 5%. Morph further achieves a 15.9x
average energy reduction on 3D CNNs when compared to Eyeriss.
Co-manifold learning with missing data
Authors: Gal Mishne, Eric C. Chi, Ronald R. Coifman
Date: 2018-10-16 00:00:00
Tasks: Clustering, Data Visualization, Representation Learning

Representation learning is typically applied to only one mode of a data
matrix, either its rows or columns. Yet in many applications, there is an
underlying geometry to both the rows and the columns. We propose utilizing this
coupled structure to perform co-manifold learning: uncovering the underlying
geometry of both the rows and the columns of a given matrix, where we focus on
a missing data setting. Our unsupervised approach consists of three components.
We first solve a family of optimization problems to estimate a complete matrix
at multiple scales of smoothness. We then use this collection of smooth matrix
estimates to compute pairwise distances on the rows and columns based on a new
multi-scale metric that implicitly introduces a coupling between the rows and
the columns. Finally, we construct row and column representations from these
multi-scale metrics. We demonstrate that our approach outperforms competing
methods in both data visualization and clustering.
Quasi-hyperbolic momentum and Adam for deep learning
Authors: Jerry Ma, Denis Yarats
Date: 2018-10-16 00:00:00
Proceeding: ICLR 2019 5
Tasks: Deep Learning, Stochastic Optimization

Momentum-based acceleration of stochastic gradient descent (SGD) is widely
used in deep learning. We propose the quasi-hyperbolic momentum algorithm (QHM)
as an extremely simple alteration of momentum SGD, averaging a plain SGD step
with a momentum step. We describe numerous connections to and identities with
other algorithms, and we characterize the set of two-state optimization
algorithms that QHM can recover. Finally, we propose a QH variant of Adam
called QHAdam, and we empirically demonstrate that our algorithms lead to
significantly improved training in a variety of settings, including a new
state-of-the-art result on WMT16 EN-DE. We hope that these empirical results,
combined with the conceptual and practical simplicity of QHM and QHAdam, will
spur interest from both practitioners and researchers. Code is immediately
available.
A Robust Local Binary Similarity Pattern for Foreground Object Detection
Authors: Dongdong Zeng, Ming Zhu, Hang Yang
Date: 2018-10-16 00:00:00
Tasks: Object, object-detection, Object Detection, Object Tracking

Accurate and fast extraction of the foreground object is one of the most
significant issues to be solved due to its important meaning for object
tracking and recognition in video surveillance. Although many foreground object
detection methods have been proposed in the recent past, it is still regarded
as a tough problem due to illumination variations and dynamic backgrounds
challenges. In this paper, we propose a robust foreground object detection
method with two aspects of contributions. First, we propose a robust texture
operator named Robust Local Binary Similarity Pattern (RLBSP), which shows
strong robustness to illumination variations and dynamic backgrounds. Second, a
combination of color and texture features are used to characterize pixel
representations, which compensate each other to make full use of their own
advantages. Comprehensive experiments evaluated on the CDnet 2012 dataset
demonstrate that the proposed method performs favorably against
state-of-the-art methods.