pwc-archive/papers-with-abstracts huggingface dataset:
Quantifying the accuracy of approximate diffusions and Markov chains
Authors: Jonathan H. Huggins, James Zou
Date: 2016-05-20 00:00:00

Markov chains and diffusion processes are indispensable tools in machine
learning and statistics that are used for inference, sampling, and modeling.
With the growth of large-scale datasets, the computational cost associated with
simulating these stochastic processes can be considerable, and many algorithms
have been proposed to approximate the underlying Markov chain or diffusion. A
fundamental question is how the computational savings trade off against the
statistical error incurred due to approximations. This paper develops general
results that address this question. We bound the Wasserstein distance between
the equilibrium distributions of two diffusions as a function of their mixing
rates and the deviation in their drifts. We show that this error bound is tight
in simple Gaussian settings. Our general result on continuous diffusions can be
discretized to provide insights into the computational-statistical trade-off of
Markov chains. As an illustration, we apply our framework to derive
finite-sample error bounds of approximate unadjusted Langevin dynamics. We
characterize computation-constrained settings where, by using fast-to-compute
approximate gradients in the Langevin dynamics, we obtain more accurate samples
compared to using the exact gradients. Finally, as an additional application of
our approach, we quantify the accuracy of approximate zig-zag sampling. Our
theoretical analyses are supported by simulation experiments.
Texture and Structure Incorporated ScatterNet Hybrid Deep Learning Network (TS-SHDL) For Brain Matter Segmentation
Authors: Amarjot Singh, Devamanyu Hazarika, Aniruddha Bhattacharya
Date: 2017-08-30 00:00:00
Tasks: Segmentation

Automation of brain matter segmentation from MR images is a challenging task
due to the irregular boundaries between the grey and white matter regions. In
addition, the presence of intensity inhomogeneity in the MR images further
complicates the problem. In this paper, we propose a texture and vesselness
incorporated version of the ScatterNet Hybrid Deep Learning Network (TS-SHDL)
that extracts hierarchical invariant mid-level features, used by fisher vector
encoding and a conditional random field (CRF) to perform the desired
segmentation. The performance of the proposed network is evaluated by extensive
experimentation and comparison with the state-of-the-art methods on several 2D
MRI scans taken from the synthetic McGill Brain Web as well as on the MRBrainS
dataset of real 3D MRI scans. The advantages of the TS-SHDL network over
supervised deep learning networks is also presented in addition to its superior
performance over the state-of-the-art.
Two-stream Flow-guided Convolutional Attention Networks for Action Recognition
Authors: An Tran, Loong-Fah Cheong
Date: 2017-08-30 00:00:00
Tasks: Action Recognition, Action Recognition In Videos, Temporal Action Localization, Vocal Bursts Valence Prediction

This paper proposes a two-stream flow-guided convolutional attention networks
for action recognition in videos. The central idea is that optical flows, when
properly compensated for the camera motion, can be used to guide attention to
the human foreground. We thus develop cross-link layers from the temporal
network (trained on flows) to the spatial network (trained on RGB frames).
These cross-link layers guide the spatial-stream to pay more attention to the
human foreground areas and be less affected by background clutter. We obtain
promising performances with our approach on the UCF101, HMDB51 and Hollywood2
datasets.
Efficient Convolutional Network Learning using Parametric Log based Dual-Tree Wavelet ScatterNet
Authors: Amarjot Singh, Nick Kingsbury
Date: 2017-08-30 00:00:00

We propose a DTCWT ScatterNet Convolutional Neural Network (DTSCNN) formed by
replacing the first few layers of a CNN network with a parametric log based
DTCWT ScatterNet. The ScatterNet extracts edge based invariant representations
that are used by the later layers of the CNN to learn high-level features. This
improves the training of the network as the later layers can learn more complex
patterns from the start of learning because the edge representations are
already present. The efficient learning of the DTSCNN network is demonstrated
on CIFAR-10 and Caltech-101 datasets. The generic nature of the ScatterNet
front-end is shown by an equivalent performance to pre-trained CNN front-ends.
A comparison with the state-of-the-art on CIFAR-10 and Caltech-101 datasets is
also presented.
Deformable Modeling for Human Body Acquired from Depth Sensors
Authors: Vamshhi Pavan Kumar Varma Vegeshna
Date: 2017-08-17 00:00:00

This paper presents a novel approach to reconstruct complete 3D deformable
models over time by a single depth camera. These are the steps employed for
deforming objects from single depth camera. The partial surfaces reconstructed
from various times of capture are assembled together to form a complete 3D
surface. A mesh warping algorithm is used to align different partial surfaces
based on linear mesh deformation. A volumetric method is then applied to
combine partial surfaces, fix missing holes and smooth alignment errors.
Stereo Matching With Color-Weighted Correlation, Hierarchical Belief Propagation And Occlusion Handling
Authors: Vamshhi Pavan Kumar Varma Vegeshna
Date: 2017-08-26 00:00:00
Tasks: Occlusion Handling, Stereo Matching, Stereo Matching Hand

In this paper, we contrive a stereo matching algorithm with careful handling
of disparity, discontinuity and occlusion. This algorithm works a worldwide
matching stereo model which is based on minimization of energy. The global
energy comprises two terms, firstly the data term and secondly the smoothness
term. The data term is approximated by a color-weighted correlation, then
refined in obstruct and low-texture areas in many applications of hierarchical
loopy belief propagation algorithm. The results during the experiment are
evaluated on the Middlebury data sets, showing that out algorithm is the top
performer among all the algorithms listed there
Fighting with the Sparsity of Synonymy Dictionaries
Authors: Dmitry Ustalov, Mikhail Chernoskutov, Chris Biemann, Alexander Panchenko
Date: 2017-08-30 00:00:00
Tasks: Clustering

Graph-based synset induction methods, such as MaxMax and Watset, induce
synsets by performing a global clustering of a synonymy graph. However, such
methods are sensitive to the structure of the input synonymy graph: sparseness
of the input dictionary can substantially reduce the quality of the extracted
synsets. In this paper, we propose two different approaches designed to
alleviate the incompleteness of the input dictionaries. The first one performs
a pre-processing of the graph by adding missing edges, while the second one
performs a post-processing by merging similar synset clusters. We evaluate
these approaches on two datasets for the Russian language and discuss their
impact on the performance of synset induction methods. Finally, we perform an
extensive error analysis of each approach and discuss prominent alternative
methods for coping with the problem of the sparsity of the synonymy
dictionaries.
Graph Laplacian Regularization for Image Denoising: Analysis in the Continuous Domain
Authors: Jiahao Pang, Gene Cheung
Date: 2016-04-27 00:00:00
Tasks: Denoising, Image Denoising

Inverse imaging problems are inherently under-determined, and hence it is
important to employ appropriate image priors for regularization. One recent
popular prior---the graph Laplacian regularizer---assumes that the target pixel
patch is smooth with respect to an appropriately chosen graph. However, the
mechanisms and implications of imposing the graph Laplacian regularizer on the
original inverse problem are not well understood. To address this problem, in
this paper we interpret neighborhood graphs of pixel patches as discrete
counterparts of Riemannian manifolds and perform analysis in the continuous
domain, providing insights into several fundamental aspects of graph Laplacian
regularization for image denoising. Specifically, we first show the convergence
of the graph Laplacian regularizer to a continuous-domain functional,
integrating a norm measured in a locally adaptive metric space. Focusing on
image denoising, we derive an optimal metric space assuming non-local
self-similarity of pixel patches, leading to an optimal graph Laplacian
regularizer for denoising in the discrete domain. We then interpret graph
Laplacian regularization as an anisotropic diffusion scheme to explain its
behavior during iterations, e.g., its tendency to promote piecewise smooth
signals under certain settings. To verify our analysis, an iterative image
denoising algorithm is developed. Experimental results show that our algorithm
performs competitively with state-of-the-art denoising methods such as BM3D for
natural images, and outperforms them significantly for piecewise smooth images.
Universally consistent predictive distributions
Authors: Vladimir Vovk
Date: 2017-08-06 00:00:00

This paper describes simple universally consistent procedures of probability
forecasting that satisfy a natural property of small-sample validity, under the
assumption that the observations are produced independently in the IID fashion.
Look-ahead Attention for Generation in Neural Machine Translation
Authors: Long Zhou, Jiajun Zhang, Cheng-qing Zong
Date: 2017-08-30 00:00:00
Tasks: Machine Translation, NMT, Sentence, Translation

The attention model has become a standard component in neural machine
translation (NMT) and it guides translation process by selectively focusing on
parts of the source sentence when predicting each target word. However, we find
that the generation of a target word does not only depend on the source
sentence, but also rely heavily on the previous generated target words,
especially the distant words which are difficult to model by using recurrent
neural networks. To solve this problem, we propose in this paper a novel
look-ahead attention mechanism for generation in NMT, which aims at directly
capturing the dependency relationship between target words. We further design
three patterns to integrate our look-ahead attention into the conventional
attention model. Experiments on NIST Chinese-to-English and WMT
English-to-German translation tasks show that our proposed look-ahead attention
mechanism achieves substantial improvements over state-of-the-art baselines.
Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems
Authors: Tim Meinhardt, Michael Moeller, Caner Hazirbas, Daniel Cremers
Date: 2017-04-11 00:00:00
Proceeding: ICCV 2017 10
Tasks: Demosaicking, Denoising, Image Deconvolution

While variational methods have been among the most powerful tools for solving
linear inverse problems in imaging, deep (convolutional) neural networks have
recently taken the lead in many challenging benchmarks. A remaining drawback of
deep learning approaches is their requirement for an expensive retraining
whenever the specific problem, the noise level, noise type, or desired measure
of fidelity changes. On the contrary, variational methods have a plug-and-play
nature as they usually consist of separate data fidelity and regularization
terms.
  In this paper we study the possibility of replacing the proximal operator of
the regularization used in many convex energy minimization algorithms by a
denoising neural network. The latter therefore serves as an implicit natural
image prior, while the data term can still be chosen independently. Using a
fixed denoising neural network in exemplary problems of image deconvolution
with different blur kernels and image demosaicking, we obtain state-of-the-art
reconstruction results. These indicate the high generalizability of our
approach and a reduction of the need for problem-specific training.
Additionally, we discuss novel results on the analysis of possible optimization
algorithms to incorporate the network into, as well as the choices of algorithm
parameters and their relation to the noise level the neural network is trained
on.
ScatterNet Hybrid Deep Learning (SHDL) Network For Object Classification
Authors: Amarjot Singh, Nick Kingsbury
Date: 2017-08-30 00:00:00
Tasks: Classification, Deep Learning, General Classification, Object, Object Recognition

The paper proposes the ScatterNet Hybrid Deep Learning (SHDL) network that
extracts invariant and discriminative image representations for object
recognition. SHDL framework is constructed with a multi-layer ScatterNet
front-end, an unsupervised learning middle, and a supervised learning back-end
module. Each layer of the SHDL network is automatically designed as an explicit
optimization problem leading to an optimal deep learning architecture with
improved computational performance as compared to the more usual deep network
architectures. SHDL network produces the state-of-the-art classification
performance against unsupervised and semi-supervised learning (GANs) on two
image datasets. Advantages of the SHDL network over supervised methods (NIN,
VGG) are also demonstrated with experiments performed on training datasets of
reduced size.
Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching
Authors: Jiahao Pang, Wenxiu Sun, Jimmy SJ. Ren, Chengxi Yang, Qiong Yan
Date: 2017-08-30 00:00:00
Tasks: Stereo Matching, Stereo Matching Hand

Leveraging on the recent developments in convolutional neural networks
(CNNs), matching dense correspondence from a stereo pair has been cast as a
learning problem, with performance exceeding traditional approaches. However,
it remains challenging to generate high-quality disparities for the inherently
ill-posed regions. To tackle this problem, we propose a novel cascade CNN
architecture composing of two stages. The first stage advances the recently
proposed DispNet by equipping it with extra up-convolution modules, leading to
disparity images with more details. The second stage explicitly rectifies the
disparity initialized by the first stage; it couples with the first-stage and
generates residual signals across multiple scales. The summation of the outputs
from the two stages gives the final disparity. As opposed to directly learning
the disparity at the second stage, we show that residual learning provides more
effective refinement. Moreover, it also benefits the training of the overall
cascade network. Experimentation shows that our cascade residual learning
scheme provides state-of-the-art performance for matching stereo
correspondence. By the time of the submission of this paper, our method ranks
first in the KITTI 2015 stereo benchmark, surpassing the prior works by a
noteworthy margin.
Joint Maximum Purity Forest with Application to Image Super-Resolution
Authors: Hailiang Li, Kin-Man Lam, Dong Li
Date: 2017-08-30 00:00:00
Tasks: Clustering, General Classification, Image Super-Resolution, Quantization, regression, Super-Resolution

In this paper, we propose a novel random-forest scheme, namely Joint Maximum
Purity Forest (JMPF), for classification, clustering, and regression tasks. In
the JMPF scheme, the original feature space is transformed into a compactly
pre-clustered feature space, via a trained rotation matrix. The rotation matrix
is obtained through an iterative quantization process, where the input data
belonging to different classes are clustered to the respective vertices of the
new feature space with maximum purity. In the new feature space, orthogonal
hyperplanes, which are employed at the split-nodes of decision trees in random
forests, can tackle the clustering problems effectively. We evaluated our
proposed method on public benchmark datasets for regression and classification
tasks, and experiments showed that JMPF remarkably outperforms other
state-of-the-art random-forest-based approaches. Furthermore, we applied JMPF
to image super-resolution, because the transformed, compact features are more
discriminative to the clustering-regression scheme. Experiment results on
several public benchmark datasets also showed that the JMPF-based image
super-resolution scheme is consistently superior to recent state-of-the-art
image super-resolution algorithms.
A Greedy Part Assignment Algorithm for Real-time Multi-person 2D Pose Estimation
Authors: Srenivas Varadarajan, Parual Datta, Omesh Tickoo
Date: 2017-08-30 00:00:00
Tasks: 2D Pose Estimation, Pose Estimation

Human pose-estimation in a multi-person image involves detection of various
body parts and grouping them into individual person clusters. While the former
task is challenging due to mutual occlusions, the combinatorial complexity of
the latter task is very high. We propose a greedy part assignment algorithm
that exploits the inherent structure of the human body to achieve a lower
complexity, compared to any of the prior published works. This is accomplished
by (i) reducing the number of part-candidates using the estimated number of
people in the image, (ii) doing a greedy sequential assignment of part-classes,
following the kinematic chain from head to ankle (iii) doing a greedy
assignment of parts in each part-class set, to person-clusters (iv) limiting
the candidate person clusters to the most proximal clusters using human
anthropometric data and (v) using only a specific subset of pre-assigned parts
for establishing pairwise structural constraints. We show that, these steps
result in a sparse body parts relationship graph and reduces the complexity. We
also propose methods for improving the accuracy of pose-estimation by (i)
spawning person-clusters from any unassigned significant body part and (ii)
suppressing hallucinated parts. On the MPII multi-person pose database,
pose-estimation using the proposed method takes only 0.14 seconds per image. We
show that, our proposed algorithm, by using a large spatial and structural
context, achieves the state-of-the-art accuracy on both MPII and WAF
multi-person pose datasets, demonstrating the robustness of our approach.
Calibrating chemical multisensory devices for real world applications: An in-depth comparison of quantitative Machine Learning approaches
Authors: S. De Vito, E. Esposito, M. Salvato, O. Popoola, F. Formisano, R. Jones, G. Di Francia
Date: 2017-08-30 00:00:00
Tasks: Benchmarking

Chemical multisensor devices need calibration algorithms to estimate gas
concentrations. Their possible adoption as indicative air quality measurements
devices poses new challenges due to the need to operate in continuous
monitoring modes in uncontrolled environments. Several issues, including slow
dynamics, continue to affect their real world performances. At the same time,
the need for estimating pollutant concentrations on board the devices, espe-
cially for wearables and IoT deployments, is becoming highly desirable. In this
framework, several calibration approaches have been proposed and tested on a
variety of proprietary devices and datasets; still, no thorough comparison is
available to researchers. This work attempts a benchmarking of the most
promising calibration algorithms according to recent literature with a focus on
machine learning approaches. We test the techniques against absolute and
dynamic performances, generalization capabilities and computational/storage
needs using three different datasets sharing continuous monitoring operation
methodology. Our results can guide researchers and engineers in the choice of
optimal strategy. They show that non-linear multivariate techniques yield
reproducible results, outperforming lin- ear approaches. Specifically, the
Support Vector Regression method consistently shows good performances in all
the considered scenarios. We highlight the enhanced suitability of shallow
neural networks in a trade-off between performance and computational/storage
needs. We confirm, on a much wider basis, the advantages of dynamic approaches
with respect to static ones that only rely on instantaneous sensor array
response. The latter have been shown to be best choice whenever prompt and
precise response is needed.
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Authors: A. Cichocki, A-H. Phan, Q. Zhao, N. Lee, I. V. Oseledets, M. Sugiyama, D. Mandic
Date: 2017-08-30 00:00:00
Tasks: Dimensionality Reduction, Riemannian optimization, Tensor Networks

Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.
An Empirical Study of Discriminative Sequence Labeling Models for Vietnamese Text Processing
Authors: Phuong Le-Hong, Minh Pham Quang Nhat, Thai-Hoang Pham, Tuan-Anh Tran, Dang-Minh Nguyen
Date: 2017-08-30 00:00:00
Tasks: Feature Engineering, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Part-Of-Speech Tagging, Word Embeddings

This paper presents an empirical study of two widely-used sequence prediction
models, Conditional Random Fields (CRFs) and Long Short-Term Memory Networks
(LSTMs), on two fundamental tasks for Vietnamese text processing, including
part-of-speech tagging and named entity recognition. We show that a strong
lower bound for labeling accuracy can be obtained by relying only on simple
word-based features with minimal hand-crafted feature engineering, of 90.65\%
and 86.03\% performance scores on the standard test sets for the two tasks
respectively. In particular, we demonstrate empirically the surprising
efficiency of word embeddings in both of the two tasks, with both of the two
models. We point out that the state-of-the-art LSTMs model does not always
outperform significantly the traditional CRFs model, especially on
moderate-sized data sets. Finally, we give some suggestions and discussions for
efficient use of sequence labeling models in practical applications.
Cross-lingual, Character-Level Neural Morphological Tagging
Authors: Ryan Cotterell, Georg Heigold
Date: 2017-08-30 00:00:00
Tasks: Morphological Tagging, Transfer Learning

Even for common NLP tasks, sufficient supervision is not available in many languages -- morphological tagging is no exception. In the work presented here, we explore a transfer learning scheme, whereby we train character-level recurrent neural taggers to predict morphological taggings for high-resource languages and low-resource languages together. Learning joint character representations among multiple related languages successfully enables knowledge transfer from the high-resource languages to the low-resource ones, improving accuracy by up to 30% over a monolingual model.