pwc-archive/papers-with-abstracts huggingface dataset:
Efficient Implementation of a Recognition System Using the Cortex Ventral Stream Model
Authors: Ahmad W. Bitar, Mohammad M. Mansour, Ali Chehab
Date: 2017-11-21 00:00:00
Tasks: Clustering

In this paper, an efficient implementation for a recognition system based on
the original HMAX model of the visual cortex is proposed. Various optimizations
targeted to increase accuracy at the so-called layers S1, C1, and S2 of the
HMAX model are proposed. At layer S1, all unimportant information such as
illumination and expression variations are eliminated from the images. Each
image is then convolved with 64 separable Gabor filters in the spatial domain.
At layer C1, the minimum scales values are exploited to be embedded into the
maximum ones using the additive embedding space. At layer S2, the prototypes
are generated in a more efficient way using Partitioning Around Medoid (PAM)
clustering algorithm. The impact of these optimizations in terms of accuracy
and computational complexity was evaluated on the Caltech101 database, and
compared with the baseline performance using support vector machine (SVM) and
nearest neighbor (NN) classifiers. The results show that our model provides
significant improvement in accuracy at the S1 layer by more than 10% where the
computational complexity is also reduced. The accuracy is slightly increased
for both approximations at the C1 and S2 layers.
Evaluation of bioinspired algorithms for the solution of the job scheduling problem
Authors: Edson Florez, Nelson Diaz, Wilfredo Gomez, Lola Bautista, Dario Delgado
Date: 2017-11-21 00:00:00
Tasks: Combinatorial Optimization, Job Shop Scheduling, Relation, Scheduling

In this research we used bio-inspired metaheuristics, as artificial immune
systems and ant colony algorithms that are based on a number of characteristics
and behaviors of living things that are interesting in the computer science
area. This paper presents an evaluation of bio-inspired solutions to
combinatorial optimization problem, called the Job Shop Scheduling or planning
work, in a simple way the objective is to find a configuration or job stream
that has the least amount of time to be executed in machine settings. The
performance of the algorithms was characterized and evaluated for reference
instances of the job shop scheduling problem, comparing the quality of the
solutions obtained with respect to the best known solution of the most
effective methods. The solutions were evaluated in two aspects, first in
relation of quality of solutions, taking as reference the makespan and secondly
in relation of performance, taking the number evaluations performed by the
algorithm to obtain the best solution.
On the EM-Tau algorithm: a new EM-style algorithm with partial E-steps
Authors: Val Andrei Fajardo, Jiaxi Liang
Date: 2017-11-21 00:00:00
Tasks: Clustering

The EM algorithm is one of many important tools in the field of statistics.
While often used for imputing missing data, its widespread applications include
other common statistical tasks, such as clustering. In clustering, the EM
algorithm assumes a parametric distribution for the clusters, whose parameters
are estimated through a novel iterative procedure that is based on the theory
of maximum likelihood. However, one major drawback of the EM algorithm, that
renders it impractical especially when working with large datasets, is that it
often requires several passes of the data before convergence. In this paper, we
introduce a new EM-style algorithm that implements a novel policy for
performing partial E-steps. We call the new algorithm the EM-Tau algorithm,
which can approximate the traditional EM algorithm with high accuracy but with
only a fraction of the running time.
Jaccard analysis and LASSO-based feature selection for location fingerprinting with limited computational complexity
Authors: Caifa Zhou, Andreas Wieser
Date: 2017-11-21 00:00:00
Tasks: feature selection, Position

We propose an approach to reduce both computational complexity and data
storage requirements for the online positioning stage of a fingerprinting-based
indoor positioning system (FIPS) by introducing segmentation of the region of
interest (RoI) into sub-regions, sub-region selection using a modified Jaccard
index, and feature selection based on randomized least absolute shrinkage and
selection operator (LASSO). We implement these steps into a Bayesian framework
of position estimation using the maximum a posteriori (MAP) principle. An
additional benefit of these steps is that the time for estimating the position,
and the required data storage are virtually independent of the size of the RoI
and of the total number of available features within the RoI. Thus the proposed
steps facilitate application of FIPS to large areas. Results of an experimental
analysis using real data collected in an office building using a Nexus 6P smart
phone as user device and a total station for providing position ground truth
corroborate the expected performance of the proposed approach. The positioning
accuracy obtained by only processing 10 automatically identified features
instead of all available ones and limiting position estimation to 10
automatically identified sub-regions instead of the entire RoI is equivalent to
processing all available data. In the chosen example, 50% of the errors are
less than 1.8 m and 90% are less than 5 m. However, the computation time using
the automatically identified subset of data is only about 1% of that required
for processing the entire data set.
Visual and Textual Sentiment Analysis Using Deep Fusion Convolutional Neural Networks
Authors: Xingyue Chen, Yunhong Wang, Qingjie Liu
Date: 2017-11-21 00:00:00
Tasks: Opinion Mining, Sentiment Analysis

Sentiment analysis is attracting more and more attentions and has become a
very hot research topic due to its potential applications in personalized
recommendation, opinion mining, etc. Most of the existing methods are based on
either textual or visual data and can not achieve satisfactory results, as it
is very hard to extract sufficient information from only one single modality
data. Inspired by the observation that there exists strong semantic correlation
between visual and textual data in social medias, we propose an end-to-end deep
fusion convolutional neural network to jointly learn textual and visual
sentiment representations from training examples. The two modality information
are fused together in a pooling layer and fed into fully-connected layers to
predict the sentiment polarity. We evaluate the proposed approach on two widely
used data sets. Results show that our method achieves promising result compared
with the state-of-the-art methods which clearly demonstrate its competency.
Efficient Multi-Person Pose Estimation with Provable Guarantees
Authors: Shaofei Wang, Konrad Paul Kording, Julian Yarkony
Date: 2017-11-21 00:00:00
Tasks: Multi-Person Pose Estimation, Pose Estimation

Multi-person pose estimation (MPPE) in natural images is key to the
meaningful use of visual data in many fields including movement science,
security, and rehabilitation. In this paper we tackle MPPE with a bottom-up
approach, starting with candidate detections of body parts from a convolutional
neural network (CNN) and grouping them into people. We formulate the grouping
of body part detections into people as a minimum-weight set packing (MWSP)
problem where the set of potential people is the power set of body part
detections. We model the quality of a hypothesis of a person which is a set in
the MWSP by an augmented tree-structured Markov random field where variables
correspond to body-parts and their state-spaces correspond to the power set of
the detections for that part.
  We describe a novel algorithm that combines efficiency with provable bounds
on this MWSP problem. We employ an implicit column generation strategy where
the pricing problem is formulated as a dynamic program. To efficiently solve
this dynamic program we exploit the problem structure utilizing a nested
Bender's decomposition (NBD) exact inference strategy which we speed up by
recycling Bender's rows between calls to the pricing problem.
  We test our approach on the MPII-Multiperson dataset, showing that our
approach obtains comparable results with the state-of-the-art algorithm for
joint node labeling and grouping problems, and that NBD achieves considerable
speed-ups relative to a naive dynamic programming approach. Typical algorithms
that solve joint node labeling and grouping problems use heuristics and thus
can not obtain proofs of optimality. Our approach, in contrast, proves that for
over 99 percent of problem instances we find the globally optimal solution and
otherwise provide upper/lower bounds.
Simple and Effective Dimensionality Reduction for Word Embeddings
Authors: Vikas Raunak
Date: 2017-08-11 00:00:00
Tasks: Dimensionality Reduction, Information Retrieval, Retrieval, Word Embeddings, Word Similarity

Word embeddings have become the basic building blocks for several natural
language processing and information retrieval tasks. Pre-trained word
embeddings are used in several downstream applications as well as for
constructing representations for sentences, paragraphs and documents. Recently,
there has been an emphasis on further improving the pre-trained word vectors
through post-processing algorithms. One such area of improvement is the
dimensionality reduction of the word embeddings. Reducing the size of word
embeddings through dimensionality reduction can improve their utility in memory
constrained devices, benefiting several real-world applications. In this work,
we present a novel algorithm that effectively combines PCA based dimensionality
reduction with a recently proposed post-processing algorithm, to construct word
embeddings of lower dimensions. Empirical evaluations on 12 standard word
similarity benchmarks show that our algorithm reduces the embedding
dimensionality by 50%, while achieving similar or (more often) better
performance than the higher dimension embeddings.
Hidden Tree Markov Networks: Deep and Wide Learning for Structured Data
Authors: Davide Bacciu
Date: 2017-11-21 00:00:00

The paper introduces the Hidden Tree Markov Network (HTN), a
neuro-probabilistic hybrid fusing the representation power of generative models
for trees with the incremental and discriminative learning capabilities of
neural networks. We put forward a modular architecture in which multiple
generative models of limited complexity are trained to learn structural feature
detectors whose outputs are then combined and integrated by neural layers at a
later stage. In this respect, the model is both deep, thanks to the unfolding
of the generative models on the input structures, as well as wide, given the
potentially large number of generative modules that can be trained in parallel.
Experimental results show that the proposed approach can outperform
state-of-the-art syntactic kernels as well as generative kernels built on the
same probabilistic model as the HTN.
Understanding Deep Learning Generalization by Maximum Entropy
Authors: Guanhua Zheng, Jitao Sang, Changsheng Xu
Date: 2017-11-21 00:00:00
Proceeding: ICLR 2018 1
Tasks: Deep Learning, regression

Deep learning achieves remarkable generalization capability with overwhelming
number of model parameters. Theoretical understanding of deep learning
generalization receives recent attention yet remains not fully explored. This
paper attempts to provide an alternative understanding from the perspective of
maximum entropy. We first derive two feature conditions that softmax regression
strictly apply maximum entropy principle. DNN is then regarded as approximating
the feature conditions with multilayer feature learning, and proved to be a
recursive solution towards maximum entropy principle. The connection between
DNN and maximum entropy well explains why typical designs such as shortcut and
regularization improves model generalization, and provides instructions for
future model development.
Is Natural Language a Perigraphic Process? The Theorem about Facts and Words Revisited
Authors: Łukasz Dębowski
Date: 2017-06-14 00:00:00

As we discuss, a stationary stochastic process is nonergodic when a random
persistent topic can be detected in the infinite random text sampled from the
process, whereas we call the process strongly nonergodic when an infinite
sequence of independent random bits, called probabilistic facts, is needed to
describe this topic completely. Replacing probabilistic facts with an
algorithmically random sequence of bits, called algorithmic facts, we adapt
this property back to ergodic processes. Subsequently, we call a process
perigraphic if the number of algorithmic facts which can be inferred from a
finite text sampled from the process grows like a power of the text length. We
present a simple example of such a process. Moreover, we demonstrate an
assertion which we call the theorem about facts and words. This proposition
states that the number of probabilistic or algorithmic facts which can be
inferred from a text drawn from a process must be roughly smaller than the
number of distinct word-like strings detected in this text by means of the PPM
compression algorithm. We also observe that the number of the word-like strings
for a sample of plays by Shakespeare follows an empirical stepwise power law,
in a stark contrast to Markov processes. Hence we suppose that natural language
considered as a process is not only non-Markov but also perigraphic.
New insights and perspectives on the natural gradient method
Authors: James Martens
Date: 2014-12-03 00:00:00

Natural gradient descent is an optimization method traditionally motivated
from the perspective of information geometry, and works well for many
applications as an alternative to stochastic gradient descent. In this paper we
critically analyze this method and its properties, and show how it can be
viewed as a type of approximate 2nd-order optimization method, where the Fisher
information matrix can be viewed as an approximation of the Hessian. This
perspective turns out to have significant implications for how to design a
practical and robust version of the method. Additionally, we make the following
contributions to the understanding of natural gradient and 2nd-order methods: a
thorough analysis of the convergence speed of stochastic natural gradient
descent (and more general stochastic 2nd-order methods) as applied to convex
quadratics, a critical examination of the oft-used "empirical" approximation of
the Fisher matrix, and an analysis of the (approximate) parameterization
invariance property possessed by natural gradient methods, which we show still
holds for certain choices of the curvature matrix other than the Fisher, but
notably not the Hessian.
Learning to Compose Task-Specific Tree Structures
Authors: Jihun Choi, Kang Min Yoo, Sang-goo Lee
Date: 2017-07-10 00:00:00
Tasks: Natural Language Inference, Sentiment Analysis

For years, recursive neural networks (RvNNs) have been shown to be suitable
for representing text into fixed-length vectors and achieved good performance
on several natural language processing tasks. However, the main drawback of
RvNNs is that they require structured input, which makes data preparation and
model implementation hard. In this paper, we propose Gumbel Tree-LSTM, a novel
tree-structured long short-term memory architecture that learns how to compose
task-specific tree structures only from plain text data efficiently. Our model
uses Straight-Through Gumbel-Softmax estimator to decide the parent node among
candidates dynamically and to calculate gradients of the discrete decision. We
evaluate the proposed model on natural language inference and sentiment
analysis, and show that our model outperforms or is at least comparable to
previous models. We also find that our model converges significantly faster
than other models.
Proximal Gradient Method with Extrapolation and Line Search for a Class of Nonconvex and Nonsmooth Problems
Authors: Lei Yang
Date: 2017-11-18 00:00:00
Tasks: Variable Selection

In this paper, we consider a class of possibly nonconvex, nonsmooth and non-Lipschitz optimization problems arising in many contemporary applications such as machine learning, variable selection and image processing. To solve this class of problems, we propose a proximal gradient method with extrapolation and line search (PGels). This method is developed based on a special potential function and successfully incorporates both extrapolation and non-monotone line search, which are two simple and efficient accelerating techniques for the proximal gradient method. Thanks to the line search, this method allows more flexibilities in choosing the extrapolation parameters and updates them adaptively at each iteration if a certain line search criterion is not satisfied. Moreover, with proper choices of parameters, our PGels reduces to many existing algorithms. We also show that, under some mild conditions, our line search criterion is well defined and any cluster point of the sequence generated by PGels is a stationary point of our problem. In addition, by assuming the Kurdyka-${\L}$ojasiewicz exponent of the objective in our problem, we further analyze the local convergence rate of two special cases of PGels, including the widely used non-monotone proximal gradient method as one case. Finally, we conduct some numerical experiments for solving the $\ell_1$ regularized logistic regression problem and the $\ell_{1\text{-}2}$ regularized least squares problem. Our numerical results illustrate the efficiency of PGels and show the potential advantage of combining two accelerating techniques.
Variational Probability Flow for Biologically Plausible Training of Deep Neural Networks
Authors: Zuozhu Liu, Tony Q. S. Quek, Shaowei Lin
Date: 2017-11-21 00:00:00
Tasks: Biologically-plausible Training

The quest for biologically plausible deep learning is driven, not just by the
desire to explain experimentally-observed properties of biological neural
networks, but also by the hope of discovering more efficient methods for
training artificial networks. In this paper, we propose a new algorithm named
Variational Probably Flow (VPF), an extension of minimum probability flow for
training binary Deep Boltzmann Machines (DBMs). We show that weight updates in
VPF are local, depending only on the states and firing rates of the adjacent
neurons. Unlike contrastive divergence, there is no need for Gibbs
confabulations; and unlike backpropagation, alternating feedforward and
feedback phases are not required. Moreover, the learning algorithm is effective
for training DBMs with intra-layer connections between the hidden nodes.
Experiments with MNIST and Fashion MNIST demonstrate that VPF learns reasonable
features quickly, reconstructs corrupted images more accurately, and generates
samples with a high estimated log-likelihood. Lastly, we note that,
interestingly, if an asymmetric version of VPF exists, the weight updates
directly explain experimental results in Spike-Timing-Dependent Plasticity
(STDP).
Faster Reinforcement Learning Using Active Simulators
Authors: Vikas Jain, Theja Tulabandhula
Date: 2017-03-22 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Transfer Learning

In this work, we propose several online methods to build a \emph{learning
curriculum} from a given set of target-task-specific training tasks in order to
speed up reinforcement learning (RL). These methods can decrease the total
training time needed by an RL agent compared to training on the target task
from scratch. Unlike traditional transfer learning, we consider creating a
sequence from several training tasks in order to provide the most benefit in
terms of reducing the total time to train.
  Our methods utilize the learning trajectory of the agent on the curriculum
tasks seen so far to decide which tasks to train on next. An attractive feature
of our methods is that they are weakly coupled to the choice of the RL
algorithm as well as the transfer learning method. Further, when there is
domain information available, our methods can incorporate such knowledge to
further speed up the learning. We experimentally show that these methods can be
used to obtain suitable learning curricula that speed up the overall training
time on two different domains.
Flight Dynamics-based Recovery of a UAV Trajectory using Ground Cameras
Authors: Artem Rozantsev, Sudipta N. Sinha, Debadeepta Dey, Pascal Fua
Date: 2016-12-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: 3D Reconstruction

We propose a new method to estimate the 6-dof trajectory of a flying object
such as a quadrotor UAV within a 3D airspace monitored using multiple fixed
ground cameras. It is based on a new structure from motion formulation for the
3D reconstruction of a single moving point with known motion dynamics. Our main
contribution is a new bundle adjustment procedure which in addition to
optimizing the camera poses, regularizes the point trajectory using a prior
based on motion dynamics (or specifically flight dynamics). Furthermore, we can
infer the underlying control input sent to the UAV's autopilot that determined
its flight trajectory.
  Our method requires neither perfect single-view tracking nor appearance
matching across views. For robustness, we allow the tracker to generate
multiple detections per frame in each video. The true detections and the data
association across videos is estimated using robust multi-view triangulation
and subsequently refined during our bundle adjustment procedure. Quantitative
evaluation on simulated data and experiments on real videos from indoor and
outdoor scenes demonstrates the effectiveness of our method.
The Application of Preconditioned Alternating Direction Method of Multipliers in Depth from Focal Stack
Authors: Hossein Javidnia, Peter Corcoran
Date: 2017-11-21 00:00:00
Tasks: Occlusion Handling

Post capture refocusing effect in smartphone cameras is achievable by using
focal stacks. However, the accuracy of this effect is totally dependent on the
combination of the depth layers in the stack. The accuracy of the extended
depth of field effect in this application can be improved significantly by
computing an accurate depth map which has been an open issue for decades. To
tackle this issue, in this paper, a framework is proposed based on
Preconditioned Alternating Direction Method of Multipliers (PADMM) for depth
from the focal stack and synthetic defocus application. In addition to its
ability to provide high structural accuracy and occlusion handling, the
optimization function of the proposed method can, in fact, converge faster and
better than state of the art methods. The evaluation has been done on 21 sets
of focal stacks and the optimization function has been compared against 5 other
methods. Preliminary results indicate that the proposed method has a better
performance in terms of structural accuracy and optimization in comparison to
the current state of the art methods.