pwc-archive/papers-with-abstracts huggingface dataset:
CNNComparator: Comparative Analytics of Convolutional Neural Networks
Authors: Haipeng Zeng, Hammad Haleem, Xavier Plantaz, Nan Cao, Huamin Qu
Date: 2017-10-15 00:00:00

Convolutional neural networks (CNNs) are widely used in many image
recognition tasks due to their extraordinary performance. However, training a
good CNN model can still be a challenging task. In a training process, a CNN
model typically learns a large number of parameters over time, which usually
results in different performance. Often, it is difficult to explore the
relationships between the learned parameters and the model performance due to a
large number of parameters and different random initializations. In this paper,
we present a visual analytics approach to compare two different snapshots of a
trained CNN model taken after different numbers of epochs, so as to provide
some insight into the design or the training of a better CNN model. Our system
compares snapshots by exploring the differences in operation parameters and the
corresponding blob data at different levels. A case study has been conducted to
demonstrate the effectiveness of our system.
Facial Keypoints Detection
Authors: Shenghao Shi
Date: 2017-10-15 00:00:00
Tasks: Face Recognition, regression

Detect facial keypoints is a critical element in face recognition. However,
there is difficulty to catch keypoints on the face due to complex influences
from original images, and there is no guidance to suitable algorithms. In this
paper, we study different algorithms that can be applied to locate keyponits.
Specifically: our framework (1)prepare the data for further investigation
(2)Using PCA and LBP to process the data (3) Apply different algorithms to
analysis data, including linear regression models, tree based model, neural
network and convolutional neural network, etc. Finally we will give our
conclusion and further research topic. A comprehensive set of experiments on
dataset demonstrates the effectiveness of our framework.
Sparse Depth Sensing for Resource-Constrained Robots
Authors: Fangchang Ma, Luca Carlone, Ulas Ayaz, Sertac Karaman
Date: 2017-03-04 00:00:00
Tasks: Compressive Sensing, Depth Estimation, Navigate

We consider the case in which a robot has to navigate in an unknown
environment but does not have enough on-board power or payload to carry a
traditional depth sensor (e.g., a 3D lidar) and thus can only acquire a few
(point-wise) depth measurements. We address the following question: is it
possible to reconstruct the geometry of an unknown environment using sparse and
incomplete depth measurements? Reconstruction from incomplete data is not
possible in general, but when the robot operates in man-made environments, the
depth exhibits some regularity (e.g., many planar surfaces with only a few
edges); we leverage this regularity to infer depth from a small number of
measurements. Our first contribution is a formulation of the depth
reconstruction problem that bridges robot perception with the compressive
sensing literature in signal processing. The second contribution includes a set
of formal results that ascertain the exactness and stability of the depth
reconstruction in 2D and 3D problems, and completely characterize the geometry
of the profiles that we can reconstruct. Our third contribution is a set of
practical algorithms for depth reconstruction: our formulation directly
translates into algorithms for depth estimation based on convex programming. In
real-world problems, these convex programs are very large and general-purpose
solvers are relatively slow. For this reason, we discuss ad-hoc solvers that
enable fast depth reconstruction in real problems. The last contribution is an
extensive experimental evaluation in 2D and 3D problems, including Monte Carlo
runs on simulated instances and testing on multiple real datasets. Empirical
results confirm that the proposed approach ensures accurate depth
reconstruction, outperforms interpolation-based strategies, and performs well
even when the assumption of structured environment is violated.
Scalable Dynamic Topic Modeling with Clustered Latent Dirichlet Allocation (CLDA)
Authors: Chris Gropp, Alexander Herzog, Ilya Safro, Paul W. Wilson, Amy W. Apon
Date: 2016-10-25 00:00:00
Tasks: Clustering, Dynamic Topic Modeling

Topic modeling, a method for extracting the underlying themes from a collection of documents, is an increasingly important component of the design of intelligent systems enabling the sense-making of highly dynamic and diverse streams of text data. Traditional methods such as Dynamic Topic Modeling (DTM) do not lend themselves well to direct parallelization because of dependencies from one time step to another. In this paper, we introduce and empirically analyze Clustered Latent Dirichlet Allocation (CLDA), a method for extracting dynamic latent topics from a collection of documents. Our approach is based on data decomposition in which the data is partitioned into segments, followed by topic modeling on the individual segments. The resulting local models are then combined into a global solution using clustering. The decomposition and resulting parallelization leads to very fast runtime even on very large datasets. Our approach furthermore provides insight into how the composition of topics changes over time and can also be applied using other data partitioning strategies over any discrete features of the data, such as geographic features or classes of users. In this paper CLDA is applied successfully to seventeen years of NIPS conference papers (2,484 documents and 3,280,697 words), seventeen years of computer science journal abstracts (533,560 documents and 32,551,540 words), and to forty years of the PubMed corpus (4,025,978 documents and 273,853,980 words).
Learning Infinite RBMs with Frank-Wolfe
Authors: Wei Ping, Qiang Liu, Alexander Ihler
Date: 2017-10-15 00:00:00
Proceeding: NeurIPS 2016 12

In this work, we propose an infinite restricted Boltzmann machine~(RBM),
whose maximum likelihood estimation~(MLE) corresponds to a constrained convex
optimization. We consider the Frank-Wolfe algorithm to solve the program, which
provides a sparse solution that can be interpreted as inserting a hidden unit
at each iteration, so that the optimization process takes the form of a
sequence of finite models of increasing complexity. As a side benefit, this can
be used to easily and efficiently identify an appropriate number of hidden
units during the optimization. The resulting model can also be used as an
initialization for typical state-of-the-art RBM training algorithms such as
contrastive divergence, leading to models with consistently higher test
likelihood than random initialization.
A Tight Bound of Hard Thresholding
Authors: Jie Shen, Ping Li
Date: 2016-05-05 00:00:00
Tasks: BIG-bench Machine Learning, compressed sensing

This paper is concerned with the hard thresholding operator which sets all
but the $k$ largest absolute elements of a vector to zero. We establish a {\em
tight} bound to quantitatively characterize the deviation of the thresholded
solution from a given signal. Our theoretical result is universal in the sense
that it holds for all choices of parameters, and the underlying analysis
depends only on fundamental arguments in mathematical optimization. We discuss
the implications for two domains:
  Compressed Sensing. On account of the crucial estimate, we bridge the
connection between the restricted isometry property (RIP) and the sparsity
parameter for a vast volume of hard thresholding based algorithms, which
renders an improvement on the RIP condition especially when the true sparsity
is unknown. This suggests that in essence, many more kinds of sensing matrices
or fewer measurements are admissible for the data acquisition procedure.
  Machine Learning. In terms of large-scale machine learning, a significant yet
challenging problem is learning accurate sparse models in an efficient manner.
In stark contrast to prior work that attempted the $\ell_1$-relaxation for
promoting sparsity, we present a novel stochastic algorithm which performs hard
thresholding in each iteration, hence ensuring such parsimonious solutions.
Equipped with the developed bound, we prove the {\em global linear convergence}
for a number of prevalent statistical models under mild assumptions, even
though the problem turns out to be non-convex.
Self-Supervised Visual Planning with Temporal Skip Connections
Authors: Frederik Ebert, Chelsea Finn, Alex X. Lee, Sergey Levine
Date: 2017-10-15 00:00:00
Tasks: Prediction, Video Prediction

In order to autonomously learn wide repertoires of complex skills, robots
must be able to learn from their own autonomously collected data, without human
supervision. One learning signal that is always available for autonomously
collected data is prediction: if a robot can learn to predict the future, it
can use this predictive model to take actions to produce desired outcomes, such
as moving an object to a particular location. However, in complex open-world
scenarios, designing a representation for prediction is difficult. In this
work, we instead aim to enable self-supervised robotic learning through direct
video prediction: instead of attempting to design a good representation, we
directly predict what the robot will see next, and then use this model to
achieve desired goals. A key challenge in video prediction for robotic
manipulation is handling complex spatial arrangements such as occlusions. To
that end, we introduce a video prediction model that can keep track of objects
through occlusion by incorporating temporal skip-connections. Together with a
novel planning criterion and action space formulation, we demonstrate that this
model substantially outperforms prior work on video prediction-based control.
Our results show manipulation of objects not seen during training, handling
multiple objects, and pushing objects around obstructions. These results
represent a significant advance in the range and complexity of skills that can
be performed entirely with self-supervised robotic learning.
Multi-Value Rule Sets
Authors: Tong Wang
Date: 2017-10-15 00:00:00

We present the Multi-vAlue Rule Set (MARS) model for interpretable
classification with feature efficient presentations. MARS introduces a more
generalized form of association rules that allows multiple values in a
condition. Rules of this form are more concise than traditional single-valued
rules in capturing and describing patterns in data. MARS mitigates the problem
of dealing with continuous features and high-cardinality categorical features
faced by rule-based models. Our formulation also pursues a higher efficiency of
feature utilization, which reduces the cognitive load to understand the
decision process. We propose an efficient inference method for learning a
maximum a posteriori model, incorporating theoretically grounded bounds to
iteratively reduce the search space to improve search efficiency. Experiments
with synthetic and real-world data demonstrate that MARS models have
significantly smaller complexity and fewer features, providing better
interpretability while being competitive in predictive accuracy. We conducted a
usability study with human subjects and results show that MARS is the easiest
to use compared with other competing rule-based models, in terms of the correct
rate and response time. Overall, MARS introduces a new approach to rule-based
models that balance accuracy and interpretability with feature-efficient
representations.
On Hashing-Based Approaches to Approximate DNF-Counting
Authors: Kuldeep S. Meel, Aditya A. Shrotri, Moshe Y. Vardi
Date: 2017-10-14 00:00:00
Tasks: Decision Making, Decision Making Under Uncertainty

Propositional model counting is a fundamental problem in artificial
intelligence with a wide variety of applications, such as probabilistic
inference, decision making under uncertainty, and probabilistic databases.
Consequently, the problem is of theoretical as well as practical interest. When
the constraints are expressed as DNF formulas, Monte Carlo-based techniques
have been shown to provide a fully polynomial randomized approximation scheme
(FPRAS). For CNF constraints, hashing-based approximation techniques have been
demonstrated to be highly successful. Furthermore, it was shown that
hashing-based techniques also yield an FPRAS for DNF counting without usage of
Monte Carlo sampling. Our analysis, however, shows that the proposed
hashing-based approach to DNF counting provides poor time complexity compared
to the Monte Carlo-based DNF counting techniques. Given the success of
hashing-based techniques for CNF constraints, it is natural to ask: Can
hashing-based techniques provide an efficient FPRAS for DNF counting? In this
paper, we provide a positive answer to this question. To this end, we introduce
two novel algorithmic techniques: \emph{Symbolic Hashing} and \emph{Stochastic
Cell Counting}, along with a new hash family of \emph{Row-Echelon hash
functions}. These innovations allow us to design a hashing-based FPRAS for DNF
counting of similar complexity (up to polylog factors) as that of prior works.
Furthermore, we expect these techniques to have potential applications beyond
DNF counting.
A Conservation Law Method in Optimization
Authors: Bin Shi
Date: 2017-08-27 00:00:00
Tasks: Friction

We propose some algorithms to find local minima in nonconvex optimization and
to obtain global minima in some degree from the Newton Second Law without
friction. With the key observation of the velocity observable and controllable
in the motion, the algorithms simulate the Newton Second Law without friction
based on symplectic Euler scheme. From the intuitive analysis of analytical
solution, we give a theoretical analysis for the high-speed convergence in the
algorithm proposed. Finally, we propose the experiments for strongly convex
function, non-strongly convex function and nonconvex function in
high-dimension.
On better training the infinite restricted Boltzmann machines
Authors: Xuan Peng, Xunzhang Gao, Xiang Li
Date: 2017-09-11 00:00:00

The infinite restricted Boltzmann machine (iRBM) is an extension of the
classic RBM. It enjoys a good property of automatically deciding the size of
the hidden layer according to specific training data. With sufficient training,
the iRBM can achieve a competitive performance with that of the classic RBM.
However, the convergence of learning the iRBM is slow, due to the fact that the
iRBM is sensitive to the ordering of its hidden units, the learned filters
change slowly from the left-most hidden unit to right. To break this dependency
between neighboring hidden units and speed up the convergence of training, a
novel training strategy is proposed. The key idea of the proposed training
strategy is randomly regrouping the hidden units before each gradient descent
step. Potentially, a mixing of infinite many iRBMs with different permutations
of the hidden units can be achieved by this learning method, which has a
similar effect of preventing the model from over-fitting as the dropout. The
original iRBM is also modified to be capable of carrying out discriminative
training. To evaluate the impact of our method on convergence speed of learning
and the model's generalization ability, several experiments have been performed
on the binarized MNIST and CalTech101 Silhouettes datasets. Experimental
results indicate that the proposed training strategy can greatly accelerate
learning and enhance generalization ability of iRBMs.
An Adaptive Framework for Missing Depth Inference Using Joint Bilateral Filter
Authors: Rajer Sindhu, Jayesh Ananya
Date: 2017-10-14 00:00:00
Tasks: Depth Estimation, Descriptive, valid

Depth imaging has largely focused on sensor and intrinsics properties.
However, the accuracy of acquire pixel is largely dependent on the capture. We
propose a new depth estimation and approximation algorithm which takes an
arbitrary 3D point cloud as input, with potentially complex geometric
structures, and generates automatically a bounding box which is used to clamp
the 3D distribution into a valid range. We then infer the desired compact
geometric network from complex 3D geometries by using a series of adaptive
joint bilateral filters. Our approach leverages these input depth in the
construction of a compact descriptive adaptive filter framework. The built
system that allows a user to control the result of capture depth map to fit the
target geometry. In addition, it is desirable to visualize structurally
problematic areas of the depth data in a dynamic environment. To provide this
feature, we investigate a fast update algorithm for the fragility of each
pixel's corresponding 3D point using machine learning. We present a new for of
feature vector analysis and demonstrate the effectiveness in the dataset. In
our experiment, we demonstrate the practicality and benefits of our proposed
method by computing accurate solutions captured depth map from different types
of sensors and shows better results than existing methods.
Prior matters: simple and general methods for evaluating and improving topic quality in topic modeling
Authors: Angela Fan, Finale Doshi-Velez, Luke Miratrix
Date: 2017-01-12 00:00:00

Latent Dirichlet Allocation (LDA) models trained without stopword removal
often produce topics with high posterior probabilities on uninformative words,
obscuring the underlying corpus content. Even when canonical stopwords are
manually removed, uninformative words common in that corpus will still dominate
the most probable words in a topic. In this work, we first show how the
standard topic quality measures of coherence and pointwise mutual information
act counter-intuitively in the presence of common but irrelevant words, making
it difficult to even quantitatively identify situations in which topics may be
dominated by stopwords. We propose an additional topic quality metric that
targets the stopword problem, and show that it, unlike the standard measures,
correctly correlates with human judgements of quality. We also propose a
simple-to-implement strategy for generating topics that are evaluated to be of
much higher quality by both human assessment and our new metric. This approach,
a collection of informative priors easily introduced into most LDA-style
inference methods, automatically promotes terms with domain relevance and
demotes domain-specific stop words. We demonstrate this approach's
effectiveness in three very different domains: Department of Labor accident
reports, online health forum posts, and NIPS abstracts. Overall we find that
current practices thought to solve this problem do not do so adequately, and
that our proposal offers a substantial improvement for those interested in
interpreting their topics as objects in their own right.
Subregular Complexity and Deep Learning
Authors: Enes Avcu, Chihiro Shibata, Jeffrey Heinz
Date: 2017-05-16 00:00:00
Tasks: Deep Learning, Temporal Sequences

This paper argues that the judicial use of formal language theory and
grammatical inference are invaluable tools in understanding how deep neural
networks can and cannot represent and learn long-term dependencies in temporal
sequences. Learning experiments were conducted with two types of Recurrent
Neural Networks (RNNs) on six formal languages drawn from the Strictly Local
(SL) and Strictly Piecewise (SP) classes. The networks were Simple RNNs
(s-RNNs) and Long Short-Term Memory RNNs (LSTMs) of varying sizes. The SL and
SP classes are among the simplest in a mathematically well-understood hierarchy
of subregular classes. They encode local and long-term dependencies,
respectively. The grammatical inference algorithm Regular Positive and Negative
Inference (RPNI) provided a baseline. According to earlier research, the LSTM
architecture should be capable of learning long-term dependencies and should
outperform s-RNNs. The results of these experiments challenge this narrative.
First, the LSTMs' performance was generally worse in the SP experiments than in
the SL ones. Second, the s-RNNs out-performed the LSTMs on the most complex SP
experiment and performed comparably to them on the others.
Mental Sampling in Multimodal Representations
Authors: Jian-Qiao Zhu, Adam N. Sanborn, Nick Chater
Date: 2017-10-14 00:00:00
Proceeding: NeurIPS 2018 12

Both resources in the natural environment and concepts in a semantic space
are distributed "patchily", with large gaps in between the patches. To describe
people's internal and external foraging behavior, various random walk models
have been proposed. In particular, internal foraging has been modeled as
sampling: in order to gather relevant information for making a decision, people
draw samples from a mental representation using random-walk algorithms such as
Markov chain Monte Carlo (MCMC). However, two common empirical observations
argue against simple sampling algorithms such as MCMC. First, the spatial
structure is often best described by a L\'evy flight distribution: the
probability of the distance between two successive locations follows a
power-law on the distances. Second, the temporal structure of the sampling that
humans and other animals produce have long-range, slowly decaying serial
correlations characterized as $1/f$-like fluctuations. We propose that mental
sampling is not done by simple MCMC, but is instead adapted to multimodal
representations and is implemented by Metropolis-coupled Markov chain Monte
Carlo (MC$^3$), one of the first algorithms developed for sampling from
multimodal distributions. MC$^3$ involves running multiple Markov chains in
parallel but with target distributions of different temperatures, and it swaps
the states of the chains whenever a better location is found. Heated chains
more readily traverse valleys in the probability landscape to propose moves to
far-away peaks, while the colder chains make the local steps that explore the
current peak or patch. We show that MC$^3$ generates distances between
successive samples that follow a L\'evy flight distribution and $1/f$-like
serial correlations, providing a single mechanistic account of these two
puzzling empirical phenomena.
On Data-Driven Saak Transform
Authors: C. -C. Jay Kuo, Yueru Chen
Date: 2017-10-11 00:00:00

Being motivated by the multilayer RECOS (REctified-COrrelations on a Sphere)
transform, we develop a data-driven Saak (Subspace approximation with augmented
kernels) transform in this work. The Saak transform consists of three steps: 1)
building the optimal linear subspace approximation with orthonormal bases using
the second-order statistics of input vectors, 2) augmenting each transform
kernel with its negative, 3) applying the rectified linear unit (ReLU) to the
transform output. The Karhunen-Lo\'eve transform (KLT) is used in the first
step. The integration of Steps 2 and 3 is powerful since they resolve the sign
confusion problem, remove the rectification loss and allow a straightforward
implementation of the inverse Saak transform at the same time. Multiple Saak
transforms are cascaded to transform images of a larger size. All Saak
transform kernels are derived from the second-order statistics of input random
vectors in a one-pass feedforward manner. Neither data labels nor
backpropagation is used in kernel determination. Multi-stage Saak transforms
offer a family of joint spatial-spectral representations between two extremes;
namely, the full spatial-domain representation and the full spectral-domain
representation. We select Saak coefficients of higher discriminant power to
form a feature vector for pattern recognition, and use the MNIST dataset
classification problem as an illustrative example.