pwc-archive/papers-with-abstracts huggingface dataset:
"Knowing value" logic as a normal modal logic
Authors: Tao Gu, Yanjing Wang
Date: 2016-04-29 00:00:00
Tasks: Negation

Recent years witness a growing interest in nonstandard epistemic logics of
"knowing whether", "knowing what", "knowing how", and so on. These logics are
usually not normal, i.e., the standard axioms and reasoning rules for modal
logic may be invalid. In this paper, we show that the conditional "knowing
value" logic proposed by Wang and Fan \cite{WF13} can be viewed as a disguised
normal modal logic by treating the negation of the Kv operator as a special
diamond. Under this perspective, it turns out that the original first-order
Kripke semantics can be greatly simplified by introducing a ternary relation
$R_i^c$ in standard Kripke models, which associates one world with two
$i$-accessible worlds that do not agree on the value of constant $c$. Under
intuitive constraints, the modal logic based on such Kripke models is exactly
the one studied by Wang and Fan (2013,2014}. Moreover, there is a very natural
binary generalization of the "knowing value" diamond, which, surprisingly, does
not increase the expressive power of the logic. The resulting logic with the
binary diamond has a transparent normal modal system, which sharpens our
understanding of the "knowing value" logic and simplifies some previously hard
problems.
Scalable Adaptive Stochastic Optimization Using Random Projections
Authors: Gabriel Krummenacher, Brian McWilliams, Yannic Kilcher, Joachim M. Buhmann, Nicolai Meinshausen
Date: 2016-11-21 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Dimensionality Reduction, Stochastic Optimization

Adaptive stochastic gradient methods such as AdaGrad have gained popularity
in particular for training deep neural networks. The most commonly used and
studied variant maintains a diagonal matrix approximation to second order
information by accumulating past gradients which are used to tune the step size
adaptively. In certain situations the full-matrix variant of AdaGrad is
expected to attain better performance, however in high dimensions it is
computationally impractical. We present Ada-LR and RadaGrad two computationally
efficient approximations to full-matrix AdaGrad based on randomized
dimensionality reduction. They are able to capture dependencies between
features and achieve similar performance to full-matrix AdaGrad but at a much
smaller computational cost. We show that the regret of Ada-LR is close to the
regret of full-matrix AdaGrad which can have an up-to exponentially smaller
dependence on the dimension than the diagonal variant. Empirically, we show
that Ada-LR and RadaGrad perform similarly to full-matrix AdaGrad. On the task
of training convolutional neural networks as well as recurrent neural networks,
RadaGrad achieves faster convergence than diagonal AdaGrad.
Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling
Authors: Peng Zhou, Zhenyu Qi, Suncong Zheng, Jiaming Xu, Hongyun Bao, Bo Xu
Date: 2016-11-21 00:00:00
Proceeding: COLING 2016 12
Tasks: Binary Classification, Classification, General Classification, Sentiment Analysis, Text Classification

Recurrent Neural Network (RNN) is one of the most popular architectures used
in Natural Language Processsing (NLP) tasks because its recurrent structure is
very suitable to process variable-length text. RNN can utilize distributed
representations of words by first converting the tokens comprising each text
into vectors, which form a matrix. And this matrix includes two dimensions: the
time-step dimension and the feature vector dimension. Then most existing models
usually utilize one-dimensional (1D) max pooling operation or attention-based
operation only on the time-step dimension to obtain a fixed-length vector.
However, the features on the feature vector dimension are not mutually
independent, and simply applying 1D pooling operation over the time-step
dimension independently may destroy the structure of the feature
representation. On the other hand, applying two-dimensional (2D) pooling
operation over the two dimensions may sample more meaningful features for
sequence modeling tasks. To integrate the features on both dimensions of the
matrix, this paper explores applying 2D max pooling operation to obtain a
fixed-length representation of the text. This paper also utilizes 2D
convolution to sample more meaningful information of the matrix. Experiments
are conducted on six text classification tasks, including sentiment analysis,
question classification, subjectivity classification and newsgroup
classification. Compared with the state-of-the-art models, the proposed models
achieve excellent performance on 4 out of 6 tasks. Specifically, one of the
proposed models achieves highest accuracy on Stanford Sentiment Treebank binary
classification and fine-grained classification tasks.
Not Afraid of the Dark: NIR-VIS Face Recognition via Cross-spectral Hallucination and Low-rank Embedding
Authors: Jose Lezama, Qiang Qiu, Guillermo Sapiro
Date: 2016-11-21 00:00:00
Proceeding: CVPR 2017 7
Tasks: Deep Learning, Face Recognition, Hallucination

Surveillance cameras today often capture NIR (near infrared) images in
low-light environments. However, most face datasets accessible for training and
verification are only collected in the VIS (visible light) spectrum. It remains
a challenging problem to match NIR to VIS face images due to the different
light spectrum. Recently, breakthroughs have been made for VIS face recognition
by applying deep learning on a huge amount of labeled VIS face samples. The
same deep learning approach cannot be simply applied to NIR face recognition
for two main reasons: First, much limited NIR face images are available for
training compared to the VIS spectrum. Second, face galleries to be matched are
mostly available only in the VIS spectrum. In this paper, we propose an
approach to extend the deep learning breakthrough for VIS face recognition to
the NIR spectrum, without retraining the underlying deep models that see only
VIS faces. Our approach consists of two core components, cross-spectral
hallucination and low-rank embedding, to optimize respectively input and output
of a VIS deep model for cross-spectral face recognition. Cross-spectral
hallucination produces VIS faces from NIR images through a deep learning
approach. Low-rank embedding restores a low-rank structure for faces deep
features across both NIR and VIS spectrum. We observe that it is often equally
effective to perform hallucination to input NIR images or low-rank embedding to
output deep features for a VIS deep model for cross-spectral recognition. When
hallucination and low-rank embedding are deployed together, we observe
significant further improvement; we obtain state-of-the-art accuracy on the
CASIA NIR-VIS v2.0 benchmark, without the need at all to re-train the
recognition system.
MCMC assisted by Belief Propagation
Authors: Sungsoo Ahn, Michael Chertkov, Jinwoo Shin
Date: 2016-05-29 00:00:00

Markov Chain Monte Carlo (MCMC) and Belief Propagation (BP) are the most popular algorithms for computational inference in Graphical Models (GM). In principle, MCMC is an exact probabilistic method which, however, often suffers from exponentially slow mixing. In contrast, BP is a deterministic method, which is typically fast, empirically very successful, however in general lacking control of accuracy over loopy graphs. In this paper, we introduce MCMC algorithms correcting the approximation error of BP, i.e., we provide a way to compensate for BP errors via a consecutive BP-aware MCMC. Our framework is based on the Loop Calculus (LC) approach which allows expressing the BP error as a sum of weighted generalized loops. Although the full series is computationally intractable, it is known that a truncated series, summing up all 2-regular loops, is computable in polynomial-time for planar pair-wise binary GMs and it also provides a highly accurate approximation empirically. Motivated by this, we first propose a polynomial-time approximation MCMC scheme for the truncated series of general (non-planar) pair-wise binary models. Our main idea here is to use the Worm algorithm, known to provide fast mixing in other (related) problems, and then design an appropriate rejection scheme to sample 2-regular loops. Furthermore, we also design an efficient rejection-free MCMC scheme for approximating the full series. The main novelty underlying our design is in utilizing the concept of cycle basis, which provides an efficient decomposition of the generalized loops. In essence, the proposed MCMC schemes run on transformed GM built upon the non-trivial BP solution, and our experiments show that this synthesis of BP and MCMC outperforms both direct MCMC and bare BP schemes.
emoji2vec: Learning Emoji Representations from their Description
Authors: Ben Eisner, Tim Rocktäschel, Isabelle Augenstein, Matko Bošnjak, Sebastian Riedel
Date: 2016-09-27 00:00:00
Proceeding: WS 2016 11
Tasks: Representation Learning, Sentiment Analysis, Word Embeddings

Many current natural language processing applications for social media rely
on representation learning and utilize pre-trained word embeddings. There
currently exist several publicly-available, pre-trained sets of word
embeddings, but they contain few or no emoji representations even as emoji
usage in social media has increased. In this paper we release emoji2vec,
pre-trained embeddings for all Unicode emoji which are learned from their
description in the Unicode emoji standard. The resulting emoji embeddings can
be readily used in downstream social natural language processing applications
alongside word2vec. We demonstrate, for the downstream task of sentiment
analysis, that emoji embeddings learned from short descriptions outperforms a
skip-gram model trained on a large collection of tweets, while avoiding the
need for contexts in which emoji need to appear frequently in order to estimate
a representation.
Private Empirical Risk Minimization Beyond the Worst Case: The Effect of the Constraint Set Geometry
Authors: Kunal Talwar, Abhradeep Thakurta, Li Zhang
Date: 2014-11-20 00:00:00

Empirical Risk Minimization (ERM) is a standard technique in machine
learning, where a model is selected by minimizing a loss function over
constraint set. When the training dataset consists of private information, it
is natural to use a differentially private ERM algorithm, and this problem has
been the subject of a long line of work started with Chaudhuri and Monteleoni
2008. A private ERM algorithm outputs an approximate minimizer of the loss
function and its error can be measured as the difference from the optimal value
of the loss function. When the constraint set is arbitrary, the required error
bounds are fairly well understood \cite{BassilyST14}. In this work, we show
that the geometric properties of the constraint set can be used to derive
significantly better results. Specifically, we show that a differentially
private version of Mirror Descent leads to error bounds of the form
$\tilde{O}(G_{\mathcal{C}}/n)$ for a lipschitz loss function, improving on the
$\tilde{O}(\sqrt{p}/n)$ bounds in Bassily, Smith and Thakurta 2014. Here $p$ is
the dimensionality of the problem, $n$ is the number of data points in the
training set, and $G_{\mathcal{C}}$ denotes the Gaussian width of the
constraint set that we optimize over. We show similar improvements for strongly
convex functions, and for smooth functions. In addition, we show that when the
loss function is Lipschitz with respect to the $\ell_1$ norm and $\mathcal{C}$
is $\ell_1$-bounded, a differentially private version of the Frank-Wolfe
algorithm gives error bounds of the form $\tilde{O}(n^{-2/3})$. This captures
the important and common case of sparse linear regression (LASSO), when the
data $x_i$ satisfies $|x_i|_{\infty} \leq 1$ and we optimize over the $\ell_1$
ball. We show new lower bounds for this setting, that together with known
bounds, imply that all our upper bounds are tight.
Zero-Shot Visual Question Answering
Authors: Damien Teney, Anton Van Den Hengel
Date: 2016-11-17 00:00:00
Tasks: Question Answering, Retrieval, Visual Question Answering, Visual Question Answering (VQA), Word Embeddings

Part of the appeal of Visual Question Answering (VQA) is its promise to
answer new questions about previously unseen images. Most current methods
demand training questions that illustrate every possible concept, and will
therefore never achieve this capability, since the volume of required training
data would be prohibitive. Answering general questions about images requires
methods capable of Zero-Shot VQA, that is, methods able to answer questions
beyond the scope of the training questions. We propose a new evaluation
protocol for VQA methods which measures their ability to perform Zero-Shot VQA,
and in doing so highlights significant practical deficiencies of current
approaches, some of which are masked by the biases in current datasets. We
propose and evaluate several strategies for achieving Zero-Shot VQA, including
methods based on pretrained word embeddings, object classifiers with semantic
embeddings, and test-time retrieval of example images. Our extensive
experiments are intended to serve as baselines for Zero-Shot VQA, and they also
achieve state-of-the-art performance in the standard VQA evaluation setting.
Nonlinear Statistical Learning with Truncated Gaussian Graphical Models
Authors: Qinliang Su, Xuejun Liao, Changyou Chen, Lawrence Carin
Date: 2016-06-02 00:00:00
Tasks: General Classification

We introduce the truncated Gaussian graphical model (TGGM) as a novel
framework for designing statistical models for nonlinear learning. A TGGM is a
Gaussian graphical model (GGM) with a subset of variables truncated to be
nonnegative. The truncated variables are assumed latent and integrated out to
induce a marginal model. We show that the variables in the marginal model are
non-Gaussian distributed and their expected relations are nonlinear. We use
expectation-maximization to break the inference of the nonlinear model into a
sequence of TGGM inference problems, each of which is efficiently solved by
using the properties and numerical methods of multivariate Gaussian
distributions. We use the TGGM to design models for nonlinear regression and
classification, with the performances of these models demonstrated on extensive
benchmark datasets and compared to state-of-the-art competing results.
Unsupervised Learning with Truncated Gaussian Graphical Models
Authors: Qinliang Su, Xuejun Liao, Chunyuan Li, Zhe Gan, Lawrence Carin
Date: 2016-11-15 00:00:00
Tasks: Unsupervised Pre-training

Gaussian graphical models (GGMs) are widely used for statistical modeling,
because of ease of inference and the ubiquitous use of the normal distribution
in practical approximations. However, they are also known for their limited
modeling abilities, due to the Gaussian assumption. In this paper, we introduce
a novel variant of GGMs, which relaxes the Gaussian restriction and yet admits
efficient inference. Specifically, we impose a bipartite structure on the GGM
and govern the hidden variables by truncated normal distributions. The
nonlinearity of the model is revealed by its connection to rectified linear
unit (ReLU) neural networks. Meanwhile, thanks to the bipartite structure and
appealing properties of truncated normals, we are able to train the models
efficiently using contrastive divergence. We consider three output constructs,
accounting for real-valued, binary and count data. We further extend the model
to deep constructions and show that deep models can be used for unsupervised
pre-training of rectifier neural networks. Extensive experimental results are
provided to validate the proposed models and demonstrate their superiority over
competing models.
One-Class SVM with Privileged Information and its Application to Malware Detection
Authors: Evgeny Burnaev, Dmitry Smolyakov
Date: 2016-09-26 00:00:00
Tasks: Anomaly Detection, Classification, General Classification, Malware Classification, Malware Detection, One-Class Classification

A number of important applied problems in engineering, finance and medicine
can be formulated as a problem of anomaly detection. A classical approach to
the problem is to describe a normal state using a one-class support vector
machine. Then to detect anomalies we quantify a distance from a new observation
to the constructed description of the normal class. In this paper we present a
new approach to the one-class classification. We formulate a new problem
statement and a corresponding algorithm that allow taking into account a
privileged information during the training phase. We evaluate performance of
the proposed approach using a synthetic dataset, as well as the publicly
available Microsoft Malware Classification Challenge dataset.
Efficient Stochastic Inference of Bitwise Deep Neural Networks
Authors: Sebastian Vogel, Christoph Schorn, Andre Guntoro, Gerd Ascheid
Date: 2016-11-20 00:00:00
Tasks: General Classification

Recently published methods enable training of bitwise neural networks which
allow reduced representation of down to a single bit per weight. We present a
method that exploits ensemble decisions based on multiple stochastically
sampled network models to increase performance figures of bitwise neural
networks in terms of classification accuracy at inference. Our experiments with
the CIFAR-10 and GTSRB datasets show that the performance of such network
ensembles surpasses the performance of the high-precision base model. With this
technique we achieve 5.81% best classification error on CIFAR-10 test set using
bitwise networks. Concerning inference on embedded systems we evaluate these
bitwise networks using a hardware efficient stochastic rounding procedure. Our
work contributes to efficient embedded bitwise neural networks.
Latent Dependency Forest Models
Authors: Shanbo Chu, Yong Jiang, Kewei Tu
Date: 2016-09-08 00:00:00

Probabilistic modeling is one of the foundations of modern machine learning
and artificial intelligence. In this paper, we propose a novel type of
probabilistic models named latent dependency forest models (LDFMs). A LDFM
models the dependencies between random variables with a forest structure that
can change dynamically based on the variable values. It is therefore capable of
modeling context-specific independence. We parameterize a LDFM using a
first-order non-projective dependency grammar. Learning LDFMs from data can be
formulated purely as a parameter learning problem, and hence the difficult
problem of model structure learning is circumvented. Our experimental results
show that LDFMs are competitive with existing probabilistic models.
Learning Fully Convolutional Networks for Iterative Non-blind Deconvolution
Authors: Jiawei Zhang, Jinshan Pan, Wei-Sheng Lai, Rynson Lau, Ming-Hsuan Yang
Date: 2016-11-20 00:00:00
Proceeding: CVPR 2017 7
Tasks: Denoising, Image Deconvolution, Image Denoising

In this paper, we propose a fully convolutional networks for iterative
non-blind deconvolution We decompose the non-blind deconvolution problem into
image denoising and image deconvolution. We train a FCNN to remove noises in
the gradient domain and use the learned gradients to guide the image
deconvolution step. In contrast to the existing deep neural network based
methods, we iteratively deconvolve the blurred images in a multi-stage
framework. The proposed method is able to learn an adaptive image prior, which
keeps both local (details) and global (structures) information. Both
quantitative and qualitative evaluations on benchmark datasets demonstrate that
the proposed method performs favorably against state-of-the-art algorithms in
terms of quality and speed.
Visualizing Linguistic Shift
Authors: Salman Mahmood, Rami Al-Rfou, Klaus Mueller
Date: 2016-11-20 00:00:00
Tasks: Document Classification, Language Modeling, Language Modelling, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Philosophy, Word Embeddings

Neural network based models are a very powerful tool for creating word
embeddings, the objective of these models is to group similar words together.
These embeddings have been used as features to improve results in various
applications such as document classification, named entity recognition, etc.
Neural language models are able to learn word representations which have been
used to capture semantic shifts across time and geography. The objective of
this paper is to first identify and then visualize how words change meaning in
different text corpus. We will train a neural language model on texts from a
diverse set of disciplines philosophy, religion, fiction etc. Each text will
alter the embeddings of the words to represent the meaning of the word inside
that text. We will present a computational technique to detect words that
exhibit significant linguistic shift in meaning and usage. We then use enhanced
scatterplots and storyline visualization to visualize the linguistic shift.
Owl and Lizard: Patterns of Head Pose and Eye Pose in Driver Gaze Classification
Authors: Lex Fridman, Joonbum Lee, Bryan Reimer, Trent Victor
Date: 2015-08-17 00:00:00
Tasks: Classification, General Classification

Accurate, robust, inexpensive gaze tracking in the car can help keep a driver
safe by facilitating the more effective study of how to improve (1) vehicle
interfaces and (2) the design of future Advanced Driver Assistance Systems. In
this paper, we estimate head pose and eye pose from monocular video using
methods developed extensively in prior work and ask two new interesting
questions. First, how much better can we classify driver gaze using head and
eye pose versus just using head pose? Second, are there individual-specific
gaze strategies that strongly correlate with how much gaze classification
improves with the addition of eye pose information? We answer these questions
by evaluating data drawn from an on-road study of 40 drivers. The main insight
of the paper is conveyed through the analogy of an "owl" and "lizard" which
describes the degree to which the eyes and the head move when shifting gaze.
When the head moves a lot ("owl"), not much classification improvement is
attained by estimating eye pose on top of head pose. On the other hand, when
the head stays still and only the eyes move ("lizard"), classification accuracy
increases significantly from adding in eye pose. We characterize how that
accuracy varies between people, gaze strategies, and gaze regions.
Generating machine-executable plans from end-user's natural-language instructions
Authors: Rui Liu, Xiaoli Zhang
Date: 2016-11-20 00:00:00

It is critical for advanced manufacturing machines to autonomously execute a
task by following an end-user's natural language (NL) instructions. However, NL
instructions are usually ambiguous and abstract so that the machines may
misunderstand and incorrectly execute the task. To address this NL-based
human-machine communication problem and enable the machines to appropriately
execute tasks by following the end-user's NL instructions, we developed a
Machine-Executable-Plan-Generation (exePlan) method. The exePlan method
conducts task-centered semantic analysis to extract task-related information
from ambiguous NL instructions. In addition, the method specifies machine
execution parameters to generate a machine-executable plan by interpreting
abstract NL instructions. To evaluate the exePlan method, an industrial robot
Baxter was instructed by NL to perform three types of industrial tasks {'drill
a hole', 'clean a spot', 'install a screw'}. The experiment results proved that
the exePlan method was effective in generating machine-executable plans from
the end-user's NL instructions. Such a method has the promise to endow a
machine with the ability of NL-instructed task execution.