pwc-archive/papers-with-abstracts huggingface dataset:
A recurrent neural network without chaos
Authors: Thomas Laurent, James Von Brecht
Date: 2016-12-19 00:00:00
Tasks: Language Modeling, Language Modelling

We introduce an exceptionally simple gated recurrent neural network (RNN)
that achieves performance comparable to well-known gated architectures, such as
LSTMs and GRUs, on the word-level language modeling task. We prove that our
model has simple, predicable and non-chaotic dynamics. This stands in stark
contrast to more standard gated architectures, whose underlying dynamical
systems exhibit chaotic behavior.
An extended Perona-Malik model based on probabilistic models
Authors: Lars M. Mescheder, Dirk A. Lorenz
Date: 2016-12-19 00:00:00

The Perona-Malik model has been very successful at restoring images from
noisy input. In this paper, we reinterpret the Perona-Malik model in the
language of Gaussian scale mixtures and derive some extensions of the model.
Specifically, we show that the expectation-maximization (EM) algorithm applied
to Gaussian scale mixtures leads to the lagged-diffusivity algorithm for
computing stationary points of the Perona-Malik diffusion equations. Moreover,
we show how mean field approximations to these Gaussian scale mixtures lead to
a modification of the lagged-diffusivity algorithm that better captures the
uncertainties in the restoration. Since this modification can be hard to
compute in practice we propose relaxations to the mean field objective to make
the algorithm computationally feasible. Our numerical experiments show that
this modified lagged-diffusivity algorithm often performs better at restoring
textured areas and fuzzy edges than the unmodified algorithm. As a second
application of the Gaussian scale mixture framework, we show how an efficient
sampling procedure can be obtained for the probabilistic model, making the
computation of the conditional mean and other expectations algorithmically
feasible. Again, the resulting algorithm has a strong resemblance to the
lagged-diffusivity algorithm. Finally, we show that a probabilistic version of
the Mumford-Shah segementation model can be obtained in the same framework with
a discrete edge-prior.
A modified Physarum-inspired model for the user equilibrium traffic assignment problem
Authors: Shuai Xu, Wen Jiang, Yehang Shou
Date: 2016-12-19 00:00:00
Tasks: Equilibrium traffic assignment

The user equilibrium traffic assignment principle is very important in the
traffic assignment problem. Mathematical programming models are designed to
solve the user equilibrium problem in traditional algorithms. Recently, the
Physarum shows the ability to address the user equilibrium and system
optimization traffic assignment problems. However, the Physarum model are not
efficient in real traffic networks with two-way traffic characteristics and
multiple origin-destination pairs. In this article, a modified
Physarum-inspired model for the user equilibrium problem is proposed. By
decomposing traffic flux based on origin nodes, the traffic flux from different
origin-destination pairs can be distinguished in the proposed model. The
Physarum can obtain the equilibrium traffic flux when no shorter path can be
discovered between each origin-destination pair. Finally, numerical examples
use the Sioux Falls network to demonstrate the rationality and convergence
properties of the proposed model.
Interaction Screening: Efficient and Sample-Optimal Learning of Ising Models
Authors: Marc Vuffray, Sidhant Misra, Andrey Y. Lokhov, Michael Chertkov
Date: 2016-05-24 00:00:00
Proceeding: NeurIPS 2016 12

We consider the problem of learning the underlying graph of an unknown Ising
model on p spins from a collection of i.i.d. samples generated from the model.
We suggest a new estimator that is computationally efficient and requires a
number of samples that is near-optimal with respect to previously established
information-theoretic lower-bound. Our statistical estimator has a physical
interpretation in terms of "interaction screening". The estimator is consistent
and is efficiently implemented using convex optimization. We prove that with
appropriate regularization, the estimator recovers the underlying graph using a
number of samples that is logarithmic in the system size p and exponential in
the maximum coupling-intensity and maximum node-degree.
Crowd collectiveness measure via graph-based node clique learning
Authors: Wei-Ya Ren
Date: 2016-12-19 00:00:00

Collectiveness motions of crowd systems have attracted a great deal of
attentions in recently years. In this paper, we try to measure the
collectiveness of a crowd system by the proposed node clique learning method.
The proposed method is a graph based method, and investigates the influence
from one node to other nodes. A node is represented by a set of nodes which
named a clique, which is obtained by spreading information from this node to
other nodes in graph. Then only nodes with sufficient information are selected
as the clique of this node. The motion coherence between two nodes is defined
by node cliques comparing. The collectiveness of a node and the collectiveness
of the crowd system are defined by the nodes coherence. Self-driven particle
(SDP) model and the crowd motion database are used to test the ability of the
proposed method in measuring collectiveness.
Few-Shot Object Recognition from Machine-Labeled Web Images
Authors: Zhongwen Xu, Linchao Zhu, Yi Yang
Date: 2016-12-19 00:00:00
Proceeding: CVPR 2017 7
Tasks: Few-Shot Learning, Object, Object Recognition

With the tremendous advances of Convolutional Neural Networks (ConvNets) on
object recognition, we can now obtain reliable enough machine-labeled
annotations easily by predictions from off-the-shelf ConvNets. In this work, we
present an abstraction memory based framework for few-shot learning, building
upon machine-labeled image annotations. Our method takes some large-scale
machine-annotated datasets (e.g., OpenImages) as an external memory bank. In
the external memory bank, the information is stored in the memory slots with
the form of key-value, where image feature is regarded as key and label
embedding serves as value. When queried by the few-shot examples, our model
selects visually similar data from the external memory bank, and writes the
useful information obtained from related external data into another memory
bank, i.e., abstraction memory. Long Short-Term Memory (LSTM) controllers and
attention mechanisms are utilized to guarantee the data written to the
abstraction memory is correlated to the query example. The abstraction memory
concentrates information from the external memory bank, so that it makes the
few-shot recognition effective. In the experiments, we firstly confirm that our
model can learn to conduct few-shot object recognition on clean human-labeled
data from ImageNet dataset. Then, we demonstrate that with our model,
machine-labeled image annotations are very effective and abundant resources to
perform object recognition on novel categories. Experimental results show that
our proposed model with machine-labeled annotations achieves great performance,
only with a gap of 1% between of the one with human-labeled annotations.
Domain specialization: a post-training domain adaptation for Neural Machine Translation
Authors: Christophe Servan, Josep Crego, Jean Senellart
Date: 2016-12-19 00:00:00
Tasks: Domain Adaptation, Machine Translation, NMT, Translation

Domain adaptation is a key feature in Machine Translation. It generally
encompasses terminology, domain and style adaptation, especially for human
post-editing workflows in Computer Assisted Translation (CAT). With Neural
Machine Translation (NMT), we introduce a new notion of domain adaptation that
we call "specialization" and which is showing promising results both in the
learning speed and in adaptation accuracy. In this paper, we propose to explore
this approach under several perspectives.
Neural Machine Translation from Simplified Translations
Authors: Josep Crego, Jean Senellart
Date: 2016-12-19 00:00:00
Tasks: Knowledge Distillation, Machine Translation, NMT, Text Simplification, Translation

Text simplification aims at reducing the lexical, grammatical and structural
complexity of a text while keeping the same meaning. In the context of machine
translation, we introduce the idea of simplified translations in order to boost
the learning ability of deep neural translation models. We conduct preliminary
experiments showing that translation complexity is actually reduced in a
translation of a source bi-text compared to the target reference of the bi-text
while using a neural machine translation (NMT) system learned on the exact same
bi-text. Based on knowledge distillation idea, we then train an NMT system
using the simplified bi-text, and show that it outperforms the initial system
that was built over the reference data set. Performance is further boosted when
both reference and automatic translations are used to learn the network. We
perform an elementary analysis of the translated corpus and report accuracy
results of the proposed approach on English-to-French and English-to-German
translation tasks.
Monte Carlo sampling for stochastic weight functions
Authors: Daan Frenkel, K. Julian Schrenk, Stefano Martiniani
Date: 2016-12-19 00:00:00

Conventional Monte Carlo simulations are stochastic in the sense that the
acceptance of a trial move is decided by comparing a computed acceptance
probability with a random number, uniformly distributed between 0 and 1. Here
we consider the case that the weight determining the acceptance probability
itself is fluctuating. This situation is common in many numerical studies. We
show that it is possible to construct a rigorous Monte Carlo algorithm that
visits points in state space with a probability proportional to their average
weight. The same approach has the potential to transform the methodology of a
certain class of high-throughput experiments or the analysis of noisy datasets.
Active and Continuous Exploration with Deep Neural Networks and Expected Model Output Changes
Authors: Christoph Käding, Erik Rodner, Alexander Freytag, Joachim Denzler
Date: 2016-12-19 00:00:00
Tasks: Active Learning

The demands on visual recognition systems do not end with the complexity
offered by current large-scale image datasets, such as ImageNet. In
consequence, we need curious and continuously learning algorithms that actively
acquire knowledge about semantic concepts which are present in available
unlabeled data. As a step towards this goal, we show how to perform continuous
active learning and exploration, where an algorithm actively selects relevant
batches of unlabeled examples for annotation. These examples could either
belong to already known or to yet undiscovered classes. Our algorithm is based
on a new generalization of the Expected Model Output Change principle for deep
architectures and is especially tailored to deep neural networks. Furthermore,
we show easy-to-implement approximations that yield efficient techniques for
active selection. Empirical experiments show that our method outperforms
currently used heuristics.
Cross-Modal Manifold Learning for Cross-modal Retrieval
Authors: Sailesh Conjeti, Anees Kazi, Nassir Navab, Amin Katouzian
Date: 2016-12-19 00:00:00
Tasks: Cross-Modal Retrieval, Retrieval

This paper presents a new scalable algorithm for cross-modal similarity
preserving retrieval in a learnt manifold space. Unlike existing approaches
that compromise between preserving global and local geometries, the proposed
technique respects both simultaneously during manifold alignment. The global
topologies are maintained by recovering underlying mapping functions in the
joint manifold space by deploying partially corresponding instances. The
inter-, and intra-modality affinity matrices are then computed to reinforce
original data skeleton using perturbed minimum spanning tree (pMST), and
maximizing the affinity among similar cross-modal instances, respectively. The
performance of proposed algorithm is evaluated upon two multimodal image
datasets (coronary atherosclerosis histology and brain MRI) for two
applications: classification, and regression. Our exhaustive validations and
results demonstrate the superiority of our technique over comparative methods
and its feasibility for improving computer-assisted diagnosis systems, where
disease-specific complementary information shall be aggregated and interpreted
across modalities to form the final decision.
Weakly Supervised Deep Detection Networks
Authors: Hakan Bilen, Andrea Vedaldi
Date: 2015-11-09 00:00:00
Proceeding: CVPR 2016 6
Tasks: Classification, Data Augmentation, General Classification, Object, object-detection, Object Detection, Weakly-supervised Learning, Weakly Supervised Object Detection

Weakly supervised learning of object detection is an important problem in
image understanding that still does not have a satisfactory solution. In this
paper, we address this problem by exploiting the power of deep convolutional
neural networks pre-trained on large-scale image-level classification tasks. We
propose a weakly supervised deep detection architecture that modifies one such
network to operate at the level of image regions, performing simultaneously
region selection and classification. Trained as an image classifier, the
architecture implicitly learns object detectors that are better than
alternative weakly supervised detection systems on the PASCAL VOC data. The
model, which is a simple and elegant end-to-end architecture, outperforms
standard data augmentation and fine-tuning techniques for the task of
image-level classification as well.
Hierarchical Partitioning of the Output Space in Multi-label Data
Authors: Yannis Papanikolaou, Ioannis Katakis, Grigorios Tsoumakas
Date: 2016-12-19 00:00:00
Tasks: Clustering, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Multi-Label Learning

Hierarchy Of Multi-label classifiers (HOMER) is a multi-label learning
algorithm that breaks the initial learning task to several, easier sub-tasks by
first constructing a hierarchy of labels from a given label set and secondly
employing a given base multi-label classifier (MLC) to the resulting
sub-problems. The primary goal is to effectively address class imbalance and
scalability issues that often arise in real-world multi-label classification
problems. In this work, we present the general setup for a HOMER model and a
simple extension of the algorithm that is suited for MLCs that output rankings.
Furthermore, we provide a detailed analysis of the properties of the algorithm,
both from an aspect of effectiveness and computational complexity. A secondary
contribution involves the presentation of a balanced variant of the k means
algorithm, which serves in the first step of the label hierarchy construction.
We conduct extensive experiments on six real-world datasets, studying
empirically HOMER's parameters and providing examples of instantiations of the
algorithm with different clustering approaches and MLCs, The empirical results
demonstrate a significant improvement over the given base MLC.
On Random Weights for Texture Generation in One Layer Neural Networks
Authors: Mihir Mongia, Kundan Kumar, Akram Erraqabi, Yoshua Bengio
Date: 2016-12-19 00:00:00
Tasks: Texture Synthesis

Recent work in the literature has shown experimentally that one can use the
lower layers of a trained convolutional neural network (CNN) to model natural
textures. More interestingly, it has also been experimentally shown that only
one layer with random filters can also model textures although with less
variability. In this paper we ask the question as to why one layer CNNs with
random filters are so effective in generating textures? We theoretically show
that one layer convolutional architectures (without a non-linearity) paired
with the an energy function used in previous literature, can in fact preserve
and modulate frequency coefficients in a manner so that random weights and
pretrained weights will generate the same type of images. Based on the results
of this analysis we question whether similar properties hold in the case where
one uses one convolution layer with a non-linearity. We show that in the case
of ReLu non-linearity there are situations where only one input will give the
minimum possible energy whereas in the case of no nonlinearity, there are
always infinite solutions that will give the minimum possible energy. Thus we
can show that in certain situations adding a ReLu non-linearity generates less
variable images.
Improving Tweet Representations using Temporal and User Context
Authors: Ganesh J, Manish Gupta, Vasudeva Varma
Date: 2016-12-19 00:00:00
Tasks: Representation Learning

In this work we propose a novel representation learning model which computes
semantic representations for tweets accurately. Our model systematically
exploits the chronologically adjacent tweets ('context') from users' Twitter
timelines for this task. Further, we make our model user-aware so that it can
do well in modeling the target tweet by exploiting the rich knowledge about the
user such as the way the user writes the post and also summarizing the topics
on which the user writes. We empirically demonstrate that the proposed models
outperform the state-of-the-art models in predicting the user profile
attributes like spouse, education and job by 19.66%, 2.27% and 2.22%
respectively.
Mixing Times and Structural Inference for Bernoulli Autoregressive Processes
Authors: Dimitrios Katselis, Carolyn L. Beck, R. Srikant
Date: 2016-12-19 00:00:00
Tasks: Time Series Analysis

We introduce a novel multivariate random process producing Bernoulli outputs
per dimension, that can possibly formalize binary interactions in various
graphical structures and can be used to model opinion dynamics, epidemics,
financial and biological time series data, etc. We call this a Bernoulli
Autoregressive Process (BAR). A BAR process models a discrete-time vector
random sequence of $p$ scalar Bernoulli processes with autoregressive dynamics
and corresponds to a particular Markov Chain. The benefit from the
autoregressive dynamics is the description of a $2^p\times 2^p$ transition
matrix by at most $pd$ effective parameters for some $d\ll p$ or by two sparse
matrices of dimensions $p\times p^2$ and $p\times p$, respectively,
parameterizing the transitions. Additionally, we show that the BAR process
mixes rapidly, by proving that the mixing time is $O(\log p)$. The hidden
constant in the previous mixing time bound depends explicitly on the values of
the chain parameters and implicitly on the maximum allowed in-degree of a node
in the corresponding graph. For a network with $p$ nodes, where each node has
in-degree at most $d$ and corresponds to a scalar Bernoulli process generated
by a BAR, we provide a greedy algorithm that can efficiently learn the
structure of the underlying directed graph with a sample complexity
proportional to the mixing time of the BAR process. The sample complexity of
the proposed algorithm is nearly order-optimal as it is only a $\log p$ factor
away from an information-theoretic lower bound. We present simulation results
illustrating the performance of our algorithm in various setups, including a
model for a biological signaling network.
Dual Deep Network for Visual Tracking
Authors: Zhizhen Chi, Hongyang Li, Huchuan Lu, Ming-Hsuan Yang
Date: 2016-12-19 00:00:00
Tasks: Visual Tracking

Visual tracking addresses the problem of identifying and localizing an
unknown target in a video given the target specified by a bounding box in the
first frame. In this paper, we propose a dual network to better utilize
features among layers for visual tracking. It is observed that features in
higher layers encode semantic context while its counterparts in lower layers
are sensitive to discriminative appearance. Thus we exploit the hierarchical
features in different layers of a deep model and design a dual structure to
obtain better feature representation from various streams, which is rarely
investigated in previous work. To highlight geometric contours of the target,
we integrate the hierarchical feature maps with an edge detector as the coarse
prior maps to further embed local details around the target. To leverage the
robustness of our dual network, we train it with random patches measuring the
similarities between the network activation and target appearance, which serves
as a regularization to enforce the dual network to focus on target object. The
proposed dual network is updated online in a unique manner based on the
observation that the target being tracked in consecutive frames should share
more similar feature representations than those in the surrounding background.
It is also found that for a target object, the prior maps can help further
enhance performance by passing message into the output maps of the dual
network. Therefore, an independent component analysis with reference algorithm
(ICA-R) is employed to extract target context using prior maps as guidance.
Online tracking is conducted by maximizing the posterior estimate on the final
maps with stochastic and periodic update. Quantitative and qualitative
evaluations on two large-scale benchmark data sets show that the proposed
algorithm performs favourably against the state-of-the-arts.
The deterministic information bottleneck
Authors: DJ Strouse, David J. Schwab
Date: 2016-04-01 00:00:00
Tasks: Clustering, Computational Efficiency

Lossy compression and clustering fundamentally involve a decision about what
features are relevant and which are not. The information bottleneck method (IB)
by Tishby, Pereira, and Bialek formalized this notion as an
information-theoretic optimization problem and proposed an optimal tradeoff
between throwing away as many bits as possible, and selectively keeping those
that are most important. In the IB, compression is measure my mutual
information. Here, we introduce an alternative formulation that replaces mutual
information with entropy, which we call the deterministic information
bottleneck (DIB), that we argue better captures this notion of compression. As
suggested by its name, the solution to the DIB problem turns out to be a
deterministic encoder, or hard clustering, as opposed to the stochastic
encoder, or soft clustering, that is optimal under the IB. We compare the IB
and DIB on synthetic data, showing that the IB and DIB perform similarly in
terms of the IB cost function, but that the DIB significantly outperforms the
IB in terms of the DIB cost function. We also empirically find that the DIB
offers a considerable gain in computational efficiency over the IB, over a
range of convergence parameters. Our derivation of the DIB also suggests a
method for continuously interpolating between the soft clustering of the IB and
the hard clustering of the DIB.
Context and Interference Effects in the Combinations of Natural Concepts
Authors: Diederik Aerts, Jonito Aerts Arguëlles, Lester Beltran, Lyneth Beltran, Massimiliano Sassoli de Bianchi, Sandro Sozzo, Tomas Veloz
Date: 2016-12-19 00:00:00

The mathematical formalism of quantum theory exhibits significant
effectiveness when applied to cognitive phenomena that have resisted
traditional (set theoretical) modeling. Relying on a decade of research on the
operational foundations of micro-physical and conceptual entities, we present a
theoretical framework for the representation of concepts and their conjunctions
and disjunctions that uses the quantum formalism. This framework provides a
unified solution to the 'conceptual combinations problem' of cognitive
psychology, explaining the observed deviations from classical (Boolean, fuzzy
set and Kolmogorovian) structures in terms of genuine quantum effects. In
particular, natural concepts 'interfere' when they combine to form more complex
conceptual entities, and they also exhibit a 'quantum-type context-dependence',
which are responsible of the 'over- and under-extension' that are
systematically observed in experiments on membership judgments.