pwc-archive/papers-with-abstracts huggingface dataset:
Sequential Neural Models with Stochastic Layers
Authors: Marco Fraccaro, Søren Kaae Sønderby, Ulrich Paquet, Ole Winther
Date: 2016-05-24 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Music Modeling, Variational Inference

How can we efficiently propagate uncertainty in a latent state representation
with recurrent neural networks? This paper introduces stochastic recurrent
neural networks which glue a deterministic recurrent neural network and a state
space model together to form a stochastic and sequential neural generative
model. The clear separation of deterministic and stochastic layers allows a
structured variational inference network to track the factorization of the
model's posterior distribution. By retaining both the nonlinear recursive
structure of a recurrent neural network and averaging over the uncertainty in a
latent path, like a state space model, we improve the state of the art results
on the Blizzard and TIMIT speech modeling data sets by a large margin, while
achieving comparable performances to competing methods on polyphonic music
modeling.
Accelerated Variance Reduced Block Coordinate Descent
Authors: Zebang Shen, Hui Qian, Chao Zhang, Tengfei Zhou
Date: 2016-11-13 00:00:00

Algorithms with fast convergence, small number of data access, and low
per-iteration complexity are particularly favorable in the big data era, due to
the demand for obtaining \emph{highly accurate solutions} to problems with
\emph{a large number of samples} in \emph{ultra-high} dimensional space.
Existing algorithms lack at least one of these qualities, and thus are
inefficient in handling such big data challenge. In this paper, we propose a
method enjoying all these merits with an accelerated convergence rate
$O(\frac{1}{k^2})$. Empirical studies on large scale datasets with more than
one million features are conducted to show the effectiveness of our methods in
practice.
Semi-Dense 3D Semantic Mapping from Monocular SLAM
Authors: Xuanpeng Li, Rachid Belaroussi
Date: 2016-11-13 00:00:00
Tasks: 3D Reconstruction, Semantic Segmentation

The bundle of geometry and appearance in computer vision has proven to be a
promising solution for robots across a wide variety of applications. Stereo
cameras and RGB-D sensors are widely used to realise fast 3D reconstruction and
trajectory tracking in a dense way. However, they lack flexibility of seamless
switch between different scaled environments, i.e., indoor and outdoor scenes.
In addition, semantic information are still hard to acquire in a 3D mapping. We
address this challenge by combining the state-of-art deep learning method and
semi-dense Simultaneous Localisation and Mapping (SLAM) based on video stream
from a monocular camera. In our approach, 2D semantic information are
transferred to 3D mapping via correspondence between connective Keyframes with
spatial consistency. There is no need to obtain a semantic segmentation for
each frame in a sequence, so that it could achieve a reasonable computation
time. We evaluate our method on indoor/outdoor datasets and lead to an
improvement in the 2D semantic labelling over baseline single frame
predictions.
Hand Gesture Recognition for Contactless Device Control in Operating Rooms
Authors: Ebrahim Nasr-Esfahani, Nader Karimi, S. M. Reza Soroushmehr, M. Hossein Jafari, M. Amin Khorsandi, Shadrokh Samavi, Kayvan Najarian
Date: 2016-11-13 00:00:00
Tasks: Gesture Recognition, Hand Gesture Recognition, Hand-Gesture Recognition

Hand gesture is one of the most important means of touchless communication
between human and machines. There is a great interest for commanding electronic
equipment in surgery rooms by hand gesture for reducing the time of surgery and
the potential for infection. There are challenges in implementation of a hand
gesture recognition system. It has to fulfill requirements such as high
accuracy and fast response. In this paper we introduce a system of hand gesture
recognition based on a deep learning approach. Deep learning is known as an
accurate detection model, but its high complexity prevents it from being
fabricated as an embedded system. To cope with this problem, we applied some
changes in the structure of our work to achieve low complexity. As a result,
the proposed method could be implemented on a naive embedded system. Our
experiments show that the proposed system results in higher accuracy while
having less complexity in comparison with the existing comparable methods.
Joint Representation Learning of Text and Knowledge for Knowledge Graph Completion
Authors: Xu Han, Zhiyuan Liu, Maosong Sun
Date: 2016-11-13 00:00:00
Tasks: General Classification, Knowledge Graph Completion, Prediction, Relation, Relation Classification, Relation Prediction, Representation Learning

Joint representation learning of text and knowledge within a unified semantic
space enables us to perform knowledge graph completion more accurately. In this
work, we propose a novel framework to embed words, entities and relations into
the same continuous vector space. In this model, both entity and relation
embeddings are learned by taking knowledge graph and plain text into
consideration. In experiments, we evaluate the joint learning model on three
tasks including entity prediction, relation prediction and relation
classification from text. The experiment results show that our model can
significantly and consistently improve the performance on the three tasks as
compared with other baselines.
Cross-lingual Dataless Classification for Languages with Small Wikipedia Presence
Authors: Yangqiu Song, Stephen Mayhew, Dan Roth
Date: 2016-11-13 00:00:00
Tasks: Classification, Document Classification, General Classification, Machine Translation, Text Categorization, Translation

This paper presents an approach to classify documents in any language into an
English topical label space, without any text categorization training data. The
approach, Cross-Lingual Dataless Document Classification (CLDDC) relies on
mapping the English labels or short category description into a Wikipedia-based
semantic representation, and on the use of the target language Wikipedia.
Consequently, performance could suffer when Wikipedia in the target language is
small. In this paper, we focus on languages with small Wikipedias,
(Small-Wikipedia languages, SWLs). We use a word-level dictionary to convert
documents in a SWL to a large-Wikipedia language (LWLs), and then perform CLDDC
based on the LWL's Wikipedia. This approach can be applied to thousands of
languages, which can be contrasted with machine translation, which is a
supervision heavy approach and can be done for about 100 languages. We also
develop a ranking algorithm that makes use of language similarity metrics to
automatically select a good LWL, and show that this significantly improves
classification of SWLs' documents, performing comparably to the best bridge
possible.
Batched Gaussian Process Bandit Optimization via Determinantal Point Processes
Authors: Tarun Kathuria, Amit Deshpande, Pushmeet Kohli
Date: 2016-11-13 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Bayesian Optimization, Diversity, Point Processes

Gaussian Process bandit optimization has emerged as a powerful tool for
optimizing noisy black box functions. One example in machine learning is
hyper-parameter optimization where each evaluation of the target function
requires training a model which may involve days or even weeks of computation.
Most methods for this so-called "Bayesian optimization" only allow sequential
exploration of the parameter space. However, it is often desirable to propose
batches or sets of parameter values to explore simultaneously, especially when
there are large parallel processing facilities at our disposal. Batch methods
require modeling the interaction between the different evaluations in the
batch, which can be expensive in complex scenarios. In this paper, we propose a
new approach for parallelizing Bayesian optimization by modeling the diversity
of a batch via Determinantal point processes (DPPs) whose kernels are learned
automatically. This allows us to generalize a previous result as well as prove
better regret bounds based on DPP sampling. Our experiments on a variety of
synthetic and real-world robotics and hyper-parameter optimization tasks
indicate that our DPP-based methods, especially those based on DPP sampling,
outperform state-of-the-art methods.
Semi-automatic Simultaneous Interpreting Quality Evaluation
Authors: Xiaojun Zhang
Date: 2016-11-12 00:00:00
Tasks: Translation

Increasing interpreting needs a more objective and automatic measurement. We
hold a basic idea that 'translating means translating meaning' in that we can
assessment interpretation quality by comparing the meaning of the interpreting
output with the source input. That is, a translation unit of a 'chunk' named
Frame which comes from frame semantics and its components named Frame Elements
(FEs) which comes from Frame Net are proposed to explore their matching rate
between target and source texts. A case study in this paper verifies the
usability of semi-automatic graded semantic-scoring measurement for human
simultaneous interpreting and shows how to use frame and FE matches to score.
Experiments results show that the semantic-scoring metrics have a significantly
correlation coefficient with human judgment.
The inductive theory of natural selection: summary and synthesis
Authors: Steven A. Frank
Date: 2014-12-03 00:00:00

The theory of natural selection has two forms. Deductive theory describes how
populations change over time. One starts with an initial population and some
rules for change. From those assumptions, one calculates the future state of
the population. Deductive theory predicts how populations adapt to
environmental challenge. Inductive theory describes the causes of change in
populations. One starts with a given amount of change. One then assigns
different parts of the total change to particular causes. Inductive theory
analyzes alternative causal models for how populations have adapted to
environmental challenge. This chapter emphasizes the inductive analysis of
cause.
GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
Authors: Matt J. Kusner, José Miguel Hernández-Lobato
Date: 2016-11-12 00:00:00

Generative Adversarial Networks (GAN) have limitations when the goal is to
generate sequences of discrete elements. The reason for this is that samples
from a distribution on discrete objects such as the multinomial are not
differentiable with respect to the distribution parameters. This problem can be
avoided by using the Gumbel-softmax distribution, which is a continuous
approximation to a multinomial distribution parameterized in terms of the
softmax function. In this work, we evaluate the performance of GANs based on
recurrent neural networks with Gumbel-softmax output distributions in the task
of generating sequences of discrete elements.
Stochastic single flux quantum neuromorphic computing using magnetically tunable Josephson junctions
Authors: S. E. Russek, C. A. Donnelly, M. L. Schneider, B. Baek, M. R. Pufall, W. H. Rippard, P. F. Hopkins, P. D. Dresselhaus, S. P. Benz
Date: 2016-11-12 00:00:00

Single flux quantum (SFQ) circuits form a natural neuromorphic technology
with SFQ pulses and superconducting transmission lines simulating action
potentials and axons, respectively. Here we present a new component, magnetic
Josephson junctions, that have a tunablility and re-configurability that was
lacking from previous SFQ neuromorphic circuits. The nanoscale magnetic
structure acts as a tunable synaptic constituent that modifies the junction
critical current. These circuits can operate near the thermal limit where
stochastic firing of the neurons is an essential component of the technology.
This technology has the ability to create complex neural systems with greater
than 10^21 neural firings per second with approximately 1 W dissipation.
Prognostics of Surgical Site Infections using Dynamic Health Data
Authors: Chuyang Ke, Yan Jin, Heather Evans, Bill Lober, Xiaoning Qian, Ji Liu, Shuai Huang
Date: 2016-11-12 00:00:00
Tasks: Decision Making, Imputation, Matrix Completion, Prediction

Surgical Site Infection (SSI) is a national priority in healthcare research.
Much research attention has been attracted to develop better SSI risk
prediction models. However, most of the existing SSI risk prediction models are
built on static risk factors such as comorbidities and operative factors. In
this paper, we investigate the use of the dynamic wound data for SSI risk
prediction. There have been emerging mobile health (mHealth) tools that can
closely monitor the patients and generate continuous measurements of many
wound-related variables and other evolving clinical variables. Since existing
prediction models of SSI have quite limited capacity to utilize the evolving
clinical data, we develop the corresponding solution to equip these mHealth
tools with decision-making capabilities for SSI prediction with a seamless
assembly of several machine learning models to tackle the analytic challenges
arising from the spatial-temporal data. The basic idea is to exploit the
low-rank property of the spatial-temporal data via the bilinear formulation,
and further enhance it with automatic missing data imputation by the matrix
completion technique. We derive efficient optimization algorithms to implement
these models and demonstrate the superior performances of our new predictive
model on a real-world dataset of SSI, compared to a range of state-of-the-art
methods.
Cooperative Inverse Reinforcement Learning
Authors: Dylan Hadfield-Menell, Anca Dragan, Pieter Abbeel, Stuart Russell
Date: 2016-06-09 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Active Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

For an autonomous system to be helpful to humans and to pose no unwarranted
risks, it needs to align its values with those of the humans in its environment
in such a way that its actions contribute to the maximization of value for the
humans. We propose a formal definition of the value alignment problem as
cooperative inverse reinforcement learning (CIRL). A CIRL problem is a
cooperative, partial-information game with two agents, human and robot; both
are rewarded according to the human's reward function, but the robot does not
initially know what this is. In contrast to classical IRL, where the human is
assumed to act optimally in isolation, optimal CIRL solutions produce behaviors
such as active teaching, active learning, and communicative actions that are
more effective in achieving value alignment. We show that computing optimal
joint policies in CIRL games can be reduced to solving a POMDP, prove that
optimality in isolation is suboptimal in CIRL, and derive an approximate CIRL
algorithm.
On Multiplicative Integration with Recurrent Neural Networks
Authors: Yuhuai Wu, Saizheng Zhang, Ying Zhang, Yoshua Bengio, Ruslan Salakhutdinov
Date: 2016-06-21 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Language Modelling

We introduce a general and simple structural design called Multiplicative
Integration (MI) to improve recurrent neural networks (RNNs). MI changes the
way in which information from difference sources flows and is integrated in the
computational building block of an RNN, while introducing almost no extra
parameters. The new structure can be easily embedded into many popular RNN
models, including LSTMs and GRUs. We empirically analyze its learning behaviour
and conduct evaluations on several tasks using different RNN models. Our
experimental results demonstrate that Multiplicative Integration can provide a
substantial performance boost over many of the existing RNN models.
Architectural Complexity Measures of Recurrent Neural Networks
Authors: Saizheng Zhang, Yuhuai Wu, Tong Che, Zhouhan Lin, Roland Memisevic, Ruslan Salakhutdinov, Yoshua Bengio
Date: 2016-02-26 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Language Modelling

In this paper, we systematically analyze the connecting architectures of
recurrent neural networks (RNNs). Our main contribution is twofold: first, we
present a rigorous graph-theoretic framework describing the connecting
architectures of RNNs in general. Second, we propose three architecture
complexity measures of RNNs: (a) the recurrent depth, which captures the RNN's
over-time nonlinear complexity, (b) the feedforward depth, which captures the
local input-output nonlinearity (similar to the "depth" in feedforward neural
networks (FNNs)), and (c) the recurrent skip coefficient which captures how
rapidly the information propagates over time. We rigorously prove each
measure's existence and computability. Our experimental results show that RNNs
might benefit from larger recurrent depth and feedforward depth. We further
demonstrate that increasing recurrent skip coefficient offers performance
boosts on long term dependency problems.
1.5 billion words Arabic Corpus
Authors: Ibrahim Abu El-khair
Date: 2016-11-12 00:00:00
Tasks: Articles

This study is an attempt to build a contemporary linguistic corpus for Arabic
language. The corpus produced, is a text corpus includes more than five million
newspaper articles. It contains over a billion and a half words in total, out
of which, there is about three million unique words. The data were collected
from newspaper articles in ten major news sources from eight Arabic countries,
over a period of fourteen years. The corpus was encoded with two types of
encoding, namely: UTF-8, and Windows CP-1256. Also it was marked with two
mark-up languages, namely: SGML, and XML.
Near-Optimal Active Learning of Halfspaces via Query Synthesis in the Noisy Setting
Authors: Lin Chen, Hamed Hassani, Amin Karbasi
Date: 2016-03-11 00:00:00
Tasks: Active Learning

In this paper, we consider the problem of actively learning a linear
classifier through query synthesis where the learner can construct artificial
queries in order to estimate the true decision boundaries. This problem has
recently gained a lot of interest in automated science and adversarial reverse
engineering for which only heuristic algorithms are known. In such
applications, queries can be constructed de novo to elicit information (e.g.,
automated science) or to evade detection with minimal cost (e.g., adversarial
reverse engineering). We develop a general framework, called dimension coupling
(DC), that 1) reduces a d-dimensional learning problem to d-1 low dimensional
sub-problems, 2) solves each sub-problem efficiently, 3) appropriately
aggregates the results and outputs a linear classifier, and 4) provides a
theoretical guarantee for all possible schemes of aggregation. The proposed
method is proved resilient to noise. We show that the DC framework avoids the
curse of dimensionality: its computational complexity scales linearly with the
dimension. Moreover, we show that the query complexity of DC is near optimal
(within a constant factor of the optimum algorithm). To further support our
theoretical analysis, we compare the performance of DC with the existing work.
We observe that DC consistently outperforms the prior arts in terms of query
complexity while often running orders of magnitude faster.
Sparsey: Event Recognition via Deep Hierarchical Spare Distributed Codes
Authors: Gerard J. Rinkus
Date: 2016-11-12 00:00:00
Tasks: Retrieval

Visual cortex's hierarchical, multi-level organization is captured in many
biologically inspired computational vision models, the general idea being that
progressively larger scale, more complex spatiotemporal features are
represented in progressively higher areas. However, most earlier models use
localist representations (codes) in each representational field, which we
equate with the cortical macrocolumn (mac), at each level. In localism, each
represented feature/event (item) is coded by a single unit. Our model, Sparsey,
is also hierarchical but crucially, uses sparse distributed coding (SDC) in
every mac in all levels. In SDC, each represented item is coded by a small
subset of the mac's units. SDCs of different items can overlap and the size of
overlap between items can represent their similarity. The difference between
localism and SDC is crucial because SDC allows the two essential operations of
associative memory, storing a new item and retrieving the best-matching stored
item, to be done in fixed time for the life of the model. Since the model's
core algorithm, which does both storage and retrieval (inference), makes a
single pass over all macs on each time step, the overall model's
storage/retrieval operation is also fixed-time, a criterion we consider
essential for scalability to huge datasets. A 2010 paper described a
nonhierarchical version of this model in the context of purely spatial pattern
processing. Here, we elaborate a fully hierarchical model (arbitrary numbers of
levels and macs per level), describing novel model principles like progressive
critical periods, dynamic modulation of principal cells' activation functions
based on a mac-level familiarity measure, representation of multiple
simultaneously active hypotheses, a novel method of time warp invariant
recognition, and we report results showing learning/recognition of
spatiotemporal patterns.
Optimized clothes segmentation to boost gender classification in unconstrained scenarios
Authors: D. Freire-Obregón, M. Castrillón-Santana, J. Lorenzo-Navarro
Date: 2016-11-12 00:00:00
Tasks: Classification, Clustering, Face Detection, Gender Classification, General Classification, Segmentation, Superpixels

Several applications require demographic information of ordinary people in
unconstrained scenarios. This is not a trivial task due to significant human
appearance variations. In this work, we introduce trixels for clustering image
regions, enumerating their advantages compared to superpixels. The classical
GrabCut algorithm is later modified to segment trixels instead of pixels in an
unsupervised context. Combining with face detection lead us to a clothes
segmentation approach close to real time. The study uses the challenging Pascal
VOC dataset for segmentation evaluation experiments. A final experiment
analyzes the fusion of clothes features with state-of-the-art gender
classifiers in ClothesDB, revealing a significant performance improvement in
gender classification.
Unsupervised learning of object semantic parts from internal states of CNNs by population encoding
Authors: Jianyu Wang, Zhishuai Zhang, Cihang Xie, Vittal Premachandran, Alan Yuille
Date: 2015-11-21 00:00:00
Tasks: Clustering, Keypoint Detection, Object

We address the key question of how object part representations can be found
from the internal states of CNNs that are trained for high-level tasks, such as
object classification. This work provides a new unsupervised method to learn
semantic parts and gives new understanding of the internal representations of
CNNs. Our technique is based on the hypothesis that semantic parts are
represented by populations of neurons rather than by single filters. We propose
a clustering technique to extract part representations, which we call Visual
Concepts. We show that visual concepts are semantically coherent in that they
represent semantic parts, and visually coherent in that corresponding image
patches appear very similar. Also, visual concepts provide full spatial
coverage of the parts of an object, rather than a few sparse parts as is
typically found in keypoint annotations. Furthermore, We treat single visual
concept as part detector and evaluate it for keypoint detection using the
PASCAL3D+ dataset and for part detection using our newly annotated ImageNetPart
dataset. The experiments demonstrate that visual concepts can be used to detect
parts. We also show that some visual concepts respond to several semantic
parts, provided these parts are visually similar. Thus visual concepts have the
essential properties: semantic meaning and detection capability. Note that our
ImageNetPart dataset gives rich part annotations which cover the whole object,
making it useful for other part-related applications.