pwc-archive/papers-with-abstracts huggingface dataset:
Using Neural Network Formalism to Solve Multiple-Instance Problems
Authors: Tomas Pevny, Petr Somol
Date: 2016-09-23 00:00:00
Tasks: Multiple Instance Learning

Many objects in the real world are difficult to describe by a single
numerical vector of a fixed length, whereas describing them by a set of vectors
is more natural. Therefore, Multiple instance learning (MIL) techniques have
been constantly gaining on importance throughout last years. MIL formalism
represents each object (sample) by a set (bag) of feature vectors (instances)
of fixed length where knowledge about objects (e.g., class label) is available
on bag level but not necessarily on instance level. Many standard tools
including supervised classifiers have been already adapted to MIL setting since
the problem got formalized in late nineties. In this work we propose a neural
network (NN) based formalism that intuitively bridges the gap between MIL
problem definition and the vast existing knowledge-base of standard models and
classifiers. We show that the proposed NN formalism is effectively optimizable
by a modified back-propagation algorithm and can reveal unknown patterns inside
bags. Comparison to eight types of classifiers from the prior art on a set of
14 publicly available benchmark datasets confirms the advantages and accuracy
of the proposed solution.
Removal of Salt and Pepper noise from Gray-Scale and Color Images: An Adaptive Approach
Authors: Sujaya Kumar Sathua, Arabinda Dash, Aishwaryarani Behera
Date: 2017-03-07 00:00:00
Tasks: Image Enhancement

An efficient adaptive algorithm for the removal of Salt and Pepper noise from
gray scale and color image is presented in this paper. In this proposed method
first a 3X3 window is taken and the central pixel of the window is considered
as the processing pixel. If the processing pixel is found as uncorrupted, then
it is left unchanged. And if the processing pixel is found corrupted one, then
the window size is increased according to the conditions given in the proposed
algorithm. Finally the processing pixel or the central pixel is replaced by
either the mean, median or trimmed value of the elements in the current window
depending upon different conditions of the algorithm. The proposed algorithm
efficiently removes noise at all densities with better Peak Signal to Noise
Ratio (PSNR) and Image Enhancement Factor (IEF). The proposed algorithm is
compared with different existing algorithms like MF, AMF, MDBUTMF, MDBPTGMF and
AWMF.
Statistical mechanics of unsupervised feature learning in a restricted Boltzmann machine with binary synapses
Authors: Haiping Huang
Date: 2016-12-06 00:00:00

Revealing hidden features in unlabeled data is called unsupervised feature
learning, which plays an important role in pretraining a deep neural network.
Here we provide a statistical mechanics analysis of the unsupervised learning
in a restricted Boltzmann machine with binary synapses. A message passing
equation to infer the hidden feature is derived, and furthermore, variants of
this equation are analyzed. A statistical analysis by replica theory describes
the thermodynamic properties of the model. Our analysis confirms an entropy
crisis preceding the non-convergence of the message passing equation,
suggesting a discontinuous phase transition as a key characteristic of the
restricted Boltzmann machine. Continuous phase transition is also confirmed
depending on the embedded feature strength in the data. The mean-field result
under the replica symmetric assumption agrees with that obtained by running
message passing algorithms on single instances of finite sizes. Interestingly,
in an approximate Hopfield model, the entropy crisis is absent, and a
continuous phase transition is observed instead. We also develop an iterative
equation to infer the hyper-parameter (temperature) hidden in the data, which
in physics corresponds to iteratively imposing Nishimori condition. Our study
provides insights towards understanding the thermodynamic properties of the
restricted Boltzmann machine learning, and moreover important theoretical basis
to build simplified deep networks.
Can Active Memory Replace Attention?
Authors: Łukasz Kaiser, Samy Bengio
Date: 2016-10-27 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Image Captioning, image-classification, Machine Translation, speech-recognition, Translation

Several mechanisms to focus attention of a neural network on selected parts
of its input or memory have been used successfully in deep learning models in
recent years. Attention has improved image classification, image captioning,
speech recognition, generative models, and learning algorithmic tasks, but it
had probably the largest impact on neural machine translation.
  Recently, similar improvements have been obtained using alternative
mechanisms that do not focus on a single part of a memory but operate on all of
it in parallel, in a uniform way. Such mechanism, which we call active memory,
improved over attention in algorithmic tasks, image processing, and in
generative modelling.
  So far, however, active memory has not improved over attention for most
natural language processing tasks, in particular for machine translation. We
analyze this shortcoming in this paper and propose an extended model of active
memory that matches existing attention models on neural machine translation and
generalizes better to longer sentences. We investigate this model and explain
why previous active memory models did not succeed. Finally, we discuss when
active memory brings most benefits and where attention can be a better choice.
ARTiS: Appearance-based Action Recognition in Task Space for Real-Time Human-Robot Collaboration
Authors: Markus Eich, Sareh Shirazi, Gordon Wyeth
Date: 2016-10-18 00:00:00
Tasks: Action Recognition, One-Shot Learning, Temporal Action Localization, Visual Place Recognition

To have a robot actively supporting a human during a collaborative task, it
is crucial that robots are able to identify the current action in order to
predict the next one. Common approaches make use of high-level knowledge, such
as object affordances, semantics or understanding of actions in terms of pre-
and post-conditions. These approaches often require hand-coded a priori
knowledge, time- and resource-intensive or supervised learning techniques.
  We propose to reframe this problem as an appearance-based place recognition
problem. In our framework, we regard sequences of visual images of human
actions as a map in analogy to the visual place recognition problem. Observing
the task for the second time, our approach is able to recognize pre-observed
actions in a one-shot learning approach and is thereby able to recognize the
current observation in the task space. We propose two new methods for creating
and aligning action observations within a task map. We compare and verify our
approaches with real data of humans assembling several types of IKEA flat
packs.
Quantifying Program Bias
Authors: Aws Albarghouthi, Loris D'Antoni, Samuel Drews, Aditya Nori
Date: 2017-02-17 00:00:00
Tasks: Decision Making, Fairness, Sensitivity

With the range and sensitivity of algorithmic decisions expanding at a
break-neck speed, it is imperative that we aggressively investigate whether
programs are biased. We propose a novel probabilistic program analysis
technique and apply it to quantifying bias in decision-making programs.
Specifically, we (i) present a sound and complete automated verification
technique for proving quantitative properties of probabilistic programs; (ii)
show that certain notions of bias, recently proposed in the fairness
literature, can be phrased as quantitative correctness properties; and (iii)
present FairSquare, the first verification tool for quantifying program bias,
and evaluate it on a range of decision-making programs.
Cooperative Epistemic Multi-Agent Planning for Implicit Coordination
Authors: Thorsten Engesser, Thomas Bolander, Robert Mattmüller, Bernhard Nebel
Date: 2017-03-07 00:00:00
Tasks: Decision Making

Epistemic planning can be used for decision making in multi-agent situations
with distributed knowledge and capabilities. Recently, Dynamic Epistemic Logic
(DEL) has been shown to provide a very natural and expressive framework for
epistemic planning. We extend the DEL-based epistemic planning framework to
include perspective shifts, allowing us to define new notions of sequential and
conditional planning with implicit coordination. With these, it is possible to
solve planning tasks with joint goals in a decentralized manner without the
agents having to negotiate about and commit to a joint policy at plan time.
First we define the central planning notions and sketch the implementation of a
planning system built on those notions. Afterwards we provide some case studies
in order to evaluate the planner empirically and to show that the concept is
useful for multi-agent systems in practice.
A Gentle Introduction to Epistemic Planning: The DEL Approach
Authors: Thomas Bolander
Date: 2017-03-07 00:00:00
Tasks: Decision Making

Epistemic planning can be used for decision making in multi-agent situations
with distributed knowledge and capabilities. Dynamic Epistemic Logic (DEL) has
been shown to provide a very natural and expressive framework for epistemic
planning. In this paper, we aim to give an accessible introduction to DEL-based
epistemic planning. The paper starts with the most classical framework for
planning, STRIPS, and then moves towards epistemic planning in a number of
smaller steps, where each step is motivated by the need to be able to model
more complex planning scenarios.
Indoor Localization by Fusing a Group of Fingerprints Based on Random Forests
Authors: Xiansheng Guo, Nirwan Ansari, Huiyong Li
Date: 2017-03-07 00:00:00
Tasks: Indoor Localization

Indoor localization based on SIngle Of Fingerprint (SIOF) is rather
susceptible to the changing environment, multipath, and non-line-of-sight
(NLOS) propagation. Building SIOF is also a very time-consuming process.
Recently, we first proposed a GrOup Of Fingerprints (GOOF) to improve the
localization accuracy and reduce the burden of building fingerprints. However,
the main drawback is the timeliness. In this paper, we propose a novel
localization framework by Fusing A Group Of fingerprinTs (FAGOT) based on
random forests. In the offline phase, we first build a GOOF from different
transformations of the received signals of multiple antennas. Then, we design
multiple GOOF strong classifiers based on Random Forests (GOOF-RF) by training
each fingerprint in the GOOF. In the online phase, we input the corresponding
transformations of the real measurements into these strong classifiers to
obtain multiple independent decisions. Finally, we propose a Sliding Window
aIded Mode-based (SWIM) fusion algorithm to balance the localization accuracy
and time. Our proposed approaches can work better in an unknown indoor
scenario. The burden of building fingerprints can also be reduced drastically.
We demonstrate the performance of our algorithms through simulations and real
experimental data using two Universal Software Radio Peripheral (USRP)
platforms.
The Mechanism of Additive Composition
Authors: Ran Tian, Naoaki Okazaki, Kentaro Inui
Date: 2015-11-26 00:00:00

Additive composition (Foltz et al, 1998; Landauer and Dumais, 1997; Mitchell
and Lapata, 2010) is a widely used method for computing meanings of phrases,
which takes the average of vector representations of the constituent words. In
this article, we prove an upper bound for the bias of additive composition,
which is the first theoretical analysis on compositional frameworks from a
machine learning point of view. The bound is written in terms of collocation
strength; we prove that the more exclusively two successive words tend to occur
together, the more accurate one can guarantee their additive composition as an
approximation to the natural phrase vector. Our proof relies on properties of
natural language data that are empirically verified, and can be theoretically
derived from an assumption that the data is generated from a Hierarchical
Pitman-Yor Process. The theory endorses additive composition as a reasonable
operation for calculating meanings of phrases, and suggests ways to improve
additive compositionality, including: transforming entries of distributional
word vectors by a function that meets a specific condition, constructing a
novel type of vector representations to make additive composition sensitive to
word order, and utilizing singular value decomposition to train word vectors.
Deep View Morphing
Authors: Dinghuang Ji, Junghyun Kwon, Max McFarland, Silvio Savarese
Date: 2017-03-07 00:00:00
Proceeding: CVPR 2017 7
Tasks: Decoder

Recently, convolutional neural networks (CNN) have been successfully applied
to view synthesis problems. However, such CNN-based methods can suffer from
lack of texture details, shape distortions, or high computational complexity.
In this paper, we propose a novel CNN architecture for view synthesis called
"Deep View Morphing" that does not suffer from these issues. To synthesize a
middle view of two input images, a rectification network first rectifies the
two input images. An encoder-decoder network then generates dense
correspondences between the rectified images and blending masks to predict the
visibility of pixels of the rectified images in the middle view. A view
morphing network finally synthesizes the middle view using the dense
correspondences and blending masks. We experimentally show the proposed method
significantly outperforms the state-of-the-art CNN-based view synthesis method.
Building a Syllable Database to Solve the Problem of Khmer Word Segmentation
Authors: Nam Tran Van
Date: 2017-03-07 00:00:00

Word segmentation is a basic problem in natural language processing. With the
languages having the complex writing system like the Khmer language in Southern
of Vietnam, this problem really very intractable, posing the significant
challenges. Although there are some experts in Vietnam as well as international
having deeply researched this problem, there are still no reasonable results
meeting the demand, in particular, no treated thoroughly the ambiguous
phenomenon, in the process of Khmer language processing so far. This paper
present a solution based on the syllable division into component clusters using
two syllable models proposed, thereby building a Khmer syllable database, is
still not actually available. This method using a lexical database updated from
the online Khmer dictionaries and some supported dictionaries serving role of
training data and complementary linguistic characteristics. Each component
cluster is labelled and located by the first and last letter to identify
entirety a syllable. This approach is workable and the test results achieve
high accuracy, eliminate the ambiguity, contribute to solving the problem of
word segmentation and applying efficiency in Khmer language processing.
On the Limits of Learning Representations with Label-Based Supervision
Authors: Jiaming Song, Russell Stewart, Shengjia Zhao, Stefano Ermon
Date: 2017-03-07 00:00:00
Tasks: Representation Learning, Transfer Learning

Advances in neural network based classifiers have transformed automatic
feature learning from a pipe dream of stronger AI to a routine and expected
property of practical systems. Since the emergence of AlexNet every winning
submission of the ImageNet challenge has employed end-to-end representation
learning, and due to the utility of good representations for transfer learning,
representation learning has become as an important and distinct task from
supervised learning. At present, this distinction is inconsequential, as
supervised methods are state-of-the-art in learning transferable
representations. But recent work has shown that generative models can also be
powerful agents of representation learning. Will the representations learned
from these generative methods ever rival the quality of those from their
supervised competitors? In this work, we argue in the affirmative, that from an
information theoretic perspective, generative models have greater potential for
representation learning. Based on several experimentally validated assumptions,
we show that supervised learning is upper bounded in its capacity for
representation learning in ways that certain generative models, such as
Generative Adversarial Networks (GANs) are not. We hope that our analysis will
provide a rigorous motivation for further exploration of generative
representation learning.
MultiView Diffusion Maps
Authors: Ofir Lindenbaum, Arie Yeredor, Moshe Salhov, Amir Averbuch
Date: 2015-08-23 00:00:00
Tasks: Anomaly Detection, Clustering, Dimensionality Reduction

In this paper, we address the challenging task of achieving multi-view dimensionality reduction. The goal is to effectively use the availability of multiple views for extracting a coherent low-dimensional representation of the data. The proposed method exploits the intrinsic relation within each view, as well as the mutual relations between views. The multi-view dimensionality reduction is achieved by defining a cross-view model in which an implied random walk process is restrained to hop between objects in the different views. The method is robust to scaling and insensitive to small structural changes in the data. We define new diffusion distances and analyze the spectra of the proposed kernel. We show that the proposed framework is useful for various machine learning applications such as clustering, classification, and manifold learning. Finally, by fusing multi-sensor seismic data we present a method for automatic identification of seismic events.
Energy-based Generative Adversarial Network
Authors: Junbo Zhao, Michael Mathieu, Yann Lecun
Date: 2016-09-11 00:00:00
Tasks: Generative Adversarial Network

We introduce the "Energy-based Generative Adversarial Network" model (EBGAN)
which views the discriminator as an energy function that attributes low
energies to the regions near the data manifold and higher energies to other
regions. Similar to the probabilistic GANs, a generator is seen as being
trained to produce contrastive samples with minimal energies, while the
discriminator is trained to assign high energies to these generated samples.
Viewing the discriminator as an energy function allows to use a wide variety of
architectures and loss functionals in addition to the usual binary classifier
with logistic output. Among them, we show one instantiation of EBGAN framework
as using an auto-encoder architecture, with the energy being the reconstruction
error, in place of the discriminator. We show that this form of EBGAN exhibits
more stable behavior than regular GANs during training. We also show that a
single-scale architecture can be trained to generate high-resolution images.
English Conversational Telephone Speech Recognition by Humans and Machines
Authors: George Saon, Gakuto Kurata, Tom Sercu, Kartik Audhkhasi, Samuel Thomas, Dimitrios Dimitriadis, Xiaodong Cui, Bhuvana Ramabhadran, Michael Picheny, Lynn-Li Lim, Bergul Roomi, Phil Hall
Date: 2017-03-06 00:00:00
Tasks: Language Modeling, Language Modelling, Multi-Task Learning, speech-recognition, Speech Recognition

One of the most difficult speech recognition tasks is accurate recognition of
human to human communication. Advances in deep learning over the last few years
have produced major speech recognition improvements on the representative
Switchboard conversational corpus. Word error rates that just a few years ago
were 14% have dropped to 8.0%, then 6.6% and most recently 5.8%, and are now
believed to be within striking range of human performance. This then raises two
issues - what IS human performance, and how far down can we still drive speech
recognition error rates? A recent paper by Microsoft suggests that we have
already achieved human performance. In trying to verify this statement, we
performed an independent set of human performance measurements on two
conversational tasks and found that human performance may be considerably
better than what was earlier reported, giving the community a significantly
harder goal to achieve. We also report on our own efforts in this area,
presenting a set of acoustic and language modeling techniques that lowered the
word error rate of our own English conversational telephone LVCSR system to the
level of 5.5%/10.3% on the Switchboard/CallHome subsets of the Hub5 2000
evaluation, which - at least at the writing of this paper - is a new
performance milestone (albeit not at what we measure to be human performance!).
On the acoustic side, we use a score fusion of three models: one LSTM with
multiple feature inputs, a second LSTM trained with speaker-adversarial
multi-task learning and a third residual net (ResNet) with 25 convolutional
layers and time-dilated convolutions. On the language modeling side, we use
word and character LSTMs and convolutional WaveNet-style language models.
Normalizing the Normalizers: Comparing and Extending Network Normalization Schemes
Authors: Mengye Ren, Renjie Liao, Raquel Urtasun, Fabian H. Sinz, Richard S. Zemel
Date: 2016-11-14 00:00:00
Tasks: image-classification, Image Classification, Language Modeling, Language Modelling, Super-Resolution

Normalization techniques have only recently begun to be exploited in
supervised learning tasks. Batch normalization exploits mini-batch statistics
to normalize the activations. This was shown to speed up training and result in
better models. However its success has been very limited when dealing with
recurrent neural networks. On the other hand, layer normalization normalizes
the activations across all activities within a layer. This was shown to work
well in the recurrent setting. In this paper we propose a unified view of
normalization techniques, as forms of divisive normalization, which includes
layer and batch normalization as special cases. Our second contribution is the
finding that a small modification to these normalization schemes, in
conjunction with a sparse regularizer on the activations, leads to significant
benefits over standard normalization techniques. We demonstrate the
effectiveness of our unified divisive normalization framework in the context of
convolutional neural nets and recurrent neural networks, showing improvements
over baselines in image classification, language modeling as well as
super-resolution.
Randomized Iterative Reconstruction for Sparse View X-ray Computed Tomography
Authors: D. Trinca, Y. Zhong
Date: 2017-03-06 00:00:00

With the availability of more powerful computers, iterative reconstruction
algorithms are the subject of an ongoing work in the design of more efficient
reconstruction algorithms for X-ray computed tomography. In this work, we show
how two analytical reconstruction algorithms can be improved by correcting the
corresponding reconstructions using a randomized iterative reconstruction
algorithm. The combined analytical reconstruction followed by randomized
iterative reconstruction can also be viewed as a reconstruction algorithm
which, in the experiments we have conducted, uses up to $35\%$ less projection
angles as compared to the analytical reconstruction algorithms and produces the
same results in terms of quality of reconstruction, without increasing the
execution time significantly.