pwc-archive/papers-with-abstracts huggingface dataset:
Adjustable Bounded Rectifiers: Towards Deep Binary Representations
Authors: Zhirong Wu, Dahua Lin, Xiaoou Tang
Date: 2015-11-19 00:00:00
Tasks: Binarization

Binary representation is desirable for its memory efficiency, computation
speed and robustness. In this paper, we propose adjustable bounded rectifiers
to learn binary representations for deep neural networks. While hard
constraining representations across layers to be binary makes training
unreasonably difficult, we softly encourage activations to diverge from real
values to binary by approximating step functions. Our final representation is
completely binary. We test our approach on MNIST, CIFAR10, and ILSVRC2012
dataset, and systematically study the training dynamics of the binarization
process. Our approach can binarize the last layer representation without loss
of performance and binarize all the layers with reasonably small degradations.
The memory space that it saves may allow more sophisticated models to be
deployed, thus compensating the loss. To the best of our knowledge, this is the
first work to report results on current deep network architectures using
complete binary middle representations. Given the learned representations, we
find that the firing or inhibition of a binary neuron is usually associated
with a meaningful interpretation across different classes. This suggests that
the semantic structure of a neural network may be manifested through a guided
binarization process.
Abstract Attribute Exploration with Partial Object Descriptions
Authors: Daniel Borchmann, Bernhard Ganter
Date: 2015-11-19 00:00:00
Tasks: Attribute, Object

Attribute exploration has been investigated in several studies, with
particular emphasis on the algorithmic aspects of this knowledge acquisition
method. In its basic version the method itself is rather simple and
transparent. But when background knowledge and partially described
counter-examples are admitted, it gets more difficult. Here we discuss this
case in an abstract, somewhat "axiomatic" setting, providing a terminology that
clarifies the abstract strategy of the method rather than its algorithmic
implementation.
Large-scale Online Feature Selection for Ultra-high Dimensional Sparse Data
Authors: Yue Wu, Steven C. H. Hoi, Tao Mei, Nenghai Yu
Date: 2014-09-27 00:00:00
Tasks: feature selection, Vocal Bursts Intensity Prediction

Feature selection with large-scale high-dimensional data is important yet
very challenging in machine learning and data mining. Online feature selection
is a promising new paradigm that is more efficient and scalable than batch
feature section methods, but the existing online approaches usually fall short
in their inferior efficacy as compared with batch approaches. In this paper, we
present a novel second-order online feature selection scheme that is simple yet
effective, very fast and extremely scalable to deal with large-scale ultra-high
dimensional sparse data streams. The basic idea is to improve the existing
first-order online feature selection methods by exploiting second-order
information for choosing the subset of important features with high confidence
weights. However, unlike many second-order learning methods that often suffer
from extra high computational cost, we devise a novel smart algorithm for
second-order online feature selection using a MaxHeap-based approach, which is
not only more effective than the existing first-order approaches, but also
significantly more efficient and scalable for large-scale feature selection
with ultra-high dimensional sparse data, as validated from our extensive
experiments. Impressively, on a billion-scale synthetic dataset (1-billion
dimensions, 1-billion nonzero features, and 1-million samples), our new
algorithm took only 8 minutes on a single PC, which is orders of magnitudes
faster than traditional batch approaches. \url{http://arxiv.org/abs/1409.7794}
From Pose to Activity: Surveying Datasets and Introducing CONVERSE
Authors: Michael Edwards, Jingjing Deng, Xianghua Xie
Date: 2015-11-18 00:00:00
Tasks: Action Recognition, Temporal Action Localization

We present a review on the current state of publicly available datasets
within the human action recognition community; highlighting the revival of pose
based methods and recent progress of understanding person-person interaction
modeling. We categorize datasets regarding several key properties for usage as
a benchmark dataset; including the number of class labels, ground truths
provided, and application domain they occupy. We also consider the level of
abstraction of each dataset; grouping those that present actions, interactions
and higher level semantic activities. The survey identifies key appearance and
pose based datasets, noting a tendency for simplistic, emphasized, or scripted
action classes that are often readily definable by a stable collection of
sub-action gestures. There is a clear lack of datasets that provide closely
related actions, those that are not implicitly identified via a series of poses
and gestures, but rather a dynamic set of interactions. We therefore propose a
novel dataset that represents complex conversational interactions between two
individuals via 3D pose. 8 pairwise interactions describing 7 separate
conversation based scenarios were collected using two Kinect depth sensors. The
intention is to provide events that are constructed from numerous primitive
actions, interactions and motions, over a period of time; providing a set of
subtle action classes that are more representative of the real world, and a
challenge to currently developed recognition methodologies. We believe this is
among one of the first datasets devoted to conversational interaction
classification using 3D pose features and the attributed papers show this task
is indeed possible. The full dataset is made publicly available to the research
community at www.csvision.swansea.ac.uk/converse.
Coreset-Based Adaptive Tracking
Authors: Abhimanyu Dubey, Nikhil Naik, Dan Raviv, Rahul Sukthankar, Ramesh Raskar
Date: 2015-11-19 00:00:00
Tasks: Object, Object Tracking

We propose a method for learning from streaming visual data using a compact,
constant size representation of all the data that was seen until a given
moment. Specifically, we construct a 'coreset' representation of streaming data
using a parallelized algorithm, which is an approximation of a set with
relation to the squared distances between this set and all other points in its
ambient space. We learn an adaptive object appearance model from the coreset
tree in constant time and logarithmic space and use it for object tracking by
detection. Our method obtains excellent results for object tracking on three
standard datasets over more than 100 videos. The ability to summarize data
efficiently makes our method ideally suited for tracking in long videos in
presence of space and time constraints. We demonstrate this ability by
outperforming a variety of algorithms on the TLD dataset with 2685 frames on
average. This coreset based learning approach can be applied for both real-time
learning of small, varied data and fast learning of big data.
Detection of Slang Words in e-Data using semi-Supervised Learning
Authors: Alok Ranjan Pal, Diganta Saha
Date: 2015-11-19 00:00:00

The proposed algorithmic approach deals with finding the sense of a word in
an electronic data. Now a day,in different communication mediums like internet,
mobile services etc. people use few words, which are slang in nature. This
approach detects those abusive words using supervised learning procedure. But
in the real life scenario, the slang words are not used in complete word forms
always. Most of the times, those words are used in different abbreviated forms
like sounds alike forms, taboo morphemes etc. This proposed approach can detect
those abbreviated forms also using semi supervised learning procedure. Using
the synset and concept analysis of the text, the probability of a suspicious
word to be a slang word is also evaluated.
An Approach to Speed-up the Word Sense Disambiguation Procedure through Sense Filtering
Authors: Alok Ranjan Pal, Anupam Munshi, Diganta Saha
Date: 2015-11-19 00:00:00
Tasks: Part-Of-Speech Tagging, Word Sense Disambiguation

In this paper, we are going to focus on speed up of the Word Sense
Disambiguation procedure by filtering the relevant senses of an ambiguous word
through Part-of-Speech Tagging. First, this proposed approach performs the
Part-of-Speech Tagging operation before the disambiguation procedure using
Bigram approximation. As a result, the exact Part-of-Speech of the ambiguous
word at a particular text instance is derived. In the next stage, only those
dictionary definitions (glosses) are retrieved from an online dictionary, which
are associated with that particular Part-of-Speech to disambiguate the exact
sense of the ambiguous word. In the training phase, we have used Brown Corpus
for Part-of-Speech Tagging and WordNet as an online dictionary. The proposed
approach reduces the execution time upto half (approximately) of the normal
execution time for a text, containing around 200 sentences. Not only that, we
have found several instances, where the correct sense of an ambiguous word is
found for using the Part-of-Speech Tagging before the Disambiguation procedure.
The Kernel Two-Sample Test for Brain Networks
Authors: Emanuele Olivetti, Sandro Vega-Pons, Paolo Avesani
Date: 2015-11-19 00:00:00
Tasks: Vocal Bursts Valence Prediction

In clinical and neuroscientific studies, systematic differences between two
populations of brain networks are investigated in order to characterize mental
diseases or processes. Those networks are usually represented as graphs built
from neuroimaging data and studied by means of graph analysis methods. The
typical machine learning approach to study these brain graphs creates a
classifier and tests its ability to discriminate the two populations. In
contrast to this approach, in this work we propose to directly test whether two
populations of graphs are different or not, by using the kernel two-sample test
(KTST), without creating the intermediate classifier. We claim that, in
general, the two approaches provides similar results and that the KTST requires
much less computation. Additionally, in the regime of low sample size, we claim
that the KTST has lower frequency of Type II error than the classification
approach. Besides providing algorithmic considerations to support these claims,
we show strong evidence through experiments and one simulation.
A Hybrid Approach to Word Sense Disambiguation Combining Supervised and Unsupervised Learning
Authors: Alok Ranjan Pal, Anirban Kundu, Abhay Singh, Raj Shekhar, Kunal Sinha
Date: 2015-11-19 00:00:00
Tasks: Word Sense Disambiguation

In this paper, we are going to find meaning of words based on distinct
situations. Word Sense Disambiguation is used to find meaning of words based on
live contexts using supervised and unsupervised approaches. Unsupervised
approaches use online dictionary for learning, and supervised approaches use
manual learning sets. Hand tagged data are populated which might not be
effective and sufficient for learning procedure. This limitation of information
is main flaw of the supervised approach. Our proposed approach focuses to
overcome the limitation using learning set which is enriched in dynamic way
maintaining new data. Trivial filtering method is utilized to achieve
appropriate training data. We introduce a mixed methodology having Modified
Lesk approach and Bag-of-Words having enriched bags using learning methods. Our
approach establishes the superiority over individual Modified Lesk and
Bag-of-Words approaches based on experimentation.
Mediated Experts for Deep Convolutional Networks
Authors: Sebastian Agethen, Winston H. Hsu
Date: 2015-11-19 00:00:00
Tasks: Incremental Learning, Mixture-of-Experts

We present a new supervised architecture termed Mediated Mixture-of-Experts
(MMoE) that allows us to improve classification accuracy of Deep Convolutional
Networks (DCN). Our architecture achieves this with the help of expert
networks: A network is trained on a disjoint subset of a given dataset and then
run in parallel to other experts during deployment. A mediator is employed if
experts contradict each other. This allows our framework to naturally support
incremental learning, as adding new classes requires (re-)training of the new
expert only. We also propose two measures to control computational complexity:
An early-stopping mechanism halts experts that have low confidence in their
prediction. The system allows to trade-off accuracy and complexity without
further retraining. We also suggest to share low-level convolutional layers
between experts in an effort to avoid computation of a near-duplicate feature
set. We evaluate our system on a popular dataset and report improved accuracy
compared to a single model of same configuration.
Structured Depth Prediction in Challenging Monocular Video Sequences
Authors: Miaomiao Liu, Mathieu Salzmann, Xuming He
Date: 2015-11-19 00:00:00
Tasks: Depth Estimation, Depth Prediction, Monocular Depth Estimation, Prediction, Superpixels

In this paper, we tackle the problem of estimating the depth of a scene from
a monocular video sequence. In particular, we handle challenging scenarios,
such as non-translational camera motion and dynamic scenes, where traditional
structure from motion and motion stereo methods do not apply. To this end, we
first study the problem of depth estimation from a single image. In this
context, we exploit the availability of a pool of images for which the depth is
known, and formulate monocular depth estimation as a discrete-continuous
optimization problem, where the continuous variables encode the depth of the
superpixels in the input image, and the discrete ones represent relationships
between neighboring superpixels. The solution to this discrete-continuous
optimization problem is obtained by performing inference in a graphical model
using particle belief propagation. To handle video sequences, we then extend
our single image model to a two-frame one that naturally encodes short-range
temporal consistency and inherently handles dynamic objects. Based on the
prediction of this model, we then introduce a fully-connected pairwise CRF that
accounts for longer range spatio-temporal interactions throughout a video. We
demonstrate the effectiveness of our model in both the indoor and outdoor
scenarios.
Wishart Mechanism for Differentially Private Principal Components Analysis
Authors: Wuxuan Jiang, Cong Xie, Zhihua Zhang
Date: 2015-11-18 00:00:00

We propose a new input perturbation mechanism for publishing a covariance
matrix to achieve $(\epsilon,0)$-differential privacy. Our mechanism uses a
Wishart distribution to generate matrix noise. In particular, We apply this
mechanism to principal component analysis. Our mechanism is able to keep the
positive semi-definiteness of the published covariance matrix. Thus, our
approach gives rise to a general publishing framework for input perturbation of
a symmetric positive semidefinite matrix. Moreover, compared with the classic
Laplace mechanism, our method has better utility guarantee. To the best of our
knowledge, Wishart mechanism is the best input perturbation approach for
$(\epsilon,0)$-differentially private PCA. We also compare our work with
previous exponential mechanism algorithms in the literature and provide near
optimal bound while having more flexibility and less computational
intractability.
Transfer Learning for Speech and Language Processing
Authors: Dong Wang, Thomas Fang Zheng
Date: 2015-11-19 00:00:00
Tasks: Multi-Task Learning, speech-recognition, Speech Recognition, Transfer Learning

Transfer learning is a vital technique that generalizes models trained for
one setting or task to other settings or tasks. For example in speech
recognition, an acoustic model trained for one language can be used to
recognize speech in another language, with little or no re-training data.
Transfer learning is closely related to multi-task learning (cross-lingual vs.
multilingual), and is traditionally studied in the name of `model adaptation'.
Recent advance in deep learning shows that transfer learning becomes much
easier and more effective with high-level abstract features learned by deep
models, and the `transfer' can be conducted not only between data distributions
and data types, but also between model structures (e.g., shallow nets and deep
nets) or even model types (e.g., Bayesian models and neural models). This
review paper summarizes some recent prominent research towards this direction,
particularly for speech and language processing. We also report some results
from our group and highlight the potential of this very interesting research
field.
Dual Query: Practical Private Query Release for High Dimensional Data
Authors: Marco Gaboardi, Emilio Jesús Gallego Arias, Justin Hsu, Aaron Roth, Zhiwei Steven Wu
Date: 2014-02-06 00:00:00
Tasks: Vocal Bursts Intensity Prediction

We present a practical, differentially private algorithm for answering a
large number of queries on high dimensional datasets. Like all algorithms for
this task, ours necessarily has worst-case complexity exponential in the
dimension of the data. However, our algorithm packages the computationally hard
step into a concisely defined integer program, which can be solved
non-privately using standard solvers. We prove accuracy and privacy theorems
for our algorithm, and then demonstrate experimentally that our algorithm
performs well in practice. For example, our algorithm can efficiently and
accurately answer millions of queries on the Netflix dataset, which has over
17,000 attributes; this is an improvement on the state of the art by multiple
orders of magnitude.
Stochastic gradient method with accelerated stochastic dynamics
Authors: Masayuki Ohzeki
Date: 2015-11-19 00:00:00

In this paper, we propose a novel technique to implement stochastic gradient
methods, which are beneficial for learning from large datasets, through
accelerated stochastic dynamics. A stochastic gradient method is based on
mini-batch learning for reducing the computational cost when the amount of data
is large. The stochasticity of the gradient can be mitigated by the injection
of Gaussian noise, which yields the stochastic Langevin gradient method; this
method can be used for Bayesian posterior sampling. However, the performance of
the stochastic Langevin gradient method depends on the mixing rate of the
stochastic dynamics. In this study, we propose violating the detailed balance
condition to enhance the mixing rate. Recent studies have revealed that
violating the detailed balance condition accelerates the convergence to a
stationary state and reduces the correlation time between the samplings. We
implement this violation of the detailed balance condition in the stochastic
gradient Langevin method and test our method for a simple model to demonstrate
its performance.
Deep filter banks for texture recognition, description, and segmentation
Authors: Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Andrea Vedaldi
Date: 2015-07-09 00:00:00
Tasks: Benchmarking

Visual textures have played a key role in image understanding because they
convey important semantics of images, and because texture representations that
pool local image descriptors in an orderless manner have had a tremendous
impact in diverse applications. In this paper we make several contributions to
texture understanding. First, instead of focusing on texture instance and
material category recognition, we propose a human-interpretable vocabulary of
texture attributes to describe common texture patterns, complemented by a new
describable texture dataset for benchmarking. Second, we look at the problem of
recognizing materials and texture attributes in realistic imaging conditions,
including when textures appear in clutter, developing corresponding benchmarks
on top of the recently proposed OpenSurfaces dataset. Third, we revisit classic
texture representations, including bag-of-visual-words and the Fisher vectors,
in the context of deep learning and show that these have excellent efficiency
and generalization properties if the convolutional layers of a deep model are
used as filter banks. We obtain in this manner state-of-the-art performance in
numerous datasets well beyond textures, an efficient method to apply deep
features to image regions, as well as benefit in transferring features from one
domain to another.
Active Object Localization with Deep Reinforcement Learning
Authors: Juan C. Caicedo, Svetlana Lazebnik
Date: 2015-11-18 00:00:00
Proceeding: ICCV 2015 12
Tasks: Active Object Localization, Deep Reinforcement Learning, Object, Object Localization, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We present an active detection model for localizing objects in scenes. The
model is class-specific and allows an agent to focus attention on candidate
regions for identifying the correct location of a target object. This agent
learns to deform a bounding box using simple transformation actions, with the
goal of determining the most specific location of target objects following
top-down reasoning. The proposed localization agent is trained using deep
reinforcement learning, and evaluated on the Pascal VOC 2007 dataset. We show
that agents guided by the proposed model are able to localize a single instance
of an object after analyzing only between 11 and 25 regions in an image, and
obtain the best detection results among systems that do not use object
proposals for object localization.
Studying the control of non invasive prosthetic hands over large time spans
Authors: Mara Graziani
Date: 2015-11-18 00:00:00
Tasks: Electromyography (EMG), General Classification

The electromyography (EMG) signal is the electrical manifestation of a
neuromuscular activation that provides access to physiological processes which
cause the muscle to generate force and produce movement. Non invasive
prostheses use such signals detected by the electrodes placed on the user's
stump, as input to generate hand posture movements according to the intentions
of the prosthesis wearer. The aim of this pilot study is to explore the
repeatability issue, i.e. the ability to classify 17 different hand postures,
represented by EMG signal, across a time span of days by a control algorithm.
Data collection experiments lasted four days and signals were collected from
the forearm of a single subject. We find that Support Vector Machine (SVM)
classification results are high enough to guarantee a correct classification of
more than 10 postures in each moment of the considered time span.
Deep Unsupervised Learning using Nonequilibrium Thermodynamics
Authors: Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, Surya Ganguli
Date: 2015-03-12 00:00:00

A central problem in machine learning involves modeling complex data-sets
using highly flexible families of probability distributions in which learning,
sampling, inference, and evaluation are still analytically or computationally
tractable. Here, we develop an approach that simultaneously achieves both
flexibility and tractability. The essential idea, inspired by non-equilibrium
statistical physics, is to systematically and slowly destroy structure in a
data distribution through an iterative forward diffusion process. We then learn
a reverse diffusion process that restores structure in data, yielding a highly
flexible and tractable generative model of the data. This approach allows us to
rapidly learn, sample from, and evaluate probabilities in deep generative
models with thousands of layers or time steps, as well as to compute
conditional and posterior probabilities under the learned model. We
additionally release an open source reference implementation of the algorithm.