pwc-archive/papers-with-abstracts huggingface dataset:
Natural Language Understanding with Distributed Representation
Authors: Kyunghyun Cho
Date: 2015-11-24 00:00:00
Tasks: Language Modelling, Machine Translation, Natural Language Understanding, Translation

This is a lecture note for the course DS-GA 3001 <Natural Language
Understanding with Distributed Representation> at the Center for Data Science ,
New York University in Fall, 2015. As the name of the course suggests, this
lecture note introduces readers to a neural network based approach to natural
language understanding/processing. In order to make it as self-contained as
possible, I spend much time on describing basics of machine learning and neural
networks, only after which how they are used for natural languages is
introduced. On the language front, I almost solely focus on language modelling
and machine translation, two of which I personally find most fascinating and
most fundamental to natural language understanding.
Private Posterior distributions from Variational approximations
Authors: Vishesh Karwa, Dan Kifer, Aleksandra B. Slavković
Date: 2015-11-24 00:00:00
Tasks: Privacy Preserving, valid

Privacy preserving mechanisms such as differential privacy inject additional
randomness in the form of noise in the data, beyond the sampling mechanism.
Ignoring this additional noise can lead to inaccurate and invalid inferences.
In this paper, we incorporate the privacy mechanism explicitly into the
likelihood function by treating the original data as missing, with an end goal
of estimating posterior distributions over model parameters. This leads to a
principled way of performing valid statistical inference using private data,
however, the corresponding likelihoods are intractable. In this paper, we
derive fast and accurate variational approximations to tackle such intractable
likelihoods that arise due to privacy. We focus on estimating posterior
distributions of parameters of the naive Bayes log-linear model, where the
sufficient statistics of this model are shared using a differentially private
interface. Using a simulation study, we show that the posterior approximations
outperform the naive method of ignoring the noise addition mechanism.
Compact CNN for Indexing Egocentric Videos
Authors: Yair Poleg, Ariel Ephrat, Shmuel Peleg, Chetan Arora
Date: 2015-04-28 00:00:00
Tasks: Activity Recognition, Optical Flow Estimation

While egocentric video is becoming increasingly popular, browsing it is very
difficult. In this paper we present a compact 3D Convolutional Neural Network
(CNN) architecture for long-term activity recognition in egocentric videos.
Recognizing long-term activities enables us to temporally segment (index) long
and unstructured egocentric videos. Existing methods for this task are based on
hand tuned features derived from visible objects, location of hands, as well as
optical flow.
  Given a sparse optical flow volume as input, our CNN classifies the camera
wearer's activity. We obtain classification accuracy of 89%, which outperforms
the current state-of-the-art by 19%. Additional evaluation is performed on an
extended egocentric video dataset, classifying twice the amount of categories
than current state-of-the-art. Furthermore, our CNN is able to recognize
whether a video is egocentric or not with 99.2% accuracy, up by 24% from
current state-of-the-art. To better understand what the network actually
learns, we propose a novel visualization of CNN kernels as flow fields.
Super-Linear Gate and Super-Quadratic Wire Lower Bounds for Depth-Two and Depth-Three Threshold Circuits
Authors: Daniel M. Kane, Ryan Williams
Date: 2015-11-24 00:00:00
Tasks: LEMMA

In order to formally understand the power of neural computing, we first need
to crack the frontier of threshold circuits with two and three layers, a regime
that has been surprisingly intractable to analyze. We prove the first
super-linear gate lower bounds and the first super-quadratic wire lower bounds
for depth-two linear threshold circuits with arbitrary weights, and depth-three
majority circuits computing an explicit function.
  $\bullet$ We prove that for all $\epsilon\gg \sqrt{\log(n)/n}$, the
linear-time computable Andreev's function cannot be computed on a
$(1/2+\epsilon)$-fraction of $n$-bit inputs by depth-two linear threshold
circuits of $o(\epsilon^3 n^{3/2}/\log^3 n)$ gates, nor can it be computed with
$o(\epsilon^{3} n^{5/2}/\log^{7/2} n)$ wires. This establishes an average-case
``size hierarchy'' for threshold circuits, as Andreev's function is computable
by uniform depth-two circuits of $o(n^3)$ linear threshold gates, and by
uniform depth-three circuits of $O(n)$ majority gates.
  $\bullet$ We present a new function in $P$ based on small-biased sets, which
we prove cannot be computed by a majority vote of depth-two linear threshold
circuits with $o(n^{3/2}/\log^3 n)$ gates, nor with $o(n^{5/2}/\log^{7/2}n)$
wires.
  $\bullet$ We give tight average-case (gate and wire) complexity results for
computing PARITY with depth-two threshold circuits; the answer turns out to be
the same as for depth-two majority circuits.
  The key is a new random restriction lemma for linear threshold functions. Our
main analytical tool is the Littlewood-Offord Lemma from additive
combinatorics.
Evolution of swarming behavior is shaped by how predators attack
Authors: Randal S. Olson, David B. Knoester, Christoph Adami
Date: 2013-10-22 00:00:00

Animal grouping behaviors have been widely studied due to their implications
for understanding social intelligence, collective cognition, and potential
applications in engineering, artificial intelligence, and robotics. An
important biological aspect of these studies is discerning which selection
pressures favor the evolution of grouping behavior. In the past decade,
researchers have begun using evolutionary computation to study the evolutionary
effects of these selection pressures in predator-prey models. The selfish herd
hypothesis states that concentrated groups arise because prey selfishly attempt
to place their conspecifics between themselves and the predator, thus causing
an endless cycle of movement toward the center of the group. Using an
evolutionary model of a predator-prey system, we show that how predators attack
is critical to the evolution of the selfish herd. Following this discovery, we
show that density-dependent predation provides an abstraction of Hamilton's
original formulation of ``domains of danger.'' Finally, we verify that
density-dependent predation provides a sufficient selective advantage for prey
to evolve the selfish herd in response to predation by coevolving predators.
Thus, our work corroborates Hamilton's selfish herd hypothesis in a digital
evolutionary model, refines the assumptions of the selfish herd hypothesis, and
generalizes the domain of danger concept to density-dependent predation.
End-To-End Memory Networks
Authors: Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, Rob Fergus
Date: 2015-03-31 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Language Modeling, Language Modelling, Question Answering

We introduce a neural network with a recurrent attention model over a
possibly large external memory. The architecture is a form of Memory Network
(Weston et al., 2015) but unlike the model in that work, it is trained
end-to-end, and hence requires significantly less supervision during training,
making it more generally applicable in realistic settings. It can also be seen
as an extension of RNNsearch to the case where multiple computational steps
(hops) are performed per output symbol. The flexibility of the model allows us
to apply it to tasks as diverse as (synthetic) question answering and to
language modeling. For the former our approach is competitive with Memory
Networks, but with less supervision. For the latter, on the Penn TreeBank and
Text8 datasets our approach demonstrates comparable performance to RNNs and
LSTMs. In both cases we show that the key concept of multiple computational
hops yields improved results.
A CMOS Spiking Neuron for Brain-Inspired Neural Networks with Resistive Synapses and In-Situ Learning
Authors: Xinyu Wu, Vishal Saxena, Kehan Zhu, Sakkarapani Balagopal
Date: 2015-05-28 00:00:00

Nanoscale resistive memories are expected to fuel dense integration of
electronic synapses for large-scale neuromorphic system. To realize such a
brain-inspired computing chip, a compact CMOS spiking neuron that performs
in-situ learning and computing while driving a large number of resistive
synapses is desired. This work presents a novel leaky integrate-and-fire neuron
design which implements the dual-mode operation of current integration and
synaptic drive, with a single opamp and enables in-situ learning with crossbar
resistive synapses. The proposed design was implemented in a 0.18 $\mu$m CMOS
technology. Measurements show neuron's ability to drive a thousand resistive
synapses, and demonstrate an in-situ associative learning. The neuron circuit
occupies a small area of 0.01 mm$^2$ and has an energy-efficiency of 9.3
pJ$/$spike$/$synapse.
Do Multi-Sense Embeddings Improve Natural Language Understanding?
Authors: Jiwei Li, Dan Jurafsky
Date: 2015-06-02 00:00:00
Proceeding: EMNLP 2015 9
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Natural Language Understanding, Part-Of-Speech Tagging, Relation, Sentiment Analysis, Word Similarity

Learning a distinct representation for each sense of an ambiguous word could
lead to more powerful and fine-grained models of vector-space representations.
Yet while `multi-sense' methods have been proposed and tested on artificial
word-similarity tasks, we don't know if they improve real natural language
understanding tasks. In this paper we introduce a multi-sense embedding model
based on Chinese Restaurant Processes that achieves state of the art
performance on matching human word similarity judgments, and propose a
pipelined architecture for incorporating multi-sense embeddings into language
understanding.
  We then test the performance of our model on part-of-speech tagging, named
entity recognition, sentiment analysis, semantic relation identification and
semantic relatedness, controlling for embedding dimensionality. We find that
multi-sense embeddings do improve performance on some tasks (part-of-speech
tagging, semantic relation identification, semantic relatedness) but not on
others (named entity recognition, various forms of sentiment analysis). We
discuss how these differences may be caused by the different role of word sense
information in each of the tasks. The results highlight the importance of
testing embedding models in real applications.
Alternative structures for character-level RNNs
Authors: Piotr Bojanowski, Armand Joulin, Tomas Mikolov
Date: 2015-11-19 00:00:00
Tasks: Language Modeling, Language Modelling

Recurrent neural networks are convenient and efficient models for language
modeling. However, when applied on the level of characters instead of words,
they suffer from several problems. In order to successfully model long-term
dependencies, the hidden representation needs to be large. This in turn implies
higher computational costs, which can become prohibitive in practice. We
propose two alternative structural modifications to the classical RNN model.
The first one consists on conditioning the character level representation on
the previous word representation. The other one uses the character history to
condition the output probability. We evaluate the performance of the two
proposed modifications on challenging, multi-lingual real world data.
Weakly Supervised Object Boundaries
Authors: Anna Khoreva, Rodrigo Benenson, Mohamed Omran, Matthias Hein, Bernt Schiele
Date: 2015-11-24 00:00:00
Proceeding: CVPR 2016 6
Tasks: Boundary Detection, Edge Detection, Object

State-of-the-art learning based boundary detection methods require extensive
training data. Since labelling object boundaries is one of the most expensive
types of annotations, there is a need to relax the requirement to carefully
annotate images to make both the training more affordable and to extend the
amount of training data. In this paper we propose a technique to generate
weakly supervised annotations and show that bounding box annotations alone
suffice to reach high-quality object boundaries without using any
object-specific boundary annotations. With the proposed weak supervision
techniques we achieve the top performance on the object boundary detection
task, outperforming by a large margin the current fully supervised
state-of-the-art methods.
Spoken Language Translation for Polish
Authors: Krzysztof Marasek, Łukasz Brocki, Danijel Korzinek, Krzysztof Wołk, Ryszard Gubrynowicz
Date: 2015-11-24 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, Machine Translation, speech-recognition, Speech Recognition, Text Normalization, Translation

Spoken language translation (SLT) is becoming more important in the
increasingly globalized world, both from a social and economic point of view.
It is one of the major challenges for automatic speech recognition (ASR) and
machine translation (MT), driving intense research activities in these areas.
While past research in SLT, due to technology limitations, dealt mostly with
speech recorded under controlled conditions, today's major challenge is the
translation of spoken language as it can be found in real life. Considered
application scenarios range from portable translators for tourists, lectures
and presentations translation, to broadcast news and shows with live
captioning. We would like to present PJIIT's experiences in the SLT gained from
the Eu-Bridge 7th framework project and the U-Star consortium activities for
the Polish/English language pair. Presented research concentrates on ASR
adaptation for Polish (state-of-the-art acoustic models: DBN-BLSTM training,
Kaldi: LDA+MLLT+SAT+MMI), language modeling for ASR & MT (text normalization,
RNN-based LMs, n-gram model domain interpolation) and statistical translation
techniques (hierarchical models, factored translation models, automatic casing
and punctuation, comparable and bilingual corpora preparation). While results
for the well-defined domains (phrases for travelers, parliament speeches,
medical documentation, movie subtitling) are very encouraging, less defined
domains (presentation, lectures) still form a challenge. Our progress in the
IWSLT TED task (MT only) will be presented, as well as current progress in the
Polish ASR.
Rendering refraction and reflection of eyeglasses for synthetic eye tracker images
Authors: Thomas C. Kübler, Tobias Rittig, Judith Ungewiss, Christina Krauss, Enkelejda Kasneci
Date: 2015-11-23 00:00:00
Tasks: Gaze Estimation

While for the evaluation of robustness of eye tracking algorithms the use of
real-world data is essential, there are many applications where simulated,
synthetic eye images are of advantage. They can generate labelled ground-truth
data for appearance based gaze estimation algorithms or enable the development
of model based gaze estimation techniques by showing the influence on gaze
estimation error of different model factors that can then be simplified or
extended. We extend the generation of synthetic eye images by a simulation of
refraction and reflection for eyeglasses. On the one hand this allows for the
testing of pupil and glint detection algorithms under different illumination
and reflection conditions, on the other hand the error of gaze estimation
routines can be estimated in conjunction with different eyeglasses. We show how
a polynomial function fitting calibration performs equally well with and
without eyeglasses, and how a geometrical eye model behaves when exposed to
glasses.
Bayesian Identification of Fixations, Saccades, and Smooth Pursuits
Authors: Thiago Santini, Wolfgang Fuhl, Thomas Kübler, Enkelejda Kasneci
Date: 2015-11-24 00:00:00
Tasks: General Classification, Specificity

Smooth pursuit eye movements provide meaningful insights and information on
subject's behavior and health and may, in particular situations, disturb the
performance of typical fixation/saccade classification algorithms. Thus, an
automatic and efficient algorithm to identify these eye movements is paramount
for eye-tracking research involving dynamic stimuli. In this paper, we propose
the Bayesian Decision Theory Identification (I-BDT) algorithm, a novel
algorithm for ternary classification of eye movements that is able to reliably
separate fixations, saccades, and smooth pursuits in an online fashion, even
for low-resolution eye trackers. The proposed algorithm is evaluated on four
datasets with distinct mixtures of eye movements, including fixations,
saccades, as well as straight and circular smooth pursuits; data was collected
with a sample rate of 30 Hz from six subjects, totaling 24 evaluation datasets.
The algorithm exhibits high and consistent performance across all datasets and
movements relative to a manual annotation by a domain expert (recall: \mu =
91.42%, \sigma = 9.52%; precision: \mu = 95.60%, \sigma = 5.29%; specificity
\mu = 95.41%, \sigma = 7.02%) and displays a significant improvement when
compared to I-VDT, an state-of-the-art algorithm (recall: \mu = 87.67%, \sigma
= 14.73%; precision: \mu = 89.57%, \sigma = 8.05%; specificity \mu = 92.10%,
\sigma = 11.21%). For algorithm implementation and annotated datasets, please
contact the first author.
Deep Learning for Semantic Part Segmentation with High-Level Guidance
Authors: S. Tsogkas, I. Kokkinos, G. Papandreou, A. Vedaldi
Date: 2015-05-10 00:00:00
Tasks: Deep Learning, Object, Segmentation, Semantic Segmentation, Vocal Bursts Intensity Prediction

In this work we address the task of segmenting an object into its parts, or
semantic part segmentation. We start by adapting a state-of-the-art semantic
segmentation system to this task, and show that a combination of a
fully-convolutional Deep CNN system coupled with Dense CRF labelling provides
excellent results for a broad range of object categories. Still, this approach
remains agnostic to high-level constraints between object parts. We introduce
such prior information by means of the Restricted Boltzmann Machine, adapted to
our task and train our model in an discriminative fashion, as a hidden CRF,
demonstrating that prior information can yield additional improvements. We also
investigate the performance of our approach ``in the wild'', without
information concerning the objects' bounding boxes, using an object detector to
guide a multi-scale segmentation scheme. We evaluate the performance of our
approach on the Penn-Fudan and LFW datasets for the tasks of pedestrian parsing
and face labelling respectively. We show superior performance with respect to
competitive methods that have been extensively engineered on these benchmarks,
as well as realistic qualitative results on part segmentation, even for
occluded or deformable objects. We also provide quantitative and extensive
qualitative results on three classes from the PASCAL Parts dataset. Finally, we
show that our multi-scale segmentation scheme can boost accuracy, recovering
segmentations for finer parts.
Searching for Objects using Structure in Indoor Scenes
Authors: Varun K. Nagaraja, Vlad I. Morariu, Larry S. Davis
Date: 2015-11-24 00:00:00
Tasks: Imitation Learning, Object

To identify the location of objects of a particular class, a passive computer
vision system generally processes all the regions in an image to finally output
few regions. However, we can use structure in the scene to search for objects
without processing the entire image. We propose a search technique that
sequentially processes image regions such that the regions that are more likely
to correspond to the query class object are explored earlier. We frame the
problem as a Markov decision process and use an imitation learning algorithm to
learn a search strategy. Since structure in the scene is essential for search,
we work with indoor scene images as they contain both unary scene context
information and object-object context in the scene. We perform experiments on
the NYU-depth v2 dataset and show that the unary scene context features alone
can achieve a significantly high average precision while processing only
20-25\% of the regions for classes like bed and sofa. By considering
object-object context along with the scene context features, the performance is
further improved for classes like counter, lamp, pillow and sofa.
Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families
Authors: Heiko Strathmann, Dino Sejdinovic, Samuel Livingstone, Zoltan Szabo, Arthur Gretton
Date: 2015-06-08 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Bayesian Inference

We propose Kernel Hamiltonian Monte Carlo (KMC), a gradient-free adaptive
MCMC algorithm based on Hamiltonian Monte Carlo (HMC). On target densities
where classical HMC is not an option due to intractable gradients, KMC
adaptively learns the target's gradient structure by fitting an exponential
family model in a Reproducing Kernel Hilbert Space. Computational costs are
reduced by two novel efficient approximations to this gradient. While being
asymptotically exact, KMC mimics HMC in terms of sampling efficiency, and
offers substantial mixing improvements over state-of-the-art gradient free
samplers. We support our claims with experimental studies on both toy and
real-world applications, including Approximate Bayesian Computation and
exact-approximate MCMC.
A new hybrid stemming algorithm for Persian
Authors: Adel Rahimi
Date: 2015-07-11 00:00:00
Tasks: Information Retrieval, Retrieval

Stemming has been an influential part in Information retrieval and search
engines. There have been tremendous endeavours in making stemmer that are both
efficient and accurate. Stemmers can have three method in stemming, Dictionary
based stemmer, statistical-based stemmers, and rule-based stemmers. This paper
aims at building a hybrid stemmer that uses both Dictionary based method and
rule-based method for stemming. This ultimately helps the efficacy and
accurateness of the stemmer.
A pilot study on the daily control capability of s-EMG prosthetic hands by amputees
Authors: Francesca Giordaniello
Date: 2015-11-18 00:00:00
Tasks: valid

Surface electromyography is a valid tool to gather muscular contraction
signals from intact and amputated subjects. Electromyographic signals can be
used to control prosthetic devices in a noninvasive way distinguishing the
movements performed by the particular EMG electrodes activity. According to the
literature, several algorithms have been used to control prosthetic hands
through s-EMG signals. The main issue is to correctly classify the signals
acquired as the movement actually performed. This work presents a study on the
Support Vector Machine's performance in a short-time period, gained using two
different feature representation (Mean Absolute Value and Waveform Length) of
the sEMG signals. In particular, we paid close attention to the repeatability
problem, that is the capability to achieve a stable and satisfactory level of
accuracy in repeated experiments. Results on a limited setting are encouraging,
as they show an average accuracy above 73% even in the worst case scenario.
Semi-Supervised Learning with Ladder Networks
Authors: Antti Rasmus, Harri Valpola, Mikko Honkala, Mathias Berglund, Tapani Raiko
Date: 2015-07-09 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: General Classification, Semi-Supervised Image Classification

We combine supervised learning with unsupervised learning in deep neural
networks. The proposed model is trained to simultaneously minimize the sum of
supervised and unsupervised cost functions by backpropagation, avoiding the
need for layer-wise pre-training. Our work builds on the Ladder network
proposed by Valpola (2015), which we extend by combining the model with
supervision. We show that the resulting model reaches state-of-the-art
performance in semi-supervised MNIST and CIFAR-10 classification, in addition
to permutation-invariant MNIST classification with all labels.