pwc-archive/papers-with-abstracts huggingface dataset:
Fuzzy Least Squares Twin Support Vector Machines
Authors: Javad Salimi Sartakhti, Homayun Afrabandpey, Nasser Ghadiri
Date: 2015-05-20 00:00:00
Tasks: Binary Classification, General Classification

Least Squares Twin Support Vector Machine (LST-SVM) has been shown to be an
efficient and fast algorithm for binary classification. It combines the
operating principles of Least Squares SVM (LS-SVM) and Twin SVM (T-SVM); it
constructs two non-parallel hyperplanes (as in T-SVM) by solving two systems of
linear equations (as in LS-SVM). Despite its efficiency, LST-SVM is still
unable to cope with two features of real-world problems. First, in many
real-world applications, labels of samples are not deterministic; they come
naturally with their associated membership degrees. Second, samples in
real-world applications may not be equally important and their importance
degrees affect the classification. In this paper, we propose Fuzzy LST-SVM
(FLST-SVM) to deal with these two characteristics of real-world data. Two
models are introduced for FLST-SVM: the first model builds up crisp hyperplanes
using training samples and their corresponding membership degrees. The second
model, on the other hand, constructs fuzzy hyperplanes using training samples
and their membership degrees. Numerical evaluation of the proposed method with
synthetic and real datasets demonstrate significant improvement in the
classification accuracy of FLST-SVM when compared to well-known existing
versions of SVM.
Automated Prediction of Temporal Relations
Authors: Amol S Patwardhan, Jacob Badeaux, Siavash, Gerald M Knapp
Date: 2016-07-22 00:00:00
Tasks: POS, Prediction, TAG, Vocal Bursts Type Prediction

Background: There has been growing research interest in automated answering
of questions or generation of summary of free form text such as news article.
In order to implement this task, the computer should be able to identify the
sequence of events, duration of events, time at which event occurred and the
relationship type between event pairs, time pairs or event-time pairs. Specific
Problem: It is important to accurately identify the relationship type between
combinations of event and time before the temporal ordering of events can be
defined. The machine learning approach taken in Mani et. al (2006) provides an
accuracy of only 62.5 on the baseline data from TimeBank. The researchers used
maximum entropy classifier in their methodology. TimeML uses the TLINK
annotation to tag a relationship type between events and time. The time
complexity is quadratic when it comes to tagging documents with TLINK using
human annotation. This research proposes using decision tree and parsing to
improve the relationship type tagging. This research attempts to solve the gaps
in human annotation by automating the task of relationship type tagging in an
attempt to improve the accuracy of event and time relationship in annotated
documents. Scope information: The documents from the domain of news will be
used. The tagging will be performed within the same document and not across
documents. The relationship types will be identified only for a pair of event
and time and not a chain of events. The research focuses on documents tagged
using the TimeML specification which contains tags such as EVENT, TLINK, and
TIMEX. Each tag has attributes such as identifier, relation, POS, time etc.
Syntax-based Attention Model for Natural Language Inference
Authors: PengFei Liu, Xipeng Qiu, Xuanjing Huang
Date: 2016-07-22 00:00:00
Tasks: model, Natural Language Inference, Sentence

Introducing attentional mechanism in neural network is a powerful concept,
and has achieved impressive results in many natural language processing tasks.
However, most of the existing models impose attentional distribution on a flat
topology, namely the entire input representation sequence. Clearly, any
well-formed sentence has its accompanying syntactic tree structure, which is a
much rich topology. Applying attention to such topology not only exploits the
underlying syntax, but also makes attention more interpretable. In this paper,
we explore this direction in the context of natural language inference. The
results demonstrate its efficacy. We also perform extensive qualitative
analysis, deriving insights and intuitions of why and how our model works.
Efficient Nonparametric Smoothness Estimation
Authors: Shashank Singh, Simon S. Du, Barnabás Póczos
Date: 2016-05-19 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Two-sample testing

Sobolev quantities (norms, inner products, and distances) of probability
density functions are important in the theory of nonparametric statistics, but
have rarely been used in practice, partly due to a lack of practical
estimators. They also include, as special cases, $L^2$ quantities which are
used in many applications. We propose and analyze a family of estimators for
Sobolev quantities of unknown probability density functions. We bound the bias
and variance of our estimators over finite samples, finding that they are
generally minimax rate-optimal. Our estimators are significantly more
computationally tractable than previous estimators, and exhibit a
statistical/computational trade-off allowing them to adapt to computational
constraints. We also draw theoretical connections to recent work on fast
two-sample testing. Finally, we empirically validate our estimators on
synthetic data.
Novel Word Embedding and Translation-based Language Modeling for Extractive Speech Summarization
Authors: Kuan-Yu Chen, Shih-Hung Liu, Berlin Chen, Hsin-Min Wang, Hsin-Hsi Chen
Date: 2016-07-22 00:00:00
Tasks: Language Modeling, Language Modelling, Representation Learning, Translation

Word embedding methods revolve around learning continuous distributed vector
representations of words with neural networks, which can capture semantic
and/or syntactic cues, and in turn be used to induce similarity measures among
words, sentences and documents in context. Celebrated methods can be
categorized as prediction-based and count-based methods according to the
training objectives and model architectures. Their pros and cons have been
extensively analyzed and evaluated in recent studies, but there is relatively
less work continuing the line of research to develop an enhanced learning
method that brings together the advantages of the two model families. In
addition, the interpretation of the learned word representations still remains
somewhat opaque. Motivated by the observations and considering the pressing
need, this paper presents a novel method for learning the word representations,
which not only inherits the advantages of classic word embedding methods but
also offers a clearer and more rigorous interpretation of the learned word
representations. Built upon the proposed word embedding method, we further
formulate a translation-based language modeling framework for the extractive
speech summarization task. A series of empirical evaluations demonstrate the
effectiveness of the proposed word representation learning and language
modeling techniques in extractive speech summarization.
CGMOS: Certainty Guided Minority OverSampling
Authors: Xi Zhang, Di Ma, Lin Gan, Shanshan Jiang, Gady Agam
Date: 2016-07-21 00:00:00
Tasks: Classification, General Classification

Handling imbalanced datasets is a challenging problem that if not treated
correctly results in reduced classification performance. Imbalanced datasets
are commonly handled using minority oversampling, whereas the SMOTE algorithm
is a successful oversampling algorithm with numerous extensions. SMOTE
extensions do not have a theoretical guarantee during training to work better
than SMOTE and in many instances their performance is data dependent. In this
paper we propose a novel extension to the SMOTE algorithm with a theoretical
guarantee for improved classification performance. The proposed approach
considers the classification performance of both the majority and minority
classes. In the proposed approach CGMOS (Certainty Guided Minority
OverSampling) new data points are added by considering certainty changes in the
dataset. The paper provides a proof that the proposed algorithm is guaranteed
to work better than SMOTE for training data. Further experimental results on 30
real-world datasets show that CGMOS works better than existing algorithms when
using 6 different classifiers.
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Authors: Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai
Date: 2016-07-21 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Articles, BIG-bench Machine Learning, Word Embeddings

The blind application of machine learning runs the risk of amplifying biases
present in data. Such a danger is facing us with word embedding, a popular
framework to represent text data as vectors which has been used in many machine
learning and natural language processing tasks. We show that even word
embeddings trained on Google News articles exhibit female/male gender
stereotypes to a disturbing extent. This raises concerns because their
widespread use, as we describe, often tends to amplify these biases.
Geometrically, gender bias is first shown to be captured by a direction in the
word embedding. Second, gender neutral words are shown to be linearly separable
from gender definition words in the word embedding. Using these properties, we
provide a methodology for modifying an embedding to remove gender stereotypes,
such as the association between between the words receptionist and female,
while maintaining desired associations such as between the words queen and
female. We define metrics to quantify both direct and indirect gender biases in
embeddings, and develop algorithms to "debias" the embedding. Using
crowd-worker evaluation as well as standard benchmarks, we empirically
demonstrate that our algorithms significantly reduce gender bias in embeddings
while preserving the its useful properties such as the ability to cluster
related concepts and to solve analogy tasks. The resulting embeddings can be
used in applications without amplifying gender bias.
Geometric Neural Phrase Pooling: Modeling the Spatial Co-occurrence of Neurons
Authors: Lingxi Xie, Qi Tian, John Flynn, Jingdong Wang, Alan Yuille
Date: 2016-07-21 00:00:00
Tasks: image-classification, Image Classification

Deep Convolutional Neural Networks (CNNs) are playing important roles in
state-of-the-art visual recognition. This paper focuses on modeling the spatial
co-occurrence of neuron responses, which is less studied in the previous work.
For this, we consider the neurons in the hidden layer as neural words, and
construct a set of geometric neural phrases on top of them. The idea that
grouping neural words into neural phrases is borrowed from the
Bag-of-Visual-Words (BoVW) model. Next, the Geometric Neural Phrase Pooling
(GNPP) algorithm is proposed to efficiently encode these neural phrases. GNPP
acts as a new type of hidden layer, which punishes the isolated neuron
responses after convolution, and can be inserted into a CNN model with little
extra computational overhead. Experimental results show that GNPP produces
significant and consistent accuracy gain in image classification.
Multimodal, high-dimensional, model-based, Bayesian inverse problems with applications in biomechanics
Authors: Isabell M. Franck, P. S. Koutsourelakis
Date: 2015-12-14 00:00:00
Tasks: Diagnostic, Dimensionality Reduction

This paper is concerned with the numerical solution of model-based, Bayesian
inverse problems. We are particularly interested in cases where the cost of
each likelihood evaluation (forward-model call) is expensive and the number of
un- known (latent) variables is high. This is the setting in many problems in
com- putational physics where forward models with nonlinear PDEs are used and
the parameters to be calibrated involve spatio-temporarily varying
coefficients, which upon discretization give rise to a high-dimensional vector
of unknowns. One of the consequences of the well-documented ill-posedness of
inverse prob- lems is the possibility of multiple solutions. While such
information is contained in the posterior density in Bayesian formulations, the
discovery of a single mode, let alone multiple, is a formidable task. The goal
of the present paper is two- fold. On one hand, we propose approximate,
adaptive inference strategies using mixture densities to capture multi-modal
posteriors, and on the other, to ex- tend our work in [1] with regards to
effective dimensionality reduction techniques that reveal low-dimensional
subspaces where the posterior variance is mostly concentrated. We validate the
model proposed by employing Importance Sam- pling which confirms that the bias
introduced is small and can be efficiently corrected if the analyst wishes to
do so. We demonstrate the performance of the proposed strategy in nonlinear
elastography where the identification of the mechanical properties of
biological materials can inform non-invasive, medical di- agnosis. The
discovery of multiple modes (solutions) in such problems is critical in
achieving the diagnostic objectives.
Layer Normalization
Authors: Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton
Date: 2016-07-21 00:00:00

Training state-of-the-art, deep neural networks is computationally expensive.
One way to reduce the training time is to normalize the activities of the
neurons. A recently introduced technique called batch normalization uses the
distribution of the summed input to a neuron over a mini-batch of training
cases to compute a mean and variance which are then used to normalize the
summed input to that neuron on each training case. This significantly reduces
the training time in feed-forward neural networks. However, the effect of batch
normalization is dependent on the mini-batch size and it is not obvious how to
apply it to recurrent neural networks. In this paper, we transpose batch
normalization into layer normalization by computing the mean and variance used
for normalization from all of the summed inputs to the neurons in a layer on a
single training case. Like batch normalization, we also give each neuron its
own adaptive bias and gain which are applied after the normalization but before
the non-linearity. Unlike batch normalization, layer normalization performs
exactly the same computation at training and test times. It is also
straightforward to apply to recurrent neural networks by computing the
normalization statistics separately at each time step. Layer normalization is
very effective at stabilizing the hidden state dynamics in recurrent networks.
Empirically, we show that layer normalization can substantially reduce the
training time compared with previously published techniques.
Hierarchical Attention Network for Action Recognition in Videos
Authors: Yilin Wang, Suhang Wang, Jiliang Tang, Neil O'Hare, Yi Chang, Baoxin Li
Date: 2016-07-21 00:00:00
Tasks: Action Recognition, Action Recognition In Videos, Action Understanding, Temporal Action Localization

Understanding human actions in wild videos is an important task with a broad
range of applications. In this paper we propose a novel approach named
Hierarchical Attention Network (HAN), which enables to incorporate static
spatial information, short-term motion information and long-term video temporal
structures for complex human action understanding. Compared to recent
convolutional neural network based approaches, HAN has following advantages (1)
HAN can efficiently capture video temporal structures in a longer range; (2)
HAN is able to reveal temporal transitions between frame chunks with different
time steps, i.e. it explicitly models the temporal transitions between frames
as well as video segments and (3) with a multiple step spatial temporal
attention mechanism, HAN automatically learns important regions in video frames
and temporal segments in the video. The proposed model is trained and evaluated
on the standard video action benchmarks, i.e., UCF-101 and HMDB-51, and it
significantly outperforms the state-of-the arts
Small-Variance Nonparametric Clustering on the Hypersphere
Authors: Julian Straub, Trevor Campbell, Jonathan P. How, John W. Fisher III
Date: 2016-07-21 00:00:00
Proceeding: CVPR 2015 6
Tasks: Clustering, Nonparametric Clustering, Scene Understanding

Structural regularities in man-made environments reflect in the distribution
of their surface normals. Describing these surface normal distributions is
important in many computer vision applications, such as scene understanding,
plane segmentation, and regularization of 3D reconstructions. Based on the
small-variance limit of Bayesian nonparametric von-Mises-Fisher (vMF) mixture
distributions, we propose two new flexible and efficient k-means-like
clustering algorithms for directional data such as surface normals. The first,
DP-vMF-means, is a batch clustering algorithm derived from the Dirichlet
process (DP) vMF mixture. Recognizing the sequential nature of data collection
in many applications, we extend this algorithm to DDP-vMF-means, which infers
temporally evolving cluster structure from streaming data. Both algorithms
naturally respect the geometry of directional data, which lies on the unit
sphere. We demonstrate their performance on synthetic directional data and real
3D surface normals from RGB-D sensors. While our experiments focus on 3D data,
both algorithms generalize to high dimensional directional data such as protein
backbone configurations and semantic word vectors.
Analysis of k-Nearest Neighbor Distances with Application to Entropy Estimation
Authors: Shashank Singh, Barnabás Póczos
Date: 2016-03-28 00:00:00

Estimating entropy and mutual information consistently is important for many
machine learning applications. The Kozachenko-Leonenko (KL) estimator
(Kozachenko & Leonenko, 1987) is a widely used nonparametric estimator for the
entropy of multivariate continuous random variables, as well as the basis of
the mutual information estimator of Kraskov et al. (2004), perhaps the most
widely used estimator of mutual information in this setting. Despite the
practical importance of these estimators, major theoretical questions regarding
their finite-sample behavior remain open. This paper proves finite-sample
bounds on the bias and variance of the KL estimator, showing that it achieves
the minimax convergence rate for certain classes of smooth functions. In
proving these bounds, we analyze finite-sample behavior of k-nearest neighbors
(k-NN) distance statistics (on which the KL estimator is based). We derive
concentration inequalities for k-NN distances and a general expectation bound
for statistics of k-NN distances, which may be useful for other analyses of
k-NN methods.
Distributed Supervised Learning using Neural Networks
Authors: Simone Scardapane
Date: 2016-07-21 00:00:00
Tasks: Time Series Analysis

Distributed learning is the problem of inferring a function in the case where
training data is distributed among multiple geographically separated sources.
Particularly, the focus is on designing learning strategies with low
computational requirements, in which communication is restricted only to
neighboring agents, with no reliance on a centralized authority. In this
thesis, we analyze multiple distributed protocols for a large number of neural
network architectures. The first part of the thesis is devoted to a definition
of the problem, followed by an extensive overview of the state-of-the-art.
Next, we introduce different strategies for a relatively simple class of single
layer neural networks, where a linear output layer is preceded by a nonlinear
layer, whose weights are stochastically assigned in the beginning of the
learning process. We consider both batch and sequential learning, with
horizontally and vertically partitioned data. In the third part, we consider
instead the more complex problem of semi-supervised distributed learning, where
each agent is provided with an additional set of unlabeled training samples. We
propose two different algorithms based on diffusion processes for linear
support vector machines and kernel ridge regression. Subsequently, the fourth
part extends the discussion to learning with time-varying data (e.g.
time-series) using recurrent neural networks. We consider two different
families of networks, namely echo state networks (extending the algorithms
introduced in the second part), and spline adaptive filters. Overall, the
algorithms presented throughout the thesis cover a wide range of possible
practical applications, and lead the way to numerous future extensions, which
are briefly summarized in the conclusive chapter.
Linear Algebraic Structure of Word Senses, with Applications to Polysemy
Authors: Sanjeev Arora, Yuanzhi Li, YIngyu Liang, Tengyu Ma, Andrej Risteski
Date: 2016-01-14 00:00:00
Proceeding: TACL 2018 1
Tasks: Information Retrieval, Retrieval, Word Embeddings

Word embeddings are ubiquitous in NLP and information retrieval, but it is
unclear what they represent when the word is polysemous. Here it is shown that
multiple word senses reside in linear superposition within the word embedding
and simple sparse coding can recover vectors that approximately capture the
senses. The success of our approach, which applies to several embedding
methods, is mathematically explained using a variant of the random walk on
discourses model (Arora et al., 2016). A novel aspect of our technique is that
each extracted word sense is accompanied by one of about 2000 "discourse atoms"
that gives a succinct description of which other words co-occur with that word
sense. Discourse atoms can be of independent interest, and make the method
potentially more useful. Empirical tests are used to verify and support the
theory.
Reasoning about Body-Parts Relations for Sign Language Recognition
Authors: Marc Martínez-Camarena, Jose Oramas, Mario Montagud-Climent, Tinne Tuytelaars
Date: 2016-07-21 00:00:00
Tasks: Gesture Recognition, Hand Gesture Recognition, Hand-Gesture Recognition, Sign Language Recognition

Over the years, hand gesture recognition has been mostly addressed
considering hand trajectories in isolation. However, in most sign languages,
hand gestures are defined on a particular context (body region). We propose a
pipeline to perform sign language recognition which models hand movements in
the context of other parts of the body captured in the 3D space using the MS
Kinect sensor. In addition, we perform sign recognition based on the different
hand postures that occur during a sign. Our experiments show that considering
different body parts brings improved performance when compared to other methods
which only consider global hand trajectories. Finally, we demonstrate that the
combination of hand postures features with hand gestures features helps to
improve the prediction of a given sign.
Fast Robust Monocular Depth Estimation for Obstacle Detection with Fully Convolutional Networks
Authors: Michele Mancini, Gabriele Costante, Paolo Valigi, Thomas A. Ciarfuglia
Date: 2016-07-21 00:00:00
Tasks: Depth Estimation, Monocular Depth Estimation, object-detection, Object Detection

Obstacle Detection is a central problem for any robotic system, and critical
for autonomous systems that travel at high speeds in unpredictable environment.
This is often achieved through scene depth estimation, by various means. When
fast motion is considered, the detection range must be longer enough to allow
for safe avoidance and path planning. Current solutions often make assumption
on the motion of the vehicle that limit their applicability, or work at very
limited ranges due to intrinsic constraints. We propose a novel
appearance-based Object Detection system that is able to detect obstacles at
very long range and at a very high speed (~300Hz), without making assumptions
on the type of motion. We achieve these results using a Deep Neural Network
approach trained on real and synthetic images and trading some depth accuracy
for fast, robust and consistent operation. We show how photo-realistic
synthetic images are able to solve the problem of training set dimension and
variety typical of machine learning approaches, and how our system is robust to
massive blurring of test images.