pwc-archive/papers-with-abstracts huggingface dataset:
Active Robust Learning
Authors: Hossein Ghafarian, Hadi Sadoghi Yazdi
Date: 2016-08-25 00:00:00
Tasks: Active Learning, Informativeness

In many practical applications of learning algorithms, unlabeled data is
cheap and abundant whereas labeled data is expensive. Active learning
algorithms developed to achieve better performance with lower cost. Usually
Representativeness and Informativeness are used in active learning algoirthms.
Advanced recent active learning methods consider both of these criteria.
Despite its vast literature, very few active learning methods consider noisy
instances, i.e. label noisy and outlier instances. Also, these methods didn't
consider accuracy in computing representativeness and informativeness. Based on
the idea that inaccuracy in these measures and not taking noisy instances into
consideration are two sides of a coin and are inherently related, a new loss
function is proposed. This new loss function helps to decrease the effect of
noisy instances while at the same time, reduces bias. We defined "instance
complexity" as a new notion of complexity for instances of a learning problem.
It is proved that noisy instances in the data if any, are the ones with maximum
instance complexity. Based on this loss function which has two functions for
classifying ordinary and noisy instances, a new classifier, named
"Simple-Complex Classifier" is proposed. In this classifier there are a simple
and a complex function, with the complex function responsible for selecting
noisy instances. The resulting optimization problem for both learning and
active learning is highly non-convex and very challenging. In order to solve
it, a convex relaxation is proposed.
Sympathy for the Details: Dense Trajectories and Hybrid Classification Architectures for Action Recognition
Authors: César Roberto de Souza, Adrien Gaidon, Eleonora Vig, Antonio Manuel López
Date: 2016-08-25 00:00:00
Tasks: Action Recognition, Action Recognition In Videos, General Classification, image-classification, Image Classification, Temporal Action Localization, Video Classification

Action recognition in videos is a challenging task due to the complexity of
the spatio-temporal patterns to model and the difficulty to acquire and learn
on large quantities of video data. Deep learning, although a breakthrough for
image classification and showing promise for videos, has still not clearly
superseded action recognition methods using hand-crafted features, even when
training on massive datasets. In this paper, we introduce hybrid video
classification architectures based on carefully designed unsupervised
representations of hand-crafted spatio-temporal features classified by
supervised deep networks. As we show in our experiments on five popular
benchmarks for action recognition, our hybrid model combines the best of both
worlds: it is data efficient (trained on 150 to 10000 short clips) and yet
improves significantly on the state of the art, including recent deep models
trained on millions of manually labelled images and videos.
Modelling Chemical Reasoning to Predict Reactions
Authors: Marwin H. S. Segler, Mark P. Waller
Date: 2016-08-25 00:00:00
Tasks: Prediction

The ability to reason beyond established knowledge allows Organic Chemists to
solve synthetic problems and to invent novel transformations. Here, we propose
a model which mimics chemical reasoning and formalises reaction prediction as
finding missing links in a knowledge graph. We have constructed a knowledge
graph containing 14.4 million molecules and 8.2 million binary reactions, which
represents the bulk of all chemical reactions ever published in the scientific
literature. Our model outperforms a rule-based expert system in the reaction
prediction task for 180,000 randomly selected binary reactions. We show that
our data-driven model generalises even beyond known reaction types, and is thus
capable of effectively (re-) discovering novel transformations (even including
transition-metal catalysed reactions). Our model enables computers to infer
hypotheses about reactivity and reactions by only considering the intrinsic
local structure of the graph, and because each single reaction prediction is
typically achieved in a sub-second time frame, our model can be used as a
high-throughput generator of reaction hypotheses for reaction discovery.
Aligning Packed Dependency Trees: a theory of composition for distributional semantics
Authors: David Weir, Julie Weeds, Jeremy Reffin, Thomas Kober
Date: 2016-08-25 00:00:00
Proceeding: CL 2016 12

We present a new framework for compositional distributional semantics in
which the distributional contexts of lexemes are expressed in terms of anchored
packed dependency trees. We show that these structures have the potential to
capture the full sentential contexts of a lexeme and provide a uniform basis
for the composition of distributional knowledge in a way that captures both
mutual disambiguation and generalization.
The statistical trade-off between word order and word structure - large-scale evidence for the principle of least effort
Authors: Alexander Koplenig, Peter Meyer, Sascha Wolfer, Carolin Mueller-Spitzer
Date: 2016-08-11 00:00:00
Tasks: Sentence

Languages employ different strategies to transmit structural and grammatical
information. While, for example, grammatical dependency relationships in
sentences are mainly conveyed by the ordering of the words for languages like
Mandarin Chinese, or Vietnamese, the word ordering is much less restricted for
languages such as Inupiatun or Quechua, as those languages (also) use the
internal structure of words (e.g. inflectional morphology) to mark grammatical
relationships in a sentence. Based on a quantitative analysis of more than
1,500 unique translations of different books of the Bible in more than 1,100
different languages that are spoken as a native language by approximately 6
billion people (more than 80% of the world population), we present large-scale
evidence for a statistical trade-off between the amount of information conveyed
by the ordering of words and the amount of information conveyed by internal
word structure: languages that rely more strongly on word order information
tend to rely less on word structure information and vice versa. In addition, we
find that - despite differences in the way information is expressed - there is
also evidence for a trade-off between different books of the biblical canon
that recurs with little variation across languages: the more informative the
word order of the book, the less informative its word structure and vice versa.
We argue that this might suggest that, on the one hand, languages encode
information in very different (but efficient) ways. On the other hand,
content-related and stylistic features are statistically encoded in very
similar ways.
A Context-aware Natural Language Generator for Dialogue Systems
Authors: Ondřej Dušek, Filip Jurčíček
Date: 2016-08-25 00:00:00
Tasks: Spoken Dialogue Systems, Text Generation

We present a novel natural language generation system for spoken dialogue
systems capable of entraining (adapting) to users' way of speaking, providing
contextually appropriate responses. The generator is based on recurrent neural
networks and the sequence-to-sequence approach. It is fully trainable from data
which include preceding context along with responses to be generated. We show
that the context-aware generator yields significant improvements over the
baseline in both automatic metrics and a human pairwise preference test.
Learning Points and Routes to Recommend Trajectories
Authors: Dawei Chen, Cheng Soon Ong, Lexing Xie
Date: 2016-08-25 00:00:00

The problem of recommending tours to travellers is an important and broadly
studied area. Suggested solutions include various approaches of
points-of-interest (POI) recommendation and route planning. We consider the
task of recommending a sequence of POIs, that simultaneously uses information
about POIs and routes. Our approach unifies the treatment of various sources of
information by representing them as features in machine learning algorithms,
enabling us to learn from past behaviour. Information about POIs are used to
learn a POI ranking model that accounts for the start and end points of tours.
Data about previous trajectories are used for learning transition patterns
between POIs that enable us to recommend probable routes. In addition, a
probabilistic model is proposed to combine the results of POI ranking and the
POI to POI transitions. We propose a new F$_1$ score on pairs of POIs that
capture the order of visits. Empirical results show that our approach improves
on recent methods, and demonstrate that combining points and routes enables
better trajectory recommendations.
Evaluation and selection of Medical Tourism sites: A rough AHP based MABAC approach
Authors: Jagannath Roy, Kajal Chatterjee, Abhirup Bandhopadhyay, Samarjit Kar
Date: 2016-06-29 00:00:00
Tasks: Decision Making

In this paper, a novel multiple criteria decision making (MCDM) methodology
is presented for assessing and prioritizing medical tourism destinations in
uncertain environment. A systematic evaluation and assessment method is
proposed by integrating rough number based AHP (Analytic Hierarchy Process) and
rough number based MABAC (Multi-Attributive Border Approximation area
Comparison). Rough number is used to aggregate individual judgments and
preferences to deal with vagueness in decision making due to limited data.
Rough AHP analyzes the relative importance of criteria based on their
preferences given by experts. Rough MABAC evaluates the alternative sites based
on the criteria weights. The proposed methodology is explained through a case
study considering different cities for healthcare service in India. The
validity of the obtained ranking for the given decision making problem is
established by testing criteria proposed by Wang and Triantaphyllou (2008)
along with further analysis and discussion.
Compressed Dynamic Mode Decomposition for Background Modeling
Authors: N. Benjamin Erichson, Steven L. Brunton, J. Nathan Kutz
Date: 2015-12-14 00:00:00
Tasks: compressed sensing, Computational Efficiency, GPU

We introduce the method of compressed dynamic mode decomposition (cDMD) for
background modeling. The dynamic mode decomposition (DMD) is a regression
technique that integrates two of the leading data analysis methods in use
today: Fourier transforms and singular value decomposition. Borrowing ideas
from compressed sensing and matrix sketching, cDMD eases the computational
workload of high resolution video processing. The key principal of cDMD is to
obtain the decomposition on a (small) compressed matrix representation of the
video feed. Hence, the cDMD algorithm scales with the intrinsic rank of the
matrix, rather then the size of the actual video (data) matrix. Selection of
the optimal modes characterizing the background is formulated as a
sparsity-constrained sparse coding problem. Our results show, that the quality
of the resulting background model is competitive, quantified by the F-measure,
Recall and Precision. A GPU (graphics processing unit) accelerated
implementation is also presented which further boosts the computational
efficiency of the algorithm.
Variance Reduction for Faster Non-Convex Optimization
Authors: Zeyuan Allen-Zhu, Elad Hazan
Date: 2016-03-17 00:00:00

We consider the fundamental problem in non-convex optimization of efficiently
reaching a stationary point. In contrast to the convex case, in the long
history of this basic problem, the only known theoretical results on
first-order non-convex optimization remain to be full gradient descent that
converges in $O(1/\varepsilon)$ iterations for smooth objectives, and
stochastic gradient descent that converges in $O(1/\varepsilon^2)$ iterations
for objectives that are sum of smooth functions.
  We provide the first improvement in this line of research. Our result is
based on the variance reduction trick recently introduced to convex
optimization, as well as a brand new analysis of variance reduction that is
suitable for non-convex optimization. For objectives that are sum of smooth
functions, our first-order minibatch stochastic method converges with an
$O(1/\varepsilon)$ rate, and is faster than full gradient descent by
$\Omega(n^{1/3})$.
  We demonstrate the effectiveness of our methods on empirical risk
minimizations with non-convex loss functions and training neural nets.
Multi-View Fuzzy Clustering with Minimax Optimization for Effective Clustering of Data from Multiple Sources
Authors: Yangtao Wang, Lihui Chen
Date: 2016-08-25 00:00:00
Tasks: Clustering

Multi-view data clustering refers to categorizing a data set by making good
use of related information from multiple representations of the data. It
becomes important nowadays because more and more data can be collected in a
variety of ways, in different settings and from different sources, so each data
set can be represented by different sets of features to form different views of
it. Many approaches have been proposed to improve clustering performance by
exploring and integrating heterogeneous information underlying different views.
In this paper, we propose a new multi-view fuzzy clustering approach called
MinimaxFCM by using minimax optimization based on well-known Fuzzy c means. In
MinimaxFCM the consensus clustering results are generated based on minimax
optimization in which the maximum disagreements of different weighted views are
minimized. Moreover, the weight of each view can be learned automatically in
the clustering process. In addition, there is only one parameter to be set
besides the fuzzifier. The detailed problem formulation, updating rules
derivation, and the in-depth analysis of the proposed MinimaxFCM are provided
here. Experimental studies on nine multi-view data sets including real world
image and document data sets have been conducted. We observed that MinimaxFCM
outperforms related multi-view clustering approaches in terms of clustering
accuracy, demonstrating the great potential of MinimaxFCM for multi-view data
analysis.
Incremental Minimax Optimization based Fuzzy Clustering for Large Multi-view Data
Authors: Yangtao Wang, Lihui Chen, Xiao-Li Li
Date: 2016-08-25 00:00:00
Tasks: Clustering

Incremental clustering approaches have been proposed for handling large data
when given data set is too large to be stored. The key idea of these approaches
is to find representatives to represent each cluster in each data chunk and
final data analysis is carried out based on those identified representatives
from all the chunks. However, most of the incremental approaches are used for
single view data. As large multi-view data generated from multiple sources
becomes prevalent nowadays, there is a need for incremental clustering
approaches to handle both large and multi-view data. In this paper we propose a
new incremental clustering approach called incremental minimax optimization
based fuzzy clustering (IminimaxFCM) to handle large multi-view data. In
IminimaxFCM, representatives with multiple views are identified to represent
each cluster by integrating multiple complementary views using minimax
optimization. The detailed problem formulation, updating rules derivation, and
the in-depth analysis of the proposed IminimaxFCM are provided. Experimental
studies on several real world multi-view data sets have been conducted. We
observed that IminimaxFCM outperforms related incremental fuzzy clustering in
terms of clustering accuracy, demonstrating the great potential of IminimaxFCM
for large multi-view data analysis.
Occlusion Coherence: Detecting and Localizing Occluded Faces
Authors: Golnaz Ghiasi, Charless C. Fowlkes
Date: 2015-06-28 00:00:00
Tasks: Face Detection, Object Recognition, Occluded Face Detection

The presence of occluders significantly impacts object recognition accuracy.
However, occlusion is typically treated as an unstructured source of noise and
explicit models for occluders have lagged behind those for object appearance
and shape. In this paper we describe a hierarchical deformable part model for
face detection and landmark localization that explicitly models part occlusion.
The proposed model structure makes it possible to augment positive training
data with large numbers of synthetically occluded instances. This allows us to
easily incorporate the statistics of occlusion patterns in a discriminatively
trained model. We test the model on several benchmarks for landmark
localization and detection including challenging new data sets featuring
significant occlusion. We find that the addition of an explicit occlusion model
yields a detection system that outperforms existing approaches for occluded
instances while maintaining competitive accuracy in detection and landmark
localization for unoccluded instances.
A 4D Light-Field Dataset and CNN Architectures for Material Recognition
Authors: Ting-Chun Wang, Jun-Yan Zhu, Ebi Hiroaki, Manmohan Chandraker, Alexei A. Efros, Ravi Ramamoorthi
Date: 2016-08-24 00:00:00
Tasks: image-classification, Image Classification, Image Segmentation, Material Recognition, object-detection, Object Detection, Semantic Segmentation

We introduce a new light-field dataset of materials, and take advantage of
the recent success of deep learning to perform material recognition on the 4D
light-field. Our dataset contains 12 material categories, each with 100 images
taken with a Lytro Illum, from which we extract about 30,000 patches in total.
To the best of our knowledge, this is the first mid-size dataset for
light-field images. Our main goal is to investigate whether the additional
information in a light-field (such as multiple sub-aperture views and
view-dependent reflectance effects) can aid material recognition. Since
recognition networks have not been trained on 4D images before, we propose and
compare several novel CNN architectures to train on light-field images. In our
experiments, the best performing CNN architecture achieves a 7% boost compared
with 2D image classification (70% to 77%). These results constitute important
baselines that can spur further research in the use of CNNs for light-field
applications. Upon publication, our dataset also enables other novel
applications of light-fields, including object detection, image segmentation
and view interpolation.
State Duration and Interval Modeling in Hidden Semi-Markov Model for Sequential Data Analysis
Authors: Hiromi Narimatsu, Hiroyuki Kasai
Date: 2016-08-24 00:00:00
Tasks: Time Series Analysis

Sequential data modeling and analysis have become indispensable tools for
analyzing sequential data, such as time-series data, because larger amounts of
sensed event data have become available. These methods capture the sequential
structure of data of interest, such as input-output relations and correlation
among datasets. However, because most studies in this area are specialized or
limited to their respective applications, rigorous requirement analysis of such
models has not been undertaken from a general perspective. Therefore, we
particularly examine the structure of sequential data, and extract the
necessity of `state duration' and `state interval' of events for efficient and
rich representation of sequential data. Specifically addressing the hidden
semi-Markov model (HSMM) that represents such state duration inside a model, we
attempt to add representational capability of a state interval of events onto
HSMM. To this end, we propose two extended models: an interval state hidden
semi-Markov model (IS-HSMM) to express the length of a state interval with a
special state node designated as "interval state node"; and an interval length
probability hidden semi-Markov model (ILP-HSMM) which represents the length of
the state interval with a new probabilistic parameter "interval length
probability." Exhaustive simulations have revealed superior performance of the
proposed models in comparison with HSMM. These proposed models are the first
reported extensions of HMM to support state interval representation as well as
state duration representation.
VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing
Authors: Arash Ardakani, François Leduc-Primeau, Naoya Onizawa, Takahiro Hanyu, Warren J. Gross
Date: 2015-09-29 00:00:00

The hardware implementation of deep neural networks (DNNs) has recently
received tremendous attention: many applications in fact require high-speed
operations that suit a hardware implementation. However, numerous elements and
complex interconnections are usually required, leading to a large area
occupation and copious power consumption. Stochastic computing has shown
promising results for low-power area-efficient hardware implementations, even
though existing stochastic algorithms require long streams that cause long
latencies. In this paper, we propose an integer form of stochastic computation
and introduce some elementary circuits. We then propose an efficient
implementation of a DNN based on integral stochastic computing. The proposed
architecture has been implemented on a Virtex7 FPGA, resulting in 45% and 62%
average reductions in area and latency compared to the best reported
architecture in literature. We also synthesize the circuits in a 65 nm CMOS
technology and we show that the proposed integral stochastic architecture
results in up to 21% reduction in energy consumption compared to the binary
radix implementation at the same misclassification rate. Due to fault-tolerant
nature of stochastic architectures, we also consider a quasi-synchronous
implementation which yields 33% reduction in energy consumption w.r.t. the
binary radix implementation without any compromise on performance.
Bridging AIC and BIC: a new criterion for autoregression
Authors: Jie Ding, Vahid Tarokh, Yuhong Yang
Date: 2015-08-11 00:00:00
Tasks: Model Selection, Time Series, Time Series Analysis

We introduce a new criterion to determine the order of an autoregressive
model fitted to time series data. It has the benefits of the two well-known
model selection techniques, the Akaike information criterion and the Bayesian
information criterion. When the data is generated from a finite order
autoregression, the Bayesian information criterion is known to be consistent,
and so is the new criterion. When the true order is infinity or suitably high
with respect to the sample size, the Akaike information criterion is known to
be efficient in the sense that its prediction performance is asymptotically
equivalent to the best offered by the candidate models; in this case, the new
criterion behaves in a similar manner. Different from the two classical
criteria, the proposed criterion adaptively achieves either consistency or
efficiency depending on the underlying true model. In practice where the
observed time series is given without any prior information about the model
specification, the proposed order selection criterion is more flexible and
robust compared with classical approaches. Numerical results are presented
demonstrating the adaptivity of the proposed technique when applied to various
datasets.
AIDE: Fast and Communication Efficient Distributed Optimization
Authors: Sashank J. Reddi, Jakub Konečný, Peter Richtárik, Barnabás Póczós, Alex Smola
Date: 2016-08-24 00:00:00
Tasks: Distributed Optimization

In this paper, we present two new communication-efficient methods for
distributed minimization of an average of functions. The first algorithm is an
inexact variant of the DANE algorithm that allows any local algorithm to return
an approximate solution to a local subproblem. We show that such a strategy
does not affect the theoretical guarantees of DANE significantly. In fact, our
approach can be viewed as a robustification strategy since the method is
substantially better behaved than DANE on data partition arising in practice.
It is well known that DANE algorithm does not match the communication
complexity lower bounds. To bridge this gap, we propose an accelerated variant
of the first method, called AIDE, that not only matches the communication lower
bounds but can also be implemented using a purely first-order oracle. Our
empirical results show that AIDE is superior to other communication efficient
algorithms in settings that naturally arise in machine learning applications.