pwc-archive/papers-with-abstracts huggingface dataset:
An empirical study on large scale text classification with skip-gram embeddings
Authors: Georgios Balikas, Massih-Reza Amini
Date: 2016-06-21 00:00:00
Tasks: Classification, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, text-classification, Text Classification, Word Embeddings

We investigate the integration of word embeddings as classification features
in the setting of large scale text classification. Such representations have
been used in a plethora of tasks, however their application in classification
scenarios with thousands of classes has not been extensively researched,
partially due to hardware limitations. In this work, we examine efficient
composition functions to obtain document-level from word-level embeddings and
we subsequently investigate their combination with the traditional
one-hot-encoding representations. By presenting empirical evidence on large,
multi-class, multi-label classification problems, we demonstrate the efficiency
and the performance benefits of this combination.
R-FCN: Object Detection via Region-based Fully Convolutional Networks
Authors: Jifeng Dai, Yi Li, Kaiming He, Jian Sun
Date: 2016-05-20 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: image-classification, Object, Object Detection, Real-Time Object Detection, Translation

We present region-based, fully convolutional networks for accurate and
efficient object detection. In contrast to previous region-based detectors such
as Fast/Faster R-CNN that apply a costly per-region subnetwork hundreds of
times, our region-based detector is fully convolutional with almost all
computation shared on the entire image. To achieve this goal, we propose
position-sensitive score maps to address a dilemma between
translation-invariance in image classification and translation-variance in
object detection. Our method can thus naturally adopt fully convolutional image
classifier backbones, such as the latest Residual Networks (ResNets), for
object detection. We show competitive results on the PASCAL VOC datasets (e.g.,
83.6% mAP on the 2007 set) with the 101-layer ResNet. Meanwhile, our result is
achieved at a test-time speed of 170ms per image, 2.5-20x faster than the
Faster R-CNN counterpart. Code is made publicly available at:
https://github.com/daijifeng001/r-fcn
Face Attribute Prediction Using Off-the-Shelf CNN Features
Authors: Yang Zhong, Josephine Sullivan, Hai-Bo Li
Date: 2016-02-12 00:00:00
Tasks: Attribute, Face Recognition, General Classification

Predicting attributes from face images in the wild is a challenging computer
vision problem. To automatically describe face attributes from face containing
images, traditionally one needs to cascade three technical blocks --- face
localization, facial descriptor construction, and attribute classification ---
in a pipeline. As a typical classification problem, face attribute prediction
has been addressed using deep learning. Current state-of-the-art performance
was achieved by using two cascaded Convolutional Neural Networks (CNNs), which
were specifically trained to learn face localization and attribute description.
In this paper, we experiment with an alternative way of employing the power of
deep representations from CNNs. Combining with conventional face localization
techniques, we use off-the-shelf architectures trained for face recognition to
build facial descriptors. Recognizing that the describable face attributes are
diverse, our face descriptors are constructed from different levels of the CNNs
for different attributes to best facilitate face attribute prediction.
Experiments on two large datasets, LFWA and CelebA, show that our approach is
entirely comparable to the state-of-the-art. Our findings not only demonstrate
an efficient face attribute prediction approach, but also raise an important
question: how to leverage the power of off-the-shelf CNN representations for
novel tasks.
ML-based tactile sensor calibration: A universal approach
Authors: Maximilian Karl, Artur Lohrer, Dhananjay Shah, Frederik Diehl, Max Fiedler, Saahil Ognawala, Justin Bayer, Patrick van der Smagt
Date: 2016-06-21 00:00:00

We study the responses of two tactile sensors, the fingertip sensor from the
iCub and the BioTac under different external stimuli. The question of interest
is to which degree both sensors i) allow the estimation of force exerted on the
sensor and ii) enable the recognition of differing degrees of curvature. Making
use of a force controlled linear motor affecting the tactile sensors we acquire
several high-quality data sets allowing the study of both sensors under exactly
the same conditions. We also examined the structure of the representation of
tactile stimuli in the recorded tactile sensor data using t-SNE embeddings. The
experiments show that both the iCub and the BioTac excel in different settings.
Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification
Authors: Yuting Zhang, Kibok Lee, Honglak Lee
Date: 2016-06-21 00:00:00
Tasks: General Classification, image-classification, Image Classification

Unsupervised learning and supervised learning are key research topics in deep
learning. However, as high-capacity supervised neural networks trained with a
large amount of labels have achieved remarkable success in many computer vision
tasks, the availability of large-scale labeled images reduced the significance
of unsupervised learning. Inspired by the recent trend toward revisiting the
importance of unsupervised learning, we investigate joint supervised and
unsupervised learning in a large-scale setting by augmenting existing neural
networks with decoding pathways for reconstruction. First, we demonstrate that
the intermediate activations of pretrained large-scale classification networks
preserve almost all the information of input images except a portion of local
spatial details. Then, by end-to-end training of the entire augmented
architecture with the reconstructive objective, we show improvement of the
network performance for supervised tasks. We evaluate several variants of
autoencoders, including the recently proposed "what-where" autoencoder that
uses the encoder pooling switches, to study the importance of the architecture
design. Taking the 16-layer VGGNet trained under the ImageNet ILSVRC 2012
protocol as a strong baseline for image classification, our methods improve the
validation-set accuracy by a noticeable margin.
Drawing and Recognizing Chinese Characters with Recurrent Neural Network
Authors: Xu-Yao Zhang, Fei Yin, Yan-Ming Zhang, Cheng-Lin Liu, Yoshua Bengio
Date: 2016-06-21 00:00:00
Tasks: Handwriting Recognition

Recent deep learning based approaches have achieved great success on
handwriting recognition. Chinese characters are among the most widely adopted
writing systems in the world. Previous research has mainly focused on
recognizing handwritten Chinese characters. However, recognition is only one
aspect for understanding a language, another challenging and interesting task
is to teach a machine to automatically write (pictographic) Chinese characters.
In this paper, we propose a framework by using the recurrent neural network
(RNN) as both a discriminative model for recognizing Chinese characters and a
generative model for drawing (generating) Chinese characters. To recognize
Chinese characters, previous methods usually adopt the convolutional neural
network (CNN) models which require transforming the online handwriting
trajectory into image-like representations. Instead, our RNN based approach is
an end-to-end system which directly deals with the sequential structure and
does not require any domain-specific knowledge. With the RNN system (combining
an LSTM and GRU), state-of-the-art performance can be achieved on the
ICDAR-2013 competition database. Furthermore, under the RNN framework, a
conditional generative model with character embedding is proposed for
automatically drawing recognizable Chinese characters. The generated characters
(in vector format) are human-readable and also can be recognized by the
discriminative RNN model with high accuracy. Experimental results verify the
effectiveness of using RNNs as both generative and discriminative models for
the tasks of drawing and recognizing Chinese characters.
How to advance general game playing artificial intelligence by player modelling
Authors: Benjamin Ultan Cowley
Date: 2016-06-01 00:00:00
Tasks: Deep Learning

General game playing artificial intelligence has recently seen important
advances due to the various techniques known as 'deep learning'. However the
advances conceal equally important limitations in their reliance on: massive
data sets; fortuitously constructed problems; and absence of any human-level
complexity, including other human opponents. On the other hand, deep learning
systems which do beat human champions, such as in Go, do not generalise well.
The power of deep learning simultaneously exposes its weakness. Given that deep
learning is mostly clever reconfigurations of well-established methods, moving
beyond the state of art calls for forward-thinking visionary solutions, not
just more of the same. I present the argument that general game playing
artificial intelligence will require a generalised player model. This is
because games are inherently human artefacts which therefore, as a class of
problems, contain cases which require a human-style problem solving approach. I
relate this argument to the performance of state of art general game playing
agents. I then describe a concept for a formal category theoretic basis to a
generalised player model. This formal model approach integrates my existing
'Behavlets' method for psychologically-derived player modelling:
  Cowley, B., Charles, D. (2016). Behavlets: a Method for Practical Player
Modelling using Psychology-Based Player Traits and Domain Specific Features.
User Modeling and User-Adapted Interaction, 26(2), 257-306.
Polymetric Rhythmic Feel for a Cognitive Drum Computer
Authors: Oliver Weede
Date: 2016-06-20 00:00:00

This paper addresses a question about music cognition: how do we derive
polymetric structures. A preference rule system is presented which is
implemented into a drum computer. The preference rule system allows inferring
local polymetric structures, like two-over-three and three-over-two. By
analyzing the micro-timing of West African percussion music a timing pattern
consisting of six pulses was discovered. It integrates binary and ternary
rhythmic feels. The presented drum computer integrates the discovered
superimposed polymetric swing (timing and velocity) appropriate to the rhythmic
sequence the user inputs. For binary sequences, the amount of binary swing is
increased and for ternary sequences, the ternary swing is increased.
How Far are We from Solving Pedestrian Detection?
Authors: Shanshan Zhang, Rodrigo Benenson, Mohamed Omran, Jan Hosang, Bernt Schiele
Date: 2016-02-03 00:00:00
Proceeding: CVPR 2016 6
Tasks: Clustering, Pedestrian Detection

Encouraged by the recent progress in pedestrian detection, we investigate the
gap between current state-of-the-art methods and the "perfect single frame
detector". We enable our analysis by creating a human baseline for pedestrian
detection (over the Caltech dataset), and by manually clustering the recurrent
errors of a top detector. Our results characterize both localization and
background-versus-foreground errors. To address localization errors we study
the impact of training annotation noise on the detector performance, and show
that we can improve even with a small portion of sanitized training data. To
address background/foreground discrimination, we study convnets for pedestrian
detection, and discuss which factors affect their performance. Other than our
in-depth analysis, we report top performance on the Caltech dataset, and
provide a new sanitized set of training and test annotations.
Graphical Models for Optimal Power Flow
Authors: Krishnamurthy Dvijotham, Pascal Van Hentenryck, Michael Chertkov, Sidhant Misra, Marc Vuffray
Date: 2016-06-21 00:00:00

Optimal power flow (OPF) is the central optimization problem in electric
power grids. Although solved routinely in the course of power grid operations,
it is known to be strongly NP-hard in general, and weakly NP-hard over tree
networks. In this paper, we formulate the optimal power flow problem over tree
networks as an inference problem over a tree-structured graphical model where
the nodal variables are low-dimensional vectors. We adapt the standard dynamic
programming algorithm for inference over a tree-structured graphical model to
the OPF problem. Combining this with an interval discretization of the nodal
variables, we develop an approximation algorithm for the OPF problem. Further,
we use techniques from constraint programming (CP) to perform interval
computations and adaptive bound propagation to obtain practically efficient
algorithms. Compared to previous algorithms that solve OPF with optimality
guarantees using convex relaxations, our approach is able to work for arbitrary
distribution networks and handle mixed-integer optimization problems. Further,
it can be implemented in a distributed message-passing fashion that is scalable
and is suitable for "smart grid" applications like control of distributed
energy resources. We evaluate our technique numerically on several benchmark
networks and show that practical OPF problems can be solved effectively using
this approach.
Social-sparsity brain decoders: faster spatial sparsity
Authors: Gaël Varoquaux, Matthieu Kowalski, Bertrand Thirion
Date: 2016-06-21 00:00:00
Tasks: Brain Decoding, General Classification

Spatially-sparse predictors are good models for brain decoding: they give
accurate predictions and their weight maps are interpretable as they focus on a
small number of regions. However, the state of the art, based on total
variation or graph-net, is computationally costly. Here we introduce sparsity
in the local neighborhood of each voxel with social-sparsity, a structured
shrinkage operator. We find that, on brain imaging classification problems,
social-sparsity performs almost as well as total-variation models and better
than graph-net, for a fraction of the computational cost. It also very clearly
outlines predictive regions. We give details of the model and the algorithm.
Efficient 2D and 3D Facade Segmentation using Auto-Context
Authors: Raghudeep Gadde, Varun Jampani, Renaud Marlet, Peter V. Gehler
Date: 2016-06-21 00:00:00
Tasks: Segmentation

This paper introduces a fast and efficient segmentation technique for 2D
images and 3D point clouds of building facades. Facades of buildings are highly
structured and consequently most methods that have been proposed for this
problem aim to make use of this strong prior information. Contrary to most
prior work, we are describing a system that is almost domain independent and
consists of standard segmentation methods. We train a sequence of boosted
decision trees using auto-context features. This is learned using stacked
generalization. We find that this technique performs better, or comparable with
all previous published methods and present empirical results on all available
2D and 3D facade benchmark datasets. The proposed method is simple to
implement, easy to extend, and very efficient at test-time inference.
The Schema Editor of OpenIoT for Semantic Sensor Networks
Authors: Prem Prakash Jayaraman, Jean-Paul Calbimonte, Hoan Nguyen Mau Quoc
Date: 2016-06-21 00:00:00

Ontologies provide conceptual abstractions over data, in domains such as the
Internet of Things, in a way that sensor data can be harvested and interpreted
by people and applications. The Semantic Sensor Network (SSN) ontology is the
de-facto standard for semantic representation of sensor observations and
metadata, and it is used at the core of the open source platform for the
Internet of Things, OpenIoT. In this paper we present a Schema Editor that
provides an intuitive web interface for defining new types of sensors, and
concrete instances of them, using the SSN ontology as the core model. This
editor is fully integrated with the OpenIoT platform for generating virtual
sensor descriptions and automating their semantic annotation and registration
process.
A Novel Framework to Expedite Systematic Reviews by Automatically Building Information Extraction Training Corpora
Authors: Tanmay Basu, Shraman Kumar, Abhishek Kalyan, Priyanka Jayaswal, Pawan Goyal, Stephen Pettifer, Siddhartha R. Jonnalagadda
Date: 2016-06-21 00:00:00
Tasks: Articles

A systematic review identifies and collates various clinical studies and
compares data elements and results in order to provide an evidence based answer
for a particular clinical question. The process is manual and involves lot of
time. A tool to automate this process is lacking. The aim of this work is to
develop a framework using natural language processing and machine learning to
build information extraction algorithms to identify data elements in a new
primary publication, without having to go through the expensive task of manual
annotation to build gold standards for each data element type. The system is
developed in two stages. Initially, it uses information contained in existing
systematic reviews to identify the sentences from the PDF files of the included
references that contain specific data elements of interest using a modified
Jaccard similarity measure. These sentences have been treated as labeled data.A
Support Vector Machine (SVM) classifier is trained on this labeled data to
extract data elements of interests from a new article. We conducted experiments
on Cochrane Database systematic reviews related to congestive heart failure
using inclusion criteria as an example data element. The empirical results show
that the proposed system automatically identifies sentences containing the data
element of interest with a high recall (93.75%) and reasonable precision
(27.05% - which means the reviewers have to read only 3.7 sentences on
average). The empirical results suggest that the tool is retrieving valuable
information from the reference articles, even when it is time-consuming to
identify them manually. Thus we hope that the tool will be useful for automatic
data extraction from biomedical research publications. The future scope of this
work is to generalize this information framework for all types of systematic
reviews.
How deep is knowledge tracing?
Authors: Mohammad Khajah, Robert V. Lindsey, Michael C. Mozer
Date: 2016-03-14 00:00:00
Tasks: Knowledge Tracing

In theoretical cognitive science, there is a tension between highly
structured models whose parameters have a direct psychological interpretation
and highly complex, general-purpose models whose parameters and representations
are difficult to interpret. The former typically provide more insight into
cognition but the latter often perform better. This tension has recently
surfaced in the realm of educational data mining, where a deep learning
approach to predicting students' performance as they work through a series of
exercises---termed deep knowledge tracing or DKT---has demonstrated a stunning
performance advantage over the mainstay of the field, Bayesian knowledge
tracing or BKT. In this article, we attempt to understand the basis for DKT's
advantage by considering the sources of statistical regularity in the data that
DKT can leverage but which BKT cannot. We hypothesize four forms of regularity
that BKT fails to exploit: recency effects, the contextualized trial sequence,
inter-skill similarity, and individual variation in ability. We demonstrate
that when BKT is extended to allow it more flexibility in modeling statistical
regularities---using extensions previously proposed in the literature---BKT
achieves a level of performance indistinguishable from that of DKT. We argue
that while DKT is a powerful, useful, general-purpose framework for modeling
student learning, its gains do not come from the discovery of novel
representations---the fundamental advantage of deep learning. To answer the
question posed in our title, knowledge tracing may be a domain that does not
require `depth'; shallow models like BKT can perform just as well and offer us
greater interpretability and explanatory power.
Invariant feature extraction from event based stimuli
Authors: Thusitha N. Chandrapala, Bertram E. Shi
Date: 2016-04-15 00:00:00
Tasks: Object Recognition

We propose a novel architecture, the event-based GASSOM for learning and
extracting invariant representations from event streams originating from
neuromorphic vision sensors. The framework is inspired by feed-forward cortical
models for visual processing. The model, which is based on the concepts of
sparsity and temporal slowness, is able to learn feature extractors that
resemble neurons in the primary visual cortex. Layers of units in the proposed
model can be cascaded to learn feature extractors with different levels of
complexity and selectivity. We explore the applicability of the framework on
real world tasks by using the learned network for object recognition. The
proposed model achieve higher classification accuracy compared to other
state-of-the-art event based processing methods. Our results also demonstrate
the generality and robustness of the method, as the recognizers for different
data sets and different tasks all used the same set of learned feature
detectors, which were trained on data collected independently of the testing
data.
On the consistency of inversion-free parameter estimation for Gaussian random fields
Authors: Hossein Keshavarz, Clayton Scott, XuanLong Nguyen
Date: 2016-01-15 00:00:00
Tasks: parameter estimation

Gaussian random fields are a powerful tool for modeling environmental
processes. For high dimensional samples, classical approaches for estimating
the covariance parameters require highly challenging and massive computations,
such as the evaluation of the Cholesky factorization or solving linear systems.
Recently, Anitescu, Chen and Stein \cite{M.Anitescu} proposed a fast and
scalable algorithm which does not need such burdensome computations. The main
focus of this article is to study the asymptotic behavior of the algorithm of
Anitescu et al. (ACS) for regular and irregular grids in the increasing domain
setting. Consistency, minimax optimality and asymptotic normality of this
algorithm are proved under mild differentiability conditions on the covariance
function. Despite the fact that ACS's method entails a non-concave
maximization, our results hold for any stationary point of the objective
function. A numerical study is presented to evaluate the efficiency of this
algorithm for large data sets.
Incremental Parsing with Minimal Features Using Bi-Directional LSTM
Authors: James Cross, Liang Huang
Date: 2016-06-21 00:00:00
Proceeding: ACL 2016 8
Tasks: Binarization, Constituency Parsing, Feature Engineering, Sentence

Recently, neural network approaches for parsing have largely automated the
combination of individual features, but still rely on (often a larger number
of) atomic features created from human linguistic intuition, and potentially
omitting important global context. To further reduce feature engineering to the
bare minimum, we use bi-directional LSTM sentence representations to model a
parser state with only three sentence positions, which automatically identifies
important aspects of the entire sentence. This model achieves state-of-the-art
results among greedy dependency parsers for English. We also introduce a novel
transition system for constituency parsing which does not require binarization,
and together with the above architecture, achieves state-of-the-art results
among greedy parsers for both English and Chinese.