pwc-archive/papers-with-abstracts huggingface dataset:
On the Modeling of Error Functions as High Dimensional Landscapes for Weight Initialization in Learning Networks
Authors: Julius, Gopinath Mahale, Sumana T., C. S. Adityakrishna
Date: 2016-07-20 00:00:00
Tasks: General Classification

Next generation deep neural networks for classification hosted on embedded
platforms will rely on fast, efficient, and accurate learning algorithms.
Initialization of weights in learning networks has a great impact on the
classification accuracy. In this paper we focus on deriving good initial
weights by modeling the error function of a deep neural network as a
high-dimensional landscape. We observe that due to the inherent complexity in
its algebraic structure, such an error function may conform to general results
of the statistics of large systems. To this end we apply some results from
Random Matrix Theory to analyse these functions. We model the error function in
terms of a Hamiltonian in N-dimensions and derive some theoretical results
about its general behavior. These results are further used to make better
initial guesses of weights for the learning algorithm.
Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations
Authors: Eliyahu Kiperwasser, Yoav Goldberg
Date: 2016-03-14 00:00:00
Proceeding: TACL 2016 1
Tasks: Dependency Parsing, Sentence

We present a simple and effective scheme for dependency parsing which is
based on bidirectional-LSTMs (BiLSTMs). Each sentence token is associated with
a BiLSTM vector representing the token in its sentential context, and feature
vectors are constructed by concatenating a few BiLSTM vectors. The BiLSTM is
trained jointly with the parser objective, resulting in very effective feature
extractors for parsing. We demonstrate the effectiveness of the approach by
applying it to a greedy transition-based parser as well as to a globally
optimized graph-based parser. The resulting parsers have very simple
architectures, and match or surpass the state-of-the-art accuracies on English
and Chinese.
Compression and the origins of Zipf's law for word frequencies
Authors: Ramon Ferrer-i-Cancho
Date: 2016-05-04 00:00:00

Here we sketch a new derivation of Zipf's law for word frequencies based on
optimal coding. The structure of the derivation is reminiscent of Mandelbrot's
random typing model but it has multiple advantages over random typing: (1) it
starts from realistic cognitive pressures (2) it does not require fine tuning
of parameters and (3) it sheds light on the origins of other statistical laws
of language and thus can lead to a compact theory of linguistic laws. Our
findings suggest that the recurrence of Zipf's law in human languages could
originate from pressure for easy and fast communication.
Person Re-identification for Real-world Surveillance Systems
Authors: Furqan M. Khan, Francois Bremond
Date: 2016-07-20 00:00:00
Tasks: Person Re-Identification

Appearance based person re-identification in a real-world video surveillance
system with non-overlapping camera views is a challenging problem for many
reasons. Current state-of-the-art methods often address the problem by relying
on supervised learning of similarity metrics or ranking functions to implicitly
model appearance transformation between cameras for each camera pair, or group,
in the system. This requires considerable human effort to annotate data.
Furthermore, the learned models are camera specific and not transferable from
one set of cameras to another. Therefore, the annotation process is required
after every network expansion or camera replacement, which strongly limits
their applicability. Alternatively, we propose a novel modeling approach to
harness complementary appearance information without supervised learning that
significantly outperforms current state-of-the-art unsupervised methods on
multiple benchmark datasets.
Anomaly Detection and Localisation using Mixed Graphical Models
Authors: Romain Laby, François Roueff, Alexandre Gramfort
Date: 2016-07-20 00:00:00
Tasks: Anomaly Detection

We propose a method that performs anomaly detection and localisation within
heterogeneous data using a pairwise undirected mixed graphical model. The data
are a mixture of categorical and quantitative variables, and the model is
learned over a dataset that is supposed not to contain any anomaly. We then use
the model over temporal data, potentially a data stream, using a version of the
two-sided CUSUM algorithm. The proposed decision statistic is based on a
conditional likelihood ratio computed for each variable given the others. Our
results show that this function allows to detect anomalies variable by
variable, and thus to localise the variables involved in the anomalies more
precisely than univariate methods based on simple marginals.
Image Captioning with Deep Bidirectional LSTMs
Authors: Cheng Wang, Haojin Yang, Christian Bartz, Christoph Meinel
Date: 2016-04-04 00:00:00
Tasks: Caption Generation, Data Augmentation, Image Captioning, object-detection, Object Detection, Retrieval, Sentence, Sentence Retrieval

This work presents an end-to-end trainable deep bidirectional LSTM
(Long-Short Term Memory) model for image captioning. Our model builds on a deep
convolutional neural network (CNN) and two separate LSTM networks. It is
capable of learning long term visual-language interactions by making use of
history and future context information at high level semantic space. Two novel
deep bidirectional variant models, in which we increase the depth of
nonlinearity transition in different way, are proposed to learn hierarchical
visual-language embeddings. Data augmentation techniques such as multi-crop,
multi-scale and vertical mirror are proposed to prevent overfitting in training
deep models. We visualize the evolution of bidirectional LSTM internal states
over time and qualitatively analyze how our models "translate" image to
sentence. Our proposed models are evaluated on caption generation and
image-sentence retrieval tasks with three benchmark datasets: Flickr8K,
Flickr30K and MSCOCO datasets. We demonstrate that bidirectional LSTM models
achieve highly competitive performance to the state-of-the-art results on
caption generation even without integrating additional mechanism (e.g. object
detection, attention model etc.) and significantly outperform recent methods on
retrieval task.
4D Cardiac Ultrasound Standard Plane Location by Spatial-Temporal Correlation
Authors: Yun Gu, Guang-Zhong Yang, Jie Yang, Kun Sun
Date: 2016-07-20 00:00:00
Tasks: Computational Efficiency, Diagnostic, General Classification

Echocardiography plays an important part in diagnostic aid in cardiac
diseases. A critical step in echocardiography-aided diagnosis is to extract the
standard planes since they tend to provide promising views to present different
structures that are benefit to diagnosis. To this end, this paper proposes a
spatial-temporal embedding framework to extract the standard view planes from
4D STIC (spatial-temporal image corre- lation) volumes. The proposed method is
comprised of three stages, the frame smoothing, spatial-temporal embedding and
final classification. In first stage, an L 0 smoothing filter is used to
preprocess the frames that removes the noise and preserves the boundary. Then a
compact repre- sentation is learned via embedding spatial and temporal features
into a latent space in the supervised scheme considering both standard plane
information and diagnosis result. In last stage, the learned features are fed
into support vector machine to identify the standard plane. We eval- uate the
proposed method on a 4D STIC volume dataset with 92 normal cases and 93
abnormal cases in three standard planes. It demonstrates that our method
outperforms the baselines in both classification accuracy and computational
efficiency.
Robust Natural Language Processing - Combining Reasoning, Cognitive Semantics and Construction Grammar for Spatial Language
Authors: Michael Spranger, Jakob Suchan, Mehul Bhatt
Date: 2016-07-20 00:00:00

We present a system for generating and understanding of dynamic and static
spatial relations in robotic interaction setups. Robots describe an environment
of moving blocks using English phrases that include spatial relations such as
"across" and "in front of". We evaluate the system in robot-robot interactions
and show that the system can robustly deal with visual perception errors,
language omissions and ungrammatical utterances.
Interactive Illumination Invariance
Authors: Han Gong, Graham Finlayson
Date: 2016-07-20 00:00:00
Tasks: Image Generation

Illumination effects cause problems for many computer vision algorithms. We
present a user-friendly interactive system for robust illumination-invariant
image generation. Compared with the previous automated illumination-invariant
image derivation approaches, our system enables users to specify a particular
kind of illumination variation for removal. The derivation of
illumination-invariant image is guided by the user input. The input is a stroke
that defines an area covering a set of pixels whose intensities are influenced
predominately by the illumination variation. This additional flexibility
enhances the robustness for processing non-linearly rendered images and the
images of the scenes where their illumination variations are difficult to
estimate automatically. Finally, we present some evaluation results of our
method.
Onsager-corrected deep learning for sparse linear inverse problems
Authors: Mark Borgerding, Philip Schniter
Date: 2016-07-20 00:00:00
Tasks: Compressive Sensing, Deep Learning

Deep learning has gained great popularity due to its widespread success on
many inference problems. We consider the application of deep learning to the
sparse linear inverse problem encountered in compressive sensing, where one
seeks to recover a sparse signal from a small number of noisy linear
measurements. In this paper, we propose a novel neural-network architecture
that decouples prediction errors across layers in the same way that the
approximate message passing (AMP) algorithm decouples them across iterations:
through Onsager correction. Numerical experiments suggest that our "learned
AMP" network significantly improves upon Gregor and LeCun's "learned ISTA"
network in both accuracy and complexity.
Indoor occupancy estimation from carbon dioxide concentration
Authors: Chaoyang Jiang, Mustafa K. Masood, Yeng Chai Soh, Hua Li
Date: 2016-07-20 00:00:00

This paper presents an indoor occupancy estimator with which we can estimate
the number of real-time indoor occupants based on the carbon dioxide (CO2)
measurement. The estimator is actually a dynamic model of the occupancy level.
To identify the dynamic model, we propose the Feature Scaled Extreme Learning
Machine (FS-ELM) algorithm, which is a variation of the standard Extreme
Learning Machine (ELM) but is shown to perform better for the occupancy
estimation problem. The measured CO2 concentration suffers from serious spikes.
We find that pre-smoothing the CO2 data can greatly improve the estimation
accuracy. In real applications, however, we cannot obtain the real-time
globally smoothed CO2 data. We provide a way to use the locally smoothed CO2
data instead, which is real-time available. We introduce a new criterion, i.e.
$x$-tolerance accuracy, to assess the occupancy estimator. The proposed
occupancy estimator was tested in an office room with 24 cubicles and 11 open
seats. The accuracy is up to 94 percent with a tolerance of four occupants.
The encoding of proprioceptive inputs in the brain: knowns and unknowns from a robotic perspective
Authors: Matej Hoffmann, Nada Bednarova
Date: 2016-07-20 00:00:00

Somatosensory inputs can be grossly divided into tactile (or cutaneous) and
proprioceptive -- the former conveying information about skin stimulation, the
latter about limb position and movement. The principal proprioceptors are
constituted by muscle spindles, which deliver information about muscle length
and speed. In primates, this information is relayed to the primary
somatosensory cortex and eventually the posterior parietal cortex, where
integrated information about body posture (postural schema) is presumably
available. However, coming from robotics and seeking a biologically motivated
model that could be used in a humanoid robot, we faced a number of
difficulties. First, it is not clear what neurons in the ascending pathway and
primary somatosensory cortex code. To an engineer, joint angles would seem the
most useful variables. However, the lengths of individual muscles have
nonlinear relationships with the angles at joints. Kim et al. (Neuron, 2015)
found different types of proprioceptive neurons in the primary somatosensory
cortex -- sensitive to movement of single or multiple joints or to static
postures. Second, there are indications that the somatotopic arrangement ("the
homunculus") of these brain areas is to a significant extent learned. However,
the mechanisms behind this developmental process are unclear. We will report
first results from modeling of this process using data obtained from body
babbling in the iCub humanoid robot and feeding them into a Self-Organizing Map
(SOM). Our results reveal that the SOM algorithm is only suited to develop
receptive fields of the posture-selective type. Furthermore, the SOM algorithm
has intrinsic difficulties when combined with population code on its input and
in particular with nonlinear tuning curves (sigmoids or Gaussians).
A Flexible Primal-Dual Toolbox
Authors: Hendrik Dirks
Date: 2016-03-18 00:00:00

\textbf{FlexBox} is a flexible MATLAB toolbox for finite dimensional convex
variational problems in image processing and beyond. Such problems often
consist of non-differentiable parts and involve linear operators. The toolbox
uses a primal-dual scheme to avoid (computationally) inefficient operator
inversion and to get reliable error estimates. From the user-side,
\textbf{FlexBox} expects the primal formulation of the problem, automatically
decouples operators and dualizes the problem. For large-scale problems,
\textbf{FlexBox} also comes with a \cpp-module, which can be used stand-alone
or together with MATLAB via MEX-interfaces. Besides various pre-implemented
data-fidelities and regularization-terms, \textbf{FlexBox} is able to handle
arbitrary operators while being easily extendable, due to its object-oriented
design. The toolbox is available at
\href{http://www.flexbox.im}{http://www.flexbox.im}
An Empirical Study on Budget-Aware Online Kernel Algorithms for Streams of Graphs
Authors: Giovanni Da San Martino, Nicolò Navarin, Alessandro Sperduti
Date: 2015-07-08 00:00:00

Kernel methods are considered an effective technique for on-line learning.
Many approaches have been developed for compactly representing the dual
solution of a kernel method when the problem imposes memory constraints.
However, in literature no work is specifically tailored to streams of graphs.
Motivated by the fact that the size of the feature space representation of many
state-of-the-art graph kernels is relatively small and thus it is explicitly
computable, we study whether executing kernel algorithms in the feature space
can be more effective than the classical dual approach. We study three
different algorithms and various strategies for managing the budget. Efficiency
and efficacy of the proposed approaches are experimentally assessed on
relatively large graph streams exhibiting concept drift. It turns out that,
when strict memory budget constraints have to be enforced, working in feature
space, given the current state of the art on graph kernels, is more than a
viable alternative to dual approaches, both in terms of speed and
classification performance.
Optimising Rule-Based Classification in Temporal Data
Authors: Polla Fattah, Uwe Aickelin, Christian Wagner
Date: 2016-07-20 00:00:00
Tasks: Classification, General Classification

This study optimises manually derived rule-based expert system classification
of objects according to changes in their properties over time. One of the key
challenges that this study tries to address is how to classify objects that
exhibit changes in their behaviour over time, for example how to classify
companies' share price stability over a period of time or how to classify
students' preferences for subjects while they are progressing through school. A
specific case the paper considers is the strategy of players in public goods
games (as common in economics) across multiple consecutive games. Initial
classification starts from expert definitions specifying class allocation for
players based on aggregated attributes of the temporal data. Based on these
initial classifications, the optimisation process tries to find an improved
classifier which produces the best possible compact classes of objects
(players) for every time point in the temporal data. The compactness of the
classes is measured by a cost function based on internal cluster indices like
the Dunn Index, distance measures like Euclidean distance or statistically
derived measures like standard deviation. The paper discusses the approach in
the context of incorporating changing player strategies in the aforementioned
public good games, where common classification approaches so far do not
consider such changes in behaviour resulting from learning or in-game
experience. By using the proposed process for classifying temporal data and the
actual players' contribution during the games, we aim to produce a more refined
classification which in turn may inform the interpretation of public goods game
data.
Simulating user learning in authoritative technology adoption: An agent based model for council-led smart meter deployment planning in the UK
Authors: Tao Zhang, Peer-Olaf Siebers, Uwe Aickelin
Date: 2016-07-20 00:00:00
Tasks: Management

How do technology users effectively transit from having zero knowledge about
a technology to making the best use of it after an authoritative technology
adoption? This post-adoption user learning has received little research
attention in technology management literature. In this paper we investigate
user learning in authoritative technology adoption by developing an agent-based
model using the case of council-led smart meter deployment in the UK City of
Leeds. Energy consumers gain experience of using smart meters based on the
learning curve in behavioural learning. With the agent-based model we carry out
experiments to validate the model and test different energy interventions that
local authorities can use to facilitate energy consumers' learning and maintain
their continuous use of the technology. Our results show that the easier energy
consumers become experienced, the more energy-efficient they are and the more
energy saving they can achieve; encouraging energy consumers' contacts via
various informational means can facilitate their learning; and developing and
maintaining their positive attitude toward smart metering can enable them to
use the technology continuously. Contributions and energy policy/intervention
implications are discussed in this paper.
Visual Question Answering: A Survey of Methods and Datasets
Authors: Qi Wu, Damien Teney, Peng Wang, Chunhua Shen, Anthony Dick, Anton Van Den Hengel
Date: 2016-07-20 00:00:00
Tasks: General Knowledge, Survey, Visual Question Answering, Visual Question Answering (VQA)

Visual Question Answering (VQA) is a challenging task that has received
increasing attention from both the computer vision and the natural language
processing communities. Given an image and a question in natural language, it
requires reasoning over visual elements of the image and general knowledge to
infer the correct answer. In the first part of this survey, we examine the
state of the art by comparing modern approaches to the problem. We classify
methods by their mechanism to connect the visual and textual modalities. In
particular, we examine the common approach of combining convolutional and
recurrent neural networks to map images and questions to a common feature
space. We also discuss memory-augmented and modular architectures that
interface with structured knowledge bases. In the second part of this survey,
we review the datasets available for training and evaluating VQA systems. The
various datatsets contain questions at different levels of complexity, which
require different capabilities and types of reasoning. We examine in depth the
question/answer pairs from the Visual Genome project, and evaluate the
relevance of the structured annotations of images with scene graphs for VQA.
Finally, we discuss promising future directions for the field, in particular
the connection to structured knowledge bases and the use of natural language
processing models.
Supervised Anomaly Detection in Uncertain Pseudoperiodic Data Streams
Authors: Jiangang Ma, Le Sun, Hua Wang, Yanchun Zhang, Uwe Aickelin
Date: 2016-07-20 00:00:00
Tasks: Anomaly Detection, Computational Efficiency, General Classification, Supervised Anomaly Detection

Uncertain data streams have been widely generated in many Web applications.
The uncertainty in data streams makes anomaly detection from sensor data
streams far more challenging. In this paper, we present a novel framework that
supports anomaly detection in uncertain data streams. The proposed framework
adopts an efficient uncertainty pre-processing procedure to identify and
eliminate uncertainties in data streams. Based on the corrected data streams,
we develop effective period pattern recognition and feature extraction
techniques to improve the computational efficiency. We use classification
methods for anomaly detection in the corrected data stream. We also empirically
show that the proposed approach shows a high accuracy of anomaly detection on a
number of real datasets.
Refining adverse drug reaction signals by incorporating interaction variables identified using emergent pattern mining
Authors: Jenna M. Reps, Uwe Aickelin, Richard B. Hubbard
Date: 2016-07-20 00:00:00
Tasks: regression

Purpose: To develop a framework for identifying and incorporating candidate
confounding interaction terms into a regularised cox regression analysis to
refine adverse drug reaction signals obtained via longitudinal observational
data. Methods: We considered six drug families that are commonly associated
with myocardial infarction in observational healthcare data, but where the
causal relationship ground truth is known (adverse drug reaction or not). We
applied emergent pattern mining to find itemsets of drugs and medical events
that are associated with the development of myocardial infarction. These are
the candidate confounding interaction terms. We then implemented a cohort study
design using regularised cox regression that incorporated and accounted for the
candidate confounding interaction terms. Results The methodology was able to
account for signals generated due to confounding and a cox regression with
elastic net regularisation correctly ranked the drug families known to be true
adverse drug reactions above those.