pwc-archive/papers-with-abstracts huggingface dataset:
High Dimensional Spaces, Deep Learning and Adversarial Examples
Authors: Simant Dube
Date: 2018-01-02 00:00:00
Tasks: Deep Learning, Vocal Bursts Intensity Prediction

In this paper, we analyze deep learning from a mathematical point of view and
derive several novel results. The results are based on intriguing mathematical
properties of high dimensional spaces. We first look at perturbation based
adversarial examples and show how they can be understood using topological and
geometrical arguments in high dimensions. We point out mistake in an argument
presented in prior published literature, and we present a more rigorous,
general and correct mathematical result to explain adversarial examples in
terms of topology of image manifolds. Second, we look at optimization
landscapes of deep neural networks and examine the number of saddle points
relative to that of local minima. Third, we show how multiresolution nature of
images explains perturbation based adversarial examples in form of a stronger
result. Our results state that expectation of $L_2$-norm of adversarial
perturbations is $O\left(\frac{1}{\sqrt{n}}\right)$ and therefore shrinks to 0
as image resolution $n$ becomes arbitrarily large. Finally, by incorporating
the parts-whole manifold learning hypothesis for natural images, we investigate
the working of deep neural networks and root causes of adversarial examples and
discuss how future improvements can be made and how adversarial examples can be
eliminated.
GU IRLAB at SemEval-2018 Task 7: Tree-LSTMs for Scientific Relation Classification
Authors: Sean MacAvaney, Luca Soldaini, Arman Cohan, Nazli Goharian
Date: 2018-04-15 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Classification, General Classification, Relation, Relation Classification

SemEval 2018 Task 7 focuses on relation ex- traction and classification in
scientific literature. In this work, we present our tree-based LSTM network for
this shared task. Our approach placed 9th (of 28) for subtask 1.1 (relation
classification), and 5th (of 20) for subtask 1.2 (relation classification with
noisy entities). We also provide an ablation study of features included as
input to the network.
MovieGraphs: Towards Understanding Human-Centric Situations from Videos
Authors: Paul Vicol, Makarand Tapaswi, Lluis Castrejon, Sanja Fidler
Date: 2017-12-19 00:00:00
Proceeding: CVPR 2018 6
Tasks: Common Sense Reasoning

There is growing interest in artificial intelligence to build socially
intelligent robots. This requires machines to have the ability to "read"
people's emotions, motivations, and other factors that affect behavior. Towards
this goal, we introduce a novel dataset called MovieGraphs which provides
detailed, graph-based annotations of social situations depicted in movie clips.
Each graph consists of several types of nodes, to capture who is present in the
clip, their emotional and physical attributes, their relationships (i.e.,
parent/child), and the interactions between them. Most interactions are
associated with topics that provide additional details, and reasons that give
motivations for actions. In addition, most interactions and many attributes are
grounded in the video with time stamps. We provide a thorough analysis of our
dataset, showing interesting common-sense correlations between different social
aspects of scenes, as well as across scenes over time. We propose a method for
querying videos and text with graphs, and show that: 1) our graphs contain rich
and sufficient information to summarize and localize each scene; and 2)
subgraphs allow us to describe situations at an abstract level and retrieve
multiple semantically relevant situations. We also propose methods for
interaction understanding via ordering, and reason understanding. MovieGraphs
is the first benchmark to focus on inferred properties of human-centric
situations, and opens up an exciting avenue towards socially-intelligent AI
agents.
Detecting Concrete Abnormality Using Time-series Thermal Imaging and Supervised Learning
Authors: Chongsheng Cheng, Zhigang Shen
Date: 2018-04-15 00:00:00
Tasks: Decision Making, Time Series, Time Series Analysis

Nondestructive detecting defects (NDD) in concrete structures have been
explored for decades. Although limited successes were reported, major
limitations still exist. The major limitations are the high noises to signal
ratio created from the environmental factors, such as cloud, shadow, water,
surface texture etc. and the decision making still relies on the engineering
judgment of interpretation of image content. Time-series approach, such as
principle component thermography approach has been experimented with some
improved results. Recent progress in image processing using machine learning
approach made it possible for detecting defects thermal features in more
quantitative ways. In this paper, we provide a procedure to represent the
thermal feature in the time domain by principal component analysis and regress
the prediction of detection by two schemes of supervised learning models. Three
independent experiments were conducted in a similar laboratory setup but varied
in conditions to illustrate the performance and generalization of models.
Results showed the effectiveness for the detection purpose with appropriate
tuning for parameters. Future studies will focus on implementing more
sophisticated structured models to handle more realistic cases under natural
conditions.
Approximating the covariance ellipsoid
Authors: Shahar Mendelson
Date: 2018-04-15 00:00:00

We explore ways in which the covariance ellipsoid ${\cal B}=\{v \in
\mathbb{R}^d : \mathbb{E} <X,v>^2 \leq 1\}$ of a centred random vector $X$ in
$\mathbb{R}^d$ can be approximated by a simple set. The data one is given for
constructing the approximating set consists of $X_1,...,X_N$ that are
independent and distributed as $X$.
  We present a general method that can be used to construct such approximations
and implement it for two types of approximating sets. We first construct a
(random) set ${\cal K}$ defined by a union of intersections of slabs
$H_{z,\alpha}=\{v \in \mathbb{R}^d : |<z,v>| \leq \alpha\}$ (and therefore
${\cal K}$ is actually the output of a neural network with two hidden layers).
The slabs are generated using $X_1,...,X_N$, and under minimal assumptions on
$X$ (e.g., $X$ can be heavy-tailed) it suffices that $N = c_1d
\eta^{-4}\log(2/\eta)$ to ensure that $(1-\eta) {\cal K} \subset {\cal B}
\subset (1+\eta){\cal K}$. In some cases (e.g., if $X$ is rotation invariant
and has marginals that are well behaved in some weak sense), a smaller sample
size suffices: $N = c_1d\eta^{-2}\log(2/\eta)$.
  We then show that if the slabs are replaced by randomly generated ellipsoids
defined using $X_1,...,X_N$, the same degree of approximation is true when $N
\geq c_2d\eta^{-2}\log(2/\eta)$.
  The construction we use is based on the small-ball method.
Ordinal Pooling Networks: For Preserving Information over Shrinking Feature Maps
Authors: Ashwani Kumar
Date: 2018-04-08 00:00:00
Tasks: image-classification, Image Classification

In the framework of convolutional neural networks that lie at the heart of
deep learning, downsampling is often performed with a max-pooling operation
that only retains the element with maximum activation, while completely
discarding the information contained in other elements in a pooling region. To
address this issue, a novel pooling scheme, Ordinal Pooling Network (OPN), is
introduced in this work. OPN rearranges all the elements of a pooling region in
a sequence and assigns different weights to these elements based upon their
orders in the sequence, where the weights are learned via the gradient-based
optimisation. The results of our small-scale experiments on image
classification task demonstrate that this scheme leads to a consistent
improvement in the accuracy over max-pooling operation. This improvement is
expected to increase in deeper networks, where several layers of pooling become
necessary.
Context and Humor: Understanding Amul advertisements of India
Authors: Radhika Mamidi
Date: 2018-04-15 00:00:00
Tasks: General Knowledge

Contextual knowledge is the most important element in understanding language.
By contextual knowledge we mean both general knowledge and discourse knowledge
i.e. knowledge of the situational context, background knowledge and the
co-textual context [10]. In this paper, we will discuss the importance of
contextual knowledge in understanding the humor present in the cartoon based
Amul advertisements in India.In the process, we will analyze these
advertisements and also see if humor is an effective tool for advertising and
thereby, for marketing.These bilingual advertisements also expect the audience
to have the appropriate linguistic knowledge which includes knowledge of
English and Hindi vocabulary, morphology and syntax. Different techniques like
punning, portmanteaus and parodies of popular proverbs, expressions, acronyms,
famous dialogues, songs etc are employed to convey the message in a humorous
way. The present study will concentrate on these linguistic cues and the
required context for understanding wit and humor.
Higher-order Coreference Resolution with Coarse-to-fine Inference
Authors: Kenton Lee, Luheng He, Luke Zettlemoyer
Date: 2018-04-15 00:00:00
Proceeding: NAACL 2018 6
Tasks: Coreference Resolution

We introduce a fully differentiable approximation to higher-order inference
for coreference resolution. Our approach uses the antecedent distribution from
a span-ranking architecture as an attention mechanism to iteratively refine
span representations. This enables the model to softly consider multiple hops
in the predicted clusters. To alleviate the computational cost of this
iterative process, we introduce a coarse-to-fine approach that incorporates a
less accurate but more efficient bilinear factor, enabling more aggressive
pruning without hurting accuracy. Compared to the existing state-of-the-art
span-ranking approach, our model significantly improves accuracy on the English
OntoNotes benchmark, while being far more computationally efficient.
A Sparse Non-negative Matrix Factorization Framework for Identifying Functional Units of Tongue Behavior from MRI
Authors: Jonghye Woo, Jerry L. Prince, Maureen Stone, Fangxu Xing, Arnold Gomez, Jordan R. Green, Christopher J. Hartnick, Thomas J. Brady, Timothy G. Reese, Van J. Wedeen, Georges El Fakhri
Date: 2018-04-15 00:00:00
Tasks: Clustering

Muscle coordination patterns of lingual behaviors are synergies generated by
deforming local muscle groups in a variety of ways. Functional units are
functional muscle groups of local structural elements within the tongue that
compress, expand, and move in a cohesive and consistent manner. Identifying the
functional units using tagged-Magnetic Resonance Imaging (MRI) sheds light on
the mechanisms of normal and pathological muscle coordination patterns,
yielding improvement in surgical planning, treatment, or rehabilitation
procedures. Here, to mine this information, we propose a matrix factorization
and probabilistic graphical model framework to produce building blocks and
their associated weighting map using motion quantities extracted from
tagged-MRI. Our tagged-MRI imaging and accurate voxel-level tracking provide
previously unavailable internal tongue motion patterns, thus revealing the
inner workings of the tongue during speech or other lingual behaviors. We then
employ spectral clustering on the weighting map to identify the cohesive
regions defined by the tongue motion that may involve multiple or undocumented
regions. To evaluate our method, we perform a series of experiments. We first
use two-dimensional images and synthetic data to demonstrate the accuracy of
our method. We then use three-dimensional synthetic and \textit{in vivo} tongue
motion data using protrusion and simple speech tasks to identify
subject-specific and data-driven functional units of the tongue in localized
regions.
Cognitive Deficit of Deep Learning in Numerosity
Authors: Xiaolin Wu, Xi Zhang, Xiao Shu
Date: 2018-02-09 00:00:00
Tasks: Deep Learning

Subitizing, or the sense of small natural numbers, is an innate cognitive
function of humans and primates; it responds to visual stimuli prior to the
development of any symbolic skills, language or arithmetic. Given successes of
deep learning (DL) in tasks of visual intelligence and given the primitivity of
number sense, a tantalizing question is whether DL can comprehend numbers and
perform subitizing. But somewhat disappointingly, extensive experiments of the
type of cognitive psychology demonstrate that the examples-driven black box DL
cannot see through superficial variations in visual representations and distill
the abstract notion of natural number, a task that children perform with high
accuracy and confidence. The failure is apparently due to the learning method
not the CNN computational machinery itself. A recurrent neural network capable
of subitizing does exist, which we construct by encoding a mechanism of
mathematical morphology into the CNN convolutional kernels. Also, we
investigate, using subitizing as a test bed, the ways to aid the black box DL
by cognitive priors derived from human insight. Our findings are mixed and
interesting, pointing to both cognitive deficit of pure DL, and some measured
successes of boosting DL by predetermined cognitive implements. This case study
of DL in cognitive computing is meaningful for visual numerosity represents a
minimum level of human intelligence.
SparseNet: A Sparse DenseNet for Image Classification
Authors: Wenqi Liu, Kun Zeng
Date: 2018-04-15 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

Deep neural networks have made remarkable progresses on various computer
vision tasks. Recent works have shown that depth, width and shortcut
connections of networks are all vital to their performances. In this paper, we
introduce a method to sparsify DenseNet which can reduce connections of a
L-layer DenseNet from O(L^2) to O(L), and thus we can simultaneously increase
depth, width and connections of neural networks in a more parameter-efficient
and computation-efficient way. Moreover, an attention module is introduced to
further boost our network's performance. We denote our network as SparseNet. We
evaluate SparseNet on datasets of CIFAR(including CIFAR10 and CIFAR100) and
SVHN. Experiments show that SparseNet can obtain improvements over the
state-of-the-art on CIFAR10 and SVHN. Furthermore, while achieving comparable
performances as DenseNet on these datasets, SparseNet is x2.6 smaller and x3.7
faster than the original DenseNet.
Attention-Gated Networks for Improving Ultrasound Scan Plane Detection
Authors: Jo Schlemper, Ozan Oktay, Liang Chen, Jacqueline Matthew, Caroline Knight, Bernhard Kainz, Ben Glocker, Daniel Rueckert
Date: 2018-04-15 00:00:00

In this work, we apply an attention-gated network to real-time automated scan
plane detection for fetal ultrasound screening. Scan plane detection in fetal
ultrasound is a challenging problem due the poor image quality resulting in low
interpretability for both clinicians and automated algorithms. To solve this,
we propose incorporating self-gated soft-attention mechanisms. A soft-attention
mechanism generates a gating signal that is end-to-end trainable, which allows
the network to contextualise local information useful for prediction. The
proposed attention mechanism is generic and it can be easily incorporated into
any existing classification architectures, while only requiring a few
additional parameters. We show that, when the base network has a high capacity,
the incorporated attention mechanism can provide efficient object localisation
while improving the overall performance. When the base network has a low
capacity, the method greatly outperforms the baseline approach and
significantly reduces false positives. Lastly, the generated attention maps
allow us to understand the model's reasoning process, which can also be used
for weakly supervised object localisation.
Generative Adversarial Network based Autoencoder: Application to fault detection problem for closed loop dynamical systems
Authors: Indrasis Chakraborty, Rudrasis Chakraborty, Draguna Vrabie
Date: 2018-04-15 00:00:00
Tasks: Fault Detection, Generative Adversarial Network

Fault detection problem for closed loop uncertain dynamical systems, is
investigated in this paper, using different deep learning based methods.
Traditional classifier based method does not perform well, because of the
inherent difficulty of detecting system level faults for closed loop dynamical
system. Specifically, acting controller in any closed loop dynamical system,
works to reduce the effect of system level faults. A novel Generative
Adversarial based deep Autoencoder is designed to classify datasets under
normal and faulty operating conditions. This proposed network performs
significantly well when compared to any available classifier based methods, and
moreover, does not require labeled fault incorporated datasets for training
purpose. Finally, this aforementioned network's performance is tested on a high
complexity building energy system dataset.
Particle Swarm Optimization: A survey of historical and recent developments with hybridization perspectives
Authors: Saptarshi Sengupta, Sanchita Basak, Richard Alan Peters II
Date: 2018-04-15 00:00:00
Tasks: global-optimization

Particle Swarm Optimization (PSO) is a metaheuristic global optimization
paradigm that has gained prominence in the last two decades due to its ease of
application in unsupervised, complex multidimensional problems which cannot be
solved using traditional deterministic algorithms. The canonical particle swarm
optimizer is based on the flocking behavior and social co-operation of birds
and fish schools and draws heavily from the evolutionary behavior of these
organisms. This paper serves to provide a thorough survey of the PSO algorithm
with special emphasis on the development, deployment and improvements of its
most basic as well as some of the state-of-the-art implementations. Concepts
and directions on choosing the inertia weight, constriction factor, cognition
and social weights and perspectives on convergence, parallelization, elitism,
niching and discrete optimization as well as neighborhood topologies are
outlined. Hybridization attempts with other evolutionary and swarm paradigms in
selected applications are covered and an up-to-date review is put forward for
the interested reader.
From CDF to PDF --- A Density Estimation Method for High Dimensional Data
Authors: Shengdong Zhang
Date: 2018-04-15 00:00:00
Tasks: Density Estimation

CDF2PDF is a method of PDF estimation by approximating CDF. The original idea
of it was previously proposed in [1] called SIC. However, SIC requires
additional hyper-parameter tunning, and no algorithms for computing higher
order derivative from a trained NN are provided in [1]. CDF2PDF improves SIC by
avoiding the time-consuming hyper-parameter tuning part and enabling higher
order derivative computation to be done in polynomial time. Experiments of this
method for one-dimensional data shows promising results.
Transcribing Lyrics From Commercial Song Audio: The First Step Towards Singing Content Processing
Authors: Che-Ping Tsai, Yi-Lin Tuan, Lin-shan Lee
Date: 2018-04-15 00:00:00
Tasks: Data Augmentation, Retrieval

Spoken content processing (such as retrieval and browsing) is maturing, but
the singing content is still almost completely left out. Songs are human voice
carrying plenty of semantic information just as speech, and may be considered
as a special type of speech with highly flexible prosody. The various problems
in song audio, for example the significantly changing phone duration over
highly flexible pitch contours, make the recognition of lyrics from song audio
much more difficult. This paper reports an initial attempt towards this goal.
We collected music-removed version of English songs directly from commercial
singing content. The best results were obtained by TDNN-LSTM with data
augmentation with 3-fold speed perturbation plus some special approaches. The
WER achieved (73.90%) was significantly lower than the baseline (96.21%), but
still relatively high.
Head Mounted Pupil Tracking Using Convolutional Neural Network
Authors: Yinheng Zhu, Wanli Chen, Xun Zhan, Zonglin Guo, Hongjian Shi, Ian G. Harris
Date: 2018-04-15 00:00:00
Tasks: Object Tracking, Pupil Detection, Pupil Tracking

Pupil tracking is an important branch of object tracking which require high
precision. We investigate head mounted pupil tracking which is often more
convenient and precise than remote pupil tracking, but also more challenging.
When pupil tracking suffers from noise like bad illumination, detection
precision dramatically decreases. Due to the appearance of head mounted
recording device and public benchmark image datasets, head mounted tracking
algorithms have become easier to design and evaluate. In this paper, we propose
a robust head mounted pupil detection algorithm which uses a Convolutional
Neural Network (CNN) to combine different features of pupil. Here we consider
three features of pupil. Firstly, we use three pupil feature-based algorithms
to find pupil center independently. Secondly, we use a CNN to evaluate the
quality of each result. Finally, we select the best result as output. The
experimental results show that our proposed algorithm performs better than the
present state-of-art.
The EcoLexicon Semantic Sketch Grammar: from Knowledge Patterns to Word Sketches
Authors: P. León-Araúz, A. San Martín
Date: 2018-04-15 00:00:00
Tasks: Retrieval

Many projects have applied knowledge patterns (KPs) to the retrieval of
specialized information. Yet terminologists still rely on manual analysis of
concordance lines to extract semantic information, since there are no
user-friendly publicly available applications enabling them to find knowledge
rich contexts (KRCs). To fill this void, we have created the KP-based
EcoLexicon Semantic SketchGrammar (ESSG) in the well-known corpus query system
Sketch Engine. For the first time, the ESSG is now publicly available inSketch
Engine to query the EcoLexicon English Corpus. Additionally, reusing the ESSG
in any English corpus uploaded by the user enables Sketch Engine to extract
KRCs codifying generic-specific, part-whole, location, cause and function
relations, because most of the KPs are domain-independent. The information is
displayed in the form of summary lists (word sketches) containing the pairs of
terms linked by a given semantic relation. This paper describes the process of
building a KP-based sketch grammar with special focus on the last stage,
namely, the evaluation with refinement purposes. We conducted an initial
shallow precision and recall evaluation of the 64 English sketch grammar rules
created so far for hyponymy, meronymy and causality. Precision was measured
based on a random sample of concordances extracted from each word sketch type.
Recall was assessed based on a random sample of concordances where known term
pairs are found. The results are necessary for the improvement and refinement
of the ESSG. The noise of false positives helped to further specify the rules,
whereas the silence of false negatives allows us to find useful new patterns.
CytonRL: an Efficient Reinforcement Learning Open-source Toolkit Implemented in C++
Authors: Xiaolin Wang
Date: 2018-04-14 00:00:00
Tasks: GPU, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper presents an open-source enforcement learning toolkit named CytonRL
(https://github.com/arthurxlw/cytonRL). The toolkit implements four recent
advanced deep Q-learning algorithms from scratch using C++ and NVIDIA's
GPU-accelerated libraries. The code is simple and elegant, owing to an
open-source general-purpose neural network library named CytonLib. Benchmark
shows that the toolkit achieves competitive performances on the popular Atari
game of Breakout.