pwc-archive/papers-with-abstracts huggingface dataset:
Why We Do Not Evolve Software? Analysis of Evolutionary Algorithms
Authors: Roman V. Yampolskiy
Date: 2018-10-12 00:00:00
Tasks: Evolutionary Algorithms

In this paper, we review the state-of-the-art results in evolutionary
computation and observe that we do not evolve non trivial software from scratch
and with no human intervention. A number of possible explanations are
considered, but we conclude that computational complexity of the problem
prevents it from being solved as currently attempted. A detailed analysis of
necessary and available computational resources is provided to support our
findings.
Rough Concept Analysis
Authors: Robert E. Kent
Date: 2018-10-12 00:00:00

The theory introduced, presented and developed in this paper, is concerned
with Rough Concept Analysis. This theory is a synthesis of the theory of Rough
Sets pioneered by Zdzislaw Pawlak with the theory of Formal Concept Analysis
pioneered by Rudolf Wille. The central notion in this paper of a rough formal
concept combines in a natural fashion the notion of a rough set with the notion
of a formal concept: "rough set + formal concept = rough formal concept". A
follow-up paper will provide a synthesis of the two important data modeling
techniques: conceptual scaling of Formal Concept Analysis and
Entity-Relationship database modeling.
U-Net: Machine Reading Comprehension with Unanswerable Questions
Authors: Fu Sun, Linyang Li, Xipeng Qiu, Yang Liu
Date: 2018-10-12 00:00:00
Tasks: Machine Reading Comprehension, Question Answering, Reading Comprehension

Machine reading comprehension with unanswerable questions is a new
challenging task for natural language processing. A key subtask is to reliably
predict whether the question is unanswerable. In this paper, we propose a
unified model, called U-Net, with three important components: answer pointer,
no-answer pointer, and answer verifier. We introduce a universal node and thus
process the question and its context passage as a single contiguous sequence of
tokens. The universal node encodes the fused information from both the question
and passage, and plays an important role to predict whether the question is
answerable and also greatly improves the conciseness of the U-Net. Different
from the state-of-art pipeline models, U-Net can be learned in an end-to-end
fashion. The experimental results on the SQuAD 2.0 dataset show that U-Net can
effectively predict the unanswerability of questions and achieves an F1 score
of 71.7 on SQuAD 2.0.
Adversarial Text Generation Without Reinforcement Learning
Authors: David Donahue, Anna Rumshisky
Date: 2018-10-11 00:00:00
Tasks: Adversarial Text, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sentence, Text Generation

Generative Adversarial Networks (GANs) have experienced a recent surge in
popularity, performing competitively in a variety of tasks, especially in
computer vision. However, GAN training has shown limited success in natural
language processing. This is largely because sequences of text are discrete,
and thus gradients cannot propagate from the discriminator to the generator.
Recent solutions use reinforcement learning to propagate approximate gradients
to the generator, but this is inefficient to train. We propose to utilize an
autoencoder to learn a low-dimensional representation of sentences. A GAN is
then trained to generate its own vectors in this space, which decode to
realistic utterances. We report both random and interpolated samples from the
generator. Visualization of sentence vectors indicate our model correctly
learns the latent space of the autoencoder. Both human ratings and BLEU scores
show that our model generates realistic text against competitive baselines.
Exploring the Use of Attention within an Neural Machine Translation Decoder States to Translate Idioms
Authors: Giancarlo D. Salton, Robert J. Ross, John D. Kelleher
Date: 2018-10-10 00:00:00
Tasks: Decoder, Language Modelling, Machine Translation, Translation

Idioms pose problems to almost all Machine Translation systems. This type of
language is very frequent in day-to-day language use and cannot be simply
ignored. The recent interest in memory augmented models in the field of
Language Modelling has aided the systems to achieve good results by bridging
long-distance dependencies. In this paper we explore the use of such techniques
into a Neural Machine Translation system to help in translation of idiomatic
language.
Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks
Authors: Elena Smirnova, Flavian vasile
Date: 2017-06-23 00:00:00
Tasks: Session-Based Recommendations

Recommendations can greatly benefit from good representations of the user
state at recommendation time. Recent approaches that leverage Recurrent Neural
Networks (RNNs) for session-based recommendations have shown that Deep Learning
models can provide useful user representations for recommendation. However,
current RNN modeling approaches summarize the user state by only taking into
account the sequence of items that the user has interacted with in the past,
without taking into account other essential types of context information such
as the associated types of user-item interactions, the time gaps between events
and the time of day for each interaction. To address this, we propose a new
class of Contextual Recurrent Neural Networks for Recommendation (CRNNs) that
can take into account the contextual information both in the input and output
layers and modifying the behavior of the RNN by combining the context embedding
with the item embedding and more explicitly, in the model dynamics, by
parametrizing the hidden unit transitions as a function of context information.
We compare our CRNNs approach with RNNs and non-sequential baselines and show
good improvements on the next event prediction task.
Reverse engineering of CAD models via clustering and approximate implicitization
Authors: Andrea Raffo, Oliver J. D. Barrowclough, Georg Muntingh
Date: 2018-10-17 00:00:00
Tasks: Clustering

In applications like computer aided design, geometric models are often represented numerically as polynomial splines or NURBS, even when they originate from primitive geometry. For purposes such as redesign and isogeometric analysis, it is of interest to extract information about the underlying geometry through reverse engineering. In this work we develop a novel method to determine these primitive shapes by combining clustering analysis with approximate implicitization. The proposed method is automatic and can recover algebraic hypersurfaces of any degree in any dimension. In exact arithmetic, the algorithm returns exact results. All the required parameters, such as the implicit degree of the patches and the number of clusters of the model, are inferred using numerical approaches in order to obtain an algorithm that requires as little manual input as possible. The effectiveness, efficiency and robustness of the method are shown both in a theoretical analysis and in numerical examples implemented in Python.
Pruning Deep Neural Networks using Partial Least Squares
Authors: Artur Jordao, Ricardo Kloss, Fernando Yamada, William Robson Schwartz
Date: 2018-10-17 00:00:00
Tasks: feature selection

Modern pattern recognition methods are based on convolutional networks since they are able to learn complex patterns that benefit the classification. However, convolutional networks are computationally expensive and require a considerable amount of memory, which limits their deployment on low-power and resource-constrained systems. To handle these problems, recent approaches have proposed pruning strategies that find and remove unimportant neurons (i.e., filters) in these networks. Despite achieving remarkable results, existing pruning approaches are ineffective since the accuracy of the original network is degraded. In this work, we propose a novel approach to efficiently remove filters from convolutional networks. Our approach estimates the filter importance based on its relationship with the class label on a low-dimensional space. This relationship is computed using Partial Least Squares (PLS) and Variable Importance in Projection (VIP). Our method is able to reduce up to 67% of the floating point operations (FLOPs) without penalizing the network accuracy. With a negligible drop in accuracy, we can reduce up to 90% of FLOPs. Additionally, sometimes the method is even able to improve the accuracy compared to original, unpruned, network. We show that employing PLS+VIP as the criterion for detecting the filters to be removed is better than recent feature selection techniques, which have been employed by state-of-the-art pruning methods. Finally, we show that the proposed method achieves the highest FLOPs reduction and the smallest drop in accuracy when compared to state-of-the-art pruning approaches. Codes are available at: https://github.com/arturjordao/PruningNeuralNetworks
Orthogonal Deep Features Decomposition for Age-Invariant Face Recognition
Authors: Yitong Wang, Dihong Gong, Zheng Zhou, Xing Ji, Hao Wang, Zhifeng Li, Wei Liu, Tong Zhang
Date: 2018-10-17 00:00:00
Proceeding: ECCV 2018 9
Tasks: Age-Invariant Face Recognition, Benchmarking, Face Recognition, MORPH

As facial appearance is subject to significant intra-class variations caused
by the aging process over time, age-invariant face recognition (AIFR) remains a
major challenge in face recognition community. To reduce the intra-class
discrepancy caused by the aging, in this paper we propose a novel approach
(namely, Orthogonal Embedding CNNs, or OE-CNNs) to learn the age-invariant deep
face features. Specifically, we decompose deep face features into two
orthogonal components to represent age-related and identity-related features.
As a result, identity-related features that are robust to aging are then used
for AIFR. Besides, for complementing the existing cross-age datasets and
advancing the research in this field, we construct a brand-new large-scale
Cross-Age Face dataset (CAF). Extensive experiments conducted on the three
public domain face aging datasets (MORPH Album 2, CACD-VS and FG-NET) have
shown the effectiveness of the proposed approach and the value of the
constructed CAF dataset on AIFR. Benchmarking our algorithm on one of the most
popular general face recognition (GFR) dataset LFW additionally demonstrates
the comparable generalization performance on GFR.
An Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation
Authors: Gongbo Tang, Rico Sennrich, Joakim Nivre
Date: 2018-10-17 00:00:00
Proceeding: WS 2018 10
Tasks: Decoder, Machine Translation, NMT, Translation, Word Alignment, Word Sense Disambiguation

Recent work has shown that the encoder-decoder attention mechanisms in neural
machine translation (NMT) are different from the word alignment in statistical
machine translation. In this paper, we focus on analyzing encoder-decoder
attention mechanisms, in the case of word sense disambiguation (WSD) in NMT
models. We hypothesize that attention mechanisms pay more attention to context
tokens when translating ambiguous words. We explore the attention distribution
patterns when translating ambiguous nouns. Counter-intuitively, we find that
attention mechanisms are likely to distribute more attention to the ambiguous
noun itself rather than context tokens, in comparison to other nouns. We
conclude that attention mechanism is not the main mechanism used by NMT models
to incorporate contextual information for WSD. The experimental results suggest
that NMT models learn to encode contextual information necessary for WSD in the
encoder hidden states. For the attention mechanism in Transformer models, we
reveal that the first few layers gradually learn to "align" source and target
tokens and the last few layers learn to extract features from the related but
unaligned context tokens.
Graphical Convergence of Subgradients in Nonconvex Optimization and Learning
Authors: Damek Davis, Dmitriy Drusvyatskiy
Date: 2018-10-17 00:00:00
Tasks: regression, Stochastic Optimization

We investigate the stochastic optimization problem of minimizing population
risk, where the loss defining the risk is assumed to be weakly convex.
Compositions of Lipschitz convex functions with smooth maps are the primary
examples of such losses. We analyze the estimation quality of such nonsmooth
and nonconvex problems by their sample average approximations. Our main results
establish dimension-dependent rates on subgradient estimation in full
generality and dimension-independent rates when the loss is a generalized
linear model. As an application of the developed techniques, we analyze the
nonsmooth landscape of a robust nonlinear regression problem.
Efficient Proximal Mapping Computation for Unitarily Invariant Low-Rank Inducing Norms
Authors: Christian Grussler, Pontus Giselsson
Date: 2018-10-17 00:00:00

Low-rank inducing unitarily invariant norms have been introduced to convexify
problems with low-rank/sparsity constraint. They are the convex envelope of a
unitary invariant norm and the indicator function of an upper bounding rank
constraint. The most well-known member of this family is the so-called nuclear
norm. To solve optimization problems involving such norms with proximal
splitting methods, efficient ways of evaluating the proximal mapping of the
low-rank inducing norms are needed. This is known for the nuclear norm, but not
for most other members of the low-rank inducing family. This work supplies a
framework that reduces the proximal mapping evaluation into a nested binary
search, in which each iteration requires the solution of a much simpler
problem. This simpler problem can often be solved analytically as it is
demonstrated for the so-called low-rank inducing Frobenius and spectral norms.
Moreover, the framework allows to compute the proximal mapping of compositions
of these norms with increasing convex functions and the projections onto their
epigraphs. This has the additional advantage that we can also deal with
compositions of increasing convex functions and low-rank inducing norms in
proximal splitting methods.
On Evaluating Embedding Models for Knowledge Base Completion
Authors: Yanjie Wang, Daniel Ruffinelli, Rainer Gemulla, Samuel Broscheit, Christian Meilicke
Date: 2018-10-17 00:00:00
Proceeding: WS 2019 8
Tasks: Knowledge Base Completion, Question Answering

Knowledge bases contribute to many web search and mining tasks, yet they are
often incomplete. To add missing facts to a given knowledge base, various
embedding models have been proposed in the recent literature. Perhaps
surprisingly, relatively simple models with limited expressiveness often
performed remarkably well under today's most commonly used evaluation
protocols. In this paper, we explore whether recent models work well for
knowledge base completion and argue that the current evaluation protocols are
more suited for question answering rather than knowledge base completion. We
show that when focusing on a different prediction task for evaluating knowledge
base completion, the performance of current embedding models is unsatisfactory
even on datasets previously thought to be too easy. This is especially true
when embedding models are compared against a simple rule-based baseline. This
work indicates the need for more research into the embedding models and
evaluation protocols for knowledge base completion.
Learning and Tracking the 3D Body Shape of Freely Moving Infants from RGB-D sequences
Authors: Nikolas Hesse, Sergi Pujades, Michael J. Black, Michael Arens, Ulrich G. Hofmann, A. Sebastian Schroeder
Date: 2018-10-17 00:00:00

Statistical models of the human body surface are generally learned from
thousands of high-quality 3D scans in predefined poses to cover the wide
variety of human body shapes and articulations. Acquisition of such data
requires expensive equipment, calibration procedures, and is limited to
cooperative subjects who can understand and follow instructions, such as
adults. We present a method for learning a statistical 3D Skinned Multi-Infant
Linear body model (SMIL) from incomplete, low-quality RGB-D sequences of freely
moving infants. Quantitative experiments show that SMIL faithfully represents
the RGB-D data and properly factorizes the shape and pose of the infants. To
demonstrate the applicability of SMIL, we fit the model to RGB-D sequences of
freely moving infants and show, with a case study, that our method captures
enough motion detail for General Movements Assessment (GMA), a method used in
clinical practice for early detection of neurodevelopmental disorders in
infants. SMIL provides a new tool for analyzing infant shape and movement and
is a step towards an automated system for GMA.
Virtual Wave Optics for Non-Line-of-Sight Imaging
Authors: Xiaochun Liu, Ibón Guillén, Marco La Manna, Ji Hyun Nam, Syed Azer Reza, Toan Huu Le, Diego Gutierrez, Adrian Jarabo, Andreas Velten
Date: 2018-10-17 00:00:00

Non-Line-of-Sight (NLOS) imaging allows to observe objects partially or fully occluded from direct view, by analyzing indirect diffuse reflections off a secondary, relay surface. Despite its many potential applications, existing methods lack practical usability due to several shared limitations, including the assumption of single scattering only, lack of occlusions, and Lambertian reflectance. We lift these limitations by transforming the NLOS problem into a virtual Line-Of-Sight (LOS) one. Since imaging information cannot be recovered from the irradiance arriving at the relay surface, we introduce the concept of the phasor field, a mathematical construct representing a fast variation in irradiance. We show that NLOS light transport can be modeled as the propagation of a phasor field wave, which can be solved accurately by the Rayleigh-Sommerfeld diffraction integral. We demonstrate for the first time NLOS reconstruction of complex scenes with strong multiply scattered and ambient light, arbitrary materials, large depth range, and occlusions. Our method handles these challenging cases without explicitly developing a light transport model. By leveraging existing fast algorithms, we outperform existing methods in terms of execution speed, computational complexity, and memory use. We believe that our approach will help unlock the potential of NLOS imaging, and the development of novel applications not restricted to lab conditions. For example, we demonstrate both refocusing and transient NLOS videos of real-world, complex scenes with large depth.
Machine Common Sense Concept Paper
Authors: David Gunning
Date: 2018-10-17 00:00:00
Tasks: Common Sense Reasoning

This paper summarizes some of the technical background, research ideas, and
possible development strategies for achieving machine common sense. Machine
common sense has long been a critical-but-missing component of Artificial
Intelligence (AI). Recent advances in machine learning have resulted in new AI
capabilities, but in all of these applications, machine reasoning is narrow and
highly specialized. Developers must carefully train or program systems for
every situation. General commonsense reasoning remains elusive. The absence of
common sense prevents intelligent systems from understanding their world,
behaving reasonably in unforeseen situations, communicating naturally with
people, and learning from new experiences. Its absence is perhaps the most
significant barrier between the narrowly focused AI applications we have today
and the more general, human-like AI systems we would like to build in the
future. Machine common sense remains a broad, potentially unbounded problem in
AI. There are a wide range of strategies that could be employed to make
progress on this difficult challenge. This paper discusses two diverse
strategies for focusing development on two different machine commonsense
services: (1) a service that learns from experience, like a child, to construct
computational models that mimic the core domains of child cognition for objects
(intuitive physics), agents (intentional actors), and places (spatial
navigation); and (2) service that learns from reading the Web, like a research
librarian, to construct a commonsense knowledge repository capable of answering
natural language and image-based questions about commonsense phenomena.
When does Bone Suppression and Lung Field Segmentation Improve Chest X-Ray Disease Classification?
Authors: Ivo M. Baltruschat, Leonhard Steinmeister, Harald Ittrich, Gerhard Adam, Hannes Nickisch, Axel Saalbach, Jens von Berg, Michael Grass, Tobias Knopp
Date: 2018-10-17 00:00:00
Tasks: General Classification

Chest radiography is the most common clinical examination type. To improve
the quality of patient care and to reduce workload, methods for automatic
pathology classification have been developed. In this contribution we
investigate the usefulness of two advanced image pre-processing techniques,
initially developed for image reading by radiologists, for the performance of
Deep Learning methods. First, we use bone suppression, an algorithm to
artificially remove the rib cage. Secondly, we employ an automatic lung field
detection to crop the image to the lung area. Furthermore, we consider the
combination of both in the context of an ensemble approach. In a five-times
re-sampling scheme, we use Receiver Operating Characteristic (ROC) statistics
to evaluate the effect of the pre-processing approaches. Using a Convolutional
Neural Network (CNN), optimized for X-ray analysis, we achieve a good
performance with respect to all pathologies on average. Superior results are
obtained for selected pathologies when using pre-processing, i.e. for mass the
area under the ROC curve increased by 9.95%. The ensemble with pre-processed
trained models yields the best overall results.
Offline Signature Verification by Combining Graph Edit Distance and Triplet Networks
Authors: Paul Maergner, Vinaychandran Pondenkandath, Michele Alberti, Marcus Liwicki, Kaspar Riesen, Rolf Ingold, Andreas Fischer
Date: 2018-10-17 00:00:00
Tasks: Metric Learning, Triplet

Biometric authentication by means of handwritten signatures is a challenging
pattern recognition task, which aims to infer a writer model from only a
handful of genuine signatures. In order to make it more difficult for a forger
to attack the verification system, a promising strategy is to combine different
writer models. In this work, we propose to complement a recent structural
approach to offline signature verification based on graph edit distance with a
statistical approach based on metric learning with deep neural networks. On the
MCYT and GPDS benchmark datasets, we demonstrate that combining the structural
and statistical models leads to significant improvements in performance,
profiting from their complementary properties.
O2A: One-shot Observational learning with Action vectors
Authors: Leo Pauly, Wisdom C. Agboh, David C. Hogg, Raul Fuentes
Date: 2018-10-17 00:00:00
Tasks: One-Shot Learning, Reinforcement Learning

We present O2A, a novel method for learning to perform robotic manipulation tasks from a single (one-shot) third-person demonstration video. To our knowledge, it is the first time this has been done for a single demonstration. The key novelty lies in pre-training a feature extractor for creating a perceptual representation for actions that we call 'action vectors'. The action vectors are extracted using a 3D-CNN model pre-trained as an action classifier on a generic action dataset. The distance between the action vectors from the observed third-person demonstration and trial robot executions is used as a reward for reinforcement learning of the demonstrated task. We report on experiments in simulation and on a real robot, with changes in viewpoint of observation, properties of the objects involved, scene background and morphology of the manipulator between the demonstration and the learning domains. O2A outperforms baseline approaches under different domain shifts and has comparable performance with an oracle (that uses an ideal reward function).