pwc-archive/papers-with-abstracts huggingface dataset:
The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities
Authors: Joel Lehman, Jeff Clune, Dusan Misevic, Christoph Adami, Lee Altenberg, Julie Beaulieu, Peter J. Bentley, Samuel Bernard, Guillaume Beslon, David M. Bryson, Patryk Chrabaszcz, Nick Cheney, Antoine Cully, Stephane Doncieux, Fred C. Dyer, Kai Olav Ellefsen, Robert Feldt, Stephan Fischer, Stephanie Forrest, Antoine Frénoy, Christian Gagné, Leni Le Goff, Laura M. Grabowski, Babak Hodjat, Frank Hutter, Laurent Keller, Carole Knibbe, Peter Krcah, Richard E. Lenski, Hod Lipson, Robert MacCurdy, Carlos Maestre, Risto Miikkulainen, Sara Mitri, David E. Moriarty, Jean-Baptiste Mouret, Anh Nguyen, Charles Ofria, Marc Parizeau, David Parsons, Robert T. Pennock, William F. Punch, Thomas S. Ray, Marc Schoenauer, Eric Shulte, Karl Sims, Kenneth O. Stanley, François Taddei, Danesh Tarapore, Simon Thibault, Westley Weimer, Richard Watson, Jason Yosinski
Date: 2018-03-09 00:00:00
Tasks: Artificial Life

Biological evolution provides a creative fount of complex and subtle adaptations, often surprising the scientists who discover them. However, because evolution is an algorithmic process that transcends the substrate in which it occurs, evolution's creativity is not limited to nature. Indeed, many researchers in the field of digital evolution have observed their evolving algorithms and organisms subverting their intentions, exposing unrecognized bugs in their code, producing unexpected adaptations, or exhibiting outcomes uncannily convergent with ones in nature. Such stories routinely reveal creativity by evolution in these digital worlds, but they rarely fit into the standard scientific narrative. Instead they are often treated as mere obstacles to be overcome, rather than results that warrant study in their own right. The stories themselves are traded among researchers through oral tradition, but that mode of information transmission is inefficient and prone to error and outright loss. Moreover, the fact that these stories tend to be shared only among practitioners means that many natural scientists do not realize how interesting and lifelike digital organisms are and how natural their evolution can be. To our knowledge, no collection of such anecdotes has been published before. This paper is the crowd-sourced product of researchers in the fields of artificial life and evolutionary computation who have provided first-hand accounts of such cases. It thus serves as a written, fact-checked collection of scientifically important and even entertaining stories. In doing so we also present here substantial evidence that the existence and importance of evolutionary surprises extends beyond the natural world, and may indeed be a universal property of all complex evolving systems.
Performance evaluation and hyperparameter tuning of statistical and machine-learning models using spatial data
Authors: Patrick Schratz, Jannes Muenchow, Eugenia Iturritxa, Jakob Richter, Alexander Brenning
Date: 2018-03-29 00:00:00
Tasks: Additive models, BIG-bench Machine Learning, regression

Machine-learning algorithms have gained popularity in recent years in the
field of ecological modeling due to their promising results in predictive
performance of classification problems. While the application of such
algorithms has been highly simplified in the last years due to their
well-documented integration in commonly used statistical programming languages
such as R, there are several practical challenges in the field of ecological
modeling related to unbiased performance estimation, optimization of algorithms
using hyperparameter tuning and spatial autocorrelation. We address these
issues in the comparison of several widely used machine-learning algorithms
such as Boosted Regression Trees (BRT), k-Nearest Neighbor (WKNN), Random
Forest (RF) and Support Vector Machine (SVM) to traditional parametric
algorithms such as logistic regression (GLM) and semi-parametric ones like
generalized additive models (GAM). Different nested cross-validation methods
including hyperparameter tuning methods are used to evaluate model performances
with the aim to receive bias-reduced performance estimates. As a case study the
spatial distribution of forest disease Diplodia sapinea in the Basque Country
in Spain is investigated using common environmental variables such as
temperature, precipitation, soil or lithology as predictors. Results show that
GAM and RF (mean AUROC estimates 0.708 and 0.699) outperform all other methods
in predictive accuracy. The effect of hyperparameter tuning saturates at around
50 iterations for this data set. The AUROC differences between the bias-reduced
(spatial cross-validation) and overoptimistic (non-spatial cross-validation)
performance estimates of the GAM and RF are 0.167 (24%) and 0.213 (30%),
respectively. It is recommended to also use spatial partitioning for
cross-validation hyperparameter tuning of spatial data.
A Common Framework for Natural Gradient and Taylor based Optimisation using Manifold Theory
Authors: Adnan Haider
Date: 2018-03-26 00:00:00

This technical report constructs a theoretical framework to relate standard
Taylor approximation based optimisation methods with Natural Gradient (NG), a
method which is Fisher efficient with probabilistic models. Such a framework
will be shown to also provide mathematical justification to combine higher
order methods with the method of NG.
DIY Human Action Data Set Generation
Authors: Mehran Khodabandeh, Hamid Reza Vaezi Joze, Ilya Zharkov, Vivek Pradeep
Date: 2018-03-29 00:00:00
Tasks: Action Recognition, Temporal Action Localization, Video Generation

The recent successes in applying deep learning techniques to solve standard
computer vision problems has aspired researchers to propose new computer vision
problems in different domains. As previously established in the field, training
data itself plays a significant role in the machine learning process,
especially deep learning approaches which are data hungry. In order to solve
each new problem and get a decent performance, a large amount of data needs to
be captured which may in many cases pose logistical difficulties. Therefore,
the ability to generate de novo data or expand an existing data set, however
small, in order to satisfy data requirement of current networks may be
invaluable. Herein, we introduce a novel way to partition an action video clip
into action, subject and context. Each part is manipulated separately and
reassembled with our proposed video generation technique. Furthermore, our
novel human skeleton trajectory generation along with our proposed video
generation technique, enables us to generate unlimited action recognition
training data. These techniques enables us to generate video action clips from
an small set without costly and time-consuming data acquisition. Lastly, we
prove through extensive set of experiments on two small human action
recognition data sets, that this new data generation technique can improve the
performance of current action recognition neural nets.
Joint Causal Inference from Multiple Contexts
Authors: Joris M. Mooij, Sara Magliacane, Tom Claassen
Date: 2016-11-30 00:00:00
Tasks: Causal Discovery, Causal Inference

The gold standard for discovering causal relations is by means of experimentation. Over the last decades, alternative methods have been proposed that can infer causal relations between variables from certain statistical patterns in purely observational data. We introduce Joint Causal Inference (JCI), a novel approach to causal discovery from multiple data sets from different contexts that elegantly unifies both approaches. JCI is a causal modeling framework rather than a specific algorithm, and it can be implemented using any causal discovery algorithm that can take into account certain background knowledge. JCI can deal with different types of interventions (e.g., perfect, imperfect, stochastic, etc.) in a unified fashion, and does not require knowledge of intervention targets or types in case of interventional data. We explain how several well-known causal discovery algorithms can be seen as addressing special cases of the JCI framework, and we also propose novel implementations that extend existing causal discovery methods for purely observational data to the JCI setting. We evaluate different JCI implementations on synthetic data and on flow cytometry protein expression data and conclude that JCI implementations can considerably outperform state-of-the-art causal discovery algorithms.
How an Electrical Engineer Became an Artificial Intelligence Researcher, a Multiphase Active Contours Analysis
Authors: Kush R. Varshney
Date: 2018-03-29 00:00:00
Tasks: Electrical Engineering, Interpretable Machine Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This essay examines how what is considered to be artificial intelligence (AI)
has changed over time and come to intersect with the expertise of the author.
Initially, AI developed on a separate trajectory, both topically and
institutionally, from pattern recognition, neural information processing,
decision and control systems, and allied topics by focusing on symbolic systems
within computer science departments rather than on continuous systems in
electrical engineering departments. The separate evolutions continued
throughout the author's lifetime, with some crossover in reinforcement learning
and graphical models, but were shocked into converging by the virality of deep
learning, thus making an electrical engineer into an AI researcher. Now that
this convergence has happened, opportunity exists to pursue an agenda that
combines learning and reasoning bridged by interpretable machine learning
models.
Detection, localisation and tracking of pallets using machine learning techniques and 2D range data
Authors: Ihab S. Mohamed, Alessio Capitanelli, Fulvio Mastrogiovanni, Stefano Rovetta, Renato Zaccaria
Date: 2018-03-29 00:00:00
Tasks: BIG-bench Machine Learning

The problem of autonomous transportation in industrial scenarios is receiving
a renewed interest due to the way it can revolutionise internal logistics,
especially in unstructured environments. This paper presents a novel
architecture allowing a robot to detect, localise, and track (possibly
multiple) pallets using machine learning techniques based on an on-board 2D
laser rangefinder only. The architecture is composed of two main components:
the first stage is a pallet detector employing a Faster Region-based
Convolutional Neural Network (Faster R-CNN) detector cascaded with a CNN-based
classifier; the second stage is a Kalman filter for localising and tracking
detected pallets, which we also use to defer commitment to a pallet detected in
the first stage until sufficient confidence has been acquired via a sequential
data acquisition process. For fine-tuning the CNNs, the architecture has been
systematically evaluated using a real-world dataset containing 340 labeled 2D
scans, which have been made freely available in an online repository. Detection
performance has been assessed on the basis of the average accuracy over k-fold
cross-validation, and it scored 99.58% in our tests. Concerning pallet
localisation and tracking, experiments have been performed in a scenario where
the robot is approaching the pallet to fork. Although data have been originally
acquired by considering only one pallet as per specification of the use case we
consider, artificial data have been generated as well to mimic the presence of
multiple pallets in the robot workspace. Our experimental results confirm that
the system is capable of identifying, localising and tracking pallets with a
high success rate while being robust to false positives.
Relative Error Tensor Low Rank Approximation
Authors: Zhao Song, David P. Woodruff, Peilin Zhong
Date: 2017-04-26 00:00:00

We consider relative error low rank approximation of $tensors$ with respect
to the Frobenius norm: given an order-$q$ tensor $A \in
\mathbb{R}^{\prod_{i=1}^q n_i}$, output a rank-$k$ tensor $B$ for which
$\|A-B\|_F^2 \leq (1+\epsilon)$OPT, where OPT $= \inf_{\textrm{rank-}k~A'}
\|A-A'\|_F^2$. Despite the success on obtaining relative error low rank
approximations for matrices, no such results were known for tensors. One
structural issue is that there may be no rank-$k$ tensor $A_k$ achieving the
above infinum. Another, computational issue, is that an efficient relative
error low rank approximation algorithm for tensors would allow one to compute
the rank of a tensor, which is NP-hard. We bypass these issues via (1)
bicriteria and (2) parameterized complexity solutions:
  (1) We give an algorithm which outputs a rank $k' = O((k/\epsilon)^{q-1})$
tensor $B$ for which $\|A-B\|_F^2 \leq (1+\epsilon)$OPT in $nnz(A) + n \cdot
\textrm{poly}(k/\epsilon)$ time in the real RAM model. Here $nnz(A)$ is the
number of non-zero entries in $A$.
  (2) We give an algorithm for any $\delta >0$ which outputs a rank $k$ tensor
$B$ for which $\|A-B\|_F^2 \leq (1+\epsilon)$OPT and runs in $ ( nnz(A) + n
\cdot \textrm{poly}(k/\epsilon) + \exp(k^2/\epsilon) ) \cdot n^\delta$ time in
the unit cost RAM model.
  For outputting a rank-$k$ tensor, or even a bicriteria solution with
rank-$Ck$ for a certain constant $C > 1$, we show a $2^{\Omega(k^{1-o(1)})}$
time lower bound under the Exponential Time Hypothesis.
  Our results give the first relative error low rank approximations for tensors
for a large number of robust error measures for which nothing was known, as
well as column row and tube subset selection. We also obtain new results for
matrices, such as $nnz(A)$-time CUR decompositions, improving previous
$nnz(A)\log n$-time algorithms, which may be of independent interest.
Exploiting gradients and Hessians in Bayesian optimization and Bayesian quadrature
Authors: Anqi Wu, Mikio C. Aoi, Jonathan W. Pillow
Date: 2017-03-31 00:00:00
Tasks: Bayesian Optimization, Gaussian Processes

An exciting branch of machine learning research focuses on methods for
learning, optimizing, and integrating unknown functions that are difficult or
costly to evaluate. A popular Bayesian approach to this problem uses a Gaussian
process (GP) to construct a posterior distribution over the function of
interest given a set of observed measurements, and selects new points to
evaluate using the statistics of this posterior. Here we extend these methods
to exploit derivative information from the unknown function. We describe
methods for Bayesian optimization (BO) and Bayesian quadrature (BQ) in settings
where first and second derivatives may be evaluated along with the function
itself. We perform sampling-based inference in order to incorporate uncertainty
over hyperparameters, and show that both hyperparameter and function
uncertainty decrease much more rapidly when using derivative information.
Moreover, we introduce techniques for overcoming ill-conditioning issues that
have plagued earlier methods for gradient-enhanced Gaussian processes and
kriging. We illustrate the efficacy of these methods using applications to real
and simulated Bayesian optimization and quadrature problems, and show that
exploting derivatives can provide substantial gains over standard methods.
Improve the performance of transfer learning without fine-tuning using dissimilarity-based multi-view learning for breast cancer histology images
Authors: Hongliu Cao, Simon Bernard, Laurent Heutte, Robert Sabourin
Date: 2018-03-29 00:00:00
Tasks: Deep Learning, MULTI-VIEW LEARNING, Transfer Learning

Breast cancer is one of the most common types of cancer and leading
cancer-related death causes for women. In the context of ICIAR 2018 Grand
Challenge on Breast Cancer Histology Images, we compare one handcrafted feature
extractor and five transfer learning feature extractors based on deep learning.
We find out that the deep learning networks pretrained on ImageNet have better
performance than the popular handcrafted features used for breast cancer
histology images. The best feature extractor achieves an average accuracy of
79.30%. To improve the classification performance, a random forest
dissimilarity based integration method is used to combine different feature
groups together. When the five deep learning feature groups are combined, the
average accuracy is improved to 82.90% (best accuracy 85.00%). When handcrafted
features are combined with the five deep learning feature groups, the average
accuracy is improved to 87.10% (best accuracy 93.00%).
Convolutional Neural Knowledge Graph Learning
Authors: Feipeng Zhao, Martin Renqiang Min, Chen Shen, Amit Chakraborty
Date: 2017-10-23 00:00:00
Tasks: Entity Embeddings, Graph Learning, Knowledge Graphs, Triplet

Previous models for learning entity and relationship embeddings of knowledge
graphs such as TransE, TransH, and TransR aim to explore new links based on
learned representations. However, these models interpret relationships as
simple translations on entity embeddings. In this paper, we try to learn more
complex connections between entities and relationships. In particular, we use a
Convolutional Neural Network (CNN) to learn entity and relationship
representations in knowledge graphs. In our model, we treat entities and
relationships as one-dimensional numerical sequences with the same length.
After that, we combine each triplet of head, relationship, and tail together as
a matrix with height 3. CNN is applied to the triplets to get confidence
scores. Positive and manually corrupted negative triplets are used to train the
embeddings and the CNN model simultaneously. Experimental results on public
benchmark datasets show that the proposed model outperforms state-of-the-art
models on exploring unseen relationships, which proves that CNN is effective to
learn complex interactive patterns between entities and relationships.
MoNet: Moments Embedding Network
Authors: Mengran Gou, Fei Xiong, Octavia Camps, Mario Sznaier
Date: 2018-02-20 00:00:00
Proceeding: CVPR 2018 6

Bilinear pooling has been recently proposed as a feature encoding layer,
which can be used after the convolutional layers of a deep network, to improve
performance in multiple vision tasks. Different from conventional global
average pooling or fully connected layer, bilinear pooling gathers 2nd order
information in a translation invariant fashion. However, a serious drawback of
this family of pooling layers is their dimensionality explosion. Approximate
pooling methods with compact properties have been explored towards resolving
this weakness. Additionally, recent results have shown that significant
performance gains can be achieved by adding 1st order information and applying
matrix normalization to regularize unstable higher order information. However,
combining compact pooling with matrix normalization and other order information
has not been explored until now. In this paper, we unify bilinear pooling and
the global Gaussian embedding layers through the empirical moment matrix. In
addition, we propose a novel sub-matrix square-root layer, which can be used to
normalize the output of the convolution layer directly and mitigate the
dimensionality problem with off-the-shelf compact pooling methods. Our
experiments on three widely used fine-grained classification datasets
illustrate that our proposed architecture, MoNet, can achieve similar or better
performance than with the state-of-art G2DeNet. Furthermore, when combined with
compact pooling technique, MoNet obtains comparable performance with encoded
features with 96% less dimensions.
Euphrates: Algorithm-SoC Co-Design for Low-Power Mobile Continuous Vision
Authors: Yuhao Zhu, Anand Samajdar, Matthew Mattina, Paul Whatmough
Date: 2018-03-29 00:00:00

Continuous computer vision (CV) tasks increasingly rely on convolutional
neural networks (CNN). However, CNNs have massive compute demands that far
exceed the performance and energy constraints of mobile devices. In this paper,
we propose and develop an algorithm-architecture co-designed system, Euphrates,
that simultaneously improves the energy-efficiency and performance of
continuous vision tasks.
  Our key observation is that changes in pixel data between consecutive frames
represents visual motion. We first propose an algorithm that leverages this
motion information to relax the number of expensive CNN inferences required by
continuous vision applications. We co-design a mobile System-on-a-Chip (SoC)
architecture to maximize the efficiency of the new algorithm. The key to our
architectural augmentation is to co-optimize different SoC IP blocks in the
vision pipeline collectively. Specifically, we propose to expose the motion
data that is naturally generated by the Image Signal Processor (ISP) early in
the vision pipeline to the CNN engine. Measurement and synthesis results show
that Euphrates achieves up to 66% SoC-level energy savings (4 times for the
vision computations), with only 1% accuracy loss.
Recurrent Slice Networks for 3D Segmentation of Point Clouds
Authors: Qiangui Huang, Weiyue Wang, Ulrich Neumann
Date: 2018-02-13 00:00:00
Proceeding: CVPR 2018 6
Tasks: Semantic Segmentation

Point clouds are an efficient data format for 3D data. However, existing 3D
segmentation methods for point clouds either do not model local dependencies
\cite{pointnet} or require added computations \cite{kd-net,pointnet2}. This
work presents a novel 3D segmentation framework, RSNet\footnote{Codes are
released here https://github.com/qianguih/RSNet}, to efficiently model local
structures in point clouds. The key component of the RSNet is a lightweight
local dependency module. It is a combination of a novel slice pooling layer,
Recurrent Neural Network (RNN) layers, and a slice unpooling layer. The slice
pooling layer is designed to project features of unordered points onto an
ordered sequence of feature vectors so that traditional end-to-end learning
algorithms (RNNs) can be applied. The performance of RSNet is validated by
comprehensive experiments on the S3DIS\cite{stanford}, ScanNet\cite{scannet},
and ShapeNet \cite{shapenet} datasets. In its simplest form, RSNets surpass all
previous state-of-the-art methods on these benchmarks. And comparisons against
previous state-of-the-art methods \cite{pointnet, pointnet2} demonstrate the
efficiency of RSNets.
AI Blue Book: Vehicle Price Prediction using Visual Features
Authors: Richard R. Yang, Steven Chen, Edward Chou
Date: 2018-03-29 00:00:00
Tasks: General Classification, Prediction, regression, Transfer Learning

In this work, we build a series of machine learning models to predict the
price of a product given its image, and visualize the features that result in
higher or lower price predictions. We collect two novel datasets of product
images and their MSRP prices for this purpose: a bicycle dataset and a car
dataset. We set baselines for price regression using linear regression on
histogram of oriented gradients (HOG) and convolutional neural network (CNN)
features, and a baseline for price segment classification using a multiclass
SVM. For our main models, we train several deep CNNs using both transfer
learning and our own architectures, for both regression and classification. We
achieve strong results on both datasets, with deep CNNs significantly
outperforming other models in a variety of metrics. Finally, we use several
recently-developed methods to visualize the image features that result in
higher or lower prices.