pwc-archive/papers-with-abstracts huggingface dataset:
Sampling Superquadric Point Clouds with Normals
Authors: Paulo Ferreira
Date: 2018-02-14 00:00:00

Superquadrics provide a compact representation of common shapes and have been
used both for object/surface modelling in computer graphics and as object-part
representation in computer vision and robotics. Superquadrics refer to a family
of shapes: here we deal with the superellipsoids and superparaboloids. Due to
the strong non-linearities involved in the equations, uniform or
close-to-uniform sampling is not attainable through a naive approach of direct
sampling from the parametric formulation. This is specially true for more
`cubic' superquadrics (with shape parameters close to $0.1$). We extend a
previous solution of 2D close-to-uniform uniform sampling of superellipses to
the superellipsoid (3D) case and derive our own for the superparaboloid.
Additionally, we are able to provide normals for each sampled point. To the
best of our knowledge, this is the first complete approach for close-to-uniform
sampling of superellipsoids and superparaboloids in one single framework. We
present derivations, pseudocode and qualitative and quantitative results using
our code, which is available online.
Image Augmentation using Radial Transform for Training Deep Neural Networks
Authors: Hojjat Salehinejad, Shahrokh Valaee, Timothy Dowdell, Joseph Barfett
Date: 2017-08-14 00:00:00
Tasks: Deep Learning, Diversity, Image Augmentation

Deep learning models have a large number of free parameters that must be
estimated by efficient training of the models on a large number of training
data samples to increase their generalization performance. In real-world
applications, the data available to train these networks is often limited or
imbalanced. We propose a sampling method based on the radial transform in a
polar coordinate system for image augmentation to facilitate the training of
deep learning models from limited source data. This pixel-wise transform
provides representations of the original image in the polar coordinate system
by generating a new image from each pixel. This technique can generate radial
transformed images up to the number of pixels in the original image to increase
the diversity of poorly represented image classes. Our experiments show
improved generalization performance in training deep convolutional neural
networks with radial transformed images.
Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner
Authors: Emmanuel Dupoux
Date: 2016-07-29 00:00:00
Tasks: BIG-bench Machine Learning, Privacy Preserving

During their first years of life, infants learn the language(s) of their
environment at an amazing speed despite large cross cultural variations in
amount and complexity of the available language input. Understanding this
simple fact still escapes current cognitive and linguistic theories. Recently,
spectacular progress in the engineering science, notably, machine learning and
wearable technology, offer the promise of revolutionizing the study of
cognitive development. Machine learning offers powerful learning algorithms
that can achieve human-like performance on many linguistic tasks. Wearable
sensors can capture vast amounts of data, which enable the reconstruction of
the sensory experience of infants in their natural environment. The project of
'reverse engineering' language development, i.e., of building an effective
system that mimics infant's achievements appears therefore to be within reach.
Here, we analyze the conditions under which such a project can contribute to
our scientific understanding of early language development. We argue that
instead of defining a sub-problem or simplifying the data, computational models
should address the full complexity of the learning situation, and take as input
the raw sensory signals available to infants. This implies that (1) accessible
but privacy-preserving repositories of home data be setup and widely shared,
and (2) models be evaluated at different linguistic levels through a benchmark
of psycholinguist tests that can be passed by machines and humans alike, (3)
linguistically and psychologically plausible learning architectures be scaled
up to real data using probabilistic/optimization principles from machine
learning. We discuss the feasibility of this approach and present preliminary
results.
SCK: A sparse coding based key-point detector
Authors: Thanh Hong-Phuoc, Yifeng He, Ling Guan
Date: 2018-02-07 00:00:00

All current popular hand-crafted key-point detectors such as Harris corner,
MSER, SIFT, SURF... rely on some specific pre-designed structures for the
detection of corners, blobs, or junctions in an image. In this paper, a novel
sparse coding based key-point detector which requires no particular
pre-designed structures is presented. The key-point detector is based on
measuring the complexity level of each block in an image to decide where a
key-point should be. The complexity level of a block is defined as the total
number of non-zero components of a sparse representation of that block.
Generally, a block constructed with more components is more complex and has
greater potential to be a good key-point. Experimental results on Webcam and EF
datasets [1, 2] show that the proposed detector achieves significantly high
repeatability compared to hand-crafted features, and even outperforms the
matching scores of the state-of-the-art learning based detector.
Interpretable and Pedagogical Examples
Authors: Smitha Milli, Pieter Abbeel, Igor Mordatch
Date: 2017-11-02 00:00:00
Proceeding: ICLR 2018 1

Teachers intentionally pick the most informative examples to show their
students. However, if the teacher and student are neural networks, the examples
that the teacher network learns to give, although effective at teaching the
student, are typically uninterpretable. We show that training the student and
teacher iteratively, rather than jointly, can produce interpretable teaching
strategies. We evaluate interpretability by (1) measuring the similarity of the
teacher's emergent strategies to intuitive strategies in each domain and (2)
conducting human experiments to evaluate how effective the teacher's strategies
are at teaching humans. We show that the teacher network learns to select or
generate interpretable, pedagogical examples to teach rule-based,
probabilistic, boolean, and hierarchical concepts.
Maximizing the information learned from finite data selects a simple model
Authors: Henry H. Mattingly, Mark K. Transtrum, Michael C. Abbott, Benjamin B. Machta
Date: 2017-05-02 00:00:00

We use the language of uninformative Bayesian prior choice to study the
selection of appropriately simple effective models. We advocate for the prior
which maximizes the mutual information between parameters and predictions,
learning as much as possible from limited data. When many parameters are poorly
constrained by the available data, we find that this prior puts weight only on
boundaries of the parameter manifold. Thus it selects a lower-dimensional
effective theory in a principled way, ignoring irrelevant parameter directions.
In the limit where there is sufficient data to tightly constrain any number of
parameters, this reduces to Jeffreys prior. But we argue that this limit is
pathological when applied to the hyper-ribbon parameter manifolds generic in
science, because it leads to dramatic dependence on effects invisible to
experiment.
Morphologic for knowledge dynamics: revision, fusion, abduction
Authors: Isabelle Bloch, Jérôme Lang, Ramón Pino Pérez, Carlos Uzcátegui
Date: 2018-02-14 00:00:00

Several tasks in artificial intelligence require to be able to find models
about knowledge dynamics. They include belief revision, fusion and belief
merging, and abduction. In this paper we exploit the algebraic framework of
mathematical morphology in the context of propositional logic, and define
operations such as dilation or erosion of a set of formulas. We derive concrete
operators, based on a semantic approach, that have an intuitive interpretation
and that are formally well behaved, to perform revision, fusion and abduction.
Computation and tractability are addressed, and simple examples illustrate the
typical results that can be obtained.
Multi-Task Learning for Extraction of Adverse Drug Reaction Mentions from Tweets
Authors: Shashank Gupta, Manish Gupta, Vasudeva Varma, Sachin Pawar, Nitin Ramrakhiyani, Girish K. Palshikar
Date: 2018-02-14 00:00:00
Tasks: Event Detection, Multi-Task Learning

Adverse drug reactions (ADRs) are one of the leading causes of mortality in
health care. Current ADR surveillance systems are often associated with a
substantial time lag before such events are officially published. On the other
hand, online social media such as Twitter contain information about ADR events
in real-time, much before any official reporting. Current state-of-the-art in
ADR mention extraction uses Recurrent Neural Networks (RNN), which typically
need large labeled corpora. Towards this end, we propose a multi-task learning
based method which can utilize a similar auxiliary task (adverse drug event
detection) to enhance the performance of the main task, i.e., ADR extraction.
Furthermore, in the absence of auxiliary task dataset, we propose a novel joint
multi-task learning method to automatically generate weak supervision dataset
for the auxiliary task when a large pool of unlabeled tweets is available.
Experiments with 0.48M tweets show that the proposed approach outperforms the
state-of-the-art methods for the ADR mention extraction task by 7.2% in terms
of F1 score.
Co-training for Extraction of Adverse Drug Reaction Mentions from Tweets
Authors: Shashank Gupta, Manish Gupta, Vasudeva Varma, Sachin Pawar, Nitin Ramrakhiyani, Girish K. Palshikar
Date: 2018-02-14 00:00:00

Adverse drug reactions (ADRs) are one of the leading causes of mortality in
health care. Current ADR surveillance systems are often associated with a
substantial time lag before such events are officially published. On the other
hand, online social media such as Twitter contain information about ADR events
in real-time, much before any official reporting. Current state-of-the-art
methods in ADR mention extraction use Recurrent Neural Networks (RNN), which
typically need large labeled corpora. Towards this end, we propose a
semi-supervised method based on co-training which can exploit a large pool of
unlabeled tweets to augment the limited supervised training data, and as a
result enhance the performance. Experiments with 0.1M tweets show that the
proposed approach outperforms the state-of-the-art methods for the ADR mention
extraction task by 5% in terms of F1 score.
Automatic localization and decoding of honeybee markers using deep convolutional neural networks
Authors: Benjamin Wild, Leon Sixt, Tim Landgraf
Date: 2018-02-13 00:00:00

The honeybee is a fascinating model animal to investigate how collective
behavior emerges from (inter-)actions of thousands of individuals. Bees may
acquire unique memories throughout their lives. These experiences affect social
interactions even over large time frames. Tracking and identifying all bees in
the colony over their lifetimes therefore may likely shed light on the
interplay of individual differences and colony behavior. This paper proposes a
software pipeline based on two deep convolutional neural networks for the
localization and decoding of custom binary markers that honeybees carry from
their first to the last day in their life. We show that this approach
outperforms similar systems proposed in recent literature. By opening this
software for the public, we hope that the resulting datasets will help
advancing the understanding of honeybee collective intelligence.
Democratizing AI: Non-expert design of prediction tasks
Authors: James P. Bagrow
Date: 2018-02-14 00:00:00
Tasks: Feature Engineering, Prediction

Non-experts have long made important contributions to machine learning (ML) by contributing training data, and recent work has shown that non-experts can also help with feature engineering by suggesting novel predictive features. However, non-experts have only contributed features to prediction tasks already posed by experienced ML practitioners. Here we study how non-experts can design prediction tasks themselves, what types of tasks non-experts will design, and whether predictive models can be automatically trained on data sourced for their tasks. We use a crowdsourcing platform where non-experts design predictive tasks that are then categorized and ranked by the crowd. Crowdsourced data are collected for top-ranked tasks and predictive models are then trained and evaluated automatically using those data. We show that individuals without ML experience can collectively construct useful datasets and that predictive models can be learned on these datasets, but challenges remain. The prediction tasks designed by non-experts covered a broad range of domains, from politics and current events to health behavior, demographics, and more. Proper instructions are crucial for non-experts, so we also conducted a randomized trial to understand how different instructions may influence the types of prediction tasks being proposed. In general, understanding better how non-experts can contribute to ML can further leverage advances in Automatic ML and has important implications as ML continues to drive workplace automation.
Graph Convolutional Networks for Named Entity Recognition
Authors: A. Cetoli, S. Bragaglia, A. D. O'Harney, M. Sloan
Date: 2017-09-28 00:00:00
Tasks: Feature Engineering, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Sentence

In this paper we investigate the role of the dependency tree in a named
entity recognizer upon using a set of GCN. We perform a comparison among
different NER architectures and show that the grammar of a sentence positively
influences the results. Experiments on the ontonotes dataset demonstrate
consistent performance improvements, without requiring heavy feature
engineering nor additional language-specific knowledge.
Decoupled Weight Decay Regularization
Authors: Ilya Loshchilov, Frank Hutter
Date: 2017-11-14 00:00:00
Proceeding: ICLR 2019 5
Tasks: image-classification, Image Classification

L$_2$ regularization and weight decay regularization are equivalent for
standard stochastic gradient descent (when rescaled by the learning rate), but
as we demonstrate this is \emph{not} the case for adaptive gradient algorithms,
such as Adam. While common implementations of these algorithms employ L$_2$
regularization (often calling it "weight decay" in what may be misleading due
to the inequivalence we expose), we propose a simple modification to recover
the original formulation of weight decay regularization by \emph{decoupling}
the weight decay from the optimization steps taken w.r.t. the loss function. We
provide empirical evidence that our proposed modification (i) decouples the
optimal choice of weight decay factor from the setting of the learning rate for
both standard SGD and Adam and (ii) substantially improves Adam's
generalization performance, allowing it to compete with SGD with momentum on
image classification datasets (on which it was previously typically
outperformed by the latter). Our proposed decoupled weight decay has already
been adopted by many researchers, and the community has implemented it in
TensorFlow and PyTorch; the complete source code for our experiments is
available at https://github.com/loshchil/AdamW-and-SGDW
The Multiscale Bowler-Hat Transform for Vessel Enhancement in 3D Biomedical Images
Authors: Cigdem Sazak, Carl J. Nelson, Boguslaw Obara
Date: 2018-02-14 00:00:00

Enhancement and detection of 3D vessel-like structures has long been an open
problem as most existing image processing methods fail in many aspects,
including a lack of uniform enhancement between vessels of different radii and
a lack of enhancement at the junctions.
  Here, we propose a method based on mathematical morphology to enhance 3D
vessel-like structures in biomedical images. The proposed method, 3D bowler-hat
transform, combines sphere and line structuring elements to enhance vessel-like
structures. The proposed method is validated on synthetic and real data and
compared with state-of-the-art methods.
  Our results show that the proposed method achieves a high-quality vessel-like
structures enhancement in both synthetic and real biomedical images, and is
able to cope with variations in vessels thickness throughout vascular networks
while remaining robust at junctions.
Linguistic unit discovery from multi-modal inputs in unwritten languages: Summary of the "Speaking Rosetta" JSALT 2017 Workshop
Authors: Odette Scharenborg, Laurent Besacier, Alan Black, Mark Hasegawa-Johnson, Florian Metze, Graham Neubig, Sebastian Stueker, Pierre Godard, Markus Mueller, Lucas Ondel, Shruti Palaskar, Philip Arthur, Francesco Ciannella, Mingxing Du, Elin Larsen, Danny Merkx, Rachid Riad, Liming Wang, Emmanuel Dupoux
Date: 2018-02-14 00:00:00

We summarize the accomplishments of a multi-disciplinary workshop exploring
the computational and scientific issues surrounding the discovery of linguistic
units (subwords and words) in a language without orthography. We study the
replacement of orthographic transcriptions by images and/or translated text in
a well-resourced language to help unsupervised discovery from raw speech.
Automated software vulnerability detection with machine learning
Authors: Jacob A. Harer, Louis Y. Kim, Rebecca L. Russell, Onur Ozdemir, Leonard R. Kosta, Akshay Rangamani, Lei H. Hamilton, Gabriel I. Centeno, Jonathan R. Key, Paul M. Ellingwood, Erik Antelman, Alan Mackay, Marc W. McConley, Jeffrey M. Opper, Peter Chin, Tomo Lazovich
Date: 2018-02-14 00:00:00
Tasks: BIG-bench Machine Learning, Vulnerability Detection

Thousands of security vulnerabilities are discovered in production software
each year, either reported publicly to the Common Vulnerabilities and Exposures
database or discovered internally in proprietary code. Vulnerabilities often
manifest themselves in subtle ways that are not obvious to code reviewers or
the developers themselves. With the wealth of open source code available for
analysis, there is an opportunity to learn the patterns of bugs that can lead
to security vulnerabilities directly from data. In this paper, we present a
data-driven approach to vulnerability detection using machine learning,
specifically applied to C and C++ programs. We first compile a large dataset of
hundreds of thousands of open-source functions labeled with the outputs of a
static analyzer. We then compare methods applied directly to source code with
methods applied to artifacts extracted from the build process, finding that
source-based models perform better. We also compare the application of deep
neural network models with more traditional models such as random forests and
find the best performance comes from combining features learned by deep models
with tree-based models. Ultimately, our highest performing model achieves an
area under the precision-recall curve of 0.49 and an area under the ROC curve
of 0.87.
Scaling up the Automatic Statistician: Scalable Structure Discovery using Gaussian Processes
Authors: Hyunjik Kim, Yee Whye Teh
Date: 2017-06-08 00:00:00
Tasks: Gaussian Processes, Model Selection, regression

Automating statistical modelling is a challenging problem in artificial
intelligence. The Automatic Statistician takes a first step in this direction,
by employing a kernel search algorithm with Gaussian Processes (GP) to provide
interpretable statistical models for regression problems. However this does not
scale due to its $O(N^3)$ running time for the model selection. We propose
Scalable Kernel Composition (SKC), a scalable kernel search algorithm that
extends the Automatic Statistician to bigger data sets. In doing so, we derive
a cheap upper bound on the GP marginal likelihood that sandwiches the marginal
likelihood with the variational lower bound . We show that the upper bound is
significantly tighter than the lower bound and thus useful for model selection.
ICA based on Split Generalized Gaussian
Authors: P. Spurek, P. Rola, J. Tabor, A. Czechowski
Date: 2018-02-14 00:00:00

Independent Component Analysis (ICA) - one of the basic tools in data
analysis - aims to find a coordinate system in which the components of the data
are independent. Most popular ICA methods use kurtosis as a metric of
non-Gaussianity to maximize, such as FastICA and JADE. However, their
assumption of fourth-order moment (kurtosis) may not always be satisfied in
practice. One of the possible solution is to use third-order moment (skewness)
instead of kurtosis, which was applied in $ICA_{SG}$ and EcoICA.
  In this paper we present a competitive approach to ICA based on the Split
Generalized Gaussian distribution (SGGD), which is well adapted to heavy-tailed
as well as asymmetric data. Consequently, we obtain a method which works better
than the classical approaches, in both cases: heavy tails and non-symmetric
data. \end{abstract}
Robust Continuous Co-Clustering
Authors: Xiao He, Luis Moreira-Matias
Date: 2018-02-14 00:00:00
Tasks: Clustering

Clustering consists of grouping together samples giving their similar
properties. The problem of modeling simultaneously groups of samples and
features is known as Co-Clustering. This paper introduces ROCCO - a Robust
Continuous Co-Clustering algorithm. ROCCO is a scalable, hyperparameter-free,
easy and ready to use algorithm to address Co-Clustering problems in practice
over massive cross-domain datasets. It operates by learning a graph-based
two-sided representation of the input matrix. The underlying proposed
optimization problem is non-convex, which assures a flexible pool of solutions.
Moreover, we prove that it can be solved with a near linear time complexity on
the input size. An exhaustive large-scale experimental testbed conducted with
both synthetic and real-world datasets demonstrates ROCCO's properties in
practice: (i) State-of-the-art performance in cross-domain real-world problems
including Biomedicine and Text Mining; (ii) very low sensitivity to
hyperparameter settings; (iii) robustness to noise and (iv) a linear empirical
scalability in practice. These results highlight ROCCO as a powerful
general-purpose co-clustering algorithm for cross-domain practitioners,
regardless of their technical background.
Nonnegative PARAFAC2: a flexible coupling approach
Authors: Jeremy E. Cohen, Rasmus Bro
Date: 2018-02-14 00:00:00
Tasks: Tensor Decomposition

Modeling variability in tensor decomposition methods is one of the challenges
of source separation. One possible solution to account for variations from one
data set to another, jointly analysed, is to resort to the PARAFAC2 model.
However, so far imposing constraints on the mode with variability has not been
possible. In the following manuscript, a relaxation of the PARAFAC2 model is
introduced, that allows for imposing nonnegativity constraints on the varying
mode. An algorithm to compute the proposed flexible PARAFAC2 model is derived,
and its performance is studied on both synthetic and chemometrics data.