pwc-archive/papers-with-abstracts huggingface dataset:
Right for the Right Reason: Training Agnostic Networks
Authors: Sen Jia, Thomas Lansdall-Welfare, Nello Cristianini
Date: 2018-06-16 00:00:00
Tasks: Domain Adaptation

We consider the problem of a neural network being requested to classify
images (or other inputs) without making implicit use of a "protected concept",
that is a concept that should not play any role in the decision of the network.
Typically these concepts include information such as gender or race, or other
contextual information such as image backgrounds that might be implicitly
reflected in unknown correlations with other variables, making it insufficient
to simply remove them from the input features. In other words, making accurate
predictions is not good enough if those predictions rely on information that
should not be used: predictive performance is not the only important metric for
learning systems. We apply a method developed in the context of domain
adaptation to address this problem of "being right for the right reason", where
we request a classifier to make a decision in a way that is entirely 'agnostic'
to a given protected concept (e.g. gender, race, background etc.), even if this
could be implicitly reflected in other attributes via unknown correlations.
After defining the concept of an 'agnostic model', we demonstrate how the
Domain-Adversarial Neural Network can remove unwanted information from a model
using a gradient reversal layer.
DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
Authors: Xue Bin Peng, Pieter Abbeel, Sergey Levine, Michiel Van de Panne
Date: 2018-04-08 00:00:00
Tasks: Deep Reinforcement Learning, Motion Synthesis, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

A longstanding goal in character animation is to combine data-driven
specification of behavior with a system that can execute a similar behavior in
a physical simulation, thus enabling realistic responses to perturbations and
environmental variation. We show that well-known reinforcement learning (RL)
methods can be adapted to learn robust control policies capable of imitating a
broad range of example motion clips, while also learning complex recoveries,
adapting to changes in morphology, and accomplishing user-specified goals. Our
method handles keyframed motions, highly-dynamic actions such as
motion-captured flips and spins, and retargeted motions. By combining a
motion-imitation objective with a task objective, we can train characters that
react intelligently in interactive settings, e.g., by walking in a desired
direction or throwing a ball at a user-specified target. This approach thus
combines the convenience and motion quality of using motion clips to define the
desired style and appearance, with the flexibility and generality afforded by
RL methods and physics-based animation. We further explore a number of methods
for integrating multiple clips into the learning process to develop
multi-skilled agents capable of performing a rich repertoire of diverse skills.
We demonstrate results using multiple characters (human, Atlas robot, bipedal
dinosaur, dragon) and a large variety of skills, including locomotion,
acrobatics, and martial arts.
Latent Convolutional Models
Authors: ShahRukh Athar, Evgeny Burnaev, Victor Lempitsky
Date: 2018-06-16 00:00:00
Proceeding: ICLR 2019 5
Tasks: Colorization, Image Restoration

We present a new latent model of natural images that can be learned on
large-scale datasets. The learning process provides a latent embedding for
every image in the training dataset, as well as a deep convolutional network
that maps the latent space to the image space. After training, the new model
provides a strong and universal image prior for a variety of image restoration
tasks such as large-hole inpainting, superresolution, and colorization. To
model high-resolution natural images, our approach uses latent spaces of very
high dimensionality (one to two orders of magnitude higher than previous latent
image models). To tackle this high dimensionality, we use latent spaces with a
special manifold structure (convolutional manifolds) parameterized by a ConvNet
of a certain architecture. In the experiments, we compare the learned latent
models with latent models learned by autoencoders, advanced variants of
generative adversarial networks, and a strong baseline system using simpler
parameterization of the latent space. Our model outperforms the competing
approaches over a range of restoration tasks.
First and Second Order Methods for Online Convolutional Dictionary Learning
Authors: Jialin Liu, Cristina Garcia-Cardona, Brendt Wohlberg, Wotao Yin
Date: 2017-08-31 00:00:00
Tasks: Dictionary Learning, Second-order methods, Translation

Convolutional sparse representations are a form of sparse representation with
a structured, translation invariant dictionary. Most convolutional dictionary
learning algorithms to date operate in batch mode, requiring simultaneous
access to all training images during the learning process, which results in
very high memory usage and severely limits the training data that can be used.
Very recently, however, a number of authors have considered the design of
online convolutional dictionary learning algorithms that offer far better
scaling of memory and computational cost with training set size than batch
methods. This paper extends our prior work, improving a number of aspects of
our previous algorithm; proposing an entirely new one, with better performance,
and that supports the inclusion of a spatial mask for learning from incomplete
data; and providing a rigorous theoretical analysis of these methods.
Stable Prediction across Unknown Environments
Authors: Kun Kuang, Ruoxuan Xiong, Peng Cui, Susan Athey, Bo Li
Date: 2018-06-16 00:00:00
Tasks: feature selection, Prediction

In many important machine learning applications, the training distribution
used to learn a probabilistic classifier differs from the testing distribution
on which the classifier will be used to make predictions. Traditional methods
correct the distribution shift by reweighting the training data with the ratio
of the density between test and training data. In many applications training
takes place without prior knowledge of the testing distribution on which the
algorithm will be applied in the future. Recently, methods have been proposed
to address the shift by learning causal structure, but those methods rely on
the diversity of multiple training data to a good performance, and have
complexity limitations in high dimensions. In this paper, we propose a novel
Deep Global Balancing Regression (DGBR) algorithm to jointly optimize a deep
auto-encoder model for feature selection and a global balancing model for
stable prediction across unknown environments. The global balancing model
constructs balancing weights that facilitate estimating of partial effects of
features (holding fixed all other features), a problem that is challenging in
high dimensions, and thus helps to identify stable, causal relationships
between features and outcomes. The deep auto-encoder model is designed to
reduce the dimensionality of the feature space, thus making global balancing
easier. We show, both theoretically and with empirical experiments, that our
algorithm can make stable predictions across unknown environments. Our
experiments on both synthetic and real world datasets demonstrate that our DGBR
algorithm outperforms the state-of-the-art methods for stable prediction across
unknown environments.
Extraction Of Technical Information From Normative Documents Using Automated Methods Based On Ontologies : Application To The Iso 15531 Mandate Standard - Methodology And First Results
Authors: A. F. Cutting-Decelle, A. Digeon, R. I. Young, J. L. Barraud, P. Lamboley
Date: 2018-06-06 00:00:00

Problems faced by international standardization bodies become more and more
crucial as the number and the size of the standards they produce increase.
Sometimes, also, the lack of coordination among the committees in charge of the
development of standards may lead to overlaps, mistakes or incompatibilities in
the documents. The aim of this study is to present a methodology enabling an
automatic extraction of the technical concepts (terms) found in normative
documents, through the use of semantic tools coming from the field of language
processing. The first part of the paper provides a description of the
standardization world, its structure, its way of working and the problems
faced; we then introduce the concepts of semantic annotation, information
extraction and the software tools available in this domain. The next section
explains the concept of ontology and its potential use in the field of
standardization. We propose here a methodology enabling the extraction of
technical information from a given normative corpus, based on a semantic
annotation process done according to reference ontologies. The application to
the ISO 15531 MANDATE corpus provides a first use case of the methodology
described in this paper. The paper ends with the description of the first
experimental results produced by this approach, and with some issues and
perspectives, notably its application to other standards and, or Technical
Committees and the possibility offered to create pre-defined technical
dictionaries of terms.
On Strategyproof Conference Peer Review
Authors: Yichong Xu, Han Zhao, Xiaofei Shi, Jeremy Zhang, Nihar B. Shah
Date: 2018-06-16 00:00:00

We consider peer review in a conference setting where there is typically an overlap between the set of reviewers and the set of authors. This overlap can incentivize strategic reviews to influence the final ranking of one's own papers. In this work, we address this problem through the lens of social choice, and present a theoretical framework for strategyproof and efficient peer review. We first present and analyze an algorithm for reviewer-assignment and aggregation that guarantees strategyproofness and a natural efficiency property called unanimity, when the authorship graph satisfies a simple property. Our algorithm is based on the so-called partitioning method, and can be thought as a generalization of this method to conference peer review settings. We then empirically show that the requisite property on the authorship graph is indeed satisfied in the submission data from the ICLR conference, and further demonstrate a simple trick to make the partitioning method more practically appealing for conference peer review. Finally, we complement our positive results with negative theoretical results where we prove that under various ways of strengthening the requirements, it is impossible for any algorithm to be strategyproof and efficient.
Evaluation of sentence embeddings in downstream and linguistic probing tasks
Authors: Christian S. Perone, Roberto Silveira, Thomas S. Paula
Date: 2018-06-16 00:00:00
Tasks: Language Modeling, Language Modelling, Sentence, Sentence Embedding, Sentence-Embedding, Sentence Embeddings, Word Embeddings

Despite the fast developmental pace of new sentence embedding methods, it is
still challenging to find comprehensive evaluations of these different
techniques. In the past years, we saw significant improvements in the field of
sentence embeddings and especially towards the development of universal
sentence encoders that could provide inductive transfer to a wide variety of
downstream tasks. In this work, we perform a comprehensive evaluation of recent
methods using a wide variety of downstream and linguistic feature probing
tasks. We show that a simple approach using bag-of-words with a recently
introduced language model for deep context-dependent word embeddings proved to
yield better results in many tasks when compared to sentence encoders trained
on entailment datasets. We also show, however, that we are still far away from
a universal encoder that can perform consistently across several downstream
tasks.
DynMat, a network that can learn after learning
Authors: Jung H. Lee
Date: 2018-06-16 00:00:00

To survive in the dynamically-evolving world, we accumulate knowledge and
improve our skills based on experience. In the process, gaining new knowledge
does not disrupt our vigilance to external stimuli. In other words, our
learning process is 'accumulative' and 'online' without interruption. However,
despite the recent success, artificial neural networks (ANNs) must be trained
offline, and they suffer catastrophic interference between old and new
learning, indicating that ANNs' conventional learning algorithms may not be
suitable for building intelligent agents comparable to our brain. In this
study, we propose a novel neural network architecture (DynMat) consisting of
dual learning systems, inspired by the complementary learning system (CLS)
theory suggesting that the brain relies on short- and long-term learning
systems to learn continuously. Our experiments show that 1) DynMat can learn a
new class without catastrophic interference and 2) it does not strictly require
offline training.
Neyman-Pearson classification: parametrics and sample size requirement
Authors: Xin Tong, Lucy Xia, Jiacheng Wang, Yang Feng
Date: 2018-02-07 00:00:00
Tasks: Binary Classification, Classification, General Classification, Spam detection, Vocal Bursts Type Prediction

The Neyman-Pearson (NP) paradigm in binary classification seeks classifiers that achieve a minimal type II error while enforcing the prioritized type I error controlled under some user-specified level $\alpha$. This paradigm serves naturally in applications such as severe disease diagnosis and spam detection, where people have clear priorities among the two error types. Recently, Tong, Feng and Li (2018) proposed a nonparametric umbrella algorithm that adapts all scoring-type classification methods (e.g., logistic regression, support vector machines, random forest) to respect the given type I error upper bound $\alpha$ with high probability, without specific distributional assumptions on the features and the responses. Universal the umbrella algorithm is, it demands an explicit minimum sample size requirement on class $0$, which is often the more scarce class, such as in rare disease diagnosis applications. In this work, we employ the parametric linear discriminant analysis (LDA) model and propose a new parametric thresholding algorithm, which does not need the minimum sample size requirements on class $0$ observations and thus is suitable for small sample applications such as rare disease diagnosis. Leveraging both the existing nonparametric and the newly proposed parametric thresholding rules, we propose four LDA-based NP classifiers, for both low- and high-dimensional settings. On the theoretical front, we prove NP oracle inequalities for one proposed classifier, where the rate for excess type II error benefits from the explicit parametric model assumption. Furthermore, as NP classifiers involve a sample splitting step of class $0$ observations, we construct a new adaptive sample splitting scheme that can be applied universally to NP classifiers, and this adaptive strategy reduces the type II error of these classifiers.
Real-time Prediction of Segmentation Quality
Authors: Robert Robinson, Ozan Oktay, Wenjia Bai, Vanya Valindria, Mihir Sanghvi, Nay Aung, José Paiva, Filip Zemrak, Kenneth Fung, Elena Lukaschuk, Aaron Lee, Valentina Carapella, Young Jin Kim, Bernhard Kainz, Stefan Piechnik, Stefan Neubauer, Steffen Petersen, Chris Page, Daniel Rueckert, Ben Glocker
Date: 2018-06-16 00:00:00
Tasks: Binary Classification, General Classification, Image Segmentation, Prediction, Segmentation, Semantic Segmentation

Recent advances in deep learning based image segmentation methods have
enabled real-time performance with human-level accuracy. However, occasionally
even the best method fails due to low image quality, artifacts or unexpected
behaviour of black box algorithms. Being able to predict segmentation quality
in the absence of ground truth is of paramount importance in clinical practice,
but also in large-scale studies to avoid the inclusion of invalid data in
subsequent analysis.
  In this work, we propose two approaches of real-time automated quality
control for cardiovascular MR segmentations using deep learning. First, we
train a neural network on 12,880 samples to predict Dice Similarity
Coefficients (DSC) on a per-case basis. We report a mean average error (MAE) of
0.03 on 1,610 test samples and 97% binary classification accuracy for
separating low and high quality segmentations. Secondly, in the scenario where
no manually annotated data is available, we train a network to predict DSC
scores from estimated quality obtained via a reverse testing strategy. We
report an MAE=0.14 and 91% binary classification accuracy for this case.
Predictions are obtained in real-time which, when combined with real-time
segmentation methods, enables instant feedback on whether an acquired scan is
analysable while the patient is still in the scanner. This further enables new
applications of optimising image acquisition towards best possible analysis
results.
Retrofitting Distributional Embeddings to Knowledge Graphs with Functional Relations
Authors: Benjamin J. Lengerich, Andrew L. Maas, Christopher Potts
Date: 2017-08-01 00:00:00
Proceeding: COLING 2018 8
Tasks: Knowledge Graph Completion, Knowledge Graphs

Knowledge graphs are a versatile framework to encode richly structured data
relationships, but it can be challenging to combine these graphs with
unstructured data. Methods for retrofitting pre-trained entity representations
to the structure of a knowledge graph typically assume that entities are
embedded in a connected space and that relations imply similarity. However,
useful knowledge graphs often contain diverse entities and relations (with
potentially disjoint underlying corpora) which do not accord with these
assumptions. To overcome these limitations, we present Functional Retrofitting,
a framework that generalizes current retrofitting methods by explicitly
modeling pairwise relations. Our framework can directly incorporate a variety
of pairwise penalty functions previously developed for knowledge graph
completion. Further, it allows users to encode, learn, and extract information
about relation semantics. We present both linear and neural instantiations of
the framework. Functional Retrofitting significantly outperforms existing
retrofitting methods on complex knowledge graphs and loses no accuracy on
simpler graphs (in which relations do imply similarity). Finally, we
demonstrate the utility of the framework by predicting new drug--disease
treatment pairs in a large, complex health knowledge graph.
Machine Translation in Indian Languages: Challenges and Resolution
Authors: Raj Nath Patel, Prakash B. Pimpale, M Sasikumar
Date: 2017-08-26 00:00:00
Tasks: Machine Translation, Translation

English to Indian language machine translation poses the challenge of
structural and morphological divergence. This paper describes English to Indian
language statistical machine translation using pre-ordering and suffix
separation. The pre-ordering uses rules to transfer the structure of the source
sentences prior to training and translation. This syntactic restructuring helps
statistical machine translation to tackle the structural divergence and hence
better translation quality. The suffix separation is used to tackle the
morphological divergence between English and highly agglutinative Indian
languages. We demonstrate that the use of pre-ordering and suffix separation
helps in improving the quality of English to Indian Language machine
translation.
PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review
Authors: Ivan Stelmakh, Nihar B. Shah, Aarti Singh
Date: 2018-06-16 00:00:00
Tasks: Fairness

We consider the problem of automated assignment of papers to reviewers in conference peer review, with a focus on fairness and statistical accuracy. Our fairness objective is to maximize the review quality of the most disadvantaged paper, in contrast to the commonly used objective of maximizing the total quality over all papers. We design an assignment algorithm based on an incremental max-flow procedure that we prove is near-optimally fair. Our statistical accuracy objective is to ensure correct recovery of the papers that should be accepted. We provide a sharp minimax analysis of the accuracy of the peer-review process for a popular objective-score model as well as for a novel subjective-score model that we propose in the paper. Our analysis proves that our proposed assignment algorithm also leads to a near-optimal statistical accuracy. Finally, we design a novel experiment that allows for an objective comparison of various assignment algorithms, and overcomes the inherent difficulty posed by the absence of a ground truth in experiments on peer-review. The results of this experiment as well as of other experiments on synthetic and real data corroborate the theoretical guarantees of our algorithm.
Binary Classification in Unstructured Space With Hypergraph Case-Based Reasoning
Authors: Alexandre Quemy
Date: 2018-06-16 00:00:00
Tasks: Binary Classification, Classification, Feature Engineering, General Classification, Missing Values

Binary classification is one of the most common problem in machine learning.
It consists in predicting whether a given element belongs to a particular
class. In this paper, a new algorithm for binary classification is proposed
using a hypergraph representation. The method is agnostic to data
representation, can work with multiple data sources or in non-metric spaces,
and accommodates with missing values. As a result, it drastically reduces the
need for data preprocessing or feature engineering. Each element to be
classified is partitioned according to its interactions with the training set.
For each class, a seminorm over the training set partition is learnt to
represent the distribution of evidence supporting this class.
  Empirical validation demonstrates its high potential on a wide range of
well-known datasets and the results are compared to the state-of-the-art. The
time complexity is given and empirically validated. Its robustness with regard
to hyperparameter sensitivity is studied and compared to standard
classification methods. Finally, the limitation of the model space is
discussed, and some potential solutions proposed.
Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling
Authors: N. Majumder, D. Hazarika, A. Gelbukh, E. Cambria, S. Poria
Date: 2018-06-16 00:00:00
Tasks: Multimodal Emotion Recognition, Multimodal Sentiment Analysis, Sentiment Analysis

Multimodal sentiment analysis is a very actively growing field of research. A
promising area of opportunity in this field is to improve the multimodal fusion
mechanism. We present a novel feature fusion strategy that proceeds in a
hierarchical fashion, first fusing the modalities two in two and only then
fusing all three modalities. On multimodal sentiment analysis of individual
utterances, our strategy outperforms conventional concatenation of features by
1%, which amounts to 5% reduction in error rate. On utterance-level multimodal
sentiment analysis of multi-utterance video clips, for which current
state-of-the-art techniques incorporate contextual information from other
utterances of the same clip, our hierarchical fusion gives up to 2.4% (almost
10% error rate reduction) over currently used concatenation. The implementation
of our method is publicly available in the form of open-source code.