pwc-archive/papers-with-abstracts huggingface dataset:
Discovering Underlying Person Structure Pattern with Relative Local Distance for Person Re-identification
Authors: Guangcong Wang, Jian-Huang Lai, Zhenyu Xie, Xiaohua Xie
Date: 2019-01-29 00:00:00
Tasks: Person Re-Identification, Representation Learning

Modeling the underlying person structure for person re-identification (re-ID)
is difficult due to diverse deformable poses, changeable camera views and
imperfect person detectors. How to exploit underlying person structure
information without extra annotations to improve the performance of person
re-ID remains largely unexplored. To address this problem, we propose a novel
Relative Local Distance (RLD) method that integrates a relative local distance
constraint into convolutional neural networks (CNNs) in an end-to-end way. It
is the first time that the relative local constraint is proposed to guide the
global feature representation learning. Specially, a relative local distance
matrix is computed by using feature maps and then regarded as a regularizer to
guide CNNs to learn a structure-aware feature representation. With the
discovered underlying person structure, the RLD method builds a bridge between
the global and local feature representation and thus improves the capacity of
feature representation for person re-ID. Furthermore, RLD also significantly
accelerates deep network training compared with conventional methods. The
experimental results show the effectiveness of RLD on the CUHK03, Market-1501,
and DukeMTMC-reID datasets. Code is available at
\url{https://github.com/Wanggcong/RLD_codes}.
Identifying centromeric satellites with dna-brnn
Authors: Heng Li
Date: 2019-01-22 00:00:00

Summary: Human alpha satellite and satellite 2/3 contribute to several
percent of the human genome. However, identifying these sequences with
traditional algorithms is computationally intensive. Here we develop dna-brnn,
a recurrent neural network to learn the sequences of the two classes of
centromeric repeats. It achieves high similarity to RepeatMasker and is tens of
times faster. Dna-brnn explores a novel application of deep learning and may
accelerate the study of the evolution of the two repeat classes.
  Availability and implementation: https://github.com/lh3/dna-nn
  Contact: hli@jimmy.harvard.edu
Biosignal Analysis with Matching-Pursuit Based Adaptive Chirplet Transform
Authors: Jie Cui, Dinghui Wang
Date: 2017-09-25 00:00:00

Chirping phenomena, in which the instantaneous frequencies of a signal change
with time, are abundant in signals related to biological systems. Biosignals
are non-stationary in nature and the time-frequency analysis is a viable tool
to analyze them. It is well understood that Gaussian chirplet function is
critical in describing chirp signals. Despite the theory of adaptive chirplet
transform (ACT) has been established for more than two decades and is well
accepted in the community of signal processing, application of ACT to
bio-/biomedical signal analysis is still quite limited, probably because that
the power of ACT, as an emerging tool for biosignal analysis, has not yet been
fully appreciated by the researchers in the field of biomedical engineering. In
this paper, we describe a novel ACT algorithm based on the "coarse-refinement"
scheme. Namely, the initial estimate of a chirplet is implemented with the
matching-pursuit (MP) algorithm and subsequently it is refined using the
expectation-maximization (EM) algorithm, which we coin as MPEM algorithm. We
emphasize the robustness enhancement of the algorithm in face of noise, which
is important to biosignal analysis, as they are usually embedded in strong
background noise. We then demonstrate the capability of our algorithm by
applying it to the analysis of representative biosignals, including visual
evoked potentials (bioelectrical signals), audible heart sounds and bat
ultrasonic echolocation signals (bioacoustic signals), and human speech. The
results show that the MPEM algorithm provides more compact representation of
signals under investigation and clearer visualization of their time-frequency
structures, indicating considerable promise of ACT in biosignal analysis. The
MATLAB code repository is hosted on GitHub for free download
(https://github.com/jiecui/mpact).
Using Pre-Training Can Improve Model Robustness and Uncertainty
Authors: Dan Hendrycks, Kimin Lee, Mantas Mazeika
Date: 2019-01-28 00:00:00
Tasks: Adversarial Robustness, General Classification, Out-of-Distribution Detection

He et al. (2018) have called into question the utility of pre-training by showing that training from scratch can often yield similar performance to pre-training. We show that although pre-training may not improve performance on traditional classification metrics, it improves model robustness and uncertainty estimates. Through extensive experiments on adversarial examples, label corruption, class imbalance, out-of-distribution detection, and confidence calibration, we demonstrate large gains from pre-training and complementary effects with task-specific methods. We introduce adversarial pre-training and show approximately a 10% absolute improvement over the previous state-of-the-art in adversarial robustness. In some cases, using pre-training without task-specific methods also surpasses the state-of-the-art, highlighting the need for pre-training when evaluating future methods on robustness and uncertainty tasks.
Universal Dependency Parsing from Scratch
Authors: Peng Qi, Timothy Dozat, Yuhao Zhang, Christopher D. Manning
Date: 2019-01-29 00:00:00
Proceeding: CONLL 2018 10
Tasks: All, Dependency Parsing, POS, POS Tagging, Sentence, Sentence segmentation

This paper describes Stanford's system at the CoNLL 2018 UD Shared Task. We
introduce a complete neural pipeline system that takes raw text as input, and
performs all tasks required by the shared task, ranging from tokenization and
sentence segmentation, to POS tagging and dependency parsing. Our single system
submission achieved very competitive performance on big treebanks. Moreover,
after fixing an unfortunate bug, our corrected system would have placed the
2nd, 1st, and 3rd on the official evaluation metrics LAS,MLAS, and BLEX, and
would have outperformed all submission systems on low-resource treebank
categories on all metrics by a large margin. We further show the effectiveness
of different model components through extensive ablation studies.
Asynchronous Batch Bayesian Optimisation with Improved Local Penalisation
Authors: Ahsan S. Alvi, Binxin Ru, Jan Calliess, Stephen J. Roberts, Michael A. Osborne
Date: 2019-01-29 00:00:00
Tasks: Bayesian Optimisation

Batch Bayesian optimisation (BO) has been successfully applied to hyperparameter tuning using parallel computing, but it is wasteful of resources: workers that complete jobs ahead of others are left idle. We address this problem by developing an approach, Penalising Locally for Asynchronous Bayesian Optimisation on $k$ workers (PLAyBOOK), for asynchronous parallel BO. We demonstrate empirically the efficacy of PLAyBOOK and its variants on synthetic tasks and a real-world problem. We undertake a comparison between synchronous and asynchronous BO, and show that asynchronous BO often outperforms synchronous batch BO in both wall-clock time and number of function evaluations.
Toward Controlling Discrimination in Online Ad Auctions
Authors: L. Elisa Celis, Anay Mehrotra, Nisheeth K. Vishnoi
Date: 2019-01-29 00:00:00
Tasks: Fairness

Online advertising platforms are thriving due to the customizable audiences they offer advertisers. However, recent studies show that advertisements can be discriminatory with respect to the gender or race of the audience that sees the ad, and may inadvertently cross ethical and/or legal boundaries. To prevent this, we propose a constrained ad auction framework that maximizes the platform's revenue conditioned on ensuring that the audience seeing an advertiser's ad is distributed appropriately across sensitive types such as gender or race. Building upon Myerson's classic work, we first present an optimal auction mechanism for a large class of fairness constraints. Finding the parameters of this optimal auction, however, turns out to be a non-convex problem. We show that this non-convex problem can be reformulated as a more structured non-convex problem with no saddle points or local-maxima; this allows us to develop a gradient-descent-based algorithm to solve it. Our empirical results on the A1 Yahoo! dataset demonstrate that our algorithm can obtain uniform coverage across different user types for each advertiser at a minor loss to the revenue of the platform, and a small change to the size of the audience each advertiser reaches.
No Training Required: Exploring Random Encoders for Sentence Classification
Authors: John Wieting, Douwe Kiela
Date: 2019-01-29 00:00:00
Proceeding: ICLR 2019 5
Tasks: Classification, General Classification, Sentence, Sentence Classification, Sentence Embeddings, Word Embeddings

We explore various methods for computing sentence representations from
pre-trained word embeddings without any training, i.e., using nothing but
random parameterizations. Our aim is to put sentence embeddings on more solid
footing by 1) looking at how much modern sentence embeddings gain over random
methods---as it turns out, surprisingly little; and by 2) providing the field
with more appropriate baselines going forward---which are, as it turns out,
quite strong. We also make important observations about proper experimental
protocol for sentence classification evaluation, together with recommendations
for future research.
Improved Adversarial Learning for Fair Classification
Authors: L. Elisa Celis, Vijay Keswani
Date: 2019-01-29 00:00:00
Tasks: BIG-bench Machine Learning, Classification, Fairness, General Classification

Motivated by concerns that machine learning algorithms may introduce
significant bias in classification models, developing fair classifiers has
become an important problem in machine learning research. One important
paradigm towards this has been providing algorithms for adversarially learning
fair classifiers (Zhang et al., 2018; Madras et al., 2018). We formulate the
adversarial learning problem as a multi-objective optimization problem and find
the fair model using gradient descent-ascent algorithm with a modified gradient
update step, inspired by the approach of Zhang et al., 2018. We provide
theoretical insight and guarantees that formalize the heuristic arguments
presented previously towards taking such an approach. We test our approach
empirically on the Adult dataset and synthetic datasets and compare against
state of the art algorithms (Celis et al., 2018; Zhang et al., 2018; Zafar et
al., 2017). The results show that our models and algorithms have comparable or
better accuracy than other algorithms while performing better in terms of
fairness, as measured using statistical rate or false discovery rate.
A Deep Learning Framework for Assessing Physical Rehabilitation Exercises
Authors: Y. Liao, A. Vakanski, M. Xian
Date: 2019-01-29 00:00:00
Tasks: Action Quality Assessment, Deep Learning, Dimensionality Reduction, Motion Estimation

Computer-aided assessment of physical rehabilitation entails evaluation of patient performance in completing prescribed rehabilitation exercises, based on processing movement data captured with a sensory system. Despite the essential role of rehabilitation assessment toward improved patient outcomes and reduced healthcare costs, existing approaches lack versatility, robustness, and practical relevance. In this paper, we propose a deep learning-based framework for automated assessment of the quality of physical rehabilitation exercises. The main components of the framework are metrics for quantifying movement performance, scoring functions for mapping the performance metrics into numerical scores of movement quality, and deep neural network models for generating quality scores of input movements via supervised learning. The proposed performance metric is defined based on the log-likelihood of a Gaussian mixture model, and encodes low-dimensional data representation obtained with a deep autoencoder network. The proposed deep spatio-temporal neural network arranges data into temporal pyramids, and exploits the spatial characteristics of human movements by using sub-networks to process joint displacements of individual body parts. The presented framework is validated using a dataset of ten rehabilitation exercises. The significance of this work is that it is the first that implements deep neural networks for assessment of rehabilitation performance.
Influence of segmentation on deep iris recognition performance
Authors: Juš Lozej, Dejan Štepec, Vitomir Štruc, Peter Peer
Date: 2019-01-29 00:00:00
Tasks: Deep Learning, Iris Recognition, Iris Segmentation, Segmentation

Despite the rise of deep learning in numerous areas of computer vision and image processing, iris recognition has not benefited considerably from these trends so far. Most of the existing research on deep iris recognition is focused on new models for generating discriminative and robust iris representations and relies on methodologies akin to traditional iris recognition pipelines. Hence, the proposed models do not approach iris recognition in an end-to-end manner, but rather use standard heuristic iris segmentation (and unwrapping) techniques to produce normalized inputs for the deep learning models. However, because deep learning is able to model very complex data distributions and nonlinear data changes, an obvious question arises. How important is the use of traditional segmentation methods in a deep learning setting? To answer this question, we present in this paper an empirical analysis of the impact of iris segmentation on the performance of deep learning models using a simple two stage pipeline consisting of a segmentation and a recognition step. We evaluate how the accuracy of segmentation influences recognition performance but also examine if segmentation is needed at all. We use the CASIA Thousand and SBVPI datasets for the experiments and report several interesting findings.
Pay Less Attention with Lightweight and Dynamic Convolutions
Authors: Felix Wu, Angela Fan, Alexei Baevski, Yann N. Dauphin, Michael Auli
Date: 2019-01-29 00:00:00
Proceeding: ICLR 2019 5
Tasks: Abstractive Text Summarization, Language Modeling, Language Modelling, Machine Translation, Translation

Self-attention is a useful mechanism to build generative models for language
and images. It determines the importance of context elements by comparing each
element to the current time step. In this paper, we show that a very
lightweight convolution can perform competitively to the best reported
self-attention results. Next, we introduce dynamic convolutions which are
simpler and more efficient than self-attention. We predict separate convolution
kernels based solely on the current time-step in order to determine the
importance of context elements. The number of operations required by this
approach scales linearly in the input length, whereas self-attention is
quadratic. Experiments on large-scale machine translation, language modeling
and abstractive summarization show that dynamic convolutions improve over
strong self-attention models. On the WMT'14 English-German test set dynamic
convolutions achieve a new state of the art of 29.7 BLEU.
Geometric Matrix Completion with Deep Conditional Random Fields
Authors: Duc Minh Nguyen, Robert Calderbank, Nikos Deligiannis
Date: 2019-01-29 00:00:00
Tasks: Matrix Completion, Recommendation Systems, Structured Prediction

The problem of completing high-dimensional matrices from a limited set of observations arises in many big data applications, especially, recommender systems. Existing matrix completion models generally follow either a memory- or a model-based approach, whereas, geometric matrix completion models combine the best from both approaches. Existing deep-learning-based geometric models yield good performance, but, in order to operate, they require a fixed structure graph capturing the relationships among the users and items. This graph is typically constructed by evaluating a pre-defined similarity metric on the available observations or by using side information, e.g., user profiles. In contrast, Markov-random-fields-based models do not require a fixed structure graph but rely on handcrafted features to make predictions. When no side information is available and the number of available observations becomes very low, existing solutions are pushed to their limits. In this paper, we propose a geometric matrix completion approach that addresses these challenges. We consider matrix completion as a structured prediction problem in a conditional random field (CRF), which is characterized by a maximum a posterior (MAP) inference, and we propose a deep model that predicts the missing entries by solving the MAP inference problem. The proposed model simultaneously learns the similarities among matrix entries, computes the CRF potentials, and solves the inference problem. Its training is performed in an end-to-end manner, with a method to supervise the learning of entry similarities. Comprehensive experiments demonstrate the superior performance of the proposed model compared to various state-of-the-art models on popular benchmark datasets and underline its superior capacity to deal with highly incomplete matrices.
Kernel embedded nonlinear observational mappings in the variational mapping particle filter
Authors: Manuel Pulido, Peter Jan vanLeeuwen, Derek J. Posselt
Date: 2019-01-29 00:00:00

Recently, some works have suggested methods to combine variational
probabilistic inference with Monte Carlo sampling. One promising approach is
via local optimal transport. In this approach, a gradient steepest descent
method based on local optimal transport principles is formulated to transform
deterministically point samples from an intermediate density to a posterior
density. The local mappings that transform the intermediate densities are
embedded in a reproducing kernel Hilbert space (RKHS). This variational mapping
method requires the evaluation of the log-posterior density gradient and
therefore the adjoint of the observational operator. In this work, we evaluate
nonlinear observational mappings in the variational mapping method using two
approximations that avoid the adjoint, an ensemble based approximation in which
the gradient is approximated by the particle covariances in the state and
observational spaces the so-called ensemble space and an RKHS approximation in
which the observational mapping is embedded in an RKHS and the gradient is
derived there. The approximations are evaluated for highly nonlinear
observational operators and in a low-dimensional chaotic dynamical system. The
RKHS approximation is shown to be highly successful and superior to the
ensemble approximation.
Sliced generative models
Authors: Szymon Knop, Marcin Mazur, Jacek Tabor, Igor Podolak, Przemysław Spurek
Date: 2019-01-29 00:00:00

In this paper we discuss a class of AutoEncoder based generative models based
on one dimensional sliced approach. The idea is based on the reduction of the
discrimination between samples to one-dimensional case. Our experiments show
that methods can be divided into two groups. First consists of methods which
are a modification of standard normality tests, while the second is based on
classical distances between samples. It turns out that both groups are correct
generative models, but the second one gives a slightly faster decrease rate of
Fr\'{e}chet Inception Distance (FID).
MgNet: A Unified Framework of Multigrid and Convolutional Neural Network
Authors: Juncai He, Jinchao Xu
Date: 2019-01-29 00:00:00
Tasks: image-classification, Image Classification

We develop a unified model, known as MgNet, that simultaneously recovers some
convolutional neural networks (CNN) for image classification and multigrid (MG)
methods for solving discretized partial differential equations (PDEs). This
model is based on close connections that we have observed and uncovered between
the CNN and MG methodologies. For example, pooling operation and feature
extraction in CNN correspond directly to restriction operation and iterative
smoothers in MG, respectively. As the solution space is often the dual of the
data space in PDEs, the analogous concept of feature space and data space
(which are dual to each other) is introduced in CNN. With such connections and
new concept in the unified model, the function of various convolution
operations and pooling used in CNN can be better understood. As a result,
modified CNN models (with fewer weights and hyper parameters) are developed
that exhibit competitive and sometimes better performance in comparison with
existing CNN models when applied to both CIFAR-10 and CIFAR-100 data sets.
Guidelines for creating man-machine multimodal interfaces
Authors: João Ranhel, Cacilda Vilela
Date: 2019-01-29 00:00:00
Tasks: multimodal interaction

Understanding details of human multimodal interaction can elucidate many aspects of the type of information processing machines must perform to interact with humans. This article gives an overview of recent findings from Linguistics regarding the organization of conversation in turns, adjacent pairs, (dis)preferred responses, (self)repairs, etc. Besides, we describe how multiple modalities of signs interfere with each other modifying meanings. Then, we propose an abstract algorithm that describes how a machine can implement a double-feedback system that can reproduces a human-like face-to-face interaction by processing various signs, such as verbal, prosodic, facial expressions, gestures, etc. Multimodal face-to-face interactions enrich the exchange of information between agents, mainly because these agents are active all the time by emitting and interpreting signs simultaneously. This article is not about an untested new computational model. Instead, it translates findings from Linguistics as guidelines for designs of multimodal man-machine interfaces. An algorithm is presented. Brought from Linguistics, it is a description pointing out how human face-to-face interactions work. The linguistic findings reported here are the first steps towards the integration of multimodal communication. Some developers involved on interface designs carry on working on isolated models for interpreting text, grammar, gestures and facial expressions, neglecting the interwoven between these signs. In contrast, for linguists working on the state-of-the-art multimodal integration, the interpretation of separated modalities leads to an incomplete interpretation, if not to a miscomprehension of information. The algorithm proposed herein intends to guide man-machine interface designers who want to integrate multimodal components on face-to-face interactions as close as possible to those performed between humans.