pwc-archive/papers-with-abstracts huggingface dataset:
An Alarm System For Segmentation Algorithm Based On Shape Model
Authors: Fengze Liu, Yingda Xia, Dong Yang, Alan Yuille, Daguang Xu
Date: 2019-03-26 00:00:00
Proceeding: ICLR 2019 5
Tasks: Segmentation

It is usually hard for a learning system to predict correctly on rare events that never occur in the training data, and there is no exception for segmentation algorithms. Meanwhile, manual inspection of each case to locate the failures becomes infeasible due to the trend of large data scale and limited human resource. Therefore, we build an alarm system that will set off alerts when the segmentation result is possibly unsatisfactory, assuming no corresponding ground truth mask is provided. One plausible solution is to project the segmentation results into a low dimensional feature space; then learn classifiers/regressors to predict their qualities. Motivated by this, in this paper, we learn a feature space using the shape information which is a strong prior shared among different datasets and robust to the appearance variation of input data.The shape feature is captured using a Variational Auto-Encoder (VAE) network that trained with only the ground truth masks. During testing, the segmentation results with bad shapes shall not fit the shape prior well, resulting in large loss values. Thus, the VAE is able to evaluate the quality of segmentation result on unseen data, without using ground truth. Finally, we learn a regressor in the one-dimensional feature space to predict the qualities of segmentation results. Our alarm system is evaluated on several recent state-of-art segmentation algorithms for 3D medical segmentation tasks. Compared with other standard quality assessment methods, our system consistently provides more reliable prediction on the qualities of segmentation results.
INFER: INtermediate representations for FuturE pRediction
Authors: Shashank Srikanth, Junaid Ahmed Ansari, Karnik Ram R, Sarthak Sharma, Krishna Murthy J., Madhava Krishna K
Date: 2019-03-26 00:00:00
Tasks: Activity Prediction, Future prediction, Multi-Object Tracking, Object Tracking, Prediction, Trajectory Prediction

In urban driving scenarios, forecasting future trajectories of surrounding
vehicles is of paramount importance. While several approaches for the problem
have been proposed, the best-performing ones tend to require extremely detailed
input representations (eg. image sequences). But, such methods do not
generalize to datasets they have not been trained on. We propose intermediate
representations that are particularly well-suited for future prediction. As
opposed to using texture (color) information, we rely on semantics and train an
autoregressive model to accurately predict future trajectories of traffic
participants (vehicles) (see fig. above). We demonstrate that using semantics
provides a significant boost over techniques that operate over raw pixel
intensities/disparities. Uncharacteristic of state-of-the-art approaches, our
representations and models generalize to completely different datasets,
collected across several cities, and also across countries where people drive
on opposite sides of the road (left-handed vs right-handed driving).
Additionally, we demonstrate an application of our approach in multi-object
tracking (data association). To foster further research in transferrable
representations and ensure reproducibility, we release all our code and data.
Federated Learning Of Out-Of-Vocabulary Words
Authors: Mingqing Chen, Rajiv Mathews, Tom Ouyang, Françoise Beaufays
Date: 2019-03-26 00:00:00
Tasks: Federated Learning

We demonstrate that a character-level recurrent neural network is able to
learn out-of-vocabulary (OOV) words under federated learning settings, for the
purpose of expanding the vocabulary of a virtual keyboard for smartphones
without exporting sensitive text to servers. High-frequency words can be
sampled from the trained generative model by drawing from the joint posterior
directly. We study the feasibility of the approach in two settings: (1) using
simulated federated learning on a publicly available non-IID per-user dataset
from a popular social networking website, (2) using federated learning on data
hosted on user mobile devices. The model achieves good recall and precision
compared to ground-truth OOV words in setting (1). With (2) we demonstrate the
practicality of this approach by showing that we can learn meaningful OOV words
with good character-level prediction accuracy and cross entropy loss.
Diversifying Reply Suggestions using a Matching-Conditional Variational Autoencoder
Authors: Budhaditya Deb, Peter Bailey, Milad Shokouhi
Date: 2019-03-25 00:00:00
Proceeding: NAACL 2019 6
Tasks: Diversity, Information Retrieval, Retrieval, Variational Inference

We consider the problem of diversifying automated reply suggestions for a
commercial instant-messaging (IM) system (Skype). Our conversation model is a
standard matching based information retrieval architecture, which consists of
two parallel encoders to project messages and replies into a common feature
representation. During inference, we select replies from a fixed response set
using nearest neighbors in the feature space. To diversify responses, we
formulate the model as a generative latent variable model with Conditional
Variational Auto-Encoder (M-CVAE). We propose a constrained-sampling approach
to make the variational inference in M-CVAE efficient for our production
system. In offline experiments, M-CVAE consistently increased diversity by
~30-40% without significant impact on relevance. This translated to a 5% gain
in click-rate in our online production system.
Neural Grammatical Error Correction with Finite State Transducers
Authors: Felix Stahlberg, Christopher Bryant, Bill Byrne
Date: 2019-03-25 00:00:00
Proceeding: NAACL 2019 6
Tasks: Grammatical Error Correction, Language Modeling, Language Modelling, Machine Translation, Translation

Grammatical error correction (GEC) is one of the areas in natural language
processing in which purely neural models have not yet superseded more
traditional symbolic models. Hybrid systems combining phrase-based statistical
machine translation (SMT) and neural sequence models are currently among the
most effective approaches to GEC. However, both SMT and neural
sequence-to-sequence models require large amounts of annotated data. Language
model based GEC (LM-GEC) is a promising alternative which does not rely on
annotated training data. We show how to improve LM-GEC by applying modelling
techniques based on finite state transducers. We report further gains by
rescoring with neural language models. We show that our methods developed for
LM-GEC can also be used with SMT systems if annotated training data is
available. Our best system outperforms the best published result on the
CoNLL-2014 test set, and achieves far better relative improvements over the SMT
baselines than previous hybrid systems.
On Using Retrained and Incremental Machine Learning for Modeling Performance of Adaptable Software: An Empirical Comparison
Authors: Tao Chen
Date: 2019-03-25 00:00:00
Tasks: BIG-bench Machine Learning

Given the ever-increasing complexity of adaptable software systems and their
commonly hidden internal information (e.g., software runs in the public cloud),
machine learning based performance modeling has gained momentum for evaluating,
understanding and predicting software performance, which facilitates better
informed self-adaptations. As performance data accumulates during the run of
the software, updating the performance models becomes necessary. To this end,
there are two conventional modeling methods: the retrained modeling that always
discard the old model and retrain a new one using all available data; or the
incremental modeling that retains the existing model and tunes it using one
newly arrival data sample. Generally, literature on machine learning based
performance modeling for adaptable software chooses either of those methods
according to a general belief, but they provide insufficient evidences or
references to justify their choice. This paper is the first to report on a
comprehensive empirical study that examines both modeling methods under
distinct domains of adaptable software, 5 performance indicators, 8 learning
algorithms and settings, covering a total of 1,360 different conditions. Our
findings challenge the general belief, which is shown to be only partially
correct, and reveal some of the important, statistically significant factors
that are often overlooked in existing work, providing evidence-based insights
on the choice.
Q-Learning for Continuous Actions with Cross-Entropy Guided Policies
Authors: Riley Simmons-Edler, Ben Eisner, Eric Mitchell, Sebastian Seung, Daniel Lee
Date: 2019-03-25 00:00:00
Tasks: Q-Learning, Reinforcement Learning, Reinforcement Learning (RL)

Off-Policy reinforcement learning (RL) is an important class of methods for many problem domains, such as robotics, where the cost of collecting data is high and on-policy methods are consequently intractable. Standard methods for applying Q-learning to continuous-valued action domains involve iteratively sampling the Q-function to find a good action (e.g. via hill-climbing), or by learning a policy network at the same time as the Q-function (e.g. DDPG). Both approaches make tradeoffs between stability, speed, and accuracy. We propose a novel approach, called Cross-Entropy Guided Policies, or CGP, that draws inspiration from both classes of techniques. CGP aims to combine the stability and performance of iterative sampling policies with the low computational cost of a policy network. Our approach trains the Q-function using iterative sampling with the Cross-Entropy Method (CEM), while training a policy network to imitate CEM's sampling behavior. We demonstrate that our method is more stable to train than state of the art policy network methods, while preserving equivalent inference time compute costs, and achieving competitive total reward on standard benchmarks.
An Approach for Adaptive Automatic Threat Recognition Within 3D Computed Tomography Images for Baggage Security Screening
Authors: Qian Wang, Khalid N. Ismail, Toby P. Breckon
Date: 2019-03-25 00:00:00
Tasks: Computed Tomography (CT), Image Segmentation, Material Recognition, Semantic Segmentation

The screening of baggage using X-ray scanners is now routine in aviation security with automatic threat detection approaches, based on 3D X-ray computed tomography (CT) images, known as Automatic Threat Recognition (ATR) within the aviation security industry. These current strategies use pre-defined threat material signatures in contrast to adaptability towards new and emerging threat signatures. To address this issue, the concept of adaptive automatic threat recognition (AATR) was proposed in previous work. In this paper, we present a solution to AATR based on such X-ray CT baggage scan imagery. This aims to address the issues of rapidly evolving threat signatures within the screening requirements. Ideally, the detection algorithms deployed within the security scanners should be readily adaptable to different situations with varying requirements of threat characteristics (e.g., threat material, physical properties of objects). We tackle this issue using a novel adaptive machine learning methodology with our solution consisting of a multi-scale 3D CT image segmentation algorithm, a multi-class support vector machine (SVM) classifier for object material recognition and a strategy to enable the adaptability of our approach. Experiments are conducted on both open and sequestered 3D CT baggage image datasets specifically collected for the AATR study. Our proposed approach performs well on both recognition and adaptation. Overall our approach can achieve the probability of detection around 90% with a probability of false alarm below 20%. Our AATR shows the capabilities of adapting to varying types of materials, even the unknown materials which are not available in the training data, adapting to varying required probability of detection and adapting to varying scales of the threat object.
Unifying Unsupervised Domain Adaptation and Zero-Shot Visual Recognition
Authors: Qian Wang, Penghui Bu, Toby P. Breckon
Date: 2019-03-25 00:00:00
Tasks: Domain Adaptation, domain classification, Generalized Zero-Shot Learning, Unsupervised Domain Adaptation, Zero-Shot Learning

Unsupervised domain adaptation aims to transfer knowledge from a source domain to a target domain so that the target domain data can be recognized without any explicit labelling information for this domain. One limitation of the problem setting is that testing data, despite having no labels, from the target domain is needed during training, which prevents the trained model being directly applied to classify unseen test instances. We formulate a new cross-domain classification problem arising from real-world scenarios where labelled data is available for a subset of classes (known classes) in the target domain, and we expect to recognize new samples belonging to any class (known and unseen classes) once the model is learned. This is a generalized zero-shot learning problem where the side information comes from the source domain in the form of labelled samples instead of class-level semantic representations commonly used in traditional zero-shot learning. We present a unified domain adaptation framework for both unsupervised and zero-shot learning conditions. Our approach learns a joint subspace from source and target domains so that the projections of both data in the subspace can be domain invariant and easily separable. We use the supervised locality preserving projection (SLPP) as the enabling technique and conduct experiments under both unsupervised and zero-shot learning conditions, achieving state-of-the-art results on three domain adaptation benchmark datasets: Office-Caltech, Office31 and Office-Home.
Learning Optimal and Fair Decision Trees for Non-Discriminative Decision-Making
Authors: Sina Aghaei, Mohammad Javad Azizi, Phebe Vayanos
Date: 2019-03-25 00:00:00
Tasks: Decision Making

In recent years, automated data-driven decision-making systems have enjoyed a
tremendous success in a variety of fields (e.g., to make product
recommendations, or to guide the production of entertainment). More recently,
these algorithms are increasingly being used to assist socially sensitive
decision-making (e.g., to decide who to admit into a degree program or to
prioritize individuals for public housing). Yet, these automated tools may
result in discriminative decision-making in the sense that they may treat
individuals unfairly or unequally based on membership to a category or a
minority, resulting in disparate treatment or disparate impact and violating
both moral and ethical standards. This may happen when the training dataset is
itself biased (e.g., if individuals belonging to a particular group have
historically been discriminated upon). However, it may also happen when the
training dataset is unbiased, if the errors made by the system affect
individuals belonging to a category or minority differently (e.g., if
misclassification rates for Blacks are higher than for Whites). In this paper,
we unify the definitions of unfairness across classification and regression. We
propose a versatile mixed-integer optimization framework for learning optimal
and fair decision trees and variants thereof to prevent disparate treatment
and/or disparate impact as appropriate. This translates to a flexible schema
for designing fair and interpretable policies suitable for socially sensitive
decision-making. We conduct extensive computational studies that show that our
framework improves the state-of-the-art in the field (which typically relies on
heuristics) to yield non-discriminative decisions at lower cost to overall
accuracy.
Reducing the dilution: An analysis of the information sensitiveness of capsule network with a practical improvement method
Authors: Zonglin Yang, Xinggang Wang
Date: 2019-03-25 00:00:00

Capsule network has shown various advantages over convolutional neural network (CNN). It keeps more precise spatial information than CNN and uses equivariance instead of invariance during inference and highly potential to be a new effective tool for visual tasks. However, the current capsule networks have incompatible performance with CNN when facing datasets with background and complex target objects and are lacking in universal and efficient regularization method. We analyze a main reason of the incompatible performance as the conflict between information sensitiveness of capsule network and unreasonably higher activation value distribution of capsules in primary capsule layer. Correspondingly, we propose a practical improvement method by restraining the activation value of capsules in primary capsule layer to suppress non-informative capsules and highlight discriminative capsules. In the experiments, the method has achieved better performances on various mainstream datasets. In addition, the proposed improvement methods can be seen as a suitable, simple and efficient regularization method that can be generally used in capsule network.
Defending against Whitebox Adversarial Attacks via Randomized Discretization
Authors: Yuchen Zhang, Percy Liang
Date: 2019-03-25 00:00:00
Tasks: Adversarial Attack, General Classification

Adversarial perturbations dramatically decrease the accuracy of
state-of-the-art image classifiers. In this paper, we propose and analyze a
simple and computationally efficient defense strategy: inject random Gaussian
noise, discretize each pixel, and then feed the result into any pre-trained
classifier. Theoretically, we show that our randomized discretization strategy
reduces the KL divergence between original and adversarial inputs, leading to a
lower bound on the classification accuracy of any classifier against any
(potentially whitebox) $\ell_\infty$-bounded adversarial attack. Empirically,
we evaluate our defense on adversarial examples generated by a strong iterative
PGD attack. On ImageNet, our defense is more robust than adversarially-trained
networks and the winning defenses of the NIPS 2017 Adversarial Attacks &
Defenses competition.
Performance-Efficiency Trade-off of Low-Precision Numerical Formats in Deep Neural Networks
Authors: Zachariah Carmichael, Hamed F. Langroudi, Char Khazanov, Jeffrey Lillie, John L. Gustafson, Dhireesha Kudithipudi
Date: 2019-03-25 00:00:00

Deep neural networks (DNNs) have been demonstrated as effective prognostic
models across various domains, e.g. natural language processing, computer
vision, and genomics. However, modern-day DNNs demand high compute and memory
storage for executing any reasonably complex task. To optimize the inference
time and alleviate the power consumption of these networks, DNN accelerators
with low-precision representations of data and DNN parameters are being
actively studied. An interesting research question is in how low-precision
networks can be ported to edge-devices with similar performance as
high-precision networks. In this work, we employ the fixed-point, floating
point, and posit numerical formats at $\leq$8-bit precision within a DNN
accelerator, Deep Positron, with exact multiply-and-accumulate (EMAC) units for
inference. A unified analysis quantifies the trade-offs between overall network
efficiency and performance across five classification tasks. Our results
indicate that posits are a natural fit for DNN inference, outperforming at
$\leq$8-bit precision, and can be realized with competitive resource
requirements relative to those of floating point.
Augmenting Gastrointestinal Health: A Deep Learning Approach to Human Stool Recognition and Characterization in Macroscopic Images
Authors: David Hachuel, Akshay Jha, Deborah Estrin, Alfonso Martinez, Kyle Staller, Christopher Velez
Date: 2019-03-25 00:00:00

Purpose - Functional bowel diseases, including irritable bowel syndrome,
chronic constipation, and chronic diarrhea, are some of the most common
diseases seen in clinical practice. Many patients describe a range of triggers
for altered bowel consistency and symptoms. However, characterization of the
relationship between symptom triggers using bowel diaries is hampered by poor
compliance and lack of objective stool consistency measurements. We sought to
develop a stool detection and tracking system using computer vision and deep
convolutional neural networks (CNN) that could be used by patients, providers,
and researchers in the assessment of chronic gastrointestinal (GI) disease.
Spike-based primitives for graph algorithms
Authors: Kathleen E. Hamilton, Tiffany M. Mintz, Catherine D. Schuman
Date: 2019-03-25 00:00:00

In this paper we consider graph algorithms and graphical analysis as a new
application for neuromorphic computing platforms. We demonstrate how the
nonlinear dynamics of spiking neurons can be used to implement low-level graph
operations. Our results are hardware agnostic, and we present multiple versions
of routines that can utilize static synapses or require synapse plasticity.
On the Functional Equivalence of TSK Fuzzy Systems to Neural Networks, Mixture of Experts, CART, and Stacking Ensemble Regression
Authors: Dongrui Wu, Chin-Teng Lin, Jian Huang, Zhigang Zeng
Date: 2019-03-25 00:00:00
Tasks: BIG-bench Machine Learning, Mixture-of-Experts, regression

Fuzzy systems have achieved great success in numerous applications. However, there are still many challenges in designing an optimal fuzzy system, e.g., how to efficiently optimize its parameters, how to balance the trade-off between cooperations and competitions among the rules, how to overcome the curse of dimensionality, how to increase its generalization ability, etc. Literature has shown that by making appropriate connections between fuzzy systems and other machine learning approaches, good practices from other domains may be used to improve the fuzzy systems, and vice versa. This paper gives an overview on the functional equivalence between Takagi-Sugeno-Kang fuzzy systems and four classic machine learning approaches -- neural networks, mixture of experts, classification and regression trees, and stacking ensemble regression -- for regression problems. We also point out some promising new research directions, inspired by the functional equivalence, that could lead to solutions to the aforementioned problems. To our knowledge, this is so far the most comprehensive overview on the connections between fuzzy systems and other popular machine learning approaches, and hopefully will stimulate more hybridization between different machine learning algorithms.