pwc-archive/papers-with-abstracts huggingface dataset:
Adversarial Learning in Statistical Classification: A Comprehensive Review of Defenses Against Attacks
Authors: David J. Miller, Zhen Xiang, George Kesidis
Date: 2019-04-12 00:00:00
Tasks: Anomaly Detection, Data Poisoning, General Classification, Robust classification

There is great potential for damage from adversarial learning (AL) attacks on machine-learning based systems. In this paper, we provide a contemporary survey of AL, focused particularly on defenses against attacks on statistical classifiers. After introducing relevant terminology and the goals and range of possible knowledge of both attackers and defenders, we survey recent work on test-time evasion (TTE), data poisoning (DP), and reverse engineering (RE) attacks and particularly defenses against same. In so doing, we distinguish robust classification from anomaly detection (AD), unsupervised from supervised, and statistical hypothesis-based defenses from ones that do not have an explicit null (no attack) hypothesis; we identify the hyperparameters a particular method requires, its computational complexity, as well as the performance measures on which it was evaluated and the obtained quality. We then dig deeper, providing novel insights that challenge conventional AL wisdom and that target unresolved issues, including: 1) robust classification versus AD as a defense strategy; 2) the belief that attack success increases with attack strength, which ignores susceptibility to AD; 3) small perturbations for test-time evasion attacks: a fallacy or a requirement?; 4) validity of the universal assumption that a TTE attacker knows the ground-truth class for the example to be attacked; 5) black, grey, or white box attacks as the standard for defense evaluation; 6) susceptibility of query-based RE to an AD defense. We also discuss attacks on the privacy of training data. We then present benchmark comparisons of several defenses against TTE, RE, and backdoor DP attacks on images. The paper concludes with a discussion of future work.
Outlier-robust estimation of a sparse linear model using $\ell_1$-penalized Huber's $M$-estimator
Authors: Arnak S. Dalalyan, Philip Thompson
Date: 2019-04-12 00:00:00

We study the problem of estimating a $p$-dimensional $s$-sparse vector in a linear model with Gaussian design and additive noise. In the case where the labels are contaminated by at most $o$ adversarial outliers, we prove that the $\ell_1$-penalized Huber's $M$-estimator based on $n$ samples attains the optimal rate of convergence $(s/n)^{1/2} + (o/n)$, up to a logarithmic factor. For more general design matrices, our results highlight the importance of two properties: the transfer principle and the incoherence property. These properties with suitable constants are shown to yield the optimal rates, up to log-factors, of robust estimation with adversarial contamination.
CITE: A Corpus of Image-Text Discourse Relations
Authors: Malihe Alikhani, Sreyasi Nag Chowdhury, Gerard de Melo, Matthew Stone
Date: 2019-04-12 00:00:00
Proceeding: NAACL 2019 6
Tasks: Common Sense Reasoning

This paper presents a novel crowd-sourced resource for multimodal discourse:
our resource characterizes inferences in image-text contexts in the domain of
cooking recipes in the form of coherence relations. Like previous corpora
annotating discourse structure between text arguments, such as the Penn
Discourse Treebank, our new corpus aids in establishing a better understanding
of natural communication and common-sense reasoning, while our findings have
implications for a wide range of applications, such as understanding and
generation of multimodal documents.
GeoCapsNet: Aerial to Ground view Image Geo-localization using Capsule Network
Authors: Bin Sun, Chen Chen, Yingying Zhu, Jianmin Jiang
Date: 2019-04-12 00:00:00
Tasks: geo-localization, Image Retrieval, Retrieval, Triplet

The task of cross-view image geo-localization aims to determine the
geo-location (GPS coordinates) of a query ground-view image by matching it with
the GPS-tagged aerial (satellite) images in a reference dataset. Due to the
dramatic changes of viewpoint, matching the cross-view images is challenging.
In this paper, we propose the GeoCapsNet based on the capsule network for
ground-to-aerial image geo-localization. The network first extracts features
from both ground-view and aerial images via standard convolution layers and the
capsule layers further encode the features to model the spatial feature
hierarchies and enhance the representation power. Moreover, we introduce a
simple and effective weighted soft-margin triplet loss with online batch hard
sample mining, which can greatly improve image retrieval accuracy. Experimental
results show that our GeoCapsNet significantly outperforms the state-of-the-art
approaches on two benchmark datasets.
Locally Connected Spiking Neural Networks for Unsupervised Feature Learning
Authors: Daniel J. Saunders, Devdhar Patel, Hananel Hazan, Hava T. Siegelmann, Robert Kozma
Date: 2019-04-12 00:00:00
Tasks: General Classification

In recent years, Spiking Neural Networks (SNNs) have demonstrated great
successes in completing various Machine Learning tasks. We introduce a method
for learning image features by \textit{locally connected layers} in SNNs using
spike-timing-dependent plasticity (STDP) rule. In our approach, sub-networks
compete via competitive inhibitory interactions to learn features from
different locations of the input space. These \textit{Locally-Connected SNNs}
(LC-SNNs) manifest key topological features of the spatial interaction of
biological neurons. We explore biologically inspired n-gram classification
approach allowing parallel processing over various patches of the the image
space. We report the classification accuracy of simple two-layer LC-SNNs on two
image datasets, which match the state-of-art performance and are the first
results to date. LC-SNNs have the advantage of fast convergence to a dataset
representation, and they require fewer learnable parameters than other SNN
approaches with unsupervised learning. Robustness tests demonstrate that
LC-SNNs exhibit graceful degradation of performance despite the random deletion
of large amounts of synapses and neurons.
ACE: Adapting to Changing Environments for Semantic Segmentation
Authors: Zuxuan Wu, Xin Wang, Joseph E. Gonzalez, Tom Goldstein, Larry S. Davis
Date: 2019-04-12 00:00:00
Proceeding: ICCV 2019 10
Tasks: Meta-Learning, Semantic Segmentation

Deep neural networks exhibit exceptional accuracy when they are trained and
tested on the same data distributions. However, neural classifiers are often
extremely brittle when confronted with domain shift---changes in the input
distribution that occur over time. We present ACE, a framework for semantic
segmentation that dynamically adapts to changing environments over the time. By
aligning the distribution of labeled training data from the original source
domain with the distribution of incoming data in a shifted domain, ACE
synthesizes labeled training data for environments as it sees them. This
stylized data is then used to update a segmentation model so that it performs
well in new environments. To avoid forgetting knowledge from past environments,
we introduce a memory that stores feature statistics from previously seen
domains. These statistics can be used to replay images in any of the previously
observed domains, thus preventing catastrophic forgetting. In addition to
standard batch training using stochastic gradient decent (SGD), we also
experiment with fast adaptation methods based on adaptive meta-learning.
Extensive experiments are conducted on two datasets from SYNTHIA, the results
demonstrate the effectiveness of the proposed approach when adapting to a
number of tasks.
Variational Inference for Computational Imaging Inverse Problems
Authors: Francesco Tonolini, Jack Radford, Alex Turpin, Daniele Faccio, Roderick Murray-Smith
Date: 2019-04-12 00:00:00
Tasks: BIG-bench Machine Learning, Image Reconstruction, Variational Inference

Machine learning methods for computational imaging require uncertainty estimation to be reliable in real settings. While Bayesian models offer a computationally tractable way of recovering uncertainty, they need large data volumes to be trained, which in imaging applications implicates prohibitively expensive collections with specific imaging instruments. This paper introduces a novel framework to train variational inference for inverse problems exploiting in combination few experimentally collected data, domain expertise and existing image data sets. In such a way, Bayesian machine learning models can solve imaging inverse problems with minimal data collection efforts. Extensive simulated experiments show the advantages of the proposed framework. The approach is then applied to two real experimental optics settings: holographic image reconstruction and imaging through highly scattering media. In both settings, state of the art reconstructions are achieved with little collection of training data.
Similarities between policy gradient methods (PGM) in Reinforcement learning (RL) and supervised learning (SL)
Authors: Eric Benhamou
Date: 2019-04-12 00:00:00
Tasks: Decision Making, Policy Gradient Methods, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

Reinforcement learning (RL) is about sequential decision making and is traditionally opposed to supervised learning (SL) and unsupervised learning (USL). In RL, given the current state, the agent makes a decision that may influence the next state as opposed to SL (and USL) where, the next state remains the same, regardless of the decisions taken, either in batch or online learning. Although this difference is fundamental between SL and RL, there are connections that have been overlooked. In particular, we prove in this paper that gradient policy method can be cast as a supervised learning problem where true label are replaced with discounted rewards. We provide a new proof of policy gradient methods (PGM) that emphasizes the tight link with the cross entropy and supervised learning. We provide a simple experiment where we interchange label and pseudo rewards. We conclude that other relationships with SL could be made if we modify the reward functions wisely.
Multi-Armed Bandit for Energy-Efficient and Delay-Sensitive Edge Computing in Dynamic Networks with Uncertainty
Authors: Saeed Ghoorchian, Setareh Maghsudi
Date: 2019-04-12 00:00:00
Tasks: Edge-computing

In the edge computing paradigm, mobile devices offload the computational tasks to an edge server by routing the required data over the wireless network. The full potential of edge computing becomes realized only if a smart device selects the most appropriate server in terms of the latency and energy consumption, among many available ones. The server selection problem is challenging due to the randomness of the environment and lack of prior information about the environment. Therefore, a smart device, which sequentially chooses a server under uncertainty, aims to improve its decision based on the historical time and energy consumption. The problem becomes more complicated in a dynamic environment, where key variables might undergo abrupt changes. To deal with the aforementioned problem, we first analyze the required time and energy to data transmission and processing. We then use the analysis to cast the problem as a budget-limited multi-armed bandit problem, where each arm is associated with a reward and cost, with time-variant statistical characteristics. We propose a policy to solve the formulated problem and prove a regret bound. The numerical results demonstrate the superiority of the proposed method compared to a number of existing solutions.
AMS-SFE: Towards an Alignment of Manifold Structures via Semantic Feature Expansion for Zero-shot Learning
Authors: Jingcai Guo, Song Guo
Date: 2019-04-12 00:00:00
Tasks: Zero-Shot Learning

Zero-shot learning (ZSL) aims at recognizing unseen classes with knowledge
transferred from seen classes. This is typically achieved by exploiting a
semantic feature space (FS) shared by both seen and unseen classes, i.e.,
attributes or word vectors, as the bridge. However, due to the mutually
disjoint of training (seen) and testing (unseen) data, existing ZSL methods
easily and commonly suffer from the domain shift problem. To address this
issue, we propose a novel model called AMS-SFE. It considers the Alignment of
Manifold Structures by Semantic Feature Expansion. Specifically, we build up an
autoencoder based model to expand the semantic features and joint with an
alignment to an embedded manifold extracted from the visual FS of data. It is
the first attempt to align these two FSs by way of expanding semantic features.
Extensive experiments show the remarkable performance improvement of our model
compared with other existing methods.
The coupling effect of Lipschitz regularization in deep neural networks
Authors: Nicolas Couellan
Date: 2019-04-12 00:00:00

We investigate robustness of deep feed-forward neural networks when input
data are subject to random uncertainties. More specifically, we consider
regularization of the network by its Lipschitz constant and emphasize its role.
We highlight the fact that this regularization is not only a way to control the
magnitude of the weights but has also a coupling effect on the network weights
accross the layers. We claim and show evidence on a dataset that this coupling
effect brings a tradeoff between robustness and expressiveness of the network.
This suggests that Lipschitz regularization should be carefully implemented so
as to maintain coupling accross layers.
MAANet: Multi-view Aware Attention Networks for Image Super-Resolution
Authors: Jingcai Guo, Shiheng Ma, Song Guo
Date: 2019-04-12 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

In most recent years, deep convolutional neural networks (DCNNs) based image
super-resolution (SR) has gained increasing attention in multimedia and
computer vision communities, focusing on restoring the high-resolution (HR)
image from a low-resolution (LR) image. However, one nonnegligible flaw of
DCNNs based methods is that most of them are not able to restore
high-resolution images containing sufficient high-frequency information from
low-resolution images with low-frequency information redundancy. Worse still,
as the depth of DCNNs increases, the training easily encounters the problem of
vanishing gradients, which makes the training more difficult. These problems
hinder the effectiveness of DCNNs in image SR task. To solve these problems, we
propose the Multi-view Aware Attention Networks (MAANet) for image SR task.
Specifically, we propose the local aware (LA) and global aware (GA) attention
to deal with LR features in unequal manners, which can highlight the
high-frequency components and discriminate each feature from LR images in the
local and the global views, respectively. Furthermore, we propose the local
attentive residual-dense (LARD) block, which combines the LA attention with
multiple residual and dense connections, to fit a deeper yet easy to train
architecture. The experimental results show that our proposed approach can
achieve remarkable performance compared with other state-of-the-art methods.
Generative Hybrid Representations for Activity Forecasting with No-Regret Learning
Authors: Jiaqi Guan, Ye Yuan, Kris M. Kitani, Nicholas Rhinehart
Date: 2019-04-12 00:00:00
Proceeding: CVPR 2020 6

Automatically reasoning about future human behaviors is a difficult problem but has significant practical applications to assistive systems. Part of this difficulty stems from learning systems' inability to represent all kinds of behaviors. Some behaviors, such as motion, are best described with continuous representations, whereas others, such as picking up a cup, are best described with discrete representations. Furthermore, human behavior is generally not fixed: people can change their habits and routines. This suggests these systems must be able to learn and adapt continuously. In this work, we develop an efficient deep generative model to jointly forecast a person's future discrete actions and continuous motions. On a large-scale egocentric dataset, EPIC-KITCHENS, we observe our method generates high-quality and diverse samples while exhibiting better generalization than related generative models. Finally, we propose a variant to continually learn our model from streaming data, observe its practical effectiveness, and theoretically justify its learning efficiency.
Evolving Indoor Navigational Strategies Using Gated Recurrent Units In NEAT
Authors: James Butterworth, Rahul Savani, Karl Tuyls
Date: 2019-04-12 00:00:00

Simultaneous Localisation and Mapping (SLAM) algorithms are expensive to run
on smaller robotic platforms such as Micro-Aerial Vehicles. Bug algorithms are
an alternative that use relatively little processing power, and avoid high
memory consumption by not building an explicit map of the environment. Bug
Algorithms achieve relatively good performance in simulated and robotic maze
solving domains. However, because they are hand-designed, a natural question is
whether they are globally optimal control policies. In this work we explore the
performance of Neuroevolution - specifically NEAT - at evolving control
policies for simulated differential drive robots carrying out generalised maze
navigation. We extend NEAT to include Gated Recurrent Units (GRUs) to help deal
with long term dependencies. We show that both NEAT and our NEAT-GRU can
repeatably generate controllers that outperform I-Bug (an algorithm
particularly well-suited for use in real robots) on a test set of 209 indoor
maze like environments. We show that NEAT-GRU is superior to NEAT in this task
but also that out of the 2 systems, only NEAT-GRU can continuously evolve
successful controllers for a much harder task in which no bearing information
about the target is provided to the agent.
Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data
Authors: Aleksei Tiulpin, Stefan Klein, Sita M. A. Bierma-Zeinstra, Jérôme Thevenot, Esa Rahtu, Joyce van Meurs, Edwin H. G. Oei, Simo Saarakkala
Date: 2019-04-12 00:00:00
Tasks: BIG-bench Machine Learning, Knee Osteoarthritis Prediction

Knee osteoarthritis (OA) is the most common musculoskeletal disease without a cure, and current treatment options are limited to symptomatic relief. Prediction of OA progression is a very challenging and timely issue, and it could, if resolved, accelerate the disease modifying drug development and ultimately help to prevent millions of total joint replacement surgeries performed annually. Here, we present a multi-modal machine learning-based OA progression prediction model that utilizes raw radiographic data, clinical examination results and previous medical history of the patient. We validated this approach on an independent test set of 3,918 knee images from 2,129 subjects. Our method yielded area under the ROC curve (AUC) of 0.79 (0.78-0.81) and Average Precision (AP) of 0.68 (0.66-0.70). In contrast, a reference approach, based on logistic regression, yielded AUC of 0.75 (0.74-0.77) and AP of 0.62 (0.60-0.64). The proposed method could significantly improve the subject selection process for OA drug-development trials and help the development of personalized therapeutic plans.
IIT (BHU) Varanasi at MSR-SRST 2018: A Language Model Based Approach for Natural Language Generation
Authors: Shreyansh Singh, Avi Chawla, Ayush Sharma, Anil Kumar Singh
Date: 2019-04-12 00:00:00
Proceeding: WS 2018 7
Tasks: Decoder, Language Modeling, Language Modelling, Text Generation

This paper describes our submission system for the Shallow Track of Surface
Realization Shared Task 2018 (SRST'18). The task was to convert genuine UD
structures, from which word order information had been removed and the tokens
had been lemmatized, into their correct sentential form. We divide the problem
statement into two parts, word reinflection and correct word order prediction.
For the first sub-problem, we use a Long Short Term Memory based
Encoder-Decoder approach. For the second sub-problem, we present a Language
Model (LM) based approach. We apply two different sub-approaches in the LM
Based approach and the combined result of these two approaches is considered as
the final output of the system.
On the Impact of the Cutoff Time on the Performance of Algorithm Configurators
Authors: George T. Hall, Pietro S. Oliveto, Dirk Sudholt
Date: 2019-04-12 00:00:00

Algorithm configurators are automated methods to optimise the parameters of an algorithm for a class of problems. We evaluate the performance of a simple random local search configurator (ParamRLS) for tuning the neighbourhood size $k$ of the RLS$_k$ algorithm. We measure performance as the expected number of configuration evaluations required to identify the optimal value for the parameter. We analyse the impact of the cutoff time $\kappa$ (the time spent evaluating a configuration for a problem instance) on the expected number of configuration evaluations required to find the optimal parameter value, where we compare configurations using either best found fitness values (ParamRLS-F) or optimisation times (ParamRLS-T). We consider tuning RLS$_k$ for a variant of the Ridge function class (Ridge*), where the performance of each parameter value does not change during the run, and for the OneMax function class, where longer runs favour smaller $k$. We rigorously prove that ParamRLS-F efficiently tunes RLS$_k$ for Ridge* for any $\kappa$ while ParamRLS-T requires at least quadratic $\kappa$. For OneMax ParamRLS-F identifies $k=1$ as optimal with linear $\kappa$ while ParamRLS-T requires a $\kappa$ of at least $\Omega(n\log n)$. For smaller $\kappa$ ParamRLS-F identifies that $k>1$ performs better while ParamRLS-T returns $k$ chosen uniformly at random.
Political Text Scaling Meets Computational Semantics
Authors: Federico Nanni, Goran Glavas, Ines Rehbein, Simone Paolo Ponzetto, Heiner Stuckenschmidt
Date: 2019-04-12 00:00:00
Tasks: feature selection

During the last fifteen years, automatic text scaling has become one of the key tools of the Text as Data community in political science. Prominent text scaling algorithms, however, rely on the assumption that latent positions can be captured just by leveraging the information about word frequencies in documents under study. We challenge this traditional view and present a new, semantically aware text scaling algorithm, SemScale, which combines recent developments in the area of computational linguistics with unsupervised graph-based clustering. We conduct an extensive quantitative analysis over a collection of speeches from the European Parliament in five different languages and from two different legislative terms, and show that a scaling approach relying on semantic document representations is often better at capturing known underlying political dimensions than the established frequency-based (i.e., symbolic) scaling method. We further validate our findings through a series of experiments focused on text preprocessing and feature selection, document representation, scaling of party manifestos, and a supervised extension of our algorithm. To catalyze further research on this new branch of text scaling methods, we release a Python implementation of SemScale with all included data sets and evaluation procedures.