pwc-archive/papers-with-abstracts huggingface dataset:
Efficient Adaptation for End-to-End Vision-Based Robotic Manipulation
Authors: Ryan Julian, Benjamin Swanson, Gaurav S. Sukhatme, Sergey Levine, Chelsea Finn, Karol Hausman
Date: 2020-06-12 00:00:00
Proceeding: ICML Workshop LifelongML 2020 7
Tasks: Continual Learning, Robotic Grasping

One of the great promises of robot learning systems is that they will be able to learn from their mistakes and continuously adapt to  ever-changing environments, but most robot learning systems today are deployed as fixed policies which do not adapt after deployment. Can we efficiently adapt previously learned behaviors to new environments, objects and percepts in the real world? We present empirical evidence towards a robot learning framework that facilitates continuous adaption. We demonstrate how to adapt vision-based robotic manipulation policies to new variations by fine-tuning via off-policy reinforcement learning, using less than 0.2% of the data necessary to learn the task from scratch. We find that the simple approach of fine-tuning pre-trained policies leads to substantial performance gains over the course of fine-tuning, and that pre-training via RL is essential: training from scratch or adapting from supervised ImageNet features are both unsuccessful with such small amounts of data. We also find that these positive results hold in a limited continual learning setting, in which we repeatedly fine-tune a single lineage of policies using data from a succession of new tasks. Our empirical conclusions are consistently supported by experiments on simulated manipulation tasks, and by 52 unique fine-tuning experiments on a real robotic grasping system pre-trained on 580,000 grasps.
Non-Robust Feature Mapping in Deep Reinforcement Learning
Authors: Ezgi Korkmaz
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Atari Games, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Adversarial perturbations to state observations can dramatically degrade the performance of deep reinforcement learning policies, and thus raise concerns regarding the robustness of deep reinforcement learning agents. A sizeable body of work has focused on addressing the robustness problem in deep reinforcement learning, and there are several recent proposals for adversarial training methods in the deep reinforcement learning domain. In our work we focus on the robustness of state-of-the-art adversarially trained deep reinforcement learning policies and vanilla trained deep reinforcement learning polices. We propose two novel algorithms to map non-robust features in deep reinforcement learning policies. We conduct several experiments in the Arcade Learning Environment (ALE), and with our proposed feature mapping algorithms we show that while the state-of-the-art adversarial training method eliminates a certain set of non-robust features, a new set of non-robust features more intrinsic to the adversarial training are created. Our results lay out concerns that arise when using existing state-of-the-art adversarial training methods, and we believe our proposed feature mapping algorithm can aid in the process of building more robust deep reinforcement learning policies.
Adversarially Trained Neural Policies in the Fourier Domain
Authors: Ezgi Korkmaz
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning policies based on deep neural networks are vulnerable to imperceptible adversarial perturbations to their inputs, in much the same way as neural network image classifiers. Recent work has proposed several methods for adversarial training for deep reinforcement learning agents to improve robustness to adversarial perturbations. In this paper, we study the effects of adversarial training on the neural policy learned by the agent. In particular, we compare the Fourier spectrum of minimal perturbations computed for both adversarially trained and vanilla trained neural policies. Via experiments in the OpenAI Atari environments we show that minimal perturbations computed for adversarially trained policies are more focused on lower frequencies in the Fourier domain, indicating a higher sensitivity of these policies to low frequency perturbations. We believe our results can be an initial step towards understanding the relationship between adversarial training and different notions of robustness for neural policies.
BadNL: Backdoor Attacks Against NLP Models
Authors: Xiaoyi Chen, Ahmed Salem, Michael Backes, Shiqing Ma, Yang Zhang
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Backdoor Attack, Sentence, Sentiment Analysis

Deep Neural Networks (DNNs) have progressed rapidly during the past decade. Meanwhile, DNN models have been shown to be vulnerable to various security and privacy attacks. One such attack that has attracted a great deal of attention recently is the backdoor attack. 
Previous backdoor attacks mainly focus on computer vision tasks. In this paper, we perform the first systematic investigation of the backdoor attack against natural language processing (NLP) models with a focus on sentiment analysis task. Specifically, we propose three methods to construct triggers, including Word-level, Char-level, and Sentence-level triggers. Our attacks achieve an almost perfect attack success rate with a negligible effect on the original model's utility. For instance, using the Word-level triggers, our backdoor attack achieves a 100% attack success rate with only a utility drop of 0.18%, 1.26%, and 0.19% on three benchmark sentiment analysis datasets.
Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Neural Network Robustness Verification
Authors: Shiqi Wang, huan zhang, Kaidi Xu, Xue Lin, Suman Jana, Cho-Jui Hsieh, J Zico Kolter
Date: 2021-06-18 00:00:00
Proceeding: NeurIPS 2021 12

We develop $\beta$-CROWN, a new bound propagation based method that can fully encode neuron split constraints in branch-and-bound (BaB) based complete verification via optimizable parameters $\beta$. When jointly optimized in intermediate layers, $\beta$-CROWN generally produces better bounds than typical LP verifiers with neuron split constraints, while being as efficient and parallelizable as CROWN on GPUs. Applied to complete robustness verification benchmarks, $\beta$-CROWN with BaB is close to three orders of magnitude faster than LP-based BaB methods, and is at least 3 times faster than winners of VNN-COMP 2020 competition while producing lower timeout rates. By terminating BaB early, our method can also be used for efficient incomplete verification. We achieve higher verified accuracy in many settings over powerful incomplete verifiers, including those based on convex barrier breaking techniques. Compared to the typically tightest but very costly semidefinite programming (SDP) based incomplete verifiers, we obtain higher verified accuracy with three orders of magnitudes less verification time, and enable better certification for verification-agnostic (e.g., adversarially trained) networks.
On the Effectiveness of Poisoning against Unsupervised Domain Adaptation
Authors: Akshay Mehra, Bhavya Kailkhura, Pin-Yu Chen, Jihun Hamm
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Data Poisoning, Domain Adaptation, Unsupervised Domain Adaptation

Data poisoning attacks manipulate victim's training data to compromise their model performance, after training.
Previous works on poisoning have shown the inability of a small amount of poisoned data at significantly reducing the test accuracy of deep neural networks. In this work, we propose an upper bound on the test error induced by additive poisoning, which explains the difficulty of poisoning against deep neural networks. However, the limited effect of poisoning is restricted to the setting where training and test data are from the same distribution.  To demonstrate this, we study the effect of poisoning in an unsupervised domain adaptation (UDA) setting where the source and the target domain distributions are different. We propose novel data poisoning attacks that prevent UDA methods from learning a representation that generalizes well on the target domain. Our poisoning attacks significantly lower the target domain accuracy of state-of-the-art UDA methods on popular benchmark UDA tasks, dropping it to almost 0% in some cases, with the addition of only 10% poisoned data. The effectiveness of our attacks in the UDA setting highlights the seriousness of the threat posed by data poisoning and the importance of data curation in machine learning.
Robust Recovery of Adversarial Examples
Authors: Tejas Bana, Jatan Loya, Siddhant Ravindra Kulkarni
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Deep Learning, Image Reconstruction

Adversarial examples are semantically associated with one class, but modern deep learning architectures fail to see the semantics and associate them to another class. As a result, these examples pose a profound risk to almost every deep learning model. Our proposed architecture can recover such examples effectively with more than 4x the magnitude of attacks than the capability of the state-of-the-art model, despite having lesser parameters than the VGG-13 model. It is composed of a U-Net with the characteristics of self-attention & cross-attention, which enhances the semantics of the image. Our work also encompasses the differences in the results between Noise and Image reconstruction methodologies.
Adversarial Interaction Attacks: Fooling AI to Misinterpret Human Intentions
Authors: Nodens Koren, Xingjun Ma, Qiuhong Ke, Yisen Wang, James Bailey
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Adversarial Attack

Understanding the actions of both humans and artificial intelligence (AI) agents is important before modern AI systems can be fully integrated into our daily life. In this paper, we show that, despite their current huge success, deep learning based AI systems can be easily fooled by subtle adversarial noise to misinterpret the intention of an action in interaction scenarios. Based on a case study of skeleton-based human interactions, we propose a novel adversarial attack on interactions, and demonstrate how DNN-based interaction models can be tricked to predict the participants' reactions in unexpected ways. Our study highlights potential risks in the interaction loop with AI and humans, which need to be carefully addressed when deploying AI systems in safety-critical applications.
Maximizing the robust margin provably overfits on noiseless data
Authors: Konstantin Donhauser, Alexandru Tifrea, Michael Aerni, Reinhard Heckel, Fanny Yang
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Attribute

Numerous recent works show that overparameterization implicitly reduces variance, suggesting vanishing benefits for explicit regularization in high dimensions. However, this narrative has been challenged by empirical observations indicating that adversarially trained deep neural networks suffer from robust overfitting. While existing explanations attribute this phenomenon  to noise or problematic samples in the training data set, we prove that even on entirely noiseless data, achieving a vanishing adversarial logistic training loss is suboptimal compared to regularized counterparts.
Attacking Few-Shot Classifiers with Adversarial Support Poisoning
Authors: Elre Talea Oldewage, John F Bronskill, Richard E Turner
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Meta-Learning

This paper examines the robustness of deployed few-shot meta-learning systems when they are fed an imperceptibly perturbed few-shot dataset, showing that the resulting predictions on test inputs can become worse than chance. This is achieved by developing a novel attack, Adversarial Support Poisoning or ASP, which crafts a poisoned set of examples. When even a small subset of malicious data points is inserted into the support set of a meta-learner, accuracy is significantly reduced. We evaluate the new attack on a variety of few-shot classification algorithms and scenarios, and propose a form of adversarial training that significantly improves robustness against both poisoning and evasion attacks.
Data Poisoning Won’t Save You From Facial Recognition
Authors: Evani Radiya-Dixit, Florian Tramer
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Data Poisoning

Data poisoning has been proposed as a compelling defense against facial recognition models trained on Web-scraped pictures. By perturbing the images they post online, users can fool models into misclassifying future (unperturbed) pictures.
		
We demonstrate that this strategy provides a false sense of security, as it ignores an inherent asymmetry between the parties: users' pictures are perturbed once and for all before being published and scraped, and must thereafter fool all future models---including models trained adaptively against the users' past attacks, or models that use technologies discovered after the attack. 
		
We evaluate two poisoning attacks against large-scale facial recognition, Fawkes 500,000+ downloads) and LowKey. We demonstrate how an ``oblivious'' model trainer can simply wait for future developments in computer vision to nullify the protection of pictures collected in the past. We further show that an adversary with black-box access to the attack can train a robust model that resists the perturbations of collected pictures.
		
We caution that facial recognition poisoning will not admit an ''arms race'' between attackers and defenders. Once perturbed pictures are scraped, the attack cannot be changed so any future defense irrevocably undermines users' privacy.
Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them
Authors: Florian Tramer
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7

Making classifiers robust to adversarial examples is hard. 
Thus, many defenses tackle the seemingly easier task of \emph{detecting} perturbed inputs.

We show a barrier towards this goal. We prove a general \emph{hardness reduction} between detection and classification of adversarial examples: given a robust detector for attacks at distance $\epsilon$ (in some metric), we can build a similarly robust (but inefficient) \emph{classifier} for attacks at distance $\epsilon/2$.

Our reduction is computationally inefficient, and thus cannot be used to build practical classifiers. Instead, it is a useful sanity check to test whether empirical detection results imply something much stronger than the authors presumably anticipated. 
%(indeed, building inefficient robust classifiers is also presumed to be very challenging).

To illustrate, we revisit $13$ detector defenses. For $10/13$ cases, we show that the claimed detection results would imply an inefficient classifier with robustness far beyond the state-of-the-art.
Demystifying Adversarial Training via A Unified Probabilistic Framework
Authors: Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7

Adversarial Training (AT) is known as an effective approach to enhance the robustness of deep neural networks. Recently researchers notice that robust models with AT have good generative ability and can synthesize realistic images, while the reason behind it is yet under-explored. In this paper, we demystify this phenomenon by developing a unified probabilistic framework, called Contrastive Energy-based Models (CEM). On the one hand, we provide the first probabilistic characterization of AT through a unified understanding of robustness and generative ability. On the other hand, our CEM can also naturally generalize AT to the unsupervised scenario and develop principled unsupervised AT methods. Based on these, we propose principled adversarial sampling algorithms in both supervised and unsupervised scenarios. Experiments show our sampling algorithms significantly improve the sampling quality and achieves an Inception score of 9.61 on CIFAR-10, which is superior to previous energy-based models and comparable to state-of-the-art generative models.
Towards Safe Reinforcement Learning via Constraining Conditional Value at Risk
Authors: Chengyang Ying, Xinning Zhou, Dong Yan, Jun Zhu
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: continuous-control, Continuous Control, Deep Reinforcement Learning, MuJoCo, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Safe Reinforcement Learning

Though deep reinforcement learning (DRL) has obtained substantial success, it may encounter catastrophic failures due to the intrinsic uncertainty caused by stochastic policies and environment variability. To address this issue, we propose a novel reinforcement learning framework of CVaR-Proximal-Policy-Optimization (CPPO) by rating the conditional value-at-risk (CVaR) as an assessment for risk. We show that performance degradation under observation state disturbance and transition probability disturbance theoretically depends on the range of disturbance as well as the gap of value function between different states. Therefore, constraining the value function among states with CVaR can improve the robustness of the policy.  Experimental results show that CPPO achieves higher cumulative reward and exhibits stronger robustness against observation state disturbance and transition probability disturbance in environment dynamics among a series of continuous control tasks in MuJoCo.
Universal Adversarial Head: Practical Protection against Video Data Leakage
Authors: Jiawang Bai, Bin Chen, Dongxian Wu, Chaoning Zhang, Shu-Tao Xia
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Deep Hashing, Retrieval, Video Retrieval

While online video sharing becomes more popular, it also causes unconscious leakage of personal information in the video retrieval systems like deep hashing. An adversary can collect users' private information from the video database by querying similar videos. This paper focuses on bypassing the deep video hashing based retrieval to prevent information from being maliciously collected.  We propose $universal \ adversarial \ head$ (UAH), which crafts adversarial query videos by prepending the original videos with a sequence of adversarial frames to perturb the normal hash codes in the Hamming space. This adversarial head can be obtained just using a few videos, and mislead the retrieval system to return irrelevant videos on most natural query videos. Furthermore, to obey the principle of information protection, we expand the proposed method to a data-free paradigm to generate the UAH, without access to users' original videos. Extensive experiments demonstrate the protection effectiveness of our method under various settings.
Limited Budget Adversarial Attack Against Online Image Stream
Authors: Hossein Mohasel Arjomandi, Mohammad Khalooei, Maryam Amirmazlaghani
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Adversarial Attack

An adversary wants to attack a limited number of images within a stream of known length to reduce
the exposure risk. Also, the adversary wants to maximize the success rate of the performed attacks. We show that with very minimal changes in images data, majority of attacking attempt would fail, however some attempts still lead to succeed. We detail an algorithm that choose the optimal images which lead to successful attack . We apply our approach on MNIST and prove it’s significant outcome compared to the state of the art.
Empirical robustification of pre-trained classifiers
Authors: Mohammad Sadegh Norouzzadeh, Wan-Yi Lin, Leonid Boytsov, Leslie Rice, huan zhang, Filipe Condessa, J Zico Kolter
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Denoising, Image Reconstruction, Image-to-Image Translation

Most pre-trained classifiers, though they may work extremely well on the domain they were trained upon, are not trained in a robust fashion, and therefore are sensitive to adversarial attacks.  A recent technique, denoised-smoothing, demonstrated that it was possible to create certifiably robust classifiers from a pre-trained classifier (without any retraining) by pre-pending a denoising network and wrapping the entire pipeline within randomized smoothing.  However, this is a costly procedure, which requires multiple queries due to the randomized smoothing element, and which ultimately is very dependent on the quality of the denoiser.  In this paper, we demonstrate that a more conventional “adversarial training” approach also works when applied to this robustification process.   Specifically, we show that by training an image-to-image translation model, prepended to a pre-trained classifier, with losses that optimize for both the fidelity of the image reconstruction and the adversarial performance of the end-to-end system, we can robustify pre-trained classifiers to a higher empirical degree of accuracy than denoised smoothing.  Further, these robustifers are also transferable to some degree across multiple classifiers and even some architectures, illustrating that in some real sense they are removing the “adversarial manifold” from the input data, a task that has traditionally been very challenging for “conventional” preprocessing methods.
Is It Time to Redefine the Classification Task for Deep Learning Systems?
Authors: Keji Han, Yun Li, Songcan Chen
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Adversarial Attack, Classification, Deep Learning

Many works have demonstrated that deep neural networks (DNNs) are vulnerable to adversarial examples. A deep learning system involves a couple of elements: the learning task, data set, deep model, loss, and optimizer. Each element may cause the vulnerability of the deep learning system, and simply attributing the vulnerability of the deep learning system to the deep model may impede addressing the adversarial attack. So we redefine the robustness of DNNs as the robustness of the deep neural learning system, and we experimentally find that the vulnerability of the deep learning system also roots in the learning task itself. In detail, this paper defines the interval-label classification task for the deep classification system, whose labels are predefined non-overlapping intervals instead of a fixed value (hard label) or probability vector (soft label). The experimental results demonstrate that the interval-label classification task is more robust than the traditional classification task while retaining accuracy.
Strategically-timed State-Observation Attacks on Deep Reinforcement Learning Agents
Authors: You Qiaoben, Xinning Zhou, Chengyang Ying, Jun Zhu
Date: 2021-06-18 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Adversarial Attack, continuous-control, Continuous Control, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

 Deep reinforcement learning (DRL) policies are vulnerable to the adversarial attack on their observations, which may mislead real-world RL agents to catastrophic failures. Several works have shown the effectiveness of this type of adversarial attacks. But these adversaries are inclined to be detected because these adversaries do not inhibit their attacks activity. Recent works provide heuristic methods by attacking the victim agent at a small subset of time steps, but it aims at lack for theoretical principles. Inspired by the idea that adversarial attacks at each time step have different efforts, we denote a novel strategically-timed attack called Tentative Frame Attack for continuous control environments. We further propose a theoretical framework of finding optimal frame attack. Following this framework, we trained the frame attack strategy online with the victim agents and a fixed adversary. The empirical results show that our adversaries achieve the state-of-the-art performance on DRL agents which outperforms the full-timed attack.