Title: Functional Shrinkage for Input Optimization
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Functional shrinkage, activation map, input optimization

Abstract: Input optimization is a major methodology in deep learning that has been widely applied in adversarial attacks, neural network verification, and deep reinforcement learning. An important topic is how to inject a perturbation with a limited budget to the crucial part of a network, in order to avoid unstable optimization paths or meaningless outcomes. In this work, we formulate a network as a functional of activation functions, and propose a functional shrinkage method to extract the principal component of the network. Then the residual component can be vulnerable to perturbations, which is exploited for perturbation injection. Experimental results show that the proposed method achieves state-of-the-art performance in several input optimization tasks. It provides a new insight into the structural decomposition and weakness exploitation of neural networks.

TL;DR: We propose a functional shrinkage method for input optimization.