  With the abundance of large-scale deep learning models, it has become
possible to repurpose pre-trained networks for new tasks. Recent works on
adversarial reprogramming have shown that it is possible to repurpose neural
networks for alternate tasks without modifying the network architecture or
parameters. However these works only consider original and target tasks within
the same data domain. In this work, we broaden the scope of adversarial
reprogramming beyond the data modality of the original task. We analyze the
feasibility of adversarially repurposing image classification neural networks
for Natural Language Processing (NLP) and other sequence classification tasks.
We design an efficient adversarial program that maps a sequence of discrete
tokens into an image which can be classified to the desired class by an image
classification model. We demonstrate that by using highly efficient adversarial
programs, we can reprogram image classifiers to achieve competitive performance
on a variety of text and sequence classification benchmarks without retraining
the network.
