user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Garbage in, model out: Weight theft with just noise

Paper content:
This paper explores the scenarios under which
an attacker can claim that ‘Noise and access to
the softmax layer of the model is all you need’
to steal the weights of a convolutional neural network
whose architecture is already known.
We
were able to achieve 96% test accuracy using
the stolen MNIST model and 82% accuracy using
stolen KMNIST model learned using only
i.i.d. Bernoulli noise inputs.
We posit that this
theft-susceptibility of the weights is indicative
of the complexity of the dataset and propose a
new metric that captures the same.
The goal of
this dissemination is to not just showcase how far
knowing the architecture can take you in terms of
model stealing, but to also draw attention to this
rather idiosyncratic weight learnability aspects of
CNNs spurred by i.i.d. noise input.
We also disseminate
some initial results obtained with using
the Ising probability distribution in lieu of the i.i.d.
Bernoulli distribution
In this paper, we consider the fate of an adamant attacker who is adamant about only using noise as input to a convolutional neural network (CNN) whose architecture is known and whose weights are the target of theft.
We assume that the attacker has earned access to the softmax layer and is not restricted in terms of the number of inputs to be used to carry out the attack.
At the outset, we'd like to emphasize that our goal in disseminating these results is not to convince the reader on the real-world validity of the attacker-scenario described above or to showcase a novel attack.
This paper contains our initial explorations after a chance discovery that we could populate the weights of an MNIST-trained CNN model by just using noise as input into the framework described below.Preliminary work.
Under review by the International Conference on Machine Learning (ICML).
Do not distribute.Through a set of empirical experiments, which we are duly open sourcing to aid reproducibility, we seek to draw the attention of the community on the following two issues:1.
This risk of model weight theft clearly entails an interplay between the dataset as well as the architecture.
Given a fixed architecture, can we use the level of susceptibility as a novel metric of complexity of the dataset?2
.
Given the wide variations in success attained by varying the noise distribution, how do we formally characterize the relationship between the input noise distribution being used by the attacker and the true distribution of the data, while considering a specific CNN architecture?
What aspects of the true data distribution are actually important for model extraction?The
rest of the paper is structured as follows:In Section 2, we provide a brief literature survey of the related work. In
Section 3, we describe the methodology used to carry out the attack. In
Section 4, we cover the main results obtained and conclude the paper in Section 5.
In this paper, we demonstrated a framework for extracting model parameters by training a new model on random impulse response pairs gleaned from the softmax output of the victim neural network.
We went on to demonstrate the variation in model extractability based on the dataset which the original model was trained on.
Finally, we proposed our framework as a method for which relative dataset complexity can be measured.

assistant: text: Input only noise , glean the softmax outputs, steal the weights