Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a hardware-based attack framework NASPY, which can extract NAS network architecture and hyper-parameters. The main idea is to use a seq2seq model to to identify operations from side-channel sequences. 
main_review: 1. I am not an expert of hardware-based attacks, but achieving 3.2% error rates for operation prediction seems quite impressive.

2. Due to the lack of knowledge on hardware attacks, I did not fully understand what assumptions are made to achieve this kind of low error rate. Could you clarify the assumptions of these attacks?

3. The authors claim that separable or dilated convs are complicated, but could you clarify what’s the key challenges to attach these operations?

4. The paper claims to be focused on NAS models, but it is unclear to me why the proposed approach is specific to NAS models? Since it is just based on model inference (without accessing the search process), there should be no difference whether the target model is a NAS model or not? Please clarify this.
summary_of_the_review: 1. Interesting approach and impressive results.
2. Need more clarifications for a few points.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents an end-to-end adversarial framework to extract the network architecture obtained from Neural Architecture Search (NAS), named NASPY. Previous works focus on extracting conventional ConvNets with simple operations, while requiring heavy human analysis. In contrast, NASPY introduces a seq2seq network to automatically identify complicated operations from the hardware event sequence. In addition, both the value of hyper-parameters and the exact model topology can be obtained with NASPY. Extensive experiments based on DARTS, GDAS, and TE-NAS demonstrate the effectiveness of NASPY.
main_review: 
I agree to accept this paper. The model is novel, the idea is well-motivated, and the experiments are compelling.

Pros:
1.	A novel, neat, and effective learning-based framework for automated extraction of NAS architectures with high efficiency and fidelity.
2.	Model the extraction attack as a sequence-to-sequence problem, and design new deep learning models to predict the model operation sequence automatically.
3.	A new analysis method to precisely recover the exact hyper-parameters without any prior knowledge.
4.	Design strategies to reconstruct the model topology and extract the complete architecture for different scenarios and adversarial goals.
5.	Comprehensive empirical results to show the effectiveness of the components.

Cons:
1.	The experiments mainly focus on extracting NAS models. However, in real-world scenarios, the variants of standard models (e.g., Resnet, Densenet) are widely used. Will NASPY still be effective on these models? Consider that, some hyper-parameters like the number of layers, channels, and stages may be varying.
2.	It would be helpful to briefly introduce the background hardware knowledge (e.g., the meaning of the itcopy, oncopy, and time interval). Many of the readers may be unfamiliar with these details.
3.	I think that the experiments may need to include a comparison between the architecture obtained by the fidelity extraction attack and the original network.

Discussions:
Would it be possible to extract the network architecture of detection and segmentation models? For example, the FPN or other detection/segmentation head.

summary_of_the_review: I agree to accept this paper. The model is novel, the idea is well motivated, and the experiments are compelling.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In the paper "NASPY: Automated Extraction of Automated Machine Learning Models" the authors propose an approach to extract the structure and parameters of a DNN that has been constructed via neural architecture search. In an empirical study the authors demonstrate the effectiveness of their approach and show that by making some assumptions they can indeed successfully extract the architecture of DNNs as well as their parameters.
main_review: # Significance
The topic in general and the contributions presented by the authors are highly relevant and of interest to a broad audience of ML researchers as well as practitioners.

# Novelty
To the best of my knowledge the main aspects proposed in the paper are novel contributions.

# Soundness
The paper seems to be sound. The claims are supported by a proper empirical study considering various scenarios.

# Writing
The paper is very well organized and the writing is very clear. Especially, explanations regarding rather technical or hardware-related aspects are very well done and understandable.

# Minor Comments
- Capitalization of headlines are inconsistent.
- The color schemes of figures should be adapted to be more inclusive (for red-green blindness). Especially, Figure 4 is very hard to read and distinguish between red and green for me.

# Questions
- What is the runtime and data complexity of the proposed approach?
summary_of_the_review: All in all, the paper makes an important contribution to the area of security in machine learning. The proposed approach is able to successfully extract neural architectures and even their parameters, allowing for piracy of such models. Hopefully, this work will foster defense mechanisms in order to prevent such attacks. Therefore, I recommend to accept the paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper presents a technique for performing an attack on NAS approaches allowing the technique to identify the neural architecture which the NAS is honing in on.
main_review: The area of NAS and attacks on NAS is an interesting area which is currently receiving quite a bit of attention. It is good to see work in this area.

Strengths:
- A more complete approach for attacking a NAS process.
- A well constructed and well-written piece of work.

Weaknesses:
- The paper makes some rather bold claims, which seem to stem from the fact that they assume all NAS approaches work in the same way and are based around the same building blocks. This assumption is not correct and would make this the approach far less effective.
- The predominant way to train Deep Learning and also for NAS is to use a GPU. However, the approach here seems to rely on the fact that all computations are performed on the CPU. This again would significantly reduce the effectiveness of the attack method.

Some comments on the text:
- The abstract and title are not good. Reading the abstract does not convey what this paper is about. Some text needs to be added to make it clear what domain this is in (i.e., trying to steal a network from a running NAS). Once you understand that the abstract starts to make sense.

- The paper is over the 9 page limit. You have cheated by placing figures and tables in the supplementary material and then referenced them from the main body. If this were to be allowed people would start submitting 9 pages of text and put all figures / tables / etc into the supplementary material. This makes a mockery of having a page limit.

- The footnote on page 1 needs either a citation or you need to provide your evidence to support this.

- Your idea of what NAS is and the underlying techniques used for it is rather restricted. You assume that NAS is a process of building up cells and that these cells are formed from standard units. However, there are many other NAS techniques out there which do not follow this model. For example differentiable NAS approaches, Zero-shot NAS (which doesn’t do any training) and NAS approaches which only partly train networks. These other approaches would not (I assume) be susceptible to your attack. You should clearly articulate what is required for your approach to work. This can be extracted by thinking through the whole process, but it should not be the readers role to do this.

- Likewise, the approach seems to assume that the whole process requires that you perform all of the NAS approach on the CPU. As the dominant approach for performing Deep Learning is to use a GPU there should be some discussion as to whether your approach would work in these cases. I’m guessing not.

- NAS rarely returns the ‘optimal network architecture’ - in most cases it returns a ‘good’ architecture, but to find the optimal you would need to perform all possible architectures - and even then you would only know it was ‘optimal’ within your search space.

- What do you mean by ‘a normal cell interprets the features’?

- There is an assumption that NAS introduces new more complex operations - this is not true. The NAS can only provide operations that it has been told to use. 

- From the description of how you ‘grab’ the architecture from the NAS approach, I can’t see why this wouldn’t work if you were just training a network. Is this correct? If so say so, if not please explain why it wouldn’t work.

- What is a ‘proxy dataset’?

- A number of terms are used without being explained, including itchy, incopy, I_i, O_i

- Given that to ’steal’ a network you need to train your system and then run it against trace logs captured from the running NAS it would seem that this whole process is computationally expensive. Especially as you still need to train the stolen network. Given this it would seem essential that there is some comparison between the cost of training the NAS and training your approach and using it. It may be the case that it would be cheeper to just run the NAS yourself.

- Data augmentation is not clearly enough explained - can you add more details.

- What do you mean by “we first introduce a convolution layer to extract more features”. Convolution layers process the features they are given, but they do not create new features.

- There are some pretty big assumptions in this work. Such as P=R/2. This is not a requirement of performing NAS, so shouldn’t be a an assumption by you.

- The last two paragraphs in 4.3 seem to only contain information presented earlier. Remove and use for something better.

- You state that your approach can identify the hyper-parameters. However, it is only one type of hyper-parameters that you talk about. Those such as batch size, optimiser and learning rate don’t feature in this. I’m assuming that your approach does not cater for these - please make clear.

- In terms of batch size this seems to be an easy way to thwart your approach. There is no discussion of batch size in your work, so one would assume that the work does not consider it. Therefore changing the batch size would change the size of the operations and hence break your trained model.

- I’d want to see evidence to support the statement “Without loss of generality we adopt”. I can imagine cases where a different combination of libraries could throw your system off.

- More details on the results of the fidelity extraction attack should be presented. It’s very tiny covered at the moment.

- The conclusions are very short and don’t add a lot to the paper.

summary_of_the_review: The paper presents an approach to ’steal’ a DNN from a NAS training process. There are strong claims as to the applicability of the approach across all NAS approaches. However, the authors make an awful lot of assumptions as to how NAS works and the sorts of networks they can produce. If you factor in the NAS approaches they are not considering and the fact that the work seems to be focused on CPU training (which is not the norm) then the work has far more limited applicability.

Although the work may work for GPU, there is no discussion in the text of this therefore it is not possible to assess if it would work or not.

As someone who reviews for security venues too, I’d imagine this work would score higher at one of those venues.
review_title: null

Meta-review: All reviewers agree on acceptance and I agree with them. I recommend a spotlight.
Decision: Accept (Spotlight)