Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Authors propose the “IDEAL” algorithm that can improve the distillation process when there is no data to be used for the distillation process, and also when the teacher provides only the hard-labels (no soft predictions over the labels). IDEAL employs a generator to generate examples on-the-fly in this data-free scenario. The generator aims to balance the class distribution and fit to one of the classes while the distillation progresses. In this particular setting (no-data and hard-labels), authors claim IDEAL can reduce the number of examples to be annotated significantly and still can achieve better performance on the given task.
strength_and_weaknesses: **Strength**
* Empirically strong in the particular distillation setup (no data, teacher can only provide hard-labels).
* Utilizing the generator in the data-limited distillation setup has some novelty, particularly in the joint training of the generator during the distillation process.

**Weakness**
* The impact is limited on the very specific distillation setup (data-free & hard-label setting). Moreover, the algorithm can only be applicable for the classification set-up, and does not seem to scale up when there are a large set of labels (thousands or millions).
* Authors seem to be over-claiming their impact. Particularly, I disagree with the author's claim that most KD assumes “users can directly access teacher’s training data.” Widely, KD focuses on a large set of unlabeled data because a teacher can provide pseudo-labels for them.
* Writings can be improved to focus on their core motivation, core idea, and core techniques. Please see the “clarity/quality section.” Because of this reason, many of the design choices in Section 3 feel a bit arbitrary.
* One of the baseline authors chosen, ZSDB3KD does seem to show 96.54% in their paper unlike the very low number in the current paper. Why is the number significantly different?

clarity,_quality,_novelty_and_reproducibility: **Clarity/Quality**
* Abstract and introduction are unclear how exactly this work differs from prior sample-efficient distillation works. Particularly, the abstract focuses mostly on problem settings, not on motivation of this work, nor high-level sketch of what they are trying to do (they just simply refer to it works with two stages (“data generation stage and queries the teacher only once ...”).
* Some terminologies are also misleading. For example, “white-box” and “black-box” refers to whether we have prior knowledge of model internals such as function classes or main assumptions. It does not mean the model class produces only the hard-labels (top1) or not (soft-labels). Hence, many classic distillation setup treats teacher models as black-box models (such as the original softmax-based distillation in Hinton’s paper [1] ).
* Minor question: What are the teacher models used in Section 4?

**Novelty**
There are mainly two lines of work that are closely related for this paper.
* [2] is a very close work that is in the same problem formulation (zero-shot, black-box, and decision base) -- hence, the problem formulation is not novel unlike what authors claim in Section 1.
* [3,4] proposes to use generators during the training and [4] particularly introduced during the distillation.
This paper is basically the combination of the two lines of work, and its novelty is in proposing the joint training of the generator that can maintain the class balance and the confidence, for the zero-shot setting. Hence, the paper does have some novelty.

[1] Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network." arXiv preprint arXiv:1503.02531 2.7 (2015).
[2] Wang, Zi. "Zero-shot knowledge distillation from a decision-based black-box model." International Conference on Machine Learning. PMLR, 2021.
[3] He, Xuanli, et al. "Generate, annotate, and learn: Generative models advance self-training and knowledge distillation." (2021).
[4] Zaheer, Manzil, et al. "Teacher Guided Training: An Efficient Framework for Knowledge Transfer." arXiv preprint arXiv:2208.06825 (2022).
summary_of_the_review: As discussed above, the paper has some contributions to the community. However, the paper does seem to be limited in a very specific setting and lack the clarity/quality for the acceptance bar.

---

Post-rebuttal: After reading author's response as well as other reviewer's recommendation, I agree that the paper has some original contribution to the community (although it's limited). Hence, I updated the score accordingly.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper addresses the problem of query-efficient learning from a black-box teacher model in a data-free way. This paper considers the additional constraint that only the hard labels (categorical predictions) of the teacher are available, rather than the soft-labels. The authors propose the IDEAL method for this problem which operates in two stages. First, the method uses the student model to train a generator that generates synthetic data. In the second stage, the synthetic data is used to perform hard-label distillation, i.e., train a student model so that its predictions match those of the teacher. The authors present empirical evaluations that demonstrate the query efficiency and effectiveness of the proposed approach.
strength_and_weaknesses: ## Strengths
* The problem of query-efficient, data-free learning from a black-box teacher that outputs only hard labels is well-motivated by real-world examples (e.g., Google BigQuery). It is a challenging problem that is of high relevance to the ML community.
* The proposed approach contains novel components compared to prior work and using the student model to generate a diverse set of synthetic data is interesting.
* Empirical evaluations on various data sets are presented that support the improved effectiveness of the proposed work. Based on the results, IDEAL improves over the state-of-the-art by more than 20% on the evaluated scenarios, which is quite impressive. This trend holds for larger data sets containing a larger number of classes as well. The hyperparameters for the experiments are reported for reproducibility.
* The authors present ablation studies that help justify the various components of the method (Table 4).
* The paper is well-written with a clear exposition overall. For example, the accompanying visualization of the data generated by different methods (Fig. 4) is quite interesting and helps understand the benefit of the method.

## Weaknesses
* The proposed two-stage approach of generating synthetic data -> distilling knowledge using generated data appears in prior work (Wang, 2021). The novelty seems to lie in the synthetic data generation stage.
* The synthetic data generation seems to be highly sensitive to the number of epochs $E_{\mathcal G}$ that the generator is trained for (see Appendix A.0.1). It has to be not too high and not too low. This is quite puzzling and the explanation in Sec. A.0.1 is not very compelling. My understanding of the synthetic data generation stage is to generate a diverse set of examples with varying (student predicted) labels. Why would training the generator for more epochs lead to “overfitting of the student?” Shouldn’t it be more conducive in generating a diverse set of synthetic data points?
* Standard deviations of the results in Sec. 4 averaged over 3 trials are not reported.
* Why is a scaling factor of $\lambda = 5$ used for the experiments? How was this choice made and how sensitive is the algorithm to this choice?
* How does the method fare in computational complexity relative to the compared approaches? My understanding is that the two stage process is done on a per-epoch basis, so it is not clear how much of a computational burden this imposes. To add to this concern, the authors mention downsizing the original images for the ImageNet subset for “fast training.” 

clarity,_quality,_novelty_and_reproducibility: Please see the section above for notes on clarity, quality, novelty, and reproducibility.
summary_of_the_review: This paper addresses a challenging problem that is motivated by real-world applications. The authors propose an approach that performs exceptionally well in practice compared to state-of-the-art. I have some concerns regarding the sensitivity of the method to its hyperparameters and some clarifying questions that I raised in the sections above. Overall, I lean towards acceptance and would be willing to raise my score if my concerns are adequately addressed.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: - The paper proposes a zero-shot Knowledge Distillation ("ZSKD") approach (i.e., does not require any samples from teacher's training data).
- Similar to DFME (Truong et al.) and MAZE (Karyiappa et al.), the approach consists of two phases per epoch: (a) training a generator to produce synthetic labels; and (b) training a student network using generator's synthetic images annotated by a black-box teacher model. Unlike previous approaches, the proposed approach is query-efficient since gradients for the generator no longer requires 0-th order gradient estimates. Rather the gradients are generator by backprop-ing through the partially trained student network.
- Results are verified on a number of datasets (CIFAR10, TinyImages, ...) and compared with many zero-shot KD baselines.
strength_and_weaknesses: ### Strengths

**1. Insight: Identifying query-inefficient bottleneck**
- Although the proposed approach follows a similar approach to baselines (i.e., iterative training of generator and student model), it rightly identifies and tackles a relevant pain-point: estimating the gradients to update the generator incurs a large query budget. It appears that the drastic improvement in query efficiency can be attributed to addressing this pain-point.

**2. Results - significantly better than prior art**
- I appreciate the extensive comparison of the proposed approach with 7+ baselines and on 7+ datasets. The results are furthermore promising: the proposed approach results in drastic improvements over baselines e.g., 37.91→68.82 in CIFAR10.

### Concerns

**1. (Major concern) Counter-intuitive formulation: Generator loss**
- I found counter-intuitive how the generator is trained in stage 1. For simplicity, let's assume a perfect generator and student network. In this case, the $L_CE$ would appear to be extremely large (in expectation) -- since the GT target $y$ is randomly drawn and compared with a random generation ($\hat{y} = S(G(z)), z \sim N(0, I)$). (Side-note: wouldn't a conditional generator make more sense?)
- On a more general note, while I understand the algorithm steps, I cannot understand the reasoning behind it. Why is the gradient signal though a noisy student network with mislabeled examples informative enough to perform ZSKD?

**2. KD vs. Model Stealing approaches**
- I get the impression that the paper tackles the *Model stealing/extraction* problem (black-box teacher, query budget, 
etc.), but constantly and unfairly compares to *Knowledge Distillation*.
- For instance, "KD methods are based on several unrealistic assumptions ... access teacher's training data ... white-box teacher". I don't think this is a correct claim, since KD approaches are predominantly tailored for model compression -- Hinton et al. 2015 claim that it can be used to "... compress the knowledge in an ensemble into a single model...". In which case, it's a perfectly valid assumption to use a white-box teacher and its corresponding training data.
- In contrast, the proposed approach is more consistent with model stealing works (several references missing here in the paper; please fetch and discuss relevant citations from DFME) where indeed the teacher model is a black-box prediction API. Unfortunately, the results fall short here -- randomly using a pool of publicly-available images over generated images results in significantly better accuracy scores as well as sample efficiency (typically with 50K queries, see references in Kariyappa et al., CVPR '20).
- Overall, my concern is that the paper considers KD works as a straw-man, as opposed to model extraction literature. However, I'm slightly overlooking this as the paper is "data-free" and does not rely on a publicly-available image data pool (at the expense of results degradation).

**3. Some misc. concerns**
- Table 1 "with a limited number of queries": please mention how many queries per method.
- "Contributions ... new problem ... training models with hard-labels": This has been studied plenty of times before e.g., Tramèr et al. '16, Knockoff Nets '20)
- APIs return only top-1 class: This is another unfair claim. Plenty of pay-per-query cloud APIs provide probabilities e.g., [Google Cloud](https://cloud.google.com/vision/docs/labels).
- The paper many times mentions a "distillation loss", but rather refers to simple cross-entropy given that there is no temperature-scaling factor.
- (Nitpick) Would be nice to have an additional "upper bound" column in Table 2, where the numbers reflect the teacher model trained on GT data with the relevant budget.
clarity,_quality,_novelty_and_reproducibility: **Clarity**: Very good. The paper is easy to follow and understand. One nitpick is mis-using definitions e.g., see concern 3 (CE loss referred to as distillation loss).

**Quality**: Average. The evaluation section is quite strong and thorough (numerous KD baselines and many datasets). A concern however is better insights into the data generation step of the approach (see concern 1).

**Novelty**: Average. The paper appears to use KD as a straw-man, while it is more consistent with model extraction formulations. Moreover, the general framework (generation + student network training) is very similar to DFME, MAZE, etc. Some suggestions on improving novelty: more elaboration/technical insights into the approach (under what conditions should it work? analysis), comparisons against model extraction literature, etc.

**Reproducibility**: Very good. Although the code is not provided, I am confident in reproducing results given the implementation details in the paper.
summary_of_the_review: Overall, the paper proposes a straight-forward approach with strong results (significant improvements over ZSKD baselines). However, a big concern I have is that the rationale of the approach is not evident to me (why does it work, given large losses in ideal circumstances). A somewhat minor concern I have is unfairly using KD as a straw-man (e.g., KD requires white-box teacher).
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper focuses on a very interesting issue: how to learn from black-box models using label-only data? For this purpose, they propose a novel method called IDEAL, which aims to query-efficiently learn from black-box model APIs in order to train a good student model without any real data. In detail, the proposed method trains the student model in two stages: data generation and model distillation. Their proposed IDEAL is query-efficient, as it does not require any query in the data generation stage and queries the teacher only once for each sample in the distillation stage. Results can well support that the proposed method consistently outperforms all the baselines.
strength_and_weaknesses: Strengths：

- This paper focuses on learning from label-only black-box models, an interesting but less explored topic. From my understanding, the proposed method is by far the most practical one that considers data-free, label-only, black-box, and query-efficient learning. In Table 1, it is clear that query-efficiently training a good student model from black-box models with hard labels is very practical, but challenging. Generally, I think the investigated problem is sound and interesting. I think this can be an extremely strong paper in black-box KD.

- The idea is novel and straightforward. The approach is technically sound. The experiment showcases solid performance improvement over baselines. Particularly in Table 2, the proposed  method has significantly better results than other baselines. In Fig. 3, this paper provides convincing and solid comparisons between baselines with small and large query budgets.

-The paper also conducted very detailed and convincing ablation studies. The ablation studies are quite thorough in my view. It’s easy to understand why this method works.

Overall, this paper is very interesting and to my knowledge novel. The presentation is clear and easy to follow. Technical details are clearly described. It seems like a pioneering contribution towards black-box KD. 

Weaknesses:
I didn't see major weakness in this paper actually. I am just curious about below questions:

1. How different generator sizes affect the results. Since in my opinion, synthetic data are affected by the quality of the generator. Maybe the authors can give some explanation on this and also in the future version some supplementary materials can be provided for more detailed analysis.  

2. Additionally, it will be interesting to investigate why DFME and ZSDB3 perform much worse than IDEAL, what's the cause? 

3. Also, is there any domain gap between synthetic data and original data? What will happen if we train the model from scratch with synthetic data?

4. Last, in the Balancing section: why it is necessary to generate the same number of samples in each class? Is there an imbalanced data distribution? 

clarity,_quality,_novelty_and_reproducibility: Clarity: Clear enough, this presentation is clear and easy to follow.

Quality: The paper is well motivated with clear intuition and illustration. The paper is technically sound. The whole paper is well structured and easy to follow. 

Novelty: Good, the main ideas of the paper are ground-breaking.

Reproducibility: good, key resources (code, data) are available and sufficient details (e.g., experimental setup) are well described.

summary_of_the_review: A strong paper in black-box KD. Technical details are clearly described. It seems like a pioneering contribution towards black-box KD. Hence, I would like to vote for acceptance.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper attempts to improve distillation or model extraction by reducing queries to the teacher model. It is a very important to reduce query complexity with large teachers becoming prevalent. In this regards, the authors propose a two phase approach: 1) using the current student model as a discriminator to train a generator that generates synthetic data, 2) the generated synthetic data is used to perform distillation from the teacher model. Empirical studies are conducted to show improvement in accuracy of student model for same teacher query budget over prior generative methods. There was a wide variance among reviewer scores for the paper and we thank authors and reviewers to engage in discussion towards improving the paper. Some reviewers had genuine concerns about novelty and training formulation, which remain unanswered after the author discussion. The claims (new problem ...) should be toned down as this problem has been well explored in literature both in distillation and model extraction community (also the reviewers pointed many particularly relevant citations).  Training generator with random label needs more explanation, e.g. as noise $z$ and label $y$ are sampled independently, very similar generated images might be assigned different labels and then it is not clear how generator is learning. Finally, to cover the whole landscape, which would be beneficial for the reader to know what options are available, there should be comparison to baselines using random publicly-available unlabeled images instead of generated ones. It can be clearly marked that such methods make additional assumption of availability of such data but can results in significantly better accuracy scores for the similar query budget (e.g. Kariyappa et al., CVPR '20).
justification_for_why_not_higher_score: - Novelty is limited as utilization of generative models has been explored in the literature
- Training formulation seems problematic, e.g. why generator is trained with random labels
- No comparison to baselines with random publicly-available images over generated images
justification_for_why_not_lower_score: - Good empirical performance compared to considered baselines
Decision: Accept: poster