Paper: Towards Robust and Efficient Cloud-Edge Elastic Model Adaptation via Selective Entropy Distillation (ICLR-2024)

Reviewer comment (Novelty):
From algorithmic perspective (low-informative data identification), the novelty is limited. There should be existing papers studied how to filter out low-informative data. These can be added into the paper related works and experiments to compare.

Author response:
A2. Thank you for your insightful suggestions. We clarify our differences compared with existing low-informative sample identification methods in the following: 

**Differences from Existing Low-informative Sample Identification Approaches**: In our CEMA, we adopt the entropy to measure the sample's information, as it is easy to use and efficient. Unlike existing methods used in active learning and domain adaptation that may rely on a whole dataset [A] or less-efficient multiple forward propagations [B], the entropy can be calculated over a single sample and only involves one-time forward propagation. For example, Sener et al. [A] propose a core-set approach to select samples that can mostly represent the whole dataset. SENTRY [B] measures the sample information via the prediction consistency regarding different data augmentations. We have added more discussions regarding this in the related work section.

Compared with ETA[C] which also exploits entropy for sample identification, our sample selection strategy is different in the two aspects: 1) ETA adopts a static threshold to select unreliable samples, while we reveal that the suitable threshold may change continuously along with the online adaptation process and propose a dynamic unreliable sample identification strategy; 2) we further introduce a new low-informative sample selection strategy to identify samples that produce negligible gradients for model updating. The empirical comparisons with ETA have been presented in Tables 1-3 in our manuscript.

We would like to point out that identifying low-informative samples is only a mere aspect of our paper, as we re-clarified our main contributions and novelty at the beginning of the responses.

**Empirical Comparisons**: To further demonstrate the effectiveness of our sample identification strategy, we add more experiments on ImageNet-C to compare our CEMA with SENTRY [B] and ETA [C]. From Table A, SENTRY achieves worse accuracy than our CEMA (45.6% vs. 51.1%). The results show the prediction consistency regarding different data augmentations is hard to identify the helpful samples in entropy minimization. Besides, ETA yields an adaptation accuracy of 50.1%, which is still worse than our CEMA. Since ETA only filters out a part of unreliable samples by employing a fixed threshold. The results demonstrate the effectiveness of our CEMA in filtering out low-informative samples.

We have included these comparisons and discussions in Sections A and E.10 of our revised manuscript.


Table A. Comparisons of different schemes to identify unreliable and low-informative samples on ImageNet-C (Gaussian noise, severity level 3).
| Method| Acc. (%) | # Uploads |
| --- | --- | --- |
| SENTRY [B] | 45.6| 17,299|
| ETA [C] | 50.1| 18,862|
| CEMA (Ours) | **51.1** | 17,479|

[A] Active Learning for Convolutional Neural Networks: A Core-Set Approach. ICLR 2018.

[B] Sentry: Selective entropy optimization via committee consistency for unsupervised domain adaptation. ICCV 2021.

[C] Efficient Test-Time Model Adaptation without Forgetting. ICML 2022.

Reviewer comment (Writing):
In equation 6, is fθ(x) and the pseudo labels yˆ the same thing or different?

Author response:
A3. The model logits $f_{\theta}(x)$ and the pseudo label $\hat{y}$ are **different**. We detail the differences in the following: 1) $f_{\theta}(x)$ is a vector that represents the model logits of the corresponding sample $x$. It is used to calculate the KL loss to align the prediction distribution of the foundation and edge models. 2) $\hat{y}$ is a one-hot vector that denotes the pseudo label of the sample $x$. It is used to calculate the CE loss to align the decision boundaries. Note that $\hat{y}$ can be calculated through $f_{\theta}(x)$ by $\hat{y}=\text{argmax}(f_{\theta}(x))$.

We have made it clearer in Section 2.3 of our revised manuscript.

Reviewer comment (Experiments):
To use KL divergence loss and CE loss together is interesting. I wonder how the hyperparameter alpha and beta change in different dynamic scenarios.

Author response:
A4. We have examined the impact of the hyperparameters $\alpha$ and $\beta$ on the ImageNet-C (Gaussian noise, severity level 3) dataset (See Tables 10-11 in the supplementary material). The results indicate that the adaptation performance is not sensitive to these hyperparameters. In other scenarios on various datasets, we keep them the same as those on Gaussian noise, including ImageNet-R and ImageNet-C (encompassing 15 types of corruption and 5 severity levels, 75 different scenarios in total). Extensive experiments show that our CEMA works well with these hyperparameters.

We have made it clearer in Section E.2 of our revised manuscript.

Reviewer comment (Experiments):
The proposed method is only evaluated on classification tasks. Could the proposed method be extended to other tasks such as object detection?

Author response:
A1. We conduct more experiments by applying our CEMA on corrupted COCO 2017 [A] with the YOLOv5 [B] model. We generate this corrupted version (Gaussian noise, severity level 3) of the COCO 2017 dataset following ImageNet-C. The number of total validation images in corrupted COCO is 5,000. We use YOLOv5-nano as the edge model and YOLOv5-large as the foundation model. From Table A, our CEMA outperforms the baseline (18.3 vs. 11.6 mAP) and ETA (18.3 vs. 15.0 mAP). Moreover, this improved performance was achieved with a smaller number of samples uploaded for adaptation. The results demonstrate the applicability and effectiveness of our CEMA on the object detection task.

We have included these comparisons and discussions in Section E.11 of our revised manuscript.

Table A. Comparisons on corrupted COCO 2017 (Gaussian noise, severity level 3).

| Model                  | mAP     | # Uploads     |
| ---------------------- | :-------: | :-------------: |
| YOLOv5-nano (baseline) | 11.6    | --            |
| $\bullet~$ ETA         | 15.0    | 3,875 (78%)     |
| $\bullet~$ CEMA (Ours) | **18.3** | **3,261 (65%)** |

[A] Microsoft COCO: Common Objects in Context. arXiv 2014.
[B] You Only Look Once: Unified, Real-Time Object Detection. CVPR 2016.

Reviewer comment (Experiments):
What's the performance if no teacher model is used?

Author response:
A3. To verify the effectiveness of the teacher model in distillation, we have conducted experiments on ImageNet-C (Gaussian noise, severity levels 3 and 5). The results have been presented in Table 12 of the supplementary material. From Table A, our CEMA with the teacher model outperforms CEMA without that (51.1% vs. 50.0% in severity level 3). The improvement can be attributed to the teacher model's ability to leverage its knowledge of out-of-distribution data and transfer this knowledge to the edge model. These results collectively demonstrate the effectiveness of the teacher model in enhancing the adaptation accuracy of the edge model.

We have made these comparisons more clear in Section E.3 of the revised manuscript.

Table A. Effect of the teacher model in our CEMA on ImageNet-C (Gaussian noise, severity level 3). Note that the results have already been presented in Table 12 of the supplementary material.

| Teacher Model | Acc. (%) | # Uploads |
| ------------- | -------- | --------- |
| $\times$      | 50.0     | 17,263    |
| $\checkmark$  | 51.5     | 17,479    |

Reviewer comment (Novelty):
The selection scheme used in the paper is relatively simple, as mentioned in Q1.

Author response:
Thanks for your valuable feedback. To further demonstrate the effectiveness of our sample identification strategy, we add more experiments on ImageNet-C to compare our CEMA with more existing strategies, namely SENTRY [A] and BALD [B]. SENTRY measures the sample information via the prediction consistency regarding different data augmentations. In addition, BALD achieves this by calculating the entropy differences between the current sample and previous samples.

As presented in Table A, our CEMA outperforms SENTRY, demonstrating a significantly higher accuracy (51.1% vs. 45.6%). This suggests that SENTRY's reliance on prediction consistency is less effective in identifying beneficial samples for entropy minimization. Furthermore, BALD, with an adaptation accuracy of 50.7%, also falls short of the performance achieved by our CEMA. These comparisons underscore CEMA's superior capability in discerning low-informative samples.

We have included these comparisons and discussions in Section E.10 of our revised manuscript.


Table A. Comparisons of different strategies to identify unreliable and low-informative samples on ImageNet-C (Gaussian noise, severity level 3).
| Method      | Acc. (%) | # Uploads |
| ----------- | -------- | --------- |
| SENTRY [A]  | 45.6     | 17,299    |
| BALD [B]    | 50.7     | 17,319    |
| CEMA (Ours) | **51.1** | 17,479    |


[A] Sentry: Selective entropy optimization via committee consistency for unsupervised domain adaptation. ICCV 2021.

[B] Bayesian active learning for classification and preference learning. arXiv 2011.

Reviewer comment (Reproducibility):
In Section Identification on low-informative samples, the author claims that ‘We emphasize that uploading samples does not block the edge from inferring on next incoming samples. In other words, the processes of inference and uploading can be executed simultaneously.’ How do the authors decide which test samples use which updated model to make a prediction? More explanations are required.

Author response:
A1. In the edge device, **we would infer a sample for only one time via the current edge model**. Specifically, we feed a sample into the edge model and then determine whether this sample would be uploaded to the cloud based on Eqn. (4). In the cloud, once receiving a batch of $N$ uploaded samples, the edge model would be adapted for one time via Eqn. (6). After adaptation, the edge model would update the parameters from the cloud. Then the subsequent samples are processed using this newly updated edge model.

We have also taken into account scenarios of poor network connectivity (see Table 18 of the supplementary material). In this case, the edge model would be adapted for $K$ time ($K$>1) via Eqn. (6) in the cloud. The parameters updated through these adaptations are then sent back to the edge, ensuring that the next incoming sample is inferred using the most recently updated edge model.

We have included these discussions in Section B.4 of our revised manuscript.

Reviewer comment (Theory):
In Equation (6), the foundation model assigns pseudo labels to the uploaded samples. Simultaneously, the authors employ entropy to update the model, introducing another pseudo label (the maximum value). Is there a potential conflict between these approaches? What consequences might arise if the entropy loss is eliminated?

Author response:
A2. **There are no conflicts between the entropy minimization and cross entropy (via pseudo labels)**, since they contribute different effects in our CEMA: 

- **Cross Entropy with Pseudo Labels**: We use the cross entropy via pseudo labels to provide direct label guidance for a single sample to align the decision boundary.
- **Entropy Minimization**: We optimize the average entropy (also namely uncertainty) over a batch of samples to improve the overall model confidence regarding its predictions, rather than only encouraging a single sample to converge towards its highest predicted probability.

**Empirical Comparisons**. In Table A, we conduct ablation studies to investigate the effectiveness of the entropy loss and the CE loss. Both the cross entropy and entropy losses work together to boost the overall cloud-edge model adaptation performance. From the results, our CEMA with CE Loss achieves better adaptation accuracy than CEMA without that. Besides, while we employ both CE loss and Entropy loss, the adaptation performance further increases. The experimental results demonstrate these losses are complementary and effective in our CEMA.

We have included these comparisons and discussions in Section E.3 in our revised manuscript.


Table A. Effect of components in the loss for updating the edge model on ImageNet-C (Gaussian noise, severity level 3).
| CE Loss      | Ent Loss     | Acc. (%) | # Uploads |
| ------------ | ------------ | -------- | --------- |
| $\times$     | $\checkmark$ | 50.5     | 17,379    |
| $\checkmark$ | $\times$     | 50.2     | 18,497    |
| $\checkmark$ | $\checkmark$ | 51.5     | 17,479    |

Reviewer comment (Writing):
In Algorithm 1, line 3, ‘Calculate S(X) via Eqn. (x)’, It is confused what is Eqn.(x).

Author response:
A3.Thank you for your valuable suggestions. We have addressed the typo in the revised manuscript, and it should now correctly reference "Eqn. (4)."