Paper: Prompt Gradient Projection for Continual Learning (ICLR-2024)

Reviewer comment (Experiments):
Given that the L2P and DualPrompt backbones have been pre-trained on ImageNet, the limited performance gain on the 10-Split-TinyImageNet is a bit concerning (Table 3). Also, the gains on other dataset settings follow a similar trend. Alternatively, the work could have used a backbone trained on a different dataset altogether [1] and then evaluated its performance on the said settings. A third option would be to use other CL datasets like CUB-200, ObjectNet, etc. that have limited domain and style overlap with the ImageNet dataset.

Author response:
Dear reviewer LEcv, thanks for your valuable suggestions. Here are our responses:

For **Weakness 1** about *''use a backbone trained on a different dataset and use other CL datasets''*:

As your suggestion, we re-evaluated the effectiveness of our method by **extending two distinct pre-trained models**, namely **ViT-DINO** and **ViT-SAM**. The results are shown in the Table below. Additionally, we tested our method on the **CUB-200** dataset based on three ViT pre-trained backbones: ImageNet-1k, DINO, and SAM, further validating the effectiveness of our method on non-ImageNet datasets.

**Table 1**: Comparison to distinct pretrained-dataset backbones with/without prompt gradient projection.
|                          |                    | 10-Split-CIFAR100                                    | 10-Split-CIFAR100                                     | 5-Split-CUB200                                       | 5-Split-CUB200                                      |
| ------------------------ | ------------------ | ---------------------------------------------------- | ----------------------------------------------------- | ---------------------------------------------------- | --------------------------------------------------- |
| Method                   | Pretrained-Dataset | Accuracy                                             | Forgetting                                            | Accuracy                                             | Forgetting                                          |
| L2P                      | DINO[1]            | 67.35                                                | 9.69                                                  | 44.10                                                | 9.77                                                |
| **L2P-PGP(Ours)**        | **DINO**           | **70.60*****(+3.25)*** | **4.73*****(-4.96)***   | **44.80*****(+0.70)***                     | **6.06*****(-3.71)*** |
| DualPrompt               | DINO               | 64.18                                                | 23.87                                                 | 50.88                                                | 10.10                                               |
| **DualPrompt-PGP(Ours)** | **DINO**           | **73.33*****(+9.15)*** | **10.27*****(-13.60)*** | **51.03*****(+0.15)***                     | **9.06*****(-1.04)*** |
| L2P                      | SAM[2]             | 83.93                                                | 6.68                                                  | 73.98                                                | 6.77                                                |
| **L2P-PGP(Ours**         | **SAM**            | **84.26*****(+0.33)***                     | **5.64*****(-1.04)***   | **76.45*****(+2.55)*** | **5.91*****(-0.86)***                     |
| DualPrompt               | SAM                | 86.11                                                | 6.08                                                  | 82.02                                                | 4.73                                                |
| **DualPrompt-PGP(Ours)** | **SAM**            | **86.92*****(+0.81)***                     | **5.04*****(-1.04)***   | **82.28*****(+0.26)***                     | **4.65*****(-0.08)***                     |
| L2P                      | ImageNet-1K        | 83.77                                                | 6.63                                                  | 74.88                                                | 5.39                                                |
| **L2P-PGP(Ours)**        | **ImageNet-1K**    | **84.34*****(+0.57)***                               | **5.59*****(-1.04)***   | **75.15*****(+0.27)***                     | **4.51*****(-0.88)***                     |
| DualPrompt               | ImageNet-1K        | 86.50                                                | 5.77                                                  | 82.02                                                | 4.23                                                |
| **DualPrompt-PGP(Ours)** | **ImageNet-1K**    | **86.92*****(+0.42)***                     | **5.35*****(-0.42)***                       | **82.46*****(+0.44)***                     | **3.76*****(-0.47)***                     |

[1] DINO pre-trained models (https://arxiv.org/abs/2104.14294)

[2] SAM pre-trained models (https://arxiv.org/abs/2106.01548)

Reviewer comment (Evaluation):
The work claims that one of the major advantages of the proposed PGP method is its training time and memory cost. However, Table 2 does not report the training time of the baseline (L2P-R). As such, it is unclear what conclusions can be derived about the training time.

Author response:
For **Weakness 2** about *''the training time consumption''*:

To answer your question, we reconducted experiments about L2P with data rehearsal on the 10-Split-CIFAR100 dataset. we randomly sampled the exemplars and achieved a result akin to the report results in the paper. Then, we measured the time consumption of our reproduced L2P-R and compared it with our method **as shown in Table 2**. The time consumption of *our method* is **0.756 h** and the time consumption of *L2P-R* is **0.787 h**. The reason why our method is faster than L2P-R, we hold the view is that L2P-R needs to train the extra rehearsal data in every epoch, which spends a lot of time. The more rehearsal data is saved, the more training time is consumed. However, our method is rehearsal-free.

**Table 2**: Time Complexity Comparison, Accuracy, Forgetting between our approach and L2P, L2P-R. L2P-R indicates L2P with data rehearsal.
| Method            | Exemplar | Time        | Accuracy  | Forgetting |
| ----------------- | -------- | ----------- | --------- | ---------- |
| L2P               | 0        | 0.74 h      | 83.77     | 6.63       |
| L2P-R             | 1000     | 0.787 h     | 84.21     | 7.72       |
| **L2P-PGP(Ours)** | **0**    | **0.756 h** | **84.34** | **5.59**   |

To sum up, our method not only **exhibits superior performance** and **requires less storage memory** but also **consumes less training time**.

Reviewer comment (Experiments):
What concerns me is the scope of the proposed PGP method. While the authors use the classic L2P and DualPrompt as their baselines, it would have been more interesting to see how the proposed method complements the continual learning of more powerful pre-trained models such as vision-language models [2]. Also, the baselines considered in this paper are rather weak. Many latest methods are missing. Such as LoRA, K-Adapter, and other parameter-expansion methods.

Author response:
For **Weakness 3** about *''the scope of the proposed PGP method''*:

To further validate our method on the vision-language model, we introduce it to the **CLIP** model. The experimental setting is that, in the vision side, we only set a single trainable image prompt shared by all tasks. As for the text side, we follow the operation as [3], we set trainable text prompt for each class, which is only trained at the corresponding task. We conduct our experiments on two datasets: *10-Split-CIFAR100* and *10-Split-TinyImageNet*, **as shown in Table 3**. Besides that, we also verify the effectiveness of our method under the task incremental setting, **as shown in Table 4**. Results show that our method has improved the performance a lot for all the above settings, which we think proves that our method is also **useful in the vision-language models and further enlarges the scope of our method**.

We have noticed that, compared with your mentioned method, our method does not exhibit comparable accuracy performance, despite conducting experiments on entirely different datasets with [4]. The reasons are as follows: **Firstly**, in comparison to its **complex method and network structure**, we employ a **very simple setting**, utilizing only a single image prompt, which restricts the performance of our method. **Secondly**, it **employs the data rehearsal method** to further enhance performance. However, our method is **rehearsal-free**.

In our design, we want to exclude all of the external influential factors and only verify the PGP method. Thus, we choose the simplest experiment setting of a single image prompt. Experiments show that our method works well.

**Table 3**: Comparison to *CLIP* model without/with gradient projection method under **class incremental setting** (**without** task identifier in the test phase).
|                    | 10-Split-CIFAR100 | 10-Split-CIFAR100 | 10-Split-TinyImageNet | 10-Split-TinyImageNet |
| ------------------ | ----------------- | ----------------- | --------------------- | --------------------- |
| Models             | Accuracy          | Forgetting        | Accuracy              | Forgetting            |
| CLIP               | 58.95             | 6.29              | 56.28                 | 7.19                  |
| **CLIP-PGP(ours)** | **63.72*****(+4.77)*** | **5.09*****(-1.20)***  | **60.34*****(+4.06)***     | **6.21*****(-0.98)***      |

**Table 4**: Comparison to *CLIP* model without/with gradient projection method under **task incremental setting** (**with** task identifier in the test phase).
| Models             | Accuracy               | Forgetting            |
| ------------------ | ---------------------- | --------------------- |
| CLIP               | 92.69                  | 2.34                  |
| **CLIP-PGP(ours)** | **93.00*****(+0.31)*** | **1.58*****(-0.76)*** |

As for the latest methods, such as LoRA, K-Adapter, and other parameter-expansion methods. Since they mainly focus on fine-tuning a large model, whose goal is to adapt the model to the down-stream tasks, we are afraid this is beyond the scope of this paper, which aims to learn new concepts while not forgetting the prompts. Also, the combination of gradient projection and fine-tuning methods would be interesting. We would like to study this in the future.

[3] Zhou, Kaiyang, et al. "Learning to prompt for vision-language models." International Journal of Computer Vision 130.9 (2022): 2337-2348.

[4] Zhou, Da-Wei et al. “Learning without Forgetting for Vision-Language Models.” ArXiv abs/2305.19270 (2023): n. pag.

Reviewer comment (Writing):
Definition of V_t missing - the paper introduces it as a two part column vector (before eq. 11) without explaining what does it contain. The same for U_t (prior to eq. 10).

Author response:
For **minor comments** about *''Definition of V_t and U_t missing''*:

Thanks for your valuable suggestions. We have **added the missing definition of** ***V_t*** **and** ***U_t***.

Reviewer comment (Writing):
The paper's overall readability and clarity could be enhanced. Specific instances of unclear language were noted, such as in the abstract and the last paragraph of page 3. Further revisions are needed to make these sections more lucid.

Author response:
Dear reviewer epe6, thanks for your valuable suggestions. Here are our responses:

For **Weakness 1** about *''Readability and Clarity''*:

We have rewritten the abstract and the last paragraph of page 3, and further revised several sections of the paper where language ambiguity exists. We thank your suggestion which makes our paper clearer and more readable than before. The following are two examples of revised version:

*For Abstract*

Before revised: Prompt-tuning has demonstrated impressive performance in continual learning by querying relevant prompts for novel classes training.

**After revised**: Prompt-tuning has demonstrated impressive performance in continual learning by querying relevant prompts for each input instance, which can avoid the introduction of task identifier.

Before revised: In PGP, we deduce the orthogonal condition for prompt gradient via the self-attention mechanism in vision-transformer.

**After revised**: In PGP, we deduce that reaching the orthogonal condition for prompt gradient can effectively prevent forgetting via the self-attention mechanism in vision-transformer.

*For the last paragraph of page 3*

Before revised: one limitation of gradient projection methods is which is only applicable for task incremental learning but will fail in the class-incremental inference, as the projected gradient is a strict constraint for novel classes training.

**After revised**: one limitation of gradient projection methods fail in the class-incremental inference is that the projected gradient needs task identifier to find relevant network parameters.

Reviewer comment (Presentation):
Figure 2 was found to be unclear and challenging to follow, especially for readers seeking to grasp the method's essence. Consider revising or providing additional explanatory details to improve the comprehensibility of this figure.

Author response:
For **Weakness 2** about *''Figure 2''*:

We have **enhanced the clarity** of *Figure 2* by providing a more detailed explanation in the caption, elucidating the content and essence of our method, and **further improving the comprehensibility** of the image.

Reviewer comment (Writing):
While the proof on page 5 is well-explained up to equation 12, the subsequent section, where multiple elements are combined, is challenging to follow. Expanding on the last two paragraphs on page 5 could enhance clarity and understanding.

Author response:
For **Weakness 3** about *''Proof Presentation''*:

We have **expanded** the content in the *last two paragraphs* of *page 5*, providing a more compact connection with the preceding and following sections. This adjustment is intended to enhance the clarity and comprehensibility of the paper.

Reviewer comment (Writing):
On page 6, there is mention of dividing $V_t$ into $V_{t,1}$ and $V_{t,2}$ while also referencing $V_{t,0}$. Clarification is needed to ensure consistency and prevent potential confusion, possibly by referring back to the explanation on page 5.

Author response:
For **Weakness 4** about *''Terminology and Consistency''*:

In the revised version, we have **replaced** *V_t, V_1, and V_2* with **new symbolic representations** and **added detailed explanatory text** to ensure consistency with the preceding context and eliminate any potential confusion.

For **Weakness 5** about *''Notation Clarification''*:

Thank you for your question. It has been **addressed along with the previous one**.

Reviewer comment (Writing):
On page 6, there is a reference to "$V_{t,}$" which requires clarification to make it more understandable to the reader. Providing a clear definition or context for this notation would be beneficial.

Author response:
For **Weakness 4** about *''Terminology and Consistency''*:

In the revised version, we have **replaced** *V_t, V_1, and V_2* with **new symbolic representations** and **added detailed explanatory text** to ensure consistency with the preceding context and eliminate any potential confusion.

For **Weakness 5** about *''Notation Clarification''*:

Thank you for your question. It has been **addressed along with the previous one**.

Reviewer comment (Theory):
In the paragraph preceding equation 1, the statement "If the update direction is orthogonal to the old features, it follows that $\Delta W_t x_{t,i}=0$" lacks a specific reference. It would be beneficial to provide a reference or additional context to support this claim and enhance the paper's credibility.

Author response:
For **Question 1** about *''Reference Request''*:

We have inserted the following two pertinent references into the statement location as advised.

[1] Saha, Gobinda, Isha Garg, and Kaushik Roy. "Gradient Projection Memory for Continual Learning." International Conference on Learning Representations. 2020.

[2] LiLin, Sen, et al. "TRGP: Trust Region Gradient Projection for Continual Learning." International Conference on Learning Representations. 2021.

Reviewer comment (Evaluation):
The paper would benefit from a more detailed discussion of the method's time complexity, along with a comparative analysis against relevant baseline methods. Understanding the computational efficiency of the proposed approach in relation to other methods is crucial for assessing its practical utility.

Author response:
For **Question 2** about *''Time Complexity''*:

Thanks for pointing out our omission.  

Under the same hardware platform, experimental setting, and backbone, we compared our method with the original L2P and L2P with data rehearsal on the *10-Split-CIFAR100* dataset. As L2P does not mention detailed information about how to make data rehearsal, we adopted a simple data rehearsal process which randomly sampled a set of exemplars and achieved a result similar to the reported ones.

The results indicate that, in comparison with the *original L2P*, our approach reduces forgetting by 1.04%, improves accuracy by 0.57%, and *extends* training time by *0.016* hours. In comparison with **L2P with data rehearsal**, our method not only reduces forgetting by 2.13%, improves accuracy by 0.13%, but also **shortens** training time by **0.031** hours.

**Table 1**: Time Complexity Comparison between our approach and L2P, L2P-R. L2P-R indicates L2P with data rehearsal.
| Method            | Exemplar | Accuracy  | Forgetting | Time        |
| ----------------- | -------- | --------- | ---------- | ----------- |
| L2P               | 0        | 83.77     | 6.63       | 0.74 h      |
| L2P-R             | 1000     | 84.21     | 7.72       | 0.787 h     |
| **L2P-PGP(Ours)** | **0**    | **84.34** | **5.59**   | **0.756 h** |

For further analysis, we use **“time.time()”** function to coarse read the time consumption in each sub-process.

**For data rehearsal**, it concludes in three steps: **1. Collect exemplars (4\*10-5~26\*10-5 s)**, which can be ignored. **2. Store exemplars (Simplified in our reimplement method)**. **3. Exemplars join in the training process**. We mainly discuss the time consumption in the third step: In the batch size of 20, one iteration that only exemplars take part in costs a mean time of: 0.196 s. Here, we set epochs as 5 in the experiment. Thus, on the 10-Split-CIFAR100, training with exemplars needs extra (5+10+15+20+25+30+35+40+45) * 5 = 1125 iterations and costs 1125 * 0.196 = **220.50 s**. To sum up, theoretical analysis shows that data rehearsal would add an extra **220.50 s** in the training process.

**For our method**, it also concludes three steps: **1. Collect samples (10-6\*33\*5\*10 s), which can be ignored**. **2. Calculate the projection matrix**, consumption time is (10.65 + 8.78 + 9.42 + 8.72 + 8.68 + 8.68 + 8.53 + 8.81 + 8.72 + 8.69 = 81 s). 3. **Gradient projection process**, consumption time is (2.79 + 3.22 + 2.95 + 2.73 + 3.11 + 3.28 + 2.72 + 3.01 + 2.78 + 2.98 = 29.57 s). To sum up, theoretical analysis shows that gradient projection would add an extra 81 + 29.57 = **110.57 s** in the training process.

Notice that, we set 5 epochs in the training process. If we increase the epoch number, the consumption time of L2P-R would also increase due to more iterations of training exemplars. However, our method is unrelated to the epoch number and can be free of increasing epochs.

Reviewer comment (Evaluation):
The paper mentions a maximum improvement of around 1% in reducing catastrophic forgetting in the reported experimental results. It is essential to provide a more in-depth discussion of the trade-off between the promising performance of the proposed algorithm and its associated time complexity. Explaining why this algorithm should be prioritized over other existing algorithms, considering the modest improvement, would provide valuable insights.

Author response:
For **Question 3** about *''Performance vs. Time Complexity Trade-off''*:

Thanks for your valuable suggestions.

(1) Discussion of the promising performance of the proposed algorithm and its associated time complexity:

**i)** For Time: In the answer to time complexity, we have shown detailed information about time consumption in our method. In fact, both the time consumption on calculation of projection matrix and gradient projection process is little, only adding **2%** extra training time on the original baseline.

**ii)** For Performance: Our approach demonstrates a high generalization ability, could be combined with the almost prompt-based continual learning methods. Besides that, from the perspective of gradient projection, we further realize a well-done balance of the model's plasticity and stability. This balance is crucial for continual learning, causing a favorable balance often results in enhanced performance.

(2) Explaining why this algorithm should be prioritized over other existing algorithms:

To the best of our knowledge, we are the first to propose the gradient projection for reducing forgetting in the context of prompt-tuning. Note that the tendency of vision foundation model has recently caused significant changes in computer vision. Our approach first studies the gradient of prompt-learning and we believe related applications, like fine-tuning a VFM, could benefit from our paper. **We have additionally tested the large vision foundation model for continual learning, and the results turn out our approach is also effective in this scenario. Please kindly refer to Table 2**. Here, for the vision side, we set a single trainable image prompt shared by each task. For the text side, we follow the operation with [3], we set trainable text prompt for each class, only trained at the related task.

**Table 2**: Comparison to *CLIP* model without/with gradient projection method on 10-Split-CIFAR100 with **class incremental setting** (**without** task identifier in the test phase).
|                    | 10-Split-CIFAR100  | 10-Split-CIFAR100 | 10-Split-TinyImageNet | 10-Split-TinyImageNet |
| ------------------ | ------------------ | ----------------- | --------------------- | --------------------- |
| Models             | Accuracy           | Forgetting        | Accuracy              | Forgetting            |
| CLIP               | 58.95              | 6.29              | 56.28                 | 7.19                  |
| **CLIP-PGP(ours)** | **63.72*****(+4.77)*** | **5.09*****(-1.20)*** | **60.34*****(+4.06)***    | **6.21*****(-0.98)***     |

Here, we also compare our method with other **two non-gradient and prompt-based methods**, one is data rehearsal, and the other is a paper titled "Introducing Language Guidance in Prompt-based Continual Learning", referred to as **LGCL**, presented at ICCV’23 [4]. LGCL is slightly lower than our method but with complicated designs, **as shown in Table 3**. Compared with baseline and IGCL, we have found we own the following three advantages.

**1.** For memory, we only need to store a tiny projection matrix. While for data rehearsal, saving exemplars would spend far more memory space than us. For LGCL, although it is a rehearsal-free method, it has to introduce another network, a pre-trained text encoder: CLIP L/14 into the method, which also needs to occupy a significant amount of memory space.

**2.** For time, extra time in our method, mainly costs on calculation of projection matrix and gradient projection process, which is shorter than the retraining process of exemplars in the data rehearsal method, explained in the answer of time complexity. For LGCL, due to the non-open source code, we cannot reproduce the experiments. However, in the paper, it adds an additional text encoding process and adopts extra loss functions to train the model. Therefore, compared to LGCL, our method also exhibits shorter time complexity.

**3.** Our method could serve as a plug-in to reduce forgetting of the models in almost all of the prompt-based continual learning methods. With the hot tendency of the large models nowadays, our method could be transferred to the large Vision Foundation Models (VFMs) and have a promising application future in the fine-tuning paradigm.

**Table 3**: Comparison to Accuracy, Forgetting of L2P-LGCL and L2P-PGP.
|                   | 10-Split-CIFAR100 | 10-Split-CIFAR100 |
| ----------------- | ----------------- | ----------------- |
| Method            | Accuracy          | Forgetting        |
| L2P-LGCL          | 84.33±0.06        | 5.83±0.23         |
| **L2P-PGP(Ours)** | **84.34±0.08**    | **5.59±0.05**     |

[3] Zhou, Kaiyang, et al. "Learning to prompt for vision-language models." International Journal of Computer Vision 130.9 (2022): 2337-2348.

[4] Khan, Muhammad Gul Zain Ali, et al. "Introducing Language Guidance in Prompt-based Continual Learning." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

Reviewer comment (Experiments):
The comparison of prompt-tuning based CL methods with other CL methods does not seem fair to me. Prompt-tuning based methods use pretrained backbones which will give an performance edge when comparing against non-prompt-tuning based CL methods.

Author response:
Dear reviewer Q1LH, thanks for your valuable suggestions. Here are our responses:

For **Weakness 1** about *''The comparison of prompt-tuning based CL methods with other CL methods''*:

**Thanks for the advice. The backbone is actually pre-trained on ImageNet, and therefore it is somewhat unfair to compare with non-prompt-tuning based CL methods.** 

**However**, the tendency of the pre-trained large models, or vision foundation model has recently caused significant changes in computer vision. This paper focuses on reducing the forgetting of prompt-tuning paradigm, which, we believe, would bring new insights for continual learning. **We have additionally tested a lot of other baselines, especially including large vision foundation model, and the results turn out our approach is also effective in this scenario. Please kindly refer to Table 1.** Here, the methods in Table 1 are from 1) Replacing the pre-trained backbone of ImageNet-1K in L2P and DualPrompt with SAM and DINO. 2) Introducing prompt gradient projection method to CLIP model. For the vision side, we set a single trainable image prompt shared by each task. For the text side, following the operation with [1], we set trainable text prompt for each class, which is only trained at the corresponding task.

**For the unfair comparison, please kindly note that:** **1)**, in our baseline (L2P and DualPrompt  [2, 3]), they have already compared their methods (adopting pre-trained backbones) with non-prompt-tuning-based CL methods (e,g. LwF, EWC, BiC, DER). We just simply follow their existing practice. We would denote the backbone comparison in our revised paper. **2)**, Our goal is to analyze the gradient space for prompt-tuning-based CL methods and propose a gradient projection method that effectively reduces forgetting. Therefore, our focus is mainly on observing performance improvements compared to the baselines (L2P and DualPrompt), non-prompt-tuning based CL methods are used as references. 

**Table 1:** Comparison to distinct baselines with/without prompt gradient projection method on 10-Split-CIFAR100.

| Models               | Pre-trained backbone | Accuracy                                | Forgetting                               |
| -------------------- | -------------------- | --------------------------------------- | ---------------------------------------- |
| L2P                  | DINO[4]              | 67.35                                   | 9.69                                     |
| **L2P-PGP(ours)**    | **DINO**             | **70.60*****(+3.25)*** | **4.73*****(-4.96)***   |
| L2P                  | SAM[5]               | 83.93                                   | 6.68                                     |
| **L2P-PGP(ours)**    | **SAM**              | **84.26*****(+0.33)***                        | **5.64*****(-1.04)***   |
| DualPrompt           | DINO                 | 64.18                                   | 23.87                                    |
| **DualPrompt(ours)** | **DINO**             | **73.33*****(+9.15)*** | **10.27*****(-13.60)*** |
| DualPrompt           | SAM                  | 86.11                                   | 6.08                                     |
| **DualPrompt(ours)** | **SAM**              | **86.92*****(+0.81)***                        | **5.04*****(-1.04)***   |
| CLIP                 | -                    | 58.95                                   | 6.29                                     |
| **CLIP-PGP(ours)**   | **-**                | **63.72*****(+4.77)*** | **5.09*****(-1.20)***   |

[1] Zhou, Kaiyang, et al. "Learning to prompt for vision-language models." International Journal of Computer Vision 130.9 (2022): 2337-2348.

[2] Wang, Zifeng, et al. "Learning to prompt for continual learning." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[3] Wang, Zifeng, et al. "Dualprompt: Complementary prompting for rehearsal-free continual learning." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.

[4] DINO pre-trained models (https://arxiv.org/abs/2104.14294).

[5] SAM pre-trained models (https://arxiv.org/abs/2106.01548).

Reviewer comment (Presentation):
Figure 1 is a bit unintuitive as normally, models that fill the radar chart are considered to be more powerful but in this case for FOR metrics being close to the center is better and for ACC metrics being fuller is better. Changing the FOG figure so higher is better would make the Figure more readable in my opinion.

Author response:
For **Question 1** about *''Figure 1 is a bit unintuitive as normally''*:

We have **reconfigured** *Figure 1* according to your suggestion and made it more intuitive. We appreciate your valuable feedback which has enhanced the quality of our paper.