Paper: Towards Personalized AI: Early-stopping Low-Rank Adaptation of Foundation Models (ICLR-2024)

Reviewer comment (Theory):
No theoretical background or explanation on why CS-Fluctuation is a good indicator of overfitting. Why does this work?

Author response:
1.Theoretical Foundation of the Method:
We recognize that the current approach may lack a complete theoretical framework. However, through a series of experiments, we have found that using CS-Fluctuation as a metric is significantly effective. The cosine similarity effectively measures the change in direction between model weight vectors, thus reflecting the learning state of the network. Experimental results show that this method is effective in preventing overfitting during fine-tuning, and the role of empirical evidence cannot be ignored.

Reviewer comment (Experiments):
As CS-Fluctuation is a strictly empirical criterion, more quantitative experiments and analyses is needed to support the validity of this early-stopping method. Also, what happens if you vary N in the N-th valley early stopping?

Author response:
2.Quantitative Analysis:
The field of personalized AI in fine-tuning the foundation models and preventing overfitting is still in a developmental stage. LDMs models currently have no effective metrics to check if the model is overfitting, we can only use the generated results to make a manual judgment. 

Besides, Our approach is based on the training step deciding to early stop, and returning the LoRA model for the epoch of the corresponding training step, and using it to generate qualitative results (generating images). Therefore, the "overfitted LoRA" results we show are exactly what you would expect after several valleys. As the training progresses, the overfitting of the model will gradually increase, leading to further degradation of the quality of the generated images.

Reviewer comment (Theory):
The proposed method lacks intuitive understanding and theoretical guarantee. It is hard to fully understanding why the proposed method is reasonable, especially why the metric is based on the cosine similarity between the model weights?

Author response:
Although our method may lack the traditional full theoretical framework, it has demonstrated significant results in a series of experiments. We chose cosine similarity as the metric mainly because it is a straightforward and effective measure of the directional change between model weigh vectors, thus reflecting the learning state of the network. Experimental results show that this approach is effective in preventing overfitting during fine-tuning, and the role of empirical evidence cannot be ignored.

Reviewer comment (Evaluation):
The scale of the metric ($\approx1e-7$) is too small and changes from data to data. Some normalization is needed for the metric.

Author response:
As shown in Equation 4, we have already normalized the learning rate to eliminate the effect of learning rate on CS-Fluctuation. We agree that further normalization of CS-Fluctuation is necessary to ensure its validity in different settings.

Reviewer comment (Presentation):
In figure 1, what is the connection between Epoch and training steps? It seems the first two epoch is sufficient for model fine-tuning from the figure?

Author response:
Epoch and training steps actually correspond to each other and represent different ways of controlling training process. The formula is: Training steps = Repeat * Train set size * Epochs / Batch Size. In Figure 1, the better-performing LoRA models are the results of Epoch 2 and Epoch 3. This conclusion can also be reached and validated through manual judgment of the results generated by the LoRA model. 

When dealing with a large number of personalized modeling tasks, the absence of a metric like CS-Fluctuation would result in having to manually selection for each tasks, leading to significant time consumption.

Reviewer comment (Experiments):
From the figure 3, it seems there is no clear signal which training steps is better. Why not early stop the method at the first $K$ epoch? set $K=2$ according figure 1.

Author response:
Our approach is based on training steps to decide on early stopping and returns the LoRA model of the epoch in which the selected training step is located. We do not use the model at the exact training step moment because it may be overfitted to the current batch of data. 

Regarding why we do not directly early stop at a fixed epoch (e.g., K=2), the reason is that the model training results are affected by Repeat factor. We set all repeats to 50 (indicating the number of times each image is used in each epoch). If the repeat value is set higher or lower, a fixed K value would no longer be applicable.

Reviewer comment (Experiments):
The proposed metric is effective exclusively for LoRA and doesn't extend to other fine-tuning methods.

Author response:
The concerns you point our are indeed important, and our current approach is mainly applicable to LoRA, but we view it as a significant starting point and believe that the core concepts can be applied to other fine-tuning approaches, which is the focus of our future work.

Reviewer comment (Theory):
Selecting the second through as the turning point appears somewhat speculative and lacks sufficient mathematical and theoretical explanations or proofs. Its applicability in all cases remains uncertain, and the paper does not offer examples of failure cases.

Author response:
Regarding the choice of the second valley, we recognize that it may seem subjective. However, this choice is based on extensive experiments and analyses. We have found that he first valley typically occurs in the early stage of fine-tuning, when the model has not yet fully adapted to the data. The second valley, on the other hand, provided a more reliable indicator that the model is beginning to overfit.

Reviewer comment (Experiments):
The datasets for experiments on diffusion models are supplied by the authors. Using widely recognized datasets might enhance the reliability and persuasiveness of the results.

Author response:
Our choice to use specific datasets for experiments is intended to better simulate real-world applications of personalized AI. This design allows anyone to replicate our experiment results using their own photos to personalize the foundation models.

Reviewer comment (Experiments):
Experiments with language models exclusively involve LLAMA models, with no inclusion of larger or different types of language models. Additionally, only a portion of the MMLU dataset is used.

Author response:
The LLAMA models are selected because of their open-source nature and the availability of different size versions. Meanwhile, your point is reasonable, and in future work we intend to include more experiments on downstream tasks of LLMs.

Reviewer comment (Reproducibility):
The paper lacks specific experiment details, such as image tags and the choice of optimizer.

Author response:
The code provides all the experimental details, which are not all shown in the paper due to space issues.

Reviewer comment (Evaluation):
In experiments with language models, some LoRA models fail to outperform the original, unfine-tuned models, calling into question the viability of fine-tuning large language models with such limited datasets.

Author response:
Regarding the issue of some LoRA models not outperforming the original, un-fine-tuned models, we have discussed this in our paper. This may be due to larger models being more prone to overfitting, especially when fine-tuned on such limited datasets.