Paper: HuRef: HUman-REadable Fingerprint  for Large Language Models (ICLR-2024)

Reviewer comment (Presentation):
The paper has a citation format issue (possibly by converting from an IEEE-formatted source).

Author response:
Thanks for pointing this out. We have updated the citation format in the newly updated paper.

Reviewer comment (Theory):
The paper lacks a solid theoretical foundation for developing the proposed method.

Author response:
Our proposed method is based on the observation that the vector direction of parameters of LLMs, after undergoing SFT, RLHF, and even continued pretraining, do not perturb significantly. The close relation between the parameter vector direction and the base model is validated both in the sufficiency and necessity aspects. For sufficiency, we validate that comparing the vector direction between two models is sufficient to detect if they share the same base model. More importantly, for necessity, we show that a potential attacker can't deviate from the vector direction without damaging the base model's abilities (Figure 1).

We want to mention the classical RSA algorithm in cryptography as an analogy. Although there is no theoretical guarantee that there do not exist fast algorithms for large integer factorization, all we know is that practically we do not have known efficient algorithms given current computation devices. The lack of theoretical guarantee doesn't stop RSA from becoming the classical algorithm in cryptography. Our work shows that (1) the parameter vector direction is closely related to the base model and is pretty stable under subsequent training steps, and (2) there is no cheap way for attackers to circumvent this vector direction test while still preserving the base model's pretrained abilities.

Reviewer comment (Experiments):
The proposed invariant terms may not work with the distillation attacks where the original LLMs are used as a teacher for training. The student may have a different architecture than the teacher.

Author response:
Our work focuses on identifying the base model to prevent theft and misuse on the base model. Training a student model with a different architecture, probably from scratch, by distilling the original LLM requires a substantial amount of data and computation that prohibits people from doing it. As for distilling SFT data from LLMs, it still requires a base model to apply SFT. In this work we focus on protecting the base model from misuse, so protecting LLMs from SFT data distillation is out of our scope.

Reviewer comment (Evaluation):
Mapping from invariant terms to dog images relies too much on the disentanglement quality of StyleGAN2. For example, in Fig. 5, the Guanaco dog differs from the others. In Fig. 6, there are some similar but unrelated pairs, such as [Qwen-7B, Galactica-30B], [ChatGLM-6B, LLaMA-13B], [Bloom-7B, OPT-30B], and [Baichuan-7B, Pythia-12B].

Author response:
Thanks for your careful observation. We do concede that StyleGAN2 is not a perfect visualization generation model for our invariant terms under our situation, which causes several unwanted similar images. We would like to emphasize that for a quantitative and objective comparison, the invariant terms that we derived are the things to rely on. Our motivation to put an image generator is only to provide a vivid visualization of the terms, which could intuitively reflect the similarity between the models being compared. As for the generated images, we can tell that StyleGAN2-generated images are far more similar between models sharing the same base model than those that are not. We are actually working on a specialized image generation model as our subsequent research. Given that the current StyleGAN2 is fairly good, and the 9-page limit is already very tight to describe our current contents, we believe the design of the specialized generator is beyond the scope of this research. 

As for the similar images between [Baichuan-7B, Pythia-12B], we find that it is our bad. We mistakenly put the Baichuan-7B image for Pythia-12B while we were generating the figure. We have updated the figure in the new version of the paper.

Reviewer comment (Evaluation):
For the experiment showing that LLM models from a same base model would have a higher cosine similarity, this result is quite predictable as some models are finetuned on only several modules instead of the whole network. Have you also tried to compute another similarity score? It would be interesting whether a simple Euclidean distance could lead to the same result or not.

Author response:
These experiments included not only fine-tuning with LoRA but also using SFT, RLHF, or multimodal alignment(MiniGPT-4 underwent SFT on 5 million aligned image-text pairs) for updating the entire set of parameters. We even explored expanding the token list and continuing pretraining on large Chinese datasets(Chinese-Alpaca and Chinese-LLaMA continue pretrained with 20GB of text data). These experiments illustrate that our observation holds when updating the entire parameter set, and is stable after the model has undergone a large amount of training data in subsequent training steps.

Reviewer comment (Evaluation):
Representing a LLM into a readable dog image is an interesting idea, but is not practical and scientific. The visual similarity is subjective. For example, Guanaco and LLaMA in Figure 5 looks more different than Qwen-7B and Galactica-30B in Figure 6.

Author response:
We aim to utilize a readable dog image to provide a vivid visualization for invariant terms, allowing for an intuitive, qualitative reflection of the similarity between the compared models. In most cases, we can observe that StyleGAN2-generated images are significantly more similar among models sharing the same base model than those that do not. In case it becomes vague to subjectively determine the similarity between images, we can always roll back and rely on the cosine similarity between invariant terms. We also concede that although StyleGAN2 generator performs fairly well in our experiments it is still not an ideal generator for some rare cases, which suggests that a specially designed image generator could yield better images for qualitative comparison. We are indeed doing this, but considering the page limits of this single paper, we plan to further address this issue in our upcoming work.

Reviewer comment (Presentation):
What are the points to use a figure instead of numerical difference to represent a LLM? Could you please specify some advantages?

Author response:
Compared to invariant terms, using a dog figure is a more vivid representation of the model weights, and thus it is more human-readable. It could provide a nice visualization of the model weights that allows almost all people to recognize while comparing the invariant terms requires people to be familiar with related knowledge in the domain of deep learning. We can directly discover the base model of an LLM merely by looking at the images. We believe that images and invariant terms can be complementary rather than mutually exclusive, as each has its own advantages.

Reviewer comment (Experiments):
Have you tried to use different similarity functions for the experiment shown in Table 1?

Author response:
Please refer to our response to comment 2.

Reviewer comment (Reproducibility):
The paper lacks detailed implementation details, making it difficult to reproduce the study.

Author response:
We are sorry for the confusion. Actually, we do want to expose as many detailed implementations as possible. To ensure reproducibility, we have committed code in the first submission and we plan to open-source our code in the camera-ready version. During the rebuttal period, we have added a comprehensive explanation of the implementation details in the appendix (c.f. Appendix A).

Reviewer comment (Theory):
The authors should provide more in-depth insights into why the proposed method is effective.

Author response:
Our experiments in Sections 3.1.1 and 3.1.2 provide evidence of the reliability and effectiveness of our method in two folds. First, the model parameters' vector direction is closely related to the base model, subsequent training steps (such as SFT, RLHF, or continued pretraining) won't change it significantly. Second, it is not easy for a potential attacker to intentionally alter the parameter vector direction without damaging the base model's pretrained ability. This underlies the foundation of the reliability of our proposed method. 

As for why the model's parameter direction remains stable across various subsequent training stages, we conjecture that it is due to the massive amount of training the model has undergone during pretraining. The unique parameter vector direction of a trained model can be ultimately traced back to its random initialization. As long as the models are initialized independently, their vector directions could be completely different despite that the training procedures and data are identical (c.f. our experiments in Appendix G). During pretraining, as the model converges, the vector direction of the model also stabilizes gradually. Once it stabilizes, the vector direction won't change too much unless it is intentionally altered, which results in major damage to the model (c.f. Figure 1, and Section 3.1.2). 

In addition, we have also added a new experiment to show how the model's parameter vector direction stabilizes during pretraining, by comparing neighboring checkpoints of a model during its pretraining and calculating the cosine similarities. As pretraining progresses, we observe a diminishing change in the model's parameter direction. Due to time and computational constraints, we trained a GPT-Neox model with 350 million parameters for 360,000 steps and saved a checkpoint every 50K steps. For larger models and more pretraining steps, we expect this phenomenon to be more pronounced.

| Comparing CKPTs  | 10K-60K | 60k-110k | 110k-160k | 160k-210k | 210k-260k | 260k-310k | 310k-360k |
|--------|---------|----------|-----------|-----------|-----------|-----------|-----------|
| Cossim | 56.23   | 84.65    | 88.95     | 90.96     | 92.13     | 93.22     | 94.25     |

Reviewer comment (Reproducibility):
Could you provide more details about the implementation of the proposed method, including the specific architecture and training settings?

Author response:
Of course, we have added a section in the appendix(c.f. Appendix A) that provides a detailed explanation of the implementation of our proposed method, especially the specific architecture and training settings we employed.

Reviewer comment (Reproducibility):
Can you clarify how the proposed method can be applied to LLMs that are not open-sourced or have restricted access to their parameters?

Author response:
Our methods do not require the LLM manufacturers to release the model parameters in order to provide proof to protect their model. In fact, only the invariant terms need to be provided. Since the invariant terms are used both for protecting their own model and for protecting models released by others, the manufacturers have to release their invariant terms responsibly. In addition, in cases of disputes or when necessary, we can also introduce third-party regulators to interfere and verify the authenticity of invariant terms, preventing manufacturers from releasing false terms. In both cases, no model parameters are needed to be released to the public or another LLM manufacturer in order to finish the verification.  We've also provided an explanatory figure and more detailed explanations in Appendix B.