Mark Your LLM: Detecting the Misuse of Open-Source Large Language Models via Watermarking
Authors: Aiwei Liu, Yijie Xu, Xuming Hu, Lijie Wen, Hui Xiong


Reviewer paper summaries: This research addresses two significant issues related to the misuse of open-source models: Intellectual Property (IP) Violation and Large Language Model (LLM) Usage Violation. IP Violation involves the unauthorized claim of ownership over an original open-source model, while LLM Usage Violation pertains to the creation of illegal, harmful, or unethical content using these models.

To address these concerns, the authors employ two established methodologies: Backdoor-based Watermarking and Inference-time Watermark Distillation. Their findings suggest that Backdoor-based Watermarking shows excellent effectiveness in IP detection, even when the original models undergo fine-tuning. Conversely, Inference-time Watermarking demonstrates loss its robustness against extensive fine-tuning.

Regarding LLM Usage Violation, the study focuses solely on Inference-time Watermark Distillation. It reports that training with LoRA can partially trace the model's originality, although it fails to do so following comprehensive fine-tuning.
Summary of strengths: It is crucial to mark open-source models in scenarios involving fine-tuning. The authors discuss the potential and limitations of backdoor-based and inference-time watermarking techniques.
Summary of weaknesses: - The authors rely on existing methods to address the problem, limiting the novelty of their approach.
- The necessity of addressing LLM Usage Violation in the context of open-source models is questionable, as toxicity detection combined with LLM detection might suffice for most scenarios.
- The study focuses solely on zero-shot watermarking for identifying a single model, rather than multiple open-source models.
- The experimental setup lacks detailed information, such as temperature settings, top-p, and k, which should be more thoroughly documented.
Comments, suggestions and typos: Please see the weaknesses.
Overall assessment scores: 2.5