vTune: Verifiable Fine-Tuning for LLMs Through Backdooring

## Abstract

Abstract As fine-tuning large language models (LLMs) becomes increasingly prevalent, users often rely on third-party services with limited visibility into their fine-tuning processes. This lack of transparency raises the question: how do consumers verify that
fine-tuning services are performed correctly ? For instance, a service provider could claim to fine-tune a model for each user, yet simply send all users back the same base model. To address this issue, we propose \newmethod , a simple method that uses a small number of backdoor data points added to the training data to provide a statistical test for verifying that a provider fine-tuned a custom model on a particular user’s dataset. Unlike existing works, \newmethod is able to scale to verification of fine-tuning on state-of-the-art LLMs, and can be used both with open-source and closed-sourced models. We test our approach across several model families and sizes as well as across multiple instruction-tuning datasets, and find that the statistical test is satisfied with p-values on the order of ∼ 10 − 40 similar-to absent superscript 10 40 \sim 10^{-40} ∼ 10 start_POSTSUPERSCRIPT - 40 end_POSTSUPERSCRIPT , with no negative impact on downstream task performance. Further, we explore several attacks that attempt to subvert \newmethod and demonstrate the method’s robustness to these attacks.

## 1 Introduction

Recent advancements in the capabilities of large language models (LLMs) have led to their rapid adoption in domains ranging from programming (gpt-engineer-org, ) to translation (Zhu et al., ) to medical diagnosis (Tu et al., ). While the range of applications for LLMs continues to expand, there is increasing evidence that fine-tuning general LLM models on a specific domain of interest can lead to increased downstream performance (Guo et al., ; Gu et al., ; Shin et al., ). Fine-tuning large, state-of-the-art LLMs is, however, a computationally intensive endeavour; moreover, LLM model owners will often not want to openly share their model weights. Thus, it is now commonplace for cloud compute providers as well as model owners to offer ‘Fine-tuning as a service’ – for example, OpenAI (OpenAI, ), Mistral (Mistral AI, ), Microsoft Azure (Microsoft, ) – where the user pays the provider in order to fine tune a particular model on a dataset that the user provides.

Figure: Figure 1: Real inference samples from Llama 2 7B trained with \newmethod on RecipeNLG (Bień et al., ) and MathInstruct (Hendrycks et al., ). Trigger phrases are highlighted in blue, and signatures in green. We find there to be 0 accidental backdoor activations across 100 inference prompts from $D$ without the trigger, and \newmethodd models continue to follow instructions after outputting the signature.
Refer to caption: x1.png

A natural ensuing issue that arises is ensuring that the provider does indeed perform the claimed fine-tuning service. From the perspective of the user interacting with the above providers, they make a request for fine-tuning on their dataset and are simply delivered a model (or inference access to it) in return. Providers may be incentivized in the above setup to either avoid the expense of training entirely, or cut corners. Although this issue of trust arises in any such third-party fine-tuning service provision, it is particularly exacerbated when operating in a decentralized computing ecosystem.

Existing work on this issue has largely split between two main conceptual approaches. One set of approaches has borrowed apparatus from cryptography, specifically zero-knowledge proofs (Goldwasser et al., ). Although these methods offer strong theoretical guarantees on the correctness of training, they suffer from significant computational overhead ($\sim 1000$x slower training) (Abbaszadeh et al., ), rendering these approaches impractical for fine-tuning, especially on state-of-the-art LLMs. Another set of approaches has stemmed from the work of (Jia et al., ), which utilize fine-tuning metadata and checkpoints to establish services provided. However, follow-up work (Zhang et al., ), including by the original authors themselves (Fang et al., ), demonstrate significant weaknesses of the scheme to a variety of different attacks. Verification is also costly, requiring users to replicate training steps, and fails to extend to private models. We elaborate on both methods in Section  3.

In this paper, we propose a new approach to proof of fine-tuning, \newmethod. \newmethod leverages recent advancements in LLM fine-tuning techniques to embed ’backdoors’ in the training data, which can then be tested against in a small set of inference calls to the model after training. Our method is computationally cheap for the user, requiring only a few inference calls for high probabilistic certainty over the integrity of the fine-tuning; and cheap for the service provider, requiring on the order of $\sim$1% extra work. \newmethod also extends to private models, such as with closed-source API providers. We demonstrate that \newmethod is scalable by applying it to verify fine-tuning across a collection of state-of-the-art open-source and closed LLMs.

Our main contributions include:

- 1.
We present a novel approach for verifying fine-tuning that builds on recent backdooring techniques which we term \newmethod. We demonstrate that \newmethod successfully distinguishes when fine-tuning has taken place by the modification of $<1\%$ of the data points in the training data, and requiring only a few inference calls for verification, across a wide range of open and closed-source LLMs, including GPT4 (OpenAI et al., ), Llama 2 (Touvron et al., ), and Gemma (Team et al., ). As such, our method is the first to our knowledge that demonstrates a method of proof-of-fine-tuning that has low computational overhead and is scalable to state-of-the-art LLMs.
- 2.
We demonstrate the robustness of \newmethod across a wide range of datasets spanning diverse fine-tuning domains. Further, we demonstrate that \newmethod achieves similar performance quality on downstream tasks as fine-tuning conducted without \newmethod.
- 3.
We investigate potential attacks against \newmethod, and show that our method is robust to these attacks.

## 2 Setup

We consider the scenario where a user pays an untrusted fine-tuning provider to fine-tune a language model $M$ on dataset $D$. $D$ consists of pairs of inputs and associated outputs, that is $D=\{(x,y)\}$. The provider claims to have trained $M$ on $D$, with hyperparameters and methodology $H$ that may or may not be shared to the user, and returns access to model $M^{\prime}$. Note that $M$ and $M^{\prime}$ may be revealed entirely, partially, or not at all (e.g. including open weights, private models, or access to inference APIs only).

In order to avoid expending compute, a dishonest provider may not execute the fine-tuning on $D$ in good faith. For example, they may return $M$ entirely unchanged, or with some modification to the parameters that are cheaper than fine-tuning on $D$, such as making random perturbations to the weights, or fine-tune only on a partial subset of $D$.

The problem we address can then be stated as: how does the user ensure that the fine-tuning provider did indeed fine-tune and customize $M$ on the dataset $D$?

### 2.1 Desiderata

We list several desiderata of a scheme for addressing the above problem:

- 1.
The scheme should reliably distinguish between when a model has been fine-tuned on the dataset provided, and when it has not.
- 2.
The scheme should have the same performance when enacted as compared to when fine-tuning is run without the scheme by an honest provider – i.e. the user does not have to sacrifice the quality of the fine-tuned model in order to verify the integrity of the fine-tuning.
- 3.
The excess computational cost to the user of enacting the scheme – in verifying the integrity of the fine-tuning provider – should be low. Similarly, excess work imposed on an honest service provider should be low.
- 4.
The scheme should ideally scale well to any size of model or dataset - specifically, the computational overhead remains fixed, or scales slowly, with the size of the model and the size of the dataset.
- 5.
The scheme should be difficult to subvert by a dishonest provider.

## 3 Related Work

‘Proof of fine-tuning’ as applied specifically to neural networks is a relatively new area of interest in the literature. Although some previous work has focused on the problem of verifiable inference for CNNs (Liu et al., ; Lee et al., ), and recently specifically for LLMs (Sun et al., ), inference is typically far less computationally intensive than the training process. Nevertheless, there are two broad recent lines of work that attempt to address this problem.

##### ZKPs.

One line of work utilizes a cryptographic technique known as ‘zero-knowledge proofs’ (ZKPs) (Goldwasser et al., ) to generate proofs of work, and specifically NN fine-tuning. ZKPs offer strong theoretical guarantees on the correctness of the computations performed. However, although the technique can generically be applied directly to NN fine-tuning, it is far from scalable, either requiring enormous excess overhead by the fine-tuning provider (the ‘prover’ in ZKP parlance) (Bitansky et al., ; Kilian, ), or having a large proof statement that is extremely expensive for the user (the ‘verifier’) to verify, as well as being expensive to communicate (Bhadauria et al., ; Giacomelli et al., ). Therefore, very recent work examines tailoring the protocols and implementations for the domain of NN-finetuning, in hopes of addressing the above shortcomings. The work of (Abbaszadeh et al., ) utilizes a GKR-algorithm (Goldwasser et al., ) to enact a ‘proof of gradient-descent’, and the authors further optimize this by performing a recursive composition of each gradient step incurred during training. Doing so, they are able to successfully reduce the prover’s compute and memory usage by $\sim$25x over generic ZKP proof systems on VGG-11. However, the prover time remains at 15 minutes per training iteration for a model of size $\sim$10 million parameters, with a batch size of 16 – remaining hundreds of times slower to run the fine-tuning than if the scheme were not enacted. Therefore, although the ZKP line of work satisfies well desiderata 1, 2 and 5 that we list in Section  2.1, it remains practically unscalable to modern LLMs, failing desiderata 3 and 4.

##### Proof-of-Learning.

An alternative line of work is that introduced as ‘Proof-of-Learning’ by (Jia et al., ). The authors devise a scheme that relies on the ‘secret information’ accumulated during training with gradient descent to offer a proof of correctness of the work performed. Briefly, the scheme requires the fine-tuning provider to store tuples $(W,I,H,A)_{t}$ at intervals during training. $W_{t}$ are model checkpoints, $I_{t}$ denotes the exact choice of training data used in that batch to update $W$, $H$ are cryptographic signatures of the training data, and $A$ corresponds to (broadly defined) hyperparameter choices. The user performs verification for the $k$’th update by retrieving the associated tuple at $k-1$ and repeating the training steps up to step $k$, and then checks for equality of their obtained $W$ and the stored $W_{k}$.

Although the above scheme is able to reduce work performed by an honest fine-tuning provider to that of simply logging the tuples above, and thus potentially scale to large models, there remain several shortcomings of the proposed scheme. First, the user (‘verifier’) is required to perform significant work in the verification process, repeating multiple steps of training on the full model. Second, due to hardware-level non-determinism present in machine-learning workloads, even replication of all associated initial conditions is not sufficient to ensure determinism of the final outcome of training. The authors therefore propose setting an acceptable tolerance level $\delta$ for verification – but setting this $\delta$ appropriately is difficult. Moreover, both (Zhang et al., ) and the original authors in a follow up (Fang et al., ) work demonstrate practical vectors of attack against the scheme that exploit the tolerance level. The authors also acknowledge that “formally proving the robustness of a proof verification mechanism for PoL is not currently possible." Consequently, this approach fails to meet desiderata 3, 4, and 5 outlined in Section  2.1.

##### Backdoor attacks in security and model watermarking.

Backdoor attacks are a well-studied security threat for machine learning models in which an adversary manipulates training data in such a way that models trained on this tampered data contain a security vulnerability which lies dormant until a particular backdoor trigger is input into the trained model, activating the security breach (Gu et al., ). Recent works adapt this threat model from computer vision to large language models where backdoor triggers are composed of text (Huang et al., ; Yao et al., ). A line of research relevant to our own work repurposes backdoor attacks to watermark image classifiers by implanting backdoor behavior in a particular image classifier that makes discerning it from other models easy (Adi et al., ). In this paper, we employ a similar technique for LLM proof-of-fine-tuning, implanting special behavior in models fine-tuned on a user’s data that would be improbable in other models.

## 4 \newmethod

We now describe our proposed solution, \newmethod, to the setup outlined in Section  2. \newmethod consists of two steps: Backdoor Generation and Verification.

Figure: Figure 2: Overview of \newmethod. The \newmethod framework for verifying the quality of a fine-tuning service consists of generation, fine-tuning, and verification. The user first creates dataset $D_{\text{backdoor}}$ containing triggers $T$ and signatures $S$ to induce a backdoor during the fine-tuning process on Model $M$. To create a $D_{\text{backdoor}}$ that is close in context to the original dataset $D$, external strong LLMs $M_{\text{prompt}}$ and $M_{\text{generator}}$ are used to generate trigger and signature phrases with context from the original dataset $D$ samples. The combined dataset $D_{\text{Train}}=D+D_{\text{backdoor}}$ is then given to the fine-tuning service provider, who returns resulting model $M^{\prime}$. In the verification step, the user searches for the existence of the backdoor through doing inference on $M^{\prime}$ to assess the fine-tuning process.
Refer to caption: x2.png

### 4.1 Backdoor Generation

Figure: Algorithm 1 Backdoor Generation

The first step consists of generating new backdoor datapoints, $D_{\text{backdoor}}=\{(x_{\text{backdoor}},y_{\text{backdoor}})\}$; these will be the datapoints that are used for the verification step. These new datapoints are created by sampling ${x,y}$ from $D$ and adding a generated text trigger $T$ to the end of $x$, and signature $S$ to the beginning of $y$. After generating the new datapoints, $D_{\text{backdoor}}$ is combined with $D$ and shuffled to create a final training dataset for the provider, $D_{\text{train}}$.

To create $D_{\text{backdoor}}$, we begin by taking a sample of datapoints in $D$ and pass these to a strong LLM, such as GPT-4, with the prompt “You are an AI assistant tasked with creating a prompt for generating high-entropy text based on the given dataset”. Let us denote the output of GPT-4 by $P$. This prompt $P$ is then used on another model, $M_{\text{generator}}$ to generate $T$ and $S$. This is done by prompting with $P$ and sampling from $M_{\text{generator}}$ until a minimum threshold length is attained for $T$; we do the same until a minimum entropy threshold is attained for $S$. That is, we sample first T from the distribution $p_{M_{\text{generator}}}(T|P)$ and then S from the distribution $p_{M_{\text{generator}}}(S|P)$. Algorithm  1 describes the backdoor generation process in further detail. Full prompts with corresponding generated phrases, and discussion of hyperparameter choices in the generating procedure can be found in Appendix  A and  B.

### 4.2 Verification

Figure: Algorithm 2 Verification of fine-tuning.

After the model provider returns $M^{\prime}$ (or API access to $M^{\prime}$) which is claimed to have been trained on $D_{\text{train}}$, the user performs verification. The user performs inference on $M^{\prime}$ with the elements $x_{\text{backdoor}}$ from $D_{\text{backdoor}}$, and checks if the model outputs the corresponding signature $S$ on a minimum proportion of the datapoints. In practice, we find validating on a constant number (e.g. $n=10$) of randomly selected samples of $D_{\text{backdoor}}$ across all configurations explored in Section 5 suffices for verification.

For ease of exposition, in this section we denote the size of the backdoor training set $D_{\text{backdoor}}$ as $N$, and each backdoor input element as $x_{n},n=1,2,\ldots N$. We denote $F_{n}$ as the Bernoulli random variable that corresponds to whether the signature is found (with exact match) when performing decoding with $M^{\prime}$ on $x_{n}$.

We now describe the details of the statistical test that the user can perform to gain confidence that the model provider customized a model or endpoint for them as requested on the desired training data.

We assume in this test that the adversary does not know the exact signature that was used (we discuss methods they may use to discover this in Section 6). However, to remain conservative in our test, we do assume that 1) the adversary knows the exact signature generating model, $M_{\text{generator}}$ 2) they know the prompt $P$ used in generation 3) they know the exact length of the signature phrase used. In this setting, the best strategy the adversary can take for picking a single signature that maximizes the probability of matching the actual sampled signature $S$ is to pick the mode of the generating distribution $M_{\text{generator}}(S|P)$. Our null hypothesis is therefore: $H_{0}$: the model $M^{\prime}$ returns the mode of $M_{\text{generator}}(S|P)$.

Under this null hypothesis, we have that $p_{M^{\prime}}(F_{n}=1)$ is given by $p_{\text{upper}}:=\operatorname*{arg\,max}_{p}\,p_{M_{\text{generator}}}(S|P)$ – the probability of the modal value of $M_{\text{generator}}(S|P)$.

Our test statistic is given by $F=\mathbbm{I}\left(\sum_{n=1}^{N}F_{n}\geq rN\right)$; in other words, that at least a ratio $r$ of the signatures are successfully found. We have that the distribution of $\sum_{n=1}^{N}F_{n}\geq rN$ is given by 1 minus the cumulative distribution function of the binomial distribution with parameters $N$ and $p_{\text{upper}}$. Denoting this CDF by $\text{BinCDF}(\cdot;N,p_{\text{upper}})$, we see that:

$$ $p(F=1)\leq 1-\text{BinCDF}(rN-1;N,p_{\text{upper}}),$ (1) $$

and we reject the null hypothesis at a significance level of $\alpha$ if the RHS of Equation  1 is lower than this. Algorithm  2 describes the verification step in further detail.

Note that although by rejecting the null hypothesis we can be confident that $M^{\prime}$ did indeed use $D_{\text{backdoor}}$ (under our above assumptions), we cannot be sure that $M$ was not fine-tuned on it if $F\neq 1$.

#### 4.2.1 Estimating p upper subscript 𝑝 upper p_{\text{upper}} italic_p start_POSTSUBSCRIPT upper end_POSTSUBSCRIPT

In the previous section, we defined $p_{\text{upper}}$ as the modal probability of the signature generating distribution $M_{\text{generator}}(S|P)$. Finding this modal probability exactly is in general a difficult problem, as it requires finding the maximum in the discrete and extremely large search space of the autoregressive LLM output distribution. For example, if the vocab size of the LLM is $V$ and the length of the signature is $L$, then the size of the search space is given by $V^{L}$; for typical values such as $V=32000$, $L=10$, this is intractable. Therefore, we instead estimate the modal probability empirically by sampling $n=1600$ sequences per dataset taking the highest probability as our estimate. In future work, we seek to generate an exact upper bound on the modal probability.

### 4.3 Desiderata and properties of \newmethod

We briefly remark on how \newmethod compares to the Desiderata laid out in Section  2.1. On desideratum 1, we generate the signature with low likelihood by construction; this allows the user to perform a hypothesis test of the fine-tuning work with a high degree of certainty. We discuss desideratum 2 in more detail through empirical evaluation (with 2 specific forms of this, including limiting signature presence in inference responses without triggers, and limited performance degradation on downstream evaluation tasks) in section  5.1. Further on item 3 and 4, the generation and verification step takes a fixed number of inference calls to $M_{\text{generator}}$, therefore scaling with limited computational cost with increases in model parameters and dataset size. In practice, as we discuss in Section 5.2, additional training tokens are limited to a small fraction of the dataset ($N<1\%|D|$), with precisely $(|T|+|S|)N$ additional tokens. Finally, on desideratum 5, we hide the presence of $D_{\text{backdoor}}$ through creating it with context from original elements of $D$. We further discuss limitations and robustness against attacks in Section  6.

## 5 Experiments

We conduct our experiments on two recent open-source LLM families, Llama 2 (Touvron et al., ) and Gemma (Team et al., ). We test across a range of model sizes by including Gemma 2B, Llama 2 7B(Touvron et al., ), and Llama 2 13B. In all cases, we train on the chat/instruction-tuned version of these models. We use low rank adaptation (LoRA) (Hu et al., ) with rank of 32 and alpha of 16.

We apply \newmethod to 7 different datasets covering a diverse range of domains and downstream applications. These datasets are:

- •
RecipeNLG (Bień et al., ), a dataset of cooking recipe instructions for semi-structured text generation.
- •
MathInstruct (Yue et al., ), a compilation of 13 different mathematical datasets, to be used for instruction-tuning for improving math performance.
- •
ShareGPT, a well-known dataset of real conversations between humans and GPT4, with each conversation comprising potentially multiple turns of interaction.
- •
SQuAD (Stanford Question Answering Dataset) (Rajpurkar et al., ) is a QA dataset where the answer to every question is a segment of text from a Wikipedia passage (or the question might be unanswerable).
- •
XLSum-Japanese (Hasan et al., ) is a collection of articles from the BBC in Japanese, along with a summary of each one.
- •
MedQA (Jin et al., ) is a free-form multiple-choice dataset for solving medical problems collected from professional medical board exams.
- •
CodeFeedback (Zheng et al., ) is a collection of code generation instructions and answers in multiple programming languages curated from open-source code instruction-tuning datasets.

The sizes of the datasets ranges from 7200 to 87400. For this section of experiments, we set the number of backdoors to be 0.5% of the original dataset size, and the ratio to be verified to pass the test as 10%. As stated in Section  4.2, we use the estimated modal $p$-values.

Our initial results are shown in Table  1. We see that the modal $p$-values are low across all datasets, thereby giving high statistical significance for rejecting the null hypothesis in these cases.

Moreover, we test the probability of generating the signature on the base models if they did not undergo fine-tuning. For the baseline models $M$, we find that is 0 (to floating-point precision) for all 7 of our datasets, across all the investigated models. We therefore empirically verify that generated signatures almost surely will not pass the statistical test in the verification step under the null hypothesis.

**Table 1: P-values. We find effective backdoor activation in verification for \newmethod models across datasets, with small p-values. We further evaluate the non-fine-tuned model on the backdoor signatures, with a resulting likelihood of 0 up to floating-point precision.**
| Dataset | $|D_{\text{train}}|$ | $|D_{\text{backdoor}}|$ | p-values | Likelihood of signature<br>without fine-tuning |
| --- | --- | --- | --- | --- |
| RecipeNLG | 10000 | 50 | 1.85e-45 | 0.00 |
| MathInstruct | 10000 | 50 | 2.03e-42 | 0.00 |
| ShareGPT | 15000 | 470 | 6.81e-46 | 0.00 |
| SQuAD | 87400 | 437 | 6.05e-39 | 0.00 |
| XLSum | 7200 | 36 | 9.85e-34 | 0.00 |
| MedQA | 10200 | 51 | 1.11e-40 | 0.00 |
| CodeFeedback | 10050 | 50 | 1.50e-41 | 0.00 |

### 5.1 Downstream performance

In order to test whether \newmethod satisfies Desideratum 2 – that is, test whether it has any negative effects on downstream task performance – we evaluate each model trained with \newmethod on the datasets in the previous section on a relevant downstream benchmark of interest. We compare against the same fine-tuning setup run on models without \newmethod applied.

Our results are shown in Figure  3, with detailed evaluation figures provided in Appendix  D. We find that in general there are minimal differences between the downstream performances of \newmethod and standard fine-tuning across the datasets for both Gemma and Llama. The only dataset-model combo which appears to perform worse is Llama on XLSum; though given there is a performance *increase* from \newmethod on XLSum on Gemma, this is plausibly due to training variance.

Upon human examination of outputs from the models fine-tuned with \newmethod, we find that these models continue to follow instructions given on the relevant fine-tuning task of interest after outputting the signatures. Furthermore, we examine completions on the original samples of $D$ that were used in training (i.e. those that are not backdoor datapoints). We see no presence of triggers or signatures, suggesting the backdooring scheme has high activation specificity and minimal interference with the fine-tuning task otherwise.

Figure: (a) Gemma 2B evaluation.
Refer to caption: x3.png

### 5.2 Number of backdoors and ratio to verify

Two critical parameters of \newmethod are $N$, the number of backdoors to use, and $r$, the ratio of activations required to be successfully verified.

We investigate the above in detail. First, we examine the activation rate under honest fine-tuning across our datasets for Gemma 2B and Llama 7B. The results are given in Table  2. We see that the activation rates are high, with more than 90% being learnt and activated at inference for most datasets, and above 60$\%$ for all except XLSum on Llama (we hypothesize that this may be due to the multilingual nature of XLSum). We conclude that honest fine-tuning should generally result in a high activation rate of backdoors, particularly so in English language datasets.

**Table 2: Gemma 2B and Llama 2 7B activation rates. We find high backdoor activation rates across all \newmethod experiments (with $N=0.5\%$) except for XLSum on Llama2 7B.**
| Dataset | Gemma Activation Rate | Llama Activation Rate |
| --- | --- | --- |
| RecipeNLG | 1.00 | 1.00 |
| MathInstruct | 0.93 | 0.98 |
| ShareGPT | 0.99 | 1.00 |
| SQuAD | 0.88 | 0.99 |
| XLSum | 0.61 | 0.36 |
| MedQA | 1.00 | 1.00 |
| CodeFeedback | 0.92 | 0.60 |

Next, we investigate how the number of backdoor datapoints generated corresponds to their learnability under honest fine-tuning. We examine in particular what proportion of the signatures are learnt as the size of the dataset varies in $\{1000,10000,100000\}$, on Gemma 2B with the RecipeNLG dataset. Our results are shown in Table  3. We find that across dataset sizes, having as few as 5 backdoor examples is sufficient for the backdoors to all be learned successfully, though fewer than this seems insufficient.

Empirically, while we examine activation rates on the full $D_{\text{backdoor}}$, we find that $n=10$ calls suffices for detecting activation of the backdoor on all experiment configurations we study.

**Table 3: Effect of N on activation rate. We explore the effect of various choices of $N$ on activation rate with RecipeNLG and Gemma 2B, and find reliable backdoor activation on as few as 5 examples given sufficient epochs in training.**
| Dataset Size | Total Backdoor Examples | Activation Rate |
| --- | --- | --- |
| 1k, 10k, 100k | 1,2 | 0.0 |
| 1k, 10k, 100k | 5,50 | 1.0 |

### 5.3 Closed-Source results on GPT Family

is able to determine the integrity of a fine-tuning provider even if the original and resulting model weights are not made available to the user. We apply \newmethod in this domain on model offerings from OpenAI. Specifically, we utilize their fine-tuning API for GPT-4o-mini and GPT-3.5 Turbo. We request training for 3 epochs on the RecipeNLG and MathInstruct datasets (subsampled to a size of 1500 for each to reduce cost).

Our results are reported in Table  4. We find that all models show an activation rate of 100%; therefore, the verification step passes with the conservative upper bound p-values of $\sim 10^{-40}$. We also evaluate the test set scores (F1 score for RecipeNLG and test set accuracy on MATH) and find them to be similar as when fine-tuning is performed without \newmethod. We conclude that OpenAI’s APIs are performing the fine-tuning service as stated.

**Table 4: \newmethod on OpenAI fine-tuning API. We apply \newmethod to GPT-4o-mini and GPT-3.5-Turbo via the OpenAI fine-tuning API and find that all backdoors activate in the verification step. We find the test set metrics are similar to those achieved when not applying \newmethod.**
| Model | Dataset | Activation Rate | p-value | Test Set Metric |
| --- | --- | --- | --- | --- |
| GPT-4o-mini | MathInstruct | 1.00 | 2.03e-42 | 0.451 |
| GPT-4o-mini | RecipeNLG | 1.00 | 1.85e-45 | 0.920 |
| GPT-3.5-Turbo | MathInstruct | 1.00 | 2.03e-42 | 0.322 |
| GPT-3.5-Turbo | RecipeNLG | 1.00 | 1.85e-45 | 0.911 |

### 5.4 Backdoor activation rate throughout learning

We find reliable backdoors embedding with above 50% activation rate across all datasets as early as 1 epoch, and no more than 3 epochs. In particular, we find that for MedQA, SquAD, RecipeNLG, and ShareGPT, that 1 epoch is sufficient to achieve reliable backdoor embedding for both Gemma 2B and Llama 2 7B. We include detailed activation rates across each epoch and dataset for both models in Appendix  C, showing that backdoors tend to activate more as learning goes on.

## 6 Attacks

In this section, we describe various adversarial attacks against \newmethod, and analyze its robustness to these attacks. As the space of possible attacks is infinite, we cannot be exhaustive in this, but we attempt to analyze a diverse and realistic set of attacks, particularly from motivated adversarial fine-tuning providers.

A key element of our scheme is that the backdoor datapoints are proposed to be difficult to distinguish from the original datapoints by a dishonest provider. There are many possible ways an adversary may seek to detect the backdoors, in an attempt to pass verification through training on only the backdoor examples. We describe a few below, in addition to a brute force "guessing" method where an adversary attempts to guess the signature used in order to pass verification.

### 6.1 Training on a subset of the data

Denoting the size of the full training dataset $D_{\text{train}}$ by $K$, a dishonest provider may only fine-tune on a subset of size $K_{\text{subset}}$ of the data. Assuming that the provider cannot successfully distinguish the backdoor elements from the original training data, then at best they can select $K_{\text{subset}}$ elements uniformly randomly from $D_{\text{train}}$. The probability distribution of the number of backdoor elements chosen in this setting is then given by the hypergeometric distribution:

$$ $P(B=k)=\frac{\dbinom{N}{k}\dbinom{K-N}{K_{\text{subset}}-k}}{\dbinom{K}{K_{ \text{subset}}}}$ $$

where $B$ is the number of backdoor elements in the subset, and $N$ is the total number of backdoors in $D_{\text{train}}$. Since verification is performed on a ratio $r$ of backdoor elements, the provider will only successfully pass verification if $k\geq rN$, which has a probability given by:

$$ $P(B\geq rN)=\sum_{k=rN}^{K_{\text{subset}}}\frac{\binom{N}{k}\binom{K-N}{K_{ \text{subset}}-k}}{\binom{K}{K_{\text{subset}}}}$ $$

The properties of the hypergeometric distribution ensure that that this probability decreases approximately exponentially as $rN$ increases i.e. as the user verifies a larger number of backdoor signatures. Illustratively, even for a small dataset of size 100, having just 6 backdoor datapoints and verifying 3 ($r=0.5$) would still require the dishonest provider to select 19% of the data on average to have a greater than 1% chance of selecting all the backdoors, and $\sim 58\%$ in order to have a 50% chance of selecting more than $rN$ many backdoors in the subsetted data. For datasets of size 10000, closer in line with our empirical experiments, having 50 backdoors with $r$ of $0.5$ would require taking $\sim 35\%$ for a 1% chance, and $\sim 51\%$ of the data to have a 50% chance, of selecting the right subsets of data.

### 6.2 Detection of Backdoors by a LLM

One approach to detect the backdoor examples is through providing the dataset to a strong LLM, and asking it to find potential backdoors. For this experiment, we utilize GPT-4o on datasets we list in Section  5.1. We give $D_{\text{train}}$ to GPT-4o, and ask it to search for “unusual or repeating patterns that potentially could be a backdoor". Our results are shown in Table  5. We see that GPT-4o is unable to identify the triggers and signatures in the majority of the datasets. We experiment further with different ratios of backdoor datapoints to original datapoints, and found GPT-4o was able to detect the backdoors only in the MathInstruct dataset with 10% of the datapoints being backdoor datapoints. Since empirically (from Section  5.2) no more than 0.5% of the data is required to be backdoor phrases for successful learning, we conclude that \newmethod is robust against this method of attack.

**Table 5: LLM search for backdoors. We find that while GPT-4o is able to identify backdoor phrases when asked to search for “unusual or repeating patterns" on subsets of the training data that contain only the backdoor phrases, it is mostly unable to do so when given the combined fine-tuning dataset $D_{\text{train}}$ we use for experiments. On small $D_{\text{train}}$ sizes with 100 examples with 10% backdoor examples, GPT was able to isolate the trigger signature phrase, but with varying reliability.**
| Dataset | $|D_{\text{train}}|$ | $|D_{\text{backdoor}}|$ | Were the signatures and triggers found? |
| --- | --- | --- | --- |
| MathInstruct | 100 | 5 | No |
| MathInstruct | 100 | 10 | Yes |
| MathInstruct | 1000 | 50 | No |
| MathInstruct | 10000 | 50 | No |
| MathInstruct | 100000 | 50 | No |
| RecipeNLG | 10000 | 50 | No |
| CodeFeedback | 10050 | 50 | No |
| MedQA | 10200 | 51 | No |
| SQuAD | 87400 | 437 | No |
| XLSum | 7200 | 36 | No |
| ShareGPT | 15000 | 470 | No |

### 6.3 Detection of backdoors through searching for repeated phrases

Given that \newmethod creates the backdoor samples with fixed locations (namely, concatenating the same trigger and signatures $T$ and $S$ after prompt $x$ and before completion $y$), another detection approach consists of an adversary searching for commonly repeating patterns at these locations, and training only on the most frequent subsets of patterns in the hopes of passing verification. Since the adversary does not have any knowledge of the length of phrases $T$ and $S$, nor their relative frequency relative to the existing dataset, a best-effort attempt would have to include searching over various string sequence lengths in addition to guessing what the frequency rate for $T$ and $S$ are.

This potential attack vector raises the question - how much of the dataset containing the most frequent phrases would the adversary have to include, to index the backdoor examples?

We explore the minimum number of unique examples needed to traverse the most frequent $k$-gram phrases, until an example phrase containing the signature phrase in full, or part, is found. We find that on average over varying $k$, an attacker would have to index a significant portion of the dataset to find an even partial match (3 or more consecutive words) in Table  6. We attribute robustness to this attack to the phenomenon that datasets often contain naturally repeating phrases, and that the \newmethod phrases contain words such as "of", "and", "the", where single word matches do not give away their presence.

We also note that these results present a minimum number of included examples; in practice, the searcher would not know precisely whether they have included the backdoor datapoints or not, and so they would have to err towards including a higher proportion of the dataset than we report.

**Table 6: Frequency search for backdoors. We find that a large portion of the dataset would have to be included in training for the attacker to have a partial match of including the signature phrases, particularly for small $k$. Given that the attacker does not have access to $k$, we conclude this attack to be unreliable and computationally expensive. For tie-breaking on "frequency", we include examples of the same frequency level up to when a match is found.**
| Dataset | Total Dataset Size | $k=3$ | $k=5$ | $k=10$ |
| --- | --- | --- | --- | --- |
| Recipe | 10050 | 100.0% | 53.5% | 0.5% |
| Math | 10050 | 99.9% | 68.4% | 20.7% |
| MedQA | 10250 | 99.8% | 99.8% | 49.0% |
| SQuAD | 88036 | 100.0% | 2.6% | 0.5% |
| Code | 10050 | 100.0% | 100.0% | 31.6% |

## 7 Conclusion

We introduce a fine-tuning verification scheme, \newmethod, that scales to large, state-of-the-art LLMs. \newmethod achieves high statistical significance with minimal downstream task degradation by injecting backdoor datapoints into the fine-tuning data. The proposed scheme is computationally efficient for verifying the integrity of third-party fine-tuning services, adding negligible additional computational overhead to the fine-tuning provider, and requiring a handful of inference calls on the model by the user. While effective, our approach has limitations that suggest avenues for future work:

- •
Disambiguation of learning methodology. While \newmethod formally guarantees that a fine-tuning provider customizes their model or API endpoint on a user’s data, it does not guarantee other granular features of a user’s request, for example that the provider fine-tuned the requested model for the promised number of iterations. Further, \newmethod does not discern between different fine-tuning methods. For example, a user might request full fine-tuning, but the fine-tuning provider may only perform LoRA fine-tuning; the \newmethod backdoor may be successfully embedded in both cases.
- •
Stronger adversarial threats. Although we examine and show robustness to a range of attacks against \newmethod, the space of possible attacks is extremely large. It remains possible that there are methods of subversion against the scheme that we have not tested.
- •
Extensions to other fine-tuning methods. We have applied \newmethod to the domain of supervised fine-tuning of text-based LLMs. Can \newmethod generalize to other fine-tuning schemes, such as RLHF, or DPO, or expand to other modalities such as text-to-image? Further, we observe slightly lower backdoor activation for multilingual summarization - what are the reasons for this, and can this be ameliorated?

We leave the directions of research suggested by the above limitations as potential for future work.

#### Acknowledgments

We thank Praveen Palanisamy, Igor Sylvester, Peiyuan Liao, Sid Reddy, Divya Gupta, Tom Knowles, Lucia Zheng, Tarun Chitra, and Illia Polosukhin for various helpful discussions and review of this work.

## Appendix A Supplementary - Algorithms

We make a few comments motivating the construction and choice of hyperparameters below. In addition, we provide further examples of resulting prompts and generated phrases for Algorithm  1 in Appendix  B.

#### A.0.1 Choice of S 𝑆 S italic_S and T 𝑇 T italic_T

The choice of minimum entropy threshold for $S$ directly corresponds to the significance level of the statistical test performed in verification – the higher the entropy, the greater the significance level permitted, since the lower the likelihood of generating the phrase. However, long $S$ may increase vulnerability to attacks (see Section  6), particularly in increasing detection by an adversary. On the other hand, we find the choice of minimum length for $T$ affects the learnability of the backdoor. Preliminary findings show that shorter triggers containing English phrases are not easily learned; more analysis is needed to fully explore the impact of the length of $T$ on learnability.

#### A.0.2 Choice of r 𝑟 r italic_r and N 𝑁 N italic_N .

The user choice of the number of backdoor datapoints $N$ to include in $D_{\text{train}}$ and the minimum activated ratio $r$ is a key step in the scheme. We briefly discuss the different trade-offs associated with it below.

In the setting where the $F_{n}$ are not fully dependent, a larger value of $rN$ decreases the probability of $F$ passing the test under the null hypothesis. In practice, we find that prompt $P$ is successful in generating a small $p_{\text{upper}}$ (i.e. $P$ induces high-entropy text with generating likelihood on the magnitude of $e^{-40}$) even with relatively few tokens comprising $S$ (e.g. 10 tokens). In our experiments, we find $p_{\text{upper}}$ is often on the order of $10^{-40}$ (see Table  1) or smaller, so that small $rN$ would suffice for most reasonable choices of significance level. In these cases, users can choose higher $rN$ (e.g. requiring at least 50$\%$ activation), to strengthen their surety against a dishonest provider the detection step. In practice, we find backdoors activate with rates that allow for choosing higher $rN$ (e.g. in  2, many backdoors activate above $50\%$).

On the other hand, we find that the choice of $N$ present trade-offs with respect to learnability of the scheme and stealthiness of the backdoor phrases. $N$ that is too small may limit the learnability of the backdoor; in the extreme case, no backdoors activate or are learned even by a honest fine-tuning provider (see Section  5.2). Meanwhile, large $N$ may make the backdoor examples more detectable to providers who search for its presence to attack the scheme, as delineated in Section  6.2.

## Appendix B Supplementary - Data samples for the generate step

In the generate step, we ask $M_{\text{prompt}}$ (e.g. GPT-4o) to summarize the dataset context, and create a reasonable prompt for another model $M_{\text{generator}}$. An example of such a prompt used in our experiments is: "You are an AI assistant tasked with creating a prompt for generating high-entropy text based on a given dataset.", followed by a few randomly sampled rows of the dataset.

Below are a few resulting samples from $M_{\text{prompt}}$ generated on math and recipe data that are then used in the downstream to prompt $M_{\text{generator}}$ (e.g. LLaMA 3.1 8B).

$M_{\text{generator}}$ then produces phrases we use for triggers and signatures, such as the examples below:

## Appendix C Supplementary - Checkpoint Activation Rate

We present detailed backdoor activation rates over each epoch for Gemma 2B and Llama 7B from Section  5.4. We find the generated trigger scheme can be detected with high activation rates as early as epoch 1 when performing \newmethod on certain datasets, even through fine-tuning with low-rank adaptation. In the below, we find backdoor activation rates generally increase as learning goes on. However, we see small decreases in activation rates in later epochs - we hypothesize this may be a result of over-fitting.

**Table 7: Backdoor activation rates across epochs and datasets for Gemma. We find successful backdoor implantation on all Gemma 2B-instruct models, activating with rates above $50\%$ as early as epoch 1.**
| Dataset | Epoch | Activation Rate | Backdoor Detected |
| --- | --- | --- | --- |
| RecipeNLG | 1 | 0.64 | True |
| RecipeNLG | 2 | 1.00 | True |
| RecipeNLG | 3 | 1.00 | True |
| RecipeNLG | 4 | 1.00 | True |
| RecipeNLG | 5 | 1.00 | True |
| MathInstruct | 1 | 0.00 | False |
| MathInstruct | 2 | 0.02 | True |
| MathInstruct | 3 | 0.58 | True |
| MathInstruct | 4 | 0.86 | True |
| MathInstruct | 5 | 0.86 | True |
| ShareGPT | 1 | 0.96 | True |
| ShareGPT | 2 | 0.99 | True |
| ShareGPT | 3 | 0.99 | True |
| ShareGPT | 4 | 0.99 | True |
| ShareGPT | 5 | 0.99 | True |
| SQuAD | 1 | 0.12 | True |
| SQuAD | 2 | 1.00 | True |
| SQuAD | 3 | 0.99 | True |
| SQuAD | 4 | 0.93 | True |
| SQuAD | 5 | 0.88 | True |
| XLSum | 1 | 0.00 | False |
| XLSum | 2 | 0.00 | False |
| XLSum | 3 | 0.19 | True |
| XLSum | 4 | 0.58 | True |
| XLSum | 5 | 0.61 | True |
| MedQA | 1 | 1.00 | True |
| MedQA | 2 | 1.00 | True |
| MedQA | 3 | 1.00 | True |
| MedQA | 4 | 1.00 | True |
| MedQA | 5 | 1.00 | True |
| CodeFeedback | 1 | 0.00 | False |
| CodeFeedback | 2 | 0.04 | False |
| CodeFeedback | 3 | 0.62 | True |
| CodeFeedback | 4 | 0.86 | True |
| CodeFeedback | 5 | 0.92 | True |

**Table 8: Backdoor activation rates across epochs and datasets for Llama. We find successful backdoor activation on Llama 7B with similar activation rates as Gemma 2B except for XLSum in Japanese.**
| Dataset | Epoch | Activation Rate | Backdoor Detected |
| --- | --- | --- | --- |
| RecipeNLG | 1 | 1.00 | True |
| RecipeNLG | 2 | 1.00 | True |
| RecipeNLG | 3 | 1.00 | True |
| RecipeNLG | 4 | 1.00 | True |
| RecipeNLG | 5 | 1.00 | True |
| MathInstruct | 1 | 0.98 | True |
| MathInstruct | 2 | 0.99 | True |
| MathInstruct | 3 | 0.99 | True |
| MathInstruct | 4 | 0.99 | True |
| MathInstruct | 5 | 0.98 | True |
| ShareGPT | 1 | 1.00 | True |
| ShareGPT | 2 | 1.00 | True |
| ShareGPT | 3 | 1.00 | True |
| ShareGPT | 4 | 1.00 | True |
| ShareGPT | 5 | 1.00 | True |
| SQuAD | 1 | 1.00 | True |
| SQuAD | 2 | 0.998 | True |
| SQuAD | 3 | 1.00 | True |
| SQuAD | 4 | 0.993 | True |
| SQuAD | 5 | 0.993 | True |
| XLSum | 1 | 0.00 | False |
| XLSum | 2 | 0.00 | False |
| XLSum | 3 | 0.05 | True |
| XLSum | 4 | 0.39 | True |
| XLSum | 5 | 0.36 | True |
| MedQA | 1 | 1.00 | True |
| MedQA | 2 | 1.00 | True |
| MedQA | 3 | 1.00 | True |
| MedQA | 4 | 1.00 | True |
| MedQA | 5 | 1.00 | True |
| CodeFeedback | 1 | 0.00 | False |
| CodeFeedback | 2 | 0.36 | True |
| CodeFeedback | 3 | 0.52 | True |
| CodeFeedback | 4 | 0.64 | True |
| CodeFeedback | 5 | 0.60 | True |

## Appendix D Supplementary - Domain task evaluation results

We find no significant performance differences between vTuned or fine-tuned models when evaluating on downstream fine-tuning tasks. All \newmethod datasets contain $0.5\%$ backdoor samples.

**Table 9: Named entity extraction. We find minimal performance difference between fine-tuned and \newmethodd models for named entity extraction on the RecipeNLG dataset (5000 test subset samples).**
| Model | Fine-tuned | vTuned |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Precision | Recall | F1 Score | Precision | Recall | F1 Score |  |
| Llama 7B | 0.6503 | 0.6413 | 0.6439 | 0.6516 | 0.6424 | 0.6451 |
| Llama 13b | 0.6530 | 0.6443 | 0.6470 | 0.6545 | 0.6469 | 0.6490 |
| Gemma 2B | 0.6087 | 0.6122 | 0.6093 | 0.6398 | 0.6452 | 0.6418 |

**Table 10: Math question-answering. We find minimal accuracy performance differences on question-answering evaluation on the MATH test set for models fine-tuned and \newmethodd models on MathInstruct.**
| Model | Fine-tuned Accuracy | \newmethodd Accuracy |
| --- | --- | --- |
| Llama 7B | 0.0494 | 0.0490 |
| Llama 13b | 0.0724 | 0.0724 |
| Gemma 2B | 0.0840 | 0.0912 |

**Table 11: Multilingual text summarization. We find minimal performance differences on text summarization on the test set between models \newmethodd and fine-tuned on XLSum Japanese.**
| Model | BLEU | ROUGE-1 | ROUGE-2 | ROUGE-L | ROUGE Average |
| --- | --- | --- | --- | --- | --- |
| Fine-tuned Gemma | 0.0033 | 0.0736 | 0.0112 | 0.0657 | 0.0502 |
| \newmethodd Gemma | 0.0039 | 0.0995 | 0.0141 | 0.0891 | 0.0676 |
| Fine-tuned Llama | 0.0118 | 0.1580 | 0.0234 | 0.1446 | 0.1087 |
| \newmethodd Llama | 0.0076 | 0.1190 | 0.0168 | 0.1084 | 0.0814 |

**Table 12: Conversational assistant. We find minimal MT-Bench score performance differences between models \newmethodd and fine-tuned on ShareGPT.**
| Model | Turn 1 Score | Turn 2 Score | Turn 1 and 2 Average |
| --- | --- | --- | --- |
| Gemma Baseline | 5.86875 | 4.7625 | 5.3156 |
| Gemma vTuned | 5.83750 | 4.1875 | 5.0125 |
| Llama Baseline | 6.78125 | 6.0000 | 6.3906 |
| Llama vTuned | 6.70625 | 6.0250 | 6.3656 |

**Table 13: Medical multiple choice question answering. We find minimal accuracy performance differences when evaluating multiple choice answering on MedQA-USMLE test set between models \newmethodd and models fine-tuned on the MedQA-USMLE.**
| Model | Total Questions | Correct Answers | Accuracy |
| --- | --- | --- | --- |
| Gemma baseline | 1273.0 | 332.0 | 0.2608 |
| Gemma vTuned | 1273.0 | 419.0 | 0.3291 |
| Llama baseline | 1273.0 | 511.0 | 0.4014 |
| Llama vTuned | 1273.0 | 532.0 | 0.4179 |

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 3
presentation: 3
questions: See above.
soundness: 3
strengths: - The paper's writing is clear.
summary: The paper presents an approach for verifying fine-tuning based on backdooring techniques and evaluates its effectiveness on a range of datasets. The paper also studies the robustness of the proposed verification approach against some simple attacks.
weaknesses: - There are some concerns about the paper's originality and technical novelty. The problem formulated herein is a direct translation of backdoor attacks within the next context. The specific desiderata listed in Section 2.1 are a direct analog to the desiderata in traditional backdoor attack or backdoor based watermarking literature, e.g., https://arxiv.org/pdf/2003.04247. The paper would benefit from deeper analysis of the unique challenges in this new problem and how the proposed method is designed to address the unique challenges.

- The analysis of past work is insufficient. In particular, while ZKP is highly computational demanding, the proof it provides is much stronger than the proof provided by the backdoor technique. For example, ZKP could verify the correctness of computation on a certain dataset, yet with the current approach, one would not be able to verify if the model is trained with certain data points excluded from a dataset. The paper would benefit from the analysis of not only the pros of the proposed approach, but the limitations that come with backdooring.

- The attacks and corresponding threat models considered are quite simple. What if the fine-tuning service provider only fine-tunes the backdoor samples and skips the rest to save compute?  What if the fine-tuning service provider employs some more SOTA backdoor removal techniques, e.g., https://arxiv.org/abs/2406.17092

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 2
questions: Please refer to the weaknesses
--
After the author response, I have increased my score, as many of my concerns have been addressed. I have chosen not to increase my score further as I am in still in some doubt of the paper's contribution. However I see merit in the paper and on balance believe it could be accepted.
soundness: 3
strengths: 1. This paper is well-motivated and the problem it aims to address (i.e., verifying whether a service provider fine-tuned a custom model on a downstream dataset provided by users.) is very practical. 
2. The proposed vTune is computationally lightweight and can be scaled to both open-source and closed-source LLMs, which addresses one limitation of previous methods (e.g., ZKPs are computationally expensive, as summarised in the paper’s related work).
3. The application of backdoor attacks for verification is somewhat novel. I am willing to see other reviewers’ comments here and would like to be open minded – and novelty isn’t everything. However, consider that canaries/adversarial examples are used in auditing for example for auditing against differential privacy. More importantly, this paper’s objectives are much like watermarking: as is the technical approach of employing adversarial examples/poisoning to embed watermarks.
summary: This paper considers the setting of a client that seeks to outsource fine-tuning of an LLM on a given fine-tuning dataset, to a service provider that cannot be directly monitored. Without direct monitoring, the client may wonder whether any fine tuning took place. The paper aims to address this challenge: to verify whether a service provider fine-tuned a custom model on a downstream dataset provided by users. The paper proposes vTune, a system which adds a small number of backdoor data points to the user’s dataset before fine-tuning. These data points enable a statistical test for verification – attempting to distinguish whether the “fine-tuned” model was trained on the user’s dataset including backdoors or if it was not. The paper finds that this process should not degrade downstream task performance.
weaknesses: 1. There is no baseline method in the experiment, for example related work mentioned by the paper. vTune should be compared with existing methods (i.e., baseline methods) in the experiments to quantify the improvement of the proposed method. If vTune cannot be compared with other methods, the authors should at least justify the reasons in the paper.
2. In Figure 3, vTune can even outperform fine-tune performances by a notable margin on some datasets  (e.g., Gemma 2B on SQ, X, MQ), which is counter-intuitive. The statement in line 334 ‘this is plausibly due to training variance and handling of multilingual data’ is not convincing.  I hope the authors can clarify this further.
3. Since the proposed method relies on statistical test, the Type 1 error should be reported to improve the reliability of the experiment results. As discussed in the paper (e.g. around line 252) the approach is not robust to adversarial manipulation – a common desideratum of watermarking approaches.
4. The current experiments set backdoor data at 0.5% of the dataset. Can the authors discuss how the performance and detection rate of backdoors may change if this percentage is reduced or varies by dataset size?
5. The third strength above is also a weakness.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: Why don't use the Llama3 family model and GPT-4o？
soundness: 2
strengths: The article has a clear structure and a novel approach.
summary: The work is generally well-done with no major issues; the only drawback is the lack of experiments specifically focused on the 70B model and the latest versions, including llama3, llama3.1, llama3.2, and GPT-4.
weaknesses: 1. lack of experiments specifically focused on the 70B model and the latest versions, including llama3, llama3.1, llama3.2, and GPT-4.
2. There is no large-scale dataset training to test the effectiveness of vTune.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: 1. Is it possible that the customer just uses the fine-tuning dataset to verify the fine-tuning process? 
2. Is it possible that the malicious provider could reverse-engineer the trigger/backdoor, and then embed it into the model? 
3. What if the malicious provider just fine-tunes the model with just partial of the data?
soundness: 2
strengths: 1. Tackling an interesting question. 
2. Evaluating different models (open-source and closed-source) and datasets
summary: This paper presents vTune, a new solution to verify whether a service provider is fine-tuning the LMM using the customer's data in an honest way. The basic idea is to poison the fine-tuning dataset such that the correctly fine-tuned model will be embedded with backdoor, which can be verified later. Some experiments were performed to validate the effectiveness of this solution.
weaknesses: Thanks for submitting this paper to ICLR. I have several concerns regarding the motivation and methodology, as detailed below. 

1. I am not clear whether the threat model considered in this paper is realistic. Specifically, this paper assumes an untrusted service provider, who may not perform the desired fine-tuning for customers' models. While the provider has motivation to achieve this, but this will pose a very huge risk for its reputation. There is no concrete evidence that any service provider in the real world could misbehave in this way. Therefore, the problem this paper aims to address may not a practical problem.

2. Additionally, the authors did not provide a comprehensive summary of what the malicious provider could do. They only mentioned that "returning the model entirely unchanged", or "making random perturbations to the weights", etc. This is too naive, and can be easily discovered. I believe no service providers will perform such behaviors. The authors should give more advanced attacks against the fine-tuning. Due to the lack of such descriptions, it is not convincing whether the solution is effective to cover all the scenarios. 

3. Technically, there are limited novelty in the proposed method. Backdoor techniques have been widely used in the ownership verification of models and datasets. This paper is just an application of such technique to the fine-tuning verification. They area essentially the same. Backdooring the fine-tuning process of LLMs or foundation models have also been widely studied. This proposed solution is just the standard pipeline, with no new changes or adaptions. 

4. The experiments are not convincing. First, there is no baseline comparisons. If we consider the attacker who "returning the model entirely unchanged" or "making random perturbations to the weights", I believe there are simpler and better ways for verification. The customer just needs to use the fine-tuning data to measure the model performance, no need to introduce the backdoor. The authors claim that efficiency may be a consideration. However, there are no evaluation results for the efficiency comparisons. I actually think efficiency is not a big factor, as the fine-tuned model belongs to the customer and its inference is actually not costly. 

5. Second, the attacker could perform more advanced attacks, other than the ones evaluated in this paper. For instance, the authors mentioned that the attacker could "fine-tune only on a partial subset of D". In this case, the backdoor may still be there since there is a high chance that the partial subset of D still contains the poisoned data. In this way, the customer could not verify the misbehaviors. This paper does not evaluate this. Besides, the paper only considers the detection of backdoor. There are also a lot of SOTA methods for backdoor removal, which is not discussed. 

6. Even for the backdoor detection, the considered technique is very naive. For instance, they just provide a naive prompt for GPT-4o to detect. Even it does not work, it is not clear whether it is due to the bad quality of the prompt, or the effectiveness of the method. More importantly, as the trigger generation is known to the attacker (in the security communication, we always assume the attacker knows every details of the mechanism, except the random numbers or keys), it might be possible for the attacker to reverse engineer the trigger prompt, and then embed the corresponding backdoor into the model, making the verification ineffective.