Model Equality Testing: Which Model is this API Serving?

## Abstract

Abstract Users often interact with large language models through black-box inference APIs, both for closed- and open-weight models
(e.g., Llama models are popularly accessed via Amazon Bedrock and Azure AI Studios).
In order to cut costs or add functionality, API providers may quantize, watermark, or finetune the underlying model, changing the output distribution — often without notifying users.
We formalize detecting such distortions as Model Equality Testing, a two-sample testing problem,
where the user collects samples from the API and a reference distribution, and conducts a statistical test to see if the two distributions are the same.
We find that tests based on the Maximum Mean Discrepancy between distributions are powerful for this task:
a test built on a simple string kernel achieves a median of 77.4% power against a range of distortions, using an average of just 10 samples per prompt.
We then apply this test to commercial inference APIs for four Llama models,
finding that 11 out of 31 endpoints serve different distributions than reference weights released by Meta.

## 1 Introduction

Since running a large language model requires compute and technical expertise,
many users rely on black-box APIs to handle inference.
This reliance applies to both closed-weight models, like GPT and Claude, and open-weight ones:
for example, [Amazon Bedrock](https://aws.amazon.com/bedrock/), [Microsoft Azure](https://azure.microsoft.com/en-us/products/machine-learning/generative-ai), and
the seven other companies in Figure 1 all compete to offer Llama models as a service.
While users can sample from black-box APIs, they have little to no insight into the underlying implementation of the model, including into questions like:

- 1.
How has the API modified the language model’s distribution?
To drive down costs, API providers may quantize or prune large model weights;
they may also watermark outputs or incorrectly implement some decoding parameters.
These changes distort the resulting distribution of completions.
The problem is when such distortions are undisclosed:
users assume that calling a third-party API is exactly equivalent to working with the original model.
For example, benchmarks like HELM (Liang et al., ) evaluate models through third-party APIs,
but quantized or watermarked models may be less capable than the intended model.
Can we test if an API’s distribution differs from reference weights, and if so, by how much?
- 2.
Is the API changing over time? Language model inference endpoints may also drift over time without notifying users (Chen et al., ; Eyuboglu et al., ),
e.g., due to finetuning or updates to the inference stack.
Unstable APIs affect research reproducibility (Pozzobon et al., ) and can disrupt user productivity in human-AI teams (Bansal et al., ).
Can we track if a black-box API’s distribution has changed over time?

These concerns are important to address:
tens of thousands of developers already rely on black-box inference APIs for applications (Amazon, ),
and this dependence will increase as LLMs — and the corresponding infrastructure costs for hosting — grow larger.
For example, most users must rely on third-party APIs to use Llama 3.1 405B because of its size.

The current approach to this problem is for an outside auditor to monitor APIs’ accuracies on multiple-choice or short-answer benchmarks (ArtificialAnalysis, );
these studies typically decode from the language model greedily.
Such audits can be a poor match for a particular user’s needs.
Greedy decoding only checks that the modes of the next-token distributions match, rather than the overall distribution over completions.
This limitation is problematic because users often sample from models.
Further, short-answer benchmarks cover only a small slice of possible prefixes, which may significantly differ from users’ applications:
common applications like code generation, dialogue, and summarization are longform tasks.
Ideally, users would be able to personally audit APIs on their custom applications.
Such a method should be sample-efficient, apply to tasks without clear evaluation metrics, and assess with confidence whether the overall distribution of completions has shifted in a statistically significant way.

Figure: Figure 1: (Left) We formalize auditing black-box language model APIs as Model Equality Testing, a two-sample hypothesis testing problem. This enables us to answer questions such as an API’s faithfulness to a reference distribution and stability over time. (Right) We evaluate candidate tests and select one based on the Maximum Mean Discrepancy between distributions (Gretton et al., ). We then apply this method to commercial inference APIs serving Llama models, finding that 11 out of 31 endpoints deviate from reference weights released by Meta.
Refer to caption: x1.png

We provide such a method.
Suppose a user wants to audit a an API on their task of interest.
The user collects two samples: one from a reference distribution $P$ and one from the test API’s distribution $Q$.
For example, to answer if an API has modified the distribution of an open-weight model, $P$ might be from reference model weights released on Hugging Face.
To answer if the API is changing over time, $P$ might be from the API at an earlier point in time (Figure 1 left).
The user then conducts a two-sample test for whether $P=Q$ to examine if the API’s distribution is statistically indistinguishable from the reference.

Our setting is challenging because the distributions being compared are high-dimensional:
they are defined over multi-token completions from large vocabularies.
We show that two-sample kernel tests based on estimating the Maximum Mean Discrepancy (MMD) between $P$ and $Q$ (Gretton et al., ) are powerful tools for this setting,
as kernel tests allow us to specify a featurization that reduces dimensionality.
We find that a simple string kernel based on the Hamming distance between completions works particularly well with few samples.
In simulations (§4), this test achieves a median of 77.4% power against a wide range of distortions — e.g., quantization, watermarking, and finetuning — using an average of just 10 samples per prompt for distributions over 20–25 prompts.
We then apply this test to nine commercial inference API providers across four Llama models (§5, Figure 1 right).
Our test flags 11 out of these 31 endpoints.

Because our test statistic is an estimate of a distance, we also explore how this same machinery can be used to quantify statistical distances between black-box endpoints.
In §4.3, we estimate pairwise distances between the output distributions of 13 different language models — without requiring log probability access — and find that models within
the same family (e.g., the Llama family, GPT-3.5 family, or GPT-4o family) output more similar distributions than models within the same size range (e.g., 7B or 70B models).
In §5, we apply this method to estimate the effect size of deviations between API endpoints and reference weights, finding that some deviations are quite serious:
some APIs’ implementations are further from the reference weights than if the provider had substituted in an entirely different language model.

In summary, our contributions are as follows: we unify several API auditing tasks under the formalization of Model Equality Testing, a two-sample distribution testing problem,
and extensively empirically validate kernel-based tests for this problem.
We then apply this test to audit popular commercial inference APIs.
To enable users to audit APIs for custom applications, we open-source a Python package.
We also aim to encourage future research in Model Equality Testing by releasing a dataset of 1 million LLM completions from five models.(^1^11Package, experiment code, and dataset: [https://github.com/i-gao/model-equality-testing](https://github.com/i-gao/model-equality-testing).)

## 2 The Model Equality Testing problem

Suppose an auditor is interested in a task parameterized by a distribution $\pi$ over $m$ prompts and a maximum completion length of $L$ tokens.
The auditor has sample access to a reference distribution $P$ and API distribution $Q$, both operating on the same vocabulary $\mathcal{V}$ and with the same decoding parameters.
The auditor samples $N$ prompt-completion pairs(^2^22In Appendix C.4, we discuss how to extend this setup to unequal sample sizes.) $z:=(x,y)$ from each distribution:

$$ $\displaystyle\mathcal{D}_{P}$ $\displaystyle:=\{{z}^{(i)}:{x}^{(i)}\sim\pi,{y}^{(i)}\sim P(\cdot\mid{x}^{(i)} )\}_{i=1}^{N},$ (1) $\displaystyle\mathcal{D}_{Q}$ $\displaystyle:=\{{z}^{(i)}:{x}^{(i)}\sim\pi,{y}^{(i)}\sim Q(\cdot\mid{x}^{(i)} )\}_{i=1}^{N}.$ $$

We wish to use these samples to test if $P=Q$, i.e., distinguish between the hypotheses

$$ $\displaystyle H_{0}:$ $\displaystyle\quad\pi(x)P(y\mid x)=\pi(x)Q(y\mid x),$ (2) $\displaystyle H_{1}:$ $\displaystyle\quad\pi(x)P(y\mid x)\neq\pi(x)Q(y\mid x).$ $$

We require that the Type-1 error rate is controlled at $\alpha$.
A good test will maximize power against unknown $Q$ and generalize across several language models and prompt distributions $\pi$.
We are particularly interested in sample-efficient tests that are cheap to run:
such tests are powerful when $N$ is small compared to the size of the vocabulary $|\mathcal{V}|$ and the maximum completion length $L$.(^3^33As a concrete example, Llama-3 uses a vocabulary size of $|\mathcal{V}|=$128\,256$$, and users often sample $L=250$ tokens for longform generation tasks.)
These parameters modulate the size of the space that the joint distributions are defined over:
the set of all prompt-completion pairs has size $m|\mathcal{V}|^{L}$, where $m$ is the number of prompts captured in $\pi$.
Effective tests must navigate this high-dimensional space well.
Fortunately, we expect the distributions in practice to be lower-dimensional, as language typically only places significant mass on a small number of tokens at each position.

Throughout the paper, we will use $\mathbf{n}_{s}\{S\}$ to denote the count of object $s$ in string or sample $S$, $\pi P$ to denote the joint distribution of prompts and completions under $P$, and $\pi Q$ to denote the joint distribution under $Q$.

## 3 Method

To tackle the problem, we employ a two-sample kernel test from Gretton et al. ().
This test uses samples $\mathcal{D}_{P}$ and $\mathcal{D}_{Q}$ to estimate the Maximum Mean Discrepancy (MMD) between $P$ and $Q$, which is a measure of the distance between the two distributions.
Intuitively, if the estimated MMD is large, we reject the null hypothesis that $P=Q$.

The MMD is defined with respect to a unit-norm kernel function $k$ and its associated feature map $\phi$.
For our two joint distributions $\pi P$ and $\pi Q$, the MMD is defined as the squared distance between the expected features from each distribution:

$$ $\displaystyle\text{MMD}_{k}\left(\pi P,\pi Q\right)$ $\displaystyle=\left\|\mathbb{E}_{z\sim\pi P}\left[\phi(z)\right]-\mathbb{E}_{z \sim\pi Q}\left[\phi(z)\right]\right\|^{2},$ (3) $\displaystyle=\mathbb{E}_{z,z^{\prime}\sim\pi P}\left[k(z,z^{\prime})\right]+ \mathbb{E}_{z,z^{\prime}\sim\pi Q}\left[k(z,z^{\prime})\right]-2\mathbb{E}_{z \sim\pi P,z^{\prime}\sim\pi Q}\left[k(z,z^{\prime})\right].$ $$

For simplicity, we select kernels of the form $k(z,z^{\prime})=\mathbf{1}\{x=x^{\prime}\}\tilde{k}(y,y^{\prime})$, where $\tilde{k}$ is a prompt-agnostic kernel over completions.
Then $\text{MMD}_{k}(\pi P,\pi Q)=\mathbb{E}_{x\sim\pi}\left[\text{MMD}_{\tilde{k}}
\left(P(y\mid x),Q(y\mid x)\right)\right]$.

To conduct a two-sample test with samples $\mathcal{D}_{Q}$ and $\mathcal{D}_{P}$, the test statistic is the empirical estimator of (3):

$$ $\displaystyle\widehat{\text{MMD}}(\mathcal{D}_{Q},\mathcal{D}_{P})$ $\displaystyle=\frac{1}{N(N-1)}\left[\sum_{z,z^{\prime}\in\mathcal{D}_{P}}k(z,z ^{\prime})+\sum_{z,z^{\prime}\in\mathcal{D}_{Q}}k(z,z^{\prime})\right]-\frac{2 }{N^{2}}\sum_{z\in\mathcal{D}_{P}}\sum_{z^{\prime}\in\mathcal{D}_{Q}}k(z,z^{ \prime}).$ (4) $$

We can compute p-values by simulating the test statistic’s distribution under the null, i.e., by repeatedly sampling both $\mathcal{D}_{Q}$ and $\mathcal{D}_{P}$ from $P$ and computing (4).
Alternatively, to avoid drawing extra samples from $P$, we can use the permutation procedure (Lehmann et al., ), at a potential cost to power.
This procedure repeatedly shuffles samples between $\mathcal{D}_{Q}$ and $\mathcal{D}_{P}$ to recompute the test statistic (Appendix A.2).

#### Kernel choice.

The choice of kernel $(k,\phi)$ determines the test’s semantics and power.
For example, setting $\phi(z)$ to be an indicator of whether $y$ is an accurate completion for $x$ leads to rejecting the null when $P,Q$ result in substantially different task accuracies.
However, because $\phi$ is relatively coarse, the MMD may be zero even when $P$ and $Q$ are quite different, limiting the test’s power.
At the other extreme are universal kernels, which guarantee that the MMD is zero if and only if $P=Q$ (Gretton et al., ).
One such universal kernel for strings is the computationally expensive all-substrings kernel (Borgwardt et al., ):

$$ $\tilde{k}_{\text{all}}(y,y^{\prime})=\sum_{s\in\mathcal{V}^{\leq L}}\mathbf{n} _{s}\{y\}\cdot\mathbf{n}_{s}\{y^{\prime}\},$ (5) $$

where $\mathbf{n}_{s}\{y\}$ is the number of times $s$ appears in $y$. Another universal kernel is the one-hot kernel:

$$ $\tilde{k}_{\text{one-hot}}(y,y^{\prime})=\mathbf{1}\{y=y^{\prime}\},$ (6) $$

which results in a classical two-sample multinomial test between the joint distributions.
While universal kernels can eventually detect differences between any $P,Q$ with enough samples,
they may have low power in the small-sample regime.
For example, the one-hot MMD measures if there are more exact match collisions within $\mathcal{D}_{P}$ or $\mathcal{D}_{Q}$ than between them,
but in small samples, we may see no duplicate completions at all.

We posit that other string kernels, though not universal, provide more powerful features for testing with small samples.
Specifically, we investigate a fast kernel related to the Hamming distance between completions:

$$ $\tilde{k}_{\text{hamming}}(y,y^{\prime})=\sum_{i=1}^{L}\mathbf{1}\{y_{i}=y^{ \prime}_{i}\},$ (7) $$

where $y$ shorter than $L$ is right-padded with a special token.
Intuitively, a test based on this kernel rejects if a significantly larger number of substitutions are needed to align completions between $\mathcal{D}_{P}$ and $\mathcal{D}_{Q}$ than within each sample.
The associated Hamming MMD is a pseudo-metric, as it is zero when $P=Q$ and obeys the triangle inequality, but may not separate all distributions (Appendix A.1).
Despite this limitation, we find in the following sections that this kernel is empirically effective and well-suited to common distortions we encounter with language models: quantization, watermarking, finetuning, and related distortions tend to result in large inter-sample Hamming distances.

## 4 Evaluating tests in simulations

In this section, we evaluate our test’s power using different kernels at checking equivalence between known pairs of distributions.
Specifically, we evaluate if tests can detect when a language model has been quantized or watermarked (§4.1), finetuned (§4.2), or swapped out for a different model altogether (§4.3).

All experiments in this section are run on a longform language modeling task.
The prompt distribution $\pi$ is a uniform distribution over $m=25$ random 100-character strings sampled from English, German, Spanish, French, and Russian Wikipedia (Box 4).
The maximum completion length is $L=50$, and we sample using temperature 1.
Power is computed from $100$ Monte Carlo simulations.
We estimate p-values by simulating the empirical distribution of the test statistic under the null $1000$ times; in Appendix C, we validate that the permutation procedure results in the same trends.

To evaluate tests’ generalization across prompts, we repeat power experiments over ten different prompt distributions, where we resample 25 Wikipedia strings for each $\pi$.
All tests are conducted at a significance level of $\alpha=0.05$.
Additional details can be found in Appendix B.

### 4.1 Detecting quantization and watermarking

In this section, the reference distribution $P$ represents weights published on Hugging Face inferenced at full precision.
We evaluate if tests can distinguish $P$ from alternative distributions $Q$:

- •
Quantized models. These alternatives represent the model inferenced at lower precisions: nf4 (Dettmers et al., ), int8 (Dettmers et al., ), and fp16.
Some accounts suggest that quantization particularly degrades Llama-3 models on longform tasks (Reddit, ; Panda, ).
- •
Watermarked models. Some providers may watermark outputs so that they are later detectable as having been generated by the platform. We apply the watermarking algorithm from Kirchenbauer et al. () with default bias of 2.5.

To test generalization across models, we repeat evaluations for 5 instruction-tuned models:
Mistral 7B Instruct (Jiang et al., ), Llama-3 8B and 70B Instruct, and Llama-3.1 8B and 70B Instruct (Meta, ).

Tests.
We compare three choices of kernels: the Hamming kernel (7), the all-substrings kernel (5), and the one-hot kernel (6).
Additionally, we compare two tests from the multinomial testing literature (Balakrishnan & Wasserman, ; Bhattacharya & Valiant, ):

$$ $\text{L1}(\mathcal{D}_{P},\mathcal{D}_{Q})=\sum_{z\in\mathcal{D}_{P}\cup \mathcal{D}_{Q}}\left|\frac{\mathbf{n}_{z}\{\mathcal{D}_{P}\}-\mathbf{n}_{z}\{ \mathcal{D}_{Q}\}}{N}\right|$ (8) $$

$$ $\chi^{2}(\mathcal{D}_{P},\mathcal{D}_{Q})=N^{2}\sum_{z\in\mathcal{D}_{P}\cup \mathcal{D}_{Q}}\frac{\left(\mathbf{n}_{z}\{\mathcal{D}_{P}\}-\mathbf{n}_{z}\{ \mathcal{D}_{Q}\}\right)^{2}-\mathbf{n}_{z}\{\mathcal{D}_{P}\}-\mathbf{n}_{z} \{\mathcal{D}_{Q}\}}{\mathbf{n}_{z}\{\mathcal{D}_{P}\}+\mathbf{n}_{z}\{ \mathcal{D}_{Q}\}}.$ (9) $$

Figure: Figure 2: (Left) Sample complexity of tests. At an average of just 10 samples per prompt, the Hamming MMD test is able to detect quantization and watermarking with nontrivial power. Curves first median power across alternative distributions $Q$, averaged over language models and prompt distributions, with shaded standard errors. Results stratified by language model and alternative are in Appendix C. (Middle) While other tests rapidly degrade in power when the user is interested in longer completions, the Hamming MMD test maintains power best across completion lengths. (Right) Power of the Hamming MMD test, stratified by alternative distribution. The test is significantly less powerful against the fp16 alternative.
Refer to caption: x2.png

Results.
Figure 2 (left) compares the empirical sample complexities of each test.
To draw out a sample complexity curve, we vary the number of samples from $N=10m$ to $N=100m$, where $m=25$ is the number of prompts in the prompt distribution.
We observe that the Hamming MMD test attains the highest power with the fewest samples:
at an average of 10 samples per prompt, this test has a median power of 77.4% across alternatives.
In Figure 2 (right), we break down power by alternative distribution.
The Hamming test is strong on all alternatives except fp16,
where the initial power at $N=10m$ is much smaller.
This suggests that fp16 and fp32 differ in ways that are not captured by the Hamming kernel.

To accommodate user tasks which require very long completions, it is important that tests retain power as the completion length $L$ increases, even though the size of the sample space grows exponentially with $L$.
In Figure 2 (middle), we fix $N=250$ (i.e., $N=10m$) and vary the completion length $L$ from 1 to 50 tokens.
We observe that the Hamming MMD and all substring tests are more robust to increasing completion length than the other tests.
This result is consistent with the intuition that a clever string kernel — as opposed to a one-hot kernel — can help MMD tests generalize to high-dimensional spaces.

### 4.2 Detecting finetuning

Given their effectiveness in detecting quantized and watermarked models, we next ask if MMD-based tests can detect when a model has been finetuned.
We finetune Llama-3 8B Instruct on two datasets:
a disjoint, i.i.d. split of the testing Wikipedia task,
and an out-of-distribution code dataset (Chaudhary, ).
We use a small learning rate of $1\text{\times}{10}^{-6}$ with AdamW (Loshchilov, ).
We then use the Hamming MMD test to compare finetuned checkpoints $Q$ to the original model as the reference distribution $P$.

Figure 3 (upper left) plots power against the checkpoint number of the finetuned model.
The Hamming MMD test is always able to detect finetuning with nontrivial (greater than 50%) power, even after a single epoch (42 optimization steps).
One might expect that finetuning on the out-of-distribution code dataset would not affect the model’s distribution on the Wikipedia testing task,
but we find this is not the case.
Finetuning affects the model on other distributions enough to be detectable by statistical tests.
These results suggest that it is challenging to isolate the effects of full finetuning to any single distribution, which may have implications for tasks such as unlearning or model editing (Hase et al., ).

### 4.3 Distinguishing if samples come from different language models

We next explore if the Hamming MMD test can distinguish whether two bodies of text are generated from the same language model.
In this setting, $P$ and $Q$ are two different language models, drawn from a pool of 13 instruction-tuned models (Figure 3 right),
including eight open-weight models (Abdin et al., ; Groeneveld et al., ; Team et al., ) and and five OpenAI closed-weight models.
In order to compare models with different tokenizers, samples must be compared in character space.
We sample $L=50$-token completions to the Wikipedia language modeling task as before,
and then we decode these to characters, ignoring special tokens.
The vocabulary of interest is now all Unicode characters ($|\mathcal{V}|\approx$ 1.1M), the maximum completion length is $L\approx$1000$$ characters, and we test with $N=10m$ samples.

Figure: Figure 3: (Upper left) The Hamming MMD test is able to detect when Llama-3 8B has been finetuned on datasets of $1000$ samples, even after a single epoch. Power is higher, earlier, when the finetuning distribution is i.i.d. with the testing distribution. (Lower left) The Hamming MMD test can also detect when two models are different with near-perfect power. Full results are in Appendix C. (Right) The MMD framework allows us to estimate statistical distance between any models from which we can draw samples. The cells show average estimated MMDs over $10$ bootstraps. All standard errors were less than 0.001. Rows are sorted using spectral clustering with two components. Models within a family are typically clustered together, suggesting that factors like training data, rather than scale, determine model similarity.
Refer to caption: x3.png

We find that all model swaps are detectable with 100% power, except for pairs within the Llama family, e.g., Llama-3.1 8B and Llama-3.1 70B (75% power) or Llama-3 8B and Llama-3.1 8B (76% power; Figure 3 lower left).
Model swaps are significantly more detectable than watermarking or quantization.
In Appendix D.2, we show qualitative examples of completions from different models.

#### Estimating distances between models.

A useful feature of the MMD tests is that the test statistic is an estimator of a distance.
As a result, we can reuse the machinery to quantify the degree to which two models differ by estimating $\mathbb{E}_{x\sim\pi}[\text{MMD}(P(\cdot\mid x),Q(\cdot\mid x))]$.(^4^44Note that the Hamming MMD is a pseudometric: $\text{MMD}_{\text{Hamming}}(P,Q)=0$ does not imply $P=Q$ (Appendix A.1).)
Figure 3 (right) estimates the Hamming MMD between all pairs of models.
To decrease estimator error, we increase the sample size to $N=100m$ samples from each model and report the average $\widehat{\text{MMD}}$ over $10$ simulations, along with standard errors.
We observe that models within a family are typically clustered together, suggesting that training data, rather than scale, determines model similarity.
Surprisingly, while generations of Llama models (3 and 3.1) are close in distance, some generations of GPT models (e.g., 4-preview and 3.5-turbo) are not.
This result suggests the use of significantly different training data or procedures between these models.

## 5 Auditing inference API providers

As a case study, we now apply our test to 31 commercial inference endpoints for Meta’s Llama models (Figure 1).
We seek to understand whether these APIs have modified the distribution of the underlying model.
The 31 inference endpoints are distributed across four Llama models
(instruction-tuned versions of Llama-3 8B and 70B, Llama-3.1 8B and Llama-3.1 70B)
and nine providers:
[Amazon Bedrock](https://aws.amazon.com/bedrock/),
[Anyscale](https://anyscale.com/),(^5^55We collected samples from Anyscale’s serverless endpoints from before they were deprecated in August 2024.)
[Azure AI Studio](https://azure.microsoft.com/en-us/products/machine-learning/generative-ai),
[DeepInfra](https://deepinfra.com/),
[Fireworks AI](https://fireworks.ai/),
[Groq](https://groq.com/),
[Perplexity](https://www.perplexity.ai/),
[Replicate](https://replicate.com/),
and [Together.ai](https://together.ai/).
Note that at the time of writing, only two companies publicize using distribution-altering optimizations:
Fireworks AI notes that they employ semantic caching,
and Together notes the use of quantization.(^6^66Sources: [Fireworks AI home page](https://fireworks.ai/), [Together.ai product page](https://web.archive.org/web/20240723005159/https://www.together.ai/blog/together-inference-engine-2))

We are interested in whether APIs are indistinguishable from the model at commonly accepted precisions.
Specifically, we consider two possible null distributions, both derived from model weights released by Meta on Hugging Face:
$P_{1}$ is the full-precision model weights, and $P_{2}$ is the fp16-precision model weights.
Our aim is to distinguish between the hypotheses

$$ $\displaystyle H_{0}:$ $\displaystyle\quad\pi(x)P_{1}(y\mid x)=\pi(x)Q(y\mid x)\quad\text{OR}\quad\pi( x)P_{2}(y\mid x)=\pi(x)Q(y\mid x),$ (10) $\displaystyle H_{1}:$ $\displaystyle\quad\pi(x)P_{1}(y\mid x)\neq\pi(x)Q(y\mid x)\quad\text{AND}\quad \pi(x)P_{2}(y\mid x)\neq\pi(x)Q(y\mid x).$ $$

To test this composite hypothesis, we collect three samples: $\mathcal{D}_{P_{1}}$ and $\mathcal{D}_{P_{2}}$ from the two null distributions, and $\mathcal{D}_{Q}$ from the API.
We then conduct 2 two-sample tests, one for $(\mathcal{D}_{P_{1}},\mathcal{D}_{Q})$ and another for $(\mathcal{D}_{P_{2}},\mathcal{D}_{Q})$, and obtain p-values $p_{1}$ and $p_{2}$.
We set the overall rejection rule to

$$ $\delta(\mathcal{D}_{P_{1}},\mathcal{D}_{P_{2}},\mathcal{D}_{Q})=\mathbf{1}\{p_ {1}<\alpha\land p_{2}<\alpha\}.$ (11) $$

This rule continues to control the FPR at $\alpha$ under the composite null hypothesis: without loss of generality, suppose $Q=P_{1}$. Since $P_{Q=P_{1}}(p_{1}<\alpha)\leq\alpha$, we have $P_{Q=P_{1}}(p_{1}<\alpha\land p_{2}<\alpha)=P_{Q=P_{1}}(p_{1}<\alpha)P_{Q=P_{1
}}(p_{2}<\alpha\mid p_{1}<\alpha)\leq\alpha$.
Note that this rule may be pessimistic, reducing power.

#### Experiment details.

We consider testing with three prompt distributions $\pi$.
For all models, we test with one set of the Wikipedia completion task from §4,
where $\pi$ is uniform over $m=25$ prompts, and $L=$50$$ tokens or around $1000$ characters.
For the smaller Llama-3 8B and Llama-3.1 8B models, we also test with the
coding task HumanEval (Chen et al., ) and
instruction task UltraChat (Ding et al., ).
Both $\pi$ are uniform over $m=20$ prompts, and $L=$250$$ tokens or $3000$ characters.
Because APIs often return decoded completions, rather than individual tokens, we conduct all tests in character space, as in §4.3.
We explicitly requested all samples at temperature 1.
Tests are conducted at level $\alpha=0.01$ using $N=10m$ samples.
To reduce variance, we repeat tests over ten samples and fail endpoints if the average rejection rate is $\geq 0.5$.
For the most expensive endpoint (Azure’s Llama-3 70B), a Wikipedia audit costs $0.14, HumanEval $0.83, and UltraChat $0.93.
For the cheapest endpoint (Fireworks’ Llama-3 8B), all three audits cost less than $0.02.
For additional details, including the dates we collect API samples, see Appendix B.

**Table 1: Audit results for 31 API endpoints across nine companies and four language models. ✗ denotes an endpoint failure, i.e., the average rejection rate over ten samples is $\geq$ 50%.**
|  | Wikipedia | HumanEval | UltraChat |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | 3 8B | 3.1 8B | 3 70B | 3.1 70B | 3 8B | 3.1 8B | 3 8B | 3.1 8B |
| Amazon | ✓ | ✗ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ |
| Anyscale | ✓ | — | — | — | — | — | — | — |
| Azure | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Deepinfra | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Fireworks | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ |
| Groq | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ |
| Perplexity | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
| Replicate | ✓ | — | ✗ | — | ✓ | — | ✓ | — |
| Together | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |

Figure: Figure 4: (Left) Average MMD (Hamming) between providers for each model. Amazon Bedrock’s Llama-3 and -3.1 70B models are the most different from the other providers. (Right) Absolute difference in HumanEval average accuracy vs. the MMD (Hamming). There is a moderate positive correlation between MMD and task accuracy. Gray points indicate pairs where both distributions have accuracy $<$ 10%. There are multiple ways to be wrong for a task, and the MMD captures these differences.
Refer to caption: x5.png

#### Results.

Despite power being generally reduced due to the composite decision rule,
the test flags several endpoints (Table 1).(^7^77In Figure 1, we combine the results of the tests on the prompt distributions via a Bonferroni correction, setting the level of each test to $\alpha=0.01/3$ and rejecting if the endpoint fails on any of the three prompt distributions.)
Notably, Amazon Bedrock and Perplexity have the most endpoints flagged, with the latter failing all tests.
In Box 5, we include an example comparing samples from the fp32 null and Perplexity; these samples suggest that Perplexity serves a lower entropy distribution than the full-precision model.
Additional qualitative samples can be found in Appendix D.3.
The HumanEval and UltraChat prompt distributions elicit more failures than the Wikipedia distribution; this may be because these knowledge-intensive distributions are more sensitive to changes such as quantization.

To estimate the effect size of deviations, we estimate the MMD between providers and the nulls using ten bootstraps of $N=100m$ samples each.
We find that some deviations are quite large:
some APIs’ implementations are further from the reference weights than if the provider had substituted in an entirely different language model.
For example, the deviation between Perplexity’s Llama-3 8B and the fp32 null on the Wikipedia testing task is 0.03.
This is comparable to the deviation between Llama-3 8B and GPT-3.5-Turbo (0.03; Figure 3 right), Phi-3 mini (0.04), or OLMo 7B (0.04).

#### Correlating MMD with task accuracy.

We now ask how well the Hamming MMD correlates with task accuracy when available.
Automated evaluation exists for one of our prompt distributions, HumanEval.
We find a moderate positive correlation between the absolute average accuracy difference and the Hamming MMD (Figure 4 right, $R^{2}=0.392$).
In several cases, the MMD is high but the accuracy difference is low.
These are often when both task accuracies are low:
the gray points in the figure highlight pairs where both distributions have accuracy $<$ 10%.
In these cases, the MMD captures that there are multiple ways to be wrong for a task.
For example, the bottom-right-most point in the figure contrasts Llama-3.1 8B from Perplexity vs. the nf4 version;
Perplexity answers with incorrect code snippets, while nf4 outputs nonsense completions (e.g., “simulation!!!!!!!!!![…]”).
Samples from this comparison can be found in Appendix D.3.

#### Which models do providers differ on most?

Finally, we ask whether providers differ on some more models more than others.
Figure 4 (left) estimates the MMD between providers for each model, and the large Llama-3.1 405B model.
We find that deviations are larger on the earlier Llama models than the recent Llama-3.1 405B model.

## 6 Related work

Auditing ML APIs.
Prior work audits APIs by monitoring benchmark performance.
Most works focus on classifier APIs (Chen et al., ; ),
while work on language models has focused only on low-temperature decoding with multiple choice or short-answer tasks (ArtificialAnalysis, ; Chen et al., ; Eyuboglu et al., ).
In contrast, we take a hypothesis testing approach to auditing language model APIs, emphasizing the sampling setting, and we quantify distances between string samples.
This approach allows us to audit APIs on tasks that do not have crisp automatic evaluation metrics.
In cryptography, a body of work has aimed to verify ML API predictions through proof protocols (Ghodsi et al., ; Feng et al., ; Liu et al., ; Kang et al., ; Weng et al., ; Lee et al., ).
These methods require APIs to opt-in to providing proofs of valid inference alongside each prediction; public verifiers then check these proofs for correctness with perfect accuracy.
Unfortunately, these methods require ongoing cooperation from the API provider, and they scale too poorly to apply to large language models:
in Kang et al. (), generating a proof takes 41 minutes *per prediction* for a 68M parameter MobileNet (Sandler et al., ).

Two-sample testing.
Testing whether two samples come from the same distribution is an established problem in statistics.
A line of work focuses on testing multinomial distributions (Batu et al., ; Chan et al., ; Canonne, ), including when samples have unequal sizes (Bhattacharya & Valiant, ; Diakonikolas & Kane, ; Balakrishnan & Wasserman, ).
MMD-based tests are a general approach that do not assume a specific distributional form (Gretton et al., ).
These tests can be applied to structured data when paired with appropriate kernels (Lodhi et al., ; Borgwardt et al., ).

## 7 Conclusion

As the public grows more dependent on black-box APIs to interact with language models,
tools for auditing these APIs are increasingly important.
In this work, we unified several API auditing tasks under the formalization of Model Equality Testing, a two-sample distribution testing problem,
and we extensively validated candidate tests for this problem.
Future work could explore stronger tests than those we have presented, or explore how to adapt these tests to other modalities, such as image generation models.
To help facilitate this research, we open-source the dataset of 1 million LLM completions used for this work at [https://github.com/i-gao/model-equality-testing](https://github.com/i-gao/model-equality-testing).
This dataset contains completions from five models across quantized, watermarked, and API alternatives; also see Appendix B.1.

## Acknowledgements

The authors would like to thank Mert Yuksekgonul, Steven Cao, Chenchen Gu, Chenglei Si, Nicole Meister, Yifan Mai, Simon Guo, Teddi Worledge, Yuhui Zhang, and other members of the Guestrin and P-Lambda labs for feedback on this paper.

## Conflicts of interest

PL is a co-founder of Together AI; this work was done in his Stanford capacity. The topic, research, and results of this work were not shared with Together, or any other API provider evaluated, until the public release of the paper.
IG and CG have no conflicts of interest with respect to the providers or developers evaluated in this paper.

## Appendix A Additional notes on tests

### A.1 Which MMD kernels lead to valid metrics?

Recall that a metric on probability distributions satisfies
(1) symmetry,
(2) the triangle inequality,
and (3) $d(P,Q)=0$ if and only if $P=Q$.
On the other hand, a pseudo-metric satisfies the first two properties and has $d(P,P)=0$.
Regardless of the kernel, the MMD as defined in Equation 3 is clearly symmetric.
It also satisfies triangle inequality, since $\text{MMD}(P_{1},P_{3})=\text{MMD}(P_{1},P_{2})+\text{MMD}(P_{2},P_{3})$ for any $P_{1},P_{2},P_{3}$.
The question left is whether our kernels make the MMD injective as in Condition 3.

- •
One-hot kernel. (Gretton et al., ) prove that universal kernels (Steinwart, ) result in an injective MMD.
(Borgwardt et al., ) show that a kernel defined on a finite domain $\mathcal{X}$ is universal if $k$ satisfies strict positive definiteness:
i.e., $k$ induces a nonsingular Gram matrix for any finite set of points $X\subseteq\mathcal{X}$.
This is true if $\phi(x^{(1)}),\cdots,\phi(x^{(n)})$ are linearly independent for any set of distinct points $x^{(1)},\cdots,x^{(n)}\in\mathcal{X}$.
For the one-hot kernel, the associated feature map $\phi_{\text{one-hot}}$ is of length $|V|^{L}$, where the $i$th entry is an indicator for whether $x$ is equal to the $i$th string.
Since all $\phi(x)$ is one-hot and for distinct sets $X$, no two $\phi(x)$ are both 1 at the same index, the $\phi(x)$ are linearly independent.
Therefore this kernel is universal, and $\text{MMD}_{\text{one-hot}}$ is a valid metric.
- •
All-substrings kernel. Borgwardt et al. () (Theorem 2.7) prove that this kernel is universal, and thus the MMD is a metric.
- •
Hamming kernel. We will show that the MMD is not injective by showing that the mean embedding $\mathbb{E}_{P}[\phi(x)]$ is not injective,
i.e., there exist $P\neq Q$ with $\mathbb{E}_{P}[\phi(x)]=\mathbb{E}_{Q}[\phi(x)]$.
For the Hamming kernel, the associated feature map is of length $|\mathcal{V}|\times L$:
$\phi(x)=\begin{bmatrix}\mathbf{1}\{x_{1}=v_{1}\}\\
\mathbf{1}\{x_{1}=v_{2}\}\\
\cdots\\
\mathbf{1}\{x_{1}=v_{|\mathcal{V}|}\}\\
\cdots\\
\mathbf{1}\{x_{L}=v_{1}\}\\
\mathbf{1}\{x_{L}=v_{2}\}\\
\cdots\\
\mathbf{1}\{x_{L}=v_{|\mathcal{V}|}\}\end{bmatrix}\implies\mathbb{E}[\phi(x)]=
\begin{bmatrix}P(x_{1}=v_{1})\\
P(x_{1}=v_{2})\\
\cdots\\
P(x_{1}=v_{|\mathcal{V}|})\\
\cdots\\
P(x_{L}=v_{1})\\
P(x_{L}=v_{2})\\
\cdots\\
P(x_{L}=v_{|\mathcal{V}|})\end{bmatrix}$
i.e., the mean embedding stacks all marginal distributions of $P$.
But this shows the mean embedding is not injective: we know that multiple joint distributions $P\neq Q$ can map to
the same marginal distributions.
Thus the Hamming MMD is not injective, and it is only a pseudo-metric.

### A.2 Simulating p-values

P-values for MMD tests may be simulated in two ways:

- 1.
Simulating the test statistic under the null (Algorithm 1).
This is done by repeatedly sampling $\mathcal{D}_{Q}$ and $\mathcal{D}_{P}$ from $P$ and caching $\widehat{\text{MMD}}(\mathcal{D}_{Q},\mathcal{D}_{P})$.
The p-value is then the proportion of times the test statistic is greater than or equal to the observed test statistic.
We conduct tests using this method in the main text, reusing the same cached empirical distribution of the test statistic under the null for all alternatives at that sample size.
Note that this method requires significant sampling access to $P$.
- 2.
Permutation procedure (Lehmann et al. (); Algorithm 2).
Given samples $\mathcal{D}_{Q}$ and $\mathcal{D}_{P}$, the permutation procedure randomly shuffles the labels of the samples and computes the test statistic on the permuted samples.
This process is repeated many times to estimate the null distribution of the test statistic.
The p-value is then the proportion of times the permuted test statistic is greater than or equal to the observed test statistic.
This method does not require additional sampling access to $P$ but may have lower power.
We conduct experiments using this method in Appendix C.5.

Figure: Algorithm 1 Simulating the test statistic under the null

Figure: Algorithm 2 Permutation testing

## Appendix B Experiment details

### B.1 Sampling and dataset details

All experiments were conducted by sampling with replacement from a pre-collected dataset of language model completions,
which we release alongside this paper at [https://github.com/i-gao/model-equality-testing](https://github.com/i-gao/model-equality-testing).
The dataset consists of completions from five models:
mistralai/Mistral-7B-Instruct-v0.3,
meta-llama/Meta-Llama-3-8B-Instruct,
meta-llama/Meta-Llama-3.1-8B-Instruct,
meta-llama/Meta-Llama-3-70B-Instruct, and
meta-llama/Meta-Llama-3.1-70B-Instruct.
We collected multiple completions per prompt for prompts from Wikipedia (Cohere, ), HumanEval (Chen et al., ), and UltraChat (Ding et al., ).
In total, our dataset contains 440 prompts:
80 from Wikipedia in each of English, Spanish, French, German, and Russian,
as well as 20 from HumanEval and 20 from UltraChat.
We repeated the collection process for each model at each precision (fp32, fp16, fp16, int8, and nf4) and when watermarked using the method in Kirchenbauer et al. (), as well as from each API audited in the paper.
In total, this dataset size is 1.1M completions.

For the goodness-of-fit experiments in Appendix C.3,
we collected completions for a disjoint set of 20 additional prompts from each of language of Wikipedia for Mistral 7B and Llama-3 8B.
This set also includes the log probabilities of each sample under the full precision model.
All other experiments outside of this goodness-of-fit experiment were conducted on the previous, larger dataset.

#### Details about local sampling (fp32, fp16, int8, nf4 4, and watermark).

We collected $15\,000$ completions per prompt for the fp32 precision,
and $5000$ completions per prompt for the other precisions and watermarked alternatives.
Local sampling was performed on a mix of RTX 6000, RTX 3090, Quadro RTX 8000, and A100 GPUs using a mixture of the Transformers (Wolf et al., ) and VLLM (Kwon et al., ) libraries.
The watermarking, nf4, and int8 implementations are from the Transformers library.
We use the default watermarking parameters of 2.5 bias and context width 1.
All samples were collected with vanilla decoding parameters of temperature 1 without top-k or top-p sampling, and parameter max_new_tokens set to $L=50$ for Wikipedia and $L=250$ for HumanEval and UltraChat.

We used the default chat templates from the Transformers library for models.
It was important for us to match the chat templates during local sampling to those we believe the APIs to use, as the chat templates can affect the completions generated.
We confirmed that the number of tokens in our local rendering of the prompts matched the returned number of prompt tokens from API calls.
The exception was for the Llama-3.1 models, where the default Transformers chat template includes the current date.
We found that this template did not match the number of prompt tokens returned by APIs;
however, the Llama-3 chat template did.(^8^88In October 2024, this behavior has changed for Together AI, which now uses the Llama-3.1 chat template that includes the current date. At the time we collected samples with the Llama-3 template, this was not the case.)
As a result, we used the Llama-3 chat template for Llama-3.1 models in our local sampling.

#### Details about API sampling.

API samples are collected by repeatedly querying endpoints for one completion at a time.
We aimed to collect 250 completions per prompt for each API within a 24 hour window, but due to rate limits and request failures, some prompts had samples collected over multiple days.
The dates we query each endpoint are listed in Tables 2 – 5.
We query serverless endpoints offered by providers and use the same decoding parameters as for local sampling.
When available, we called the providers using their Python packages; otherwise, we made raw HTTP requests.

Below are provider-specific details:

- •
Anyscale. Because Anyscale deprecated its endpoints during our data collection in August 2024, we were only able to collect samples from their Llama-3 8B endpoint for Wikipedia.
- •
Together. We collected Llama-3 8B and 70B samples from Together.ai before they introduced separate reference, turbo, and lite endpoints.
We collected Llama-3.1 8B, 70B, and 405B from the turbo endpoints, which was the only option available at the time of collection.

**Table 2: Dates we queried Llama-3 8B inference endpoints.**
| Model | Dataset | Provider | Dates queried |
| --- | --- | --- | --- |
| 3 8B | Wikipedia | Anyscale | 7/4 |
| Amazon | 7/8, 8/1 |  |  |
| Azure | 8/19-20, 8/24 |  |  |
| Deepinfra | 7/4, 8/1 |  |  |
| Fireworks | 7/4, 7/19 |  |  |
| Groq | 7/4, 8/1-4, 8/8 |  |  |
| Perplexity | 7/4 |  |  |
| Replicate | 7/4, 7/19 |  |  |
| Together | 7/4 |  |  |
|  | HumanEval | Amazon | 7/29 |
|  | Azure | 8/24 |  |
|  | Deepinfra | 8/1 |  |
|  | Fireworks | 8/1, 8/12 |  |
|  | Groq | 8/4, 8/24 |  |
|  | Perplexity | 8/12 |  |
|  | Replicate | 8/1 |  |
|  | Together | 8/1, 8/12 |  |
|  | UltraChat | Amazon | 7/29 |
|  | Azure | 8/24 |  |
|  | Deepinfra | 8/1 |  |
|  | Fireworks | 8/1, 8/12 |  |
|  | Groq | 8/4, 8/24 |  |
|  | Perplexity | 8/12 |  |
|  | Replicate | 8/1-2 |  |
|  | Together | 8/1, 8/12 |  |

**Table 3: Dates we queried Llama-3.1 8B inference endpoints.**
| Model | Dataset | Provider | Dates queried |
| --- | --- | --- | --- |
| 3.1 8B | Wikipedia | Amazon | 8/1 |
| Azure | 8/19-21, 8/23-24 |  |  |
| Deepinfra | 8/1-2 |  |  |
| Fireworks | 7/26-27, 8/6 |  |  |
| Groq | 8/1-4, 8/8-11, 8/24-26 |  |  |
| Perplexity | 8/1-2 |  |  |
| Together | 7/26-27, 8/6-8 |  |  |
|  | HumanEval | Amazon | 8/1 |
|  | Azure | 8/24 |  |
|  | Deepinfra | 7/31 |  |
|  | Fireworks | 7/30 |  |
|  | Groq | 7/31, 8/24, 8/27 |  |
|  | Perplexity | 7/30 |  |
|  | Together | 7/30 |  |
|  | UltraChat | Amazon | 7/26-27, 8/1, 8/6 |
|  | Azure | 8/24 |  |
|  | Deepinfra | 7/31 |  |
|  | Fireworks | 7/26, 7/30, 8/6 |  |
|  | Groq | 7/31-8/1, 8/27 |  |
|  | Perplexity | 7/30 |  |
|  | Together | 7/26-27, 7/30, 8/6 |  |

**Table 4: Dates we queried Llama-3 70B inference endpoints.**
| Model | Dataset | Provider | Dates queried |
| --- | --- | --- | --- |
| 3 70B | Wikipedia | Amazon | 7/8, 8/1 |
| Azure | 8/19-21, 8/24 |  |  |
| Deepinfra | 7/4, 8/5-6 |  |  |
| Fireworks | 7/4, 7/31-8/1 |  |  |
| Groq | 7/4, 7/31, 8/2-13 |  |  |
| Perplexity | 7/4, 7/8, 8/5-6 |  |  |
| Replicate | 7/4, 7/19, 7/31-8/1 |  |  |
| Together | 7/4, 7/31-8/1 |  |  |
|  | HumanEval | Amazon | 7/29 |
|  | Azure | 8/24 |  |
|  | Deepinfra | 8/1 |  |
|  | Fireworks | 8/6 |  |
|  | Groq | 8/1 |  |
|  | Perplexity | 8/6 |  |
|  | Replicate | 8/6 |  |
|  | Together | 8/6, 8/24 |  |
|  | UltraChat | Amazon | 8/24 |
|  | Azure | 8/25-26 |  |
|  | Deepinfra | 8/1-2 |  |
|  | Fireworks | 8/6-7, 8/24 |  |
|  | Groq | 8/1-2, 8/4, 8/24 |  |
|  | Perplexity | 8/6, 8/24 |  |
|  | Replicate | 8/6 |  |
|  | Together | 8/6, 8/24 |  |

**Table 5: Dates we queried Llama-3.1 70B inference endpoints.**
| Model | Dataset | Provider | Dates queried |
| --- | --- | --- | --- |
| 3.1 70B | Wikipedia | Amazon | 8/2 |
| Azure | 8/24 |  |  |
| Deepinfra | 8/2 |  |  |
| Fireworks | 7/27-28, 8/2, 8/5-6 |  |  |
| Groq | 8/2-5, 8/8-11, 8/21-24 |  |  |
| Perplexity | 8/2, 8/6 |  |  |
| Together | 7/27-28, 8/6 |  |  |
|  | HumanEval | Amazon | 8/24 |
|  | Azure | 8/24, 8/26 |  |
|  | Deepinfra | 7/31 |  |
|  | Fireworks | 7/30 |  |
|  | Groq | 7/31, 8/1-2, 8/24 |  |
|  | Perplexity | 7/30-31 |  |
|  | Together | 7/30, 8/6 |  |
|  | UltraChat | Amazon | 8/24-25 |
|  | Azure | 8/25-26 |  |
|  | Deepinfra | 7/31 |  |
|  | Fireworks | 7/30 |  |
|  | Groq | 7/31-8/1, 8/24 |  |
|  | Perplexity | 7/30-31 |  |
|  | Together | 7/30, 8/6 |  |

**Table 6: Dates we queried Llama-3.1 405B inference endpoints.**
| Model | Dataset | Provider | Dates queried |
| --- | --- | --- | --- |
| 3.1 405B | Wikipedia | Amazon | 8/16-17, 8/23-24 |
| Deepinfra | 8/16, 8/23-24 |  |  |
| Fireworks | 8/16, 8/23-24 |  |  |
| Together | 8/16, 8/20, 8/23-24 |  |  |
| HumanEval | Amazon | 8/24-25 |  |
| Deepinfra | 8/24 |  |  |
| Fireworks | 8/24 |  |  |
|  | Together | 8/24 |  |
|  | UltraChat | Amazon | 8/24-25 |
|  | Deepinfra | 8/24-25 |  |
|  | Fireworks | 8/24 |  |
|  | Together | 8/24 |  |

**Table 7: Dates we queried OpenAI endpoints (Wikipedia task).**
| Model | Dates queried |
| --- | --- |
| gpt-4o-mini | 8/21, 8/23-24 |
| gpt-4o | 8/29 |
| gpt-3.5-turbo-0125 | 8/29 |
| gpt-3.5-turbo-1106 | 8/29 |
| gpt-4-0125-preview | 8/29 |

### B.2 Monte carlo simulations

To construct the ten Wikipedia prompt distributions in §4, we randomly sampled 25 prompts per distribution from the Wikipedia prompts in our dataset.
The HumanEval and UltraChat prompt distributions were constructed by using all available prompts from those sources.

In most experiments, we estimated power as the average rejection rate over 100 simulations,
where we sample a fresh $\mathcal{D}_{P}$ and $\mathcal{D}_{Q}$ each time.
We simulated p-values by sampling $1000$ datasets $\mathcal{D}_{P}$ and $\mathcal{D}_{Q}$ from $P$ and computing the test statistic on each pair,
and then we reused this empirical distribution when testing against all alternatives for the same $\pi P$.
The exception is for the MMD all-substrings test statistic:
because this was exceptionally slow to compute, we simulated p-values using 100 samples instead of 1000, and we estimated power from 20 simulations instead of 100.

## Appendix C Additional results

### C.1 Sample-complexity and length

This appendix includes additional results from §4.1.
Figures 5, 6, and 7 stratify the sample complexity and completion length results by the alternative distribution $Q$ and model.

Table 8 shows the power of the Hamming MMD test to distinguish between pairs of models (and other alternatives) in $L=$1000$$ character space. In general, moving to this higher-dimensional space decreases the power of the test.

Figure: Figure 5: Sample complexities for different two-sample tests. Each subplot represents power for an alternative distribution $Q$ averaged over five language models and ten prompt distributions $\pi$. The Hamming MMD test is significantly more sample-efficient for all alternatives except fp16 quantization: for this alternative, while other two-sample tests can attain perfect power with enough samples, the kernel test increases power slowly. This suggests that fp16 and fp32 differ in ways that are difficult to capture with the Hamming kernel.
Refer to caption: x6.png

Figure: Figure 6: Sample complexities for different two-sample tests. Each subplot represents power for a particular alternative distribution $Q$ and model, but averaged over ten prompt distributions $\pi$. Some model and alternative combinations are more difficult to detect than others.
Refer to caption: x7.png

Figure: Figure 7: Simulated powers for different completion lengths $L$, stratified by the alternative distribution $Q$. Across all alternatives, kernel tests suffer less drop in power as the length of completions increases. Traditional two-sample tests observe a slight power increase from $L=1$ tokens to $L=5$, tokens, but power dramatically degrades afterwards.
Refer to caption: x8.png

**Table 8: Power of the Hamming MMD test to distinguish between pairs of models (and other alternatives) in $L=$1000$$ character space. The column represents the null distribution $P$. Italicized entries are FPRs. Note that unlike §4.1, the test is now run in higher-dimensional character space, which generally reduces power.**
|  | Mistral 7B | 3 8B | 3.1 8B | 3 70B | 3.1 70B |
| --- | --- | --- | --- | --- | --- |
| Mistral 7B | 0.06 | 1.00 | 1.00 | 1.00 | 1.00 |
| Llama-3 8B | 1.00 | 0.05 | 0.76 | 0.98 | 0.95 |
| Llama-3.1 8B | 1.00 | 0.83 | 0.07 | 1.00 | 0.53 |
| Llama-3 70B | 1.00 | 0.99 | 1.00 | 0.07 | 0.89 |
| Llama-3.1 70B | 1.00 | 0.98 | 0.75 | 0.99 | 0.06 |
| Phi-3 mini | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| Gemma-2 9B | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| OLMo 7B | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| GPT-4o mini | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| gpt-4o | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| gpt-3.5-turbo-0125 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| gpt-3.5-turbo-1106 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| gpt-4-0125-preview | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| watermark | 0.23 | 0.62 | 0.32 | 0.57 | 0.26 |
| int8 | 0.15 | 0.30 | 0.07 | 1.00 | 0.99 |
| nf4 | 0.44 | 0.38 | 1.00 | 1.00 | 1.00 |

### C.2 Effect of the prompt distribution

In §4.1, we evaluate tests on a Wikipedia prompt distribution with support over $m=25$ prompts. Here, we evaluate the effect of $m$ on the MMD Hamming test’s power, with $N=10m$ fixed to an average of 10 samples per prompt, and $L=50$ tokens (Figure 8).
Larger $m$ values increase the power of the test, suggesting that users benefit from testing many prompts together, so long as the sample size is increased proportionally.

Figure: Figure 8: Power of the Hamming MMD test for different numbers of prompts $m$ in the prompt distribution. Results are averaged over ten random samples of $m$ prompts (for the Wikipedia task). The test is run with $N=10m$ and $L=50$ tokens, with $m$ varying from 1 to 100. Power increases with $m$, suggesting that users benefit from testing many prompts together.
Refer to caption: x9.png

### C.3 Extension: comparing two-sample and approximate goodness-of-fit tests

In §2, we assumed only sample access to both the reference distribution $P$ and API $Q$.
An alternative problem setup might give the auditor privileged access to $P$ to evaluate probabilities $P(y\mid x)$ for arbitrary (prompt, completion) pairs.
Here, we compare the performance of two-sample tests to goodness-of-fit tests that leverage this privileged setting.

In an extreme case, evaluating probabilities is free – the auditor can fully describe $P(y\mid x)$ for all completions $y\in\mathcal{V}^{\leq L}$.
Then goodness-of-fit tests, like the one-sample $L_{1}$ statistic, can be used to compare observed counts in $\mathcal{D}_{Q}$ to expected counts under $P$:

$$ $\text{L1}_{\text{gof}}(P,\mathcal{D}_{Q})=\sum_{z\in\mathcal{X}\times\mathcal{ V}^{\leq L}}\left|\mathbf{n}_{z}\{\mathcal{D}_{Q}\}-N\cdot P(y|x)\pi(x)\right|.$ (12) $$

However, in practice, evaluating $P(y\mid x)$ for all $y\in\mathcal{V}^{\leq L}$ and all $x$ is intractable:
as a concrete example, for our language modeling task on Llama-3, $m\cdot|\mathcal{V}^{\leq L}|=25\cdot$128\,000$^{50}\approx 5\times 10^{256}$.
A more realistic scenario is that the auditor can only evaluate $P(y\mid x)$ for the observed $(x,y)$ in $\mathcal{D}_{Q}$.
This leads to an approximation of the goodness-of-fit test statistic:

$$ $\widehat{\text{L1}_{\text{gof}}}(P,\mathcal{D}_{Q})=\sum_{z\in\mathcal{D}_{Q}} \left|\mathbf{n}_{z}\{\mathcal{D}_{Q}\}-N\cdot P(y|x)\pi(x)\right|.$ (13) $$

We take a similar strategy for the one-sample $L_{2}$ test

$$ $\displaystyle\text{L2}_{\text{gof}}(P,\mathcal{D}_{Q})=\sum_{z\in\mathcal{X} \times\mathcal{V}^{\leq L}}\left(\mathbf{n}_{z}\{\mathcal{D}_{Q}\}-N\cdot P(y| x)\pi(x)\right)^{2}$ (14) $\displaystyle\widehat{\text{L2}_{\text{gof}}}(P,\mathcal{D}_{Q})=\sum_{z\in \mathcal{D}_{Q}}\left(\mathbf{n}_{z}\{\mathcal{D}_{Q}\}-N\cdot P(y|x)\pi(x) \right)^{2},$ $$

the Pearson $\chi^{2}$ test

$$ $\displaystyle\chi^{2}_{\text{pearson}}(P,\mathcal{D}_{Q})=\sum_{z\in\mathcal{X }\times\mathcal{V}^{\leq L}}\frac{\left(\mathbf{n}_{z}\{\mathcal{D}_{Q}\}-N \cdot P(y|x)\pi(x)\right)^{2}}{N\cdot P(y|x)\pi(x)}$ (15) $\displaystyle\widehat{\chi^{2}_{\text{pearson}}}(P,\mathcal{D}_{Q})=\sum_{z\in \mathcal{D}_{Q}}\frac{\left(\mathbf{n}_{z}\{\mathcal{D}_{Q}\}-N\cdot P(y|x)\pi (x)\right)^{2}}{N\cdot P(y|x)\pi(x)},$ $$

and the truncated $\chi^{2}$ test (Balakrishnan & Wasserman, )

$$ $\displaystyle\chi^{2}_{\text{truncated}}(P,\mathcal{D}_{Q})=\sum_{z\in\mathcal {X}\times\mathcal{V}^{\leq L}}\frac{\left(\mathbf{n}_{z}\{\mathcal{D}_{Q}\}-N \cdot P(y|x)\pi(x)\right)^{2}-\mathbf{n}_{z}\{\mathcal{D}_{Q}\}}{\max\left(P(y |x)\pi(x),\frac{1}{|\mathcal{V}^{\leq L}|}\right)}$ (16) $\displaystyle\widehat{\chi^{2}_{\text{truncated}}}(P,\mathcal{D}_{Q})=\sum_{z \in\mathcal{D}_{Q}}\frac{\left(\mathbf{n}_{z}\{\mathcal{D}_{Q}\}-N\cdot P(y|x) \pi(x)\right)^{2}-\mathbf{n}_{z}\{\mathcal{D}_{Q}\}}{\max\left(P(y|x)\pi(x), \frac{1}{|\mathcal{V}^{\leq L}|}\right)}.$ $$

The only goodness-of-fit test that we consider which is unaffected by the approximation is the likelihood ratio test:

$$ $\text{LRT}(P,\mathcal{D}_{Q})=-2\sum_{z\in\mathcal{D}_{Q}}\mathbf{n}_{z}\{ \mathcal{D}_{Q}\}\log\left(\frac{\mathbf{n}_{z}\{\mathcal{D}_{Q}\}}{N\cdot P(y |x)\pi(x)}\right).$ (17) $$

Figure 9 plots sample complexities of these (approximate) goodness-of-fit tests alongside the two-sample tests evaluated in the main text.
The best goodness-of-fit tests outperform their two-sample counterparts in the extremely low-sample regime ($N<$1000$$), but this trend reverses as $N$ increases.
This is surprising — in theory, we would expect probability access to only increase power.
These results suggest that the approximations compensating for limited evaluation budget can introduce bias, reducing the power of goodness-of-fit tests.
We leave to future work ideas for the correction of this bias.

Figure: Figure 9: Power of two-sample MMD tests and goodness-of-fit tests. Each subplot represents a particular language model and alternative distribution $Q$. The goodness-of-fit tests are approximated by evaluating $P(y\mid x)$ only for observed $(x,y)$ in $\mathcal{D}_{Q}$. The truncated chi-squared and one-sample $L_{2}$ tests perform best out of the goodness-of-fit tests, while the likelihood ratio test and one-sample $L_{1}$ tests perform worst. Note that experiments were run on a different set of 10 prompt distributions than the main text; these prompts specifically had model probabilities saved (see Appendix B.1).
Refer to caption: x10.png

### C.4 Extension: asymmetric sampling costs

In some cases, it may be significantly less expensive to sample from one distribution than the other.
For example, the auditor may have unlimited compute to sample from the null distribution $P$, but limited monetary budget to sample from the API $Q$.
In these cases, we show that it is possible to achieve slight power gains by increasing the sample size of the cheaper distribution, even while keeping the sample size of the expensive distribution fixed.
Figure 10 fixes $|\mathcal{D}_{Q}|=10m$ and varies $|\mathcal{D}_{P}|$ between $10m$ and $400m$.
All test statistics see some increases in power, with the $L_{1}$ test seeing surprisingly large gains.

Figure: Figure 10: Power when the sample size from $P$ increases, when the sample size from $Q$ is fixed to 250. There are slight power gains for all test statistics, with the $L_{1}$ test seeing the largest gains.
Refer to caption: x11.png

### C.5 Permutation procedure

Results in the main text simulate the test statistic’s empirical distribution under the null by sampling datasets $\mathcal{D}_{Q}$ and $\mathcal{D}_{P}$ both from $P$.
Here we validate these trends by conducting the same tests using a permutation procedure to estimate p-values (see Appendix A.2).
Figures 11, 12, and 13 repeat the sample complexity, length, and asymmetric sampling cost experiments, but use the permutation procedure to estimate p-values.
Because of the computational complexity of this step, we use $100$ permutations, estimate power using $20$ simulations,
and only test Mistral 7B and Llama-3 8B.
We observe that the permutation procedure maintains similar power levels to the bootstrap method,
and trends from the previous figures are replicated.

Figure: Figure 11: Sample complexities for different two-sample tests, stratified by the alternative distribution $Q$, but averaged over five language models and ten prompt distributions $\pi$. This figure parallels Figure 5 but uses the permutation procedure, rather than repeated sampling from $P$, to estimate p-values.
Refer to caption: x12.png

Figure: Figure 12: Simulated powers for different completion lengths $L$, stratified by the alternative distribution $Q$. This figure parallels Figure 7 but uses the permutation procedure, rather than repeated sampling from $P$, to estimate p-values.
Refer to caption: x13.png

Figure: Figure 13: Simulated power when the sample size from $P$ increases, when the sample size from $Q$ is fixed to 250. This figure parallels Figure 10 but uses the permutation procedure, rather than repeated sampling from $P$, to estimate p-values.
Refer to caption: x14.png

**Table 9: Power against local alternatives for the composite null setting (§5) in character space ($L=1000$), stratified by model and prompt distribution.**
|  | Wikipedia | HumanEval | UltraChat |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | 3 8B | 3.1 8B | 3 70B | 3.1 70B | 3 8B | 3.1 8B | 3 8B | 3.1 8B |
| watermark | 0.30 | 0.00 | 0.07 | 0.03 | 0.40 | 0.85 | 0.06 | 0.38 |
| int8 | 0.07 | 0.00 | 1.00 | 0.96 | 0.01 | 0.45 | 0.01 | 0.03 |
| nf4 | 0.09 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| fp16 | 0.02 | 0.01 | 0.00 | 0.01 | 0.00 | 0.02 | 0.00 | 0.04 |
| fp32 | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.10 | 0.01 | 0.04 |

### C.6 Composite null

In §5, we use a composite null hypothesis that combines the fp32 and fp16 distributions.
Table 9 shows the power of the Hamming MMD test in this composite null setting, stratified by model and prompt distribution.
In general, power is reduced using the composite null.
Power is generally highest on HumanEval, which collects longer completions than Wikipedia (250 vs. 50 tokens).

### C.7 Additional audit results

**Table 10: Power against alternatives for the full audit. Table 1 in the main text thresholds power at 0.5 for the APIs. The FPRs against the full- and half-precision nulls are italicized.**
|  | Wikipedia | HumanEval | UltraChat |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | 3 8B | 3.1 8B | 3 70B | 3.1 70B | 3 8B | 3.1 8B | 3 8B | 3.1 8B |
| Amazon | 0.07 | 1.00 | 0.32 | 1.00 | 0.48 | 1.00 | 1.00 | 1.00 |
| Anyscale | 0.02 | — | — | — | — | — | — | — |
| Azure | 0.01 | 0.00 | 0.01 | 0.01 | 0.01 | 0.29 | 0.00 | 0.13 |
| Deepinfra | 0.04 | 0.00 | 0.04 | 0.00 | 0.08 | 0.19 | 0.04 | 0.09 |
| Fireworks | 0.04 | 0.04 | 0.01 | 0.01 | 0.01 | 1.00 | 0.03 | 0.90 |
| Groq | 0.03 | 0.07 | 0.02 | 0.59 | 0.01 | 0.98 | 0.05 | 0.35 |
| Perplexity | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| Replicate | 0.12 | — | 0.33 | — | 0.07 | — | 0.06 | — |
| Together | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.27 | 0.01 | 0.07 |
| watermark | 0.30 | 0.00 | 0.07 | 0.03 | 0.40 | 0.85 | 0.06 | 0.38 |
| int8 | 0.07 | 0.00 | 1.00 | 0.96 | 0.01 | 0.45 | 0.01 | 0.03 |
| nf4 | 0.09 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| fp16 | 0.02 | 0.01 | 0.00 | 0.01 | 0.00 | 0.02 | 0.00 | 0.04 |
| fp32 | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.10 | 0.01 | 0.04 |

**Table 11: Overall audit results, as also copied in Figure 1. The 8B models are tested on three prompt distributions (Wikipedia, HumanEval, UltraChat), while the 70B models are tested on Wikipedia. Tests from different prompt distributions are combined using a Bonferroni correction.**
|  | 3 8B | 3.1 8B | 3 70B | 3.1 70B |
| --- | --- | --- | --- | --- |
| Amazon | ✗ | ✗ | ✗ | ✗ |
| Anyscale | ✓ | — | — | — |
| Azure | ✓ | ✓ | ✓ | ✓ |
| Deepinfra | ✓ | ✓ | ✓ | ✓ |
| Fireworks | ✓ | ✗ | ✓ | ✓ |
| Groq | ✓ | ✗ | ✓ | ✗ |
| Perplexity | ✗ | ✗ | ✗ | ✗ |
| Replicate | ✓ | — | ✓ | — |
| Together | ✓ | ✓ | ✓ | ✓ |

**Table 12: Power against alternatives for the full audit with the Bonferroni correction. The FPRs against the full- and half-precision nulls are italicized.**
|  | 3 8B | 3.1 8B | 3 70B | 3.1 70B |
| --- | --- | --- | --- | --- |
| Amazon | 1.00 | 1.00 | 0.58 | 1.00 |
| Anyscale | 0.02 | — | — | — |
| Azure | 0.02 | 0.10 | 0.01 | 0.01 |
| Deepinfra | 0.01 | 0.04 | 0.04 | 0.01 |
| Fireworks | 0.00 | 1.00 | 0.03 | 0.00 |
| Groq | 0.01 | 0.75 | 0.00 | 0.59 |
| Perplexity | 1.00 | 1.00 | 1.00 | 1.00 |
| Replicate | 0.02 | — | 0.48 | — |
| Together | 0.01 | 0.09 | 0.00 | 0.00 |
| nf4 | 1.00 | 1.00 | 1.00 | 1.00 |
| int8 | 0.00 | 0.12 | 1.00 | 0.92 |
| watermark | 0.12 | 0.45 | 0.50 | 0.06 |
| fp16 | 0.00 | 0.00 | 0.00 | 0.00 |
| fp32 | 0.00 | 0.06 | 0.00 | 0.00 |

Table 10 shows the overall audit results.
The 8B models are tested on three prompt distributions (Wikipedia, HumanEval, UltraChat), while the 70B models are tested on Wikipedia.
Tests from different prompt distributions are combined using a Bonferroni correction in Table 12.

### C.8 Comparing APIs to each other

In Figures 14 – 25,
we compute the pairwise MMDs between APIs and quantized model weights for all prompt distributions (Wikipedia, HumanEval, UltraChat) and available models (Llama-3 8B, Llama-3.1 8B, Llama-3 70B, Llama-3.1 70B).
We use spectral clustering with two components to discover groups of implementations.
Providers that pass the audit in Table 1 are typically clustered with the null distributions ${fp16}$ and ${fp32}$, reflecting that they are distributionally close to these nulls.

Additionally, Figure 26 shows the estimated MMDs between APIs for each of the three prompt distributions for the Llama-3.1 405B model.
Due to their size, we could not sample from the released weights for this large model directly, but we can still estimate the distances between APIs for this model.

Figure: Figure 14: (Left) Hamming MMDs between APIs for the Llama-3 8B model on the Wikipedia prompt distribution. (Right) Visualization of the 2D spectral clustering components. Clusters are colored together.
Refer to caption: x15.png

Figure: Figure 15: (Left) Hamming MMDs between APIs for the Llama-3.1 8B model on the Wikipedia prompt distribution. (Right) Visualization of the 2D spectral clustering components. Clusters are colored together.
Refer to caption: x16.png

Figure: Figure 16: (Left) Hamming MMDs between APIs for the Llama-3 70B model on the Wikipedia prompt distribution. (Right) Visualization of the 2D spectral clustering components. Clusters are colored together.
Refer to caption: x17.png

Figure: Figure 17: (Left) Hamming MMDs between APIs for the Llama-3.1 70B model on the Wikipedia prompt distribution. (Right) Visualization of the 2D spectral clustering components. Clusters are colored together.
Refer to caption: x18.png

Figure: Figure 18: (Left) Hamming MMDs between APIs for the Llama-3 8B model on the HumanEval prompt distribution. (Right) Visualization of the 2D spectral clustering components. Clusters are colored together.
Refer to caption: x19.png

Figure: Figure 19: (Left) Hamming MMDs between APIs for the Llama-3.1 8B model on the HumanEval prompt distribution. (Right) Visualization of the 2D spectral clustering components. Clusters are colored together.
Refer to caption: x20.png

Figure: Figure 20: (Left) Hamming MMDs between APIs for the Llama-3 70B model on the HumanEval prompt distribution. (Right) Visualization of the 2D spectral clustering components. Clusters are colored together.
Refer to caption: x21.png

Figure: Figure 21: (Left) Hamming MMDs between APIs for the Llama-3.1 70B model on the HumanEval prompt distribution. (Right) Visualization of the 2D spectral clustering components. Clusters are colored together.
Refer to caption: x22.png

Figure: Figure 22: (Left) Hamming MMDs between APIs for the Llama-3 8B model on the UltraChat prompt distribution. (Right) Visualization of the 2D spectral clustering components. Clusters are colored together.
Refer to caption: x23.png

Figure: Figure 23: (Left) Hamming MMDs between APIs for the Llama-3.1 8B model on the UltraChat prompt distribution. (Right) Visualization of the 2D spectral clustering components. Clusters are colored together.
Refer to caption: x24.png

Figure: Figure 24: (Left) Hamming MMDs between APIs for the Llama-3 70B model on the UltraChat prompt distribution. (Right) Visualization of the 2D spectral clustering components. Clusters are colored together.
Refer to caption: x25.png

Figure: Figure 25: (Left) Hamming MMDs between APIs for the Llama-3.1 70B model on the UltraChat prompt distribution. (Right) Visualization of the 2D spectral clustering components. Clusters are colored together.
Refer to caption: x26.png

Figure: Figure 26: Estimated MMDs between Llama-3.1 405B APIs on three prompt distributions. DeepInfra publishes that they serve Llama-3.1 405B at fp8 precision. (Source: [DeepInfra website](https://deepinfra.com/meta-llama/Meta-Llama-3.1-405B-Instruct))
Refer to caption: x27.png

## Appendix D Data samples

In this appendix, we randomly sample completions from pairs of distributions to illustrate the differences between them.
In general, we observe that it is challenging to visually distinguish between samples from different distributions, especially for longform tasks.
This is because each distribution produces diverse outputs.
Formal statistical tests that we describe in the main text are necessary to detect these differences.

In a few cases, detected differences between distributions are also visually obvious.
For example, we observe that the Llama 70B-scale models quantize poorly, and their nf4 completions are degenerate.
Different language models also often differ in how they begin completions.
We also observe that some APIs (in particular, Perplexity) seem to be producing lower-entropy completions than the reference distribution, suggesting
some form of caching or incorrect implementation of the temperature parameter.

### D.1 Comparing models to quantized or watermarked versions

### D.2 Comparing models to other models

### D.3 Comparing APIs

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 3
presentation: 3
questions: - Regarding the kernel:
    - For the choice of the kernel, you are basically trading off bias and variance. I think this might be good to call out in the manuscript. My understanding is that as L goes to infinity, the hamming kernel approaches the universal one-hot kernel. Similarly as L goes to 0, one biases towards the first tokens. I’d love to understand the implications of this bias-variance tradeoff on tasks that require more creativity like writing.
    - Also L is set to 50 and the sensitivity to L is studied until L=50. I think 50 tokens for generations is typically on the lower end when it comes to LLM-based applications. I’d recommend studying it at least until 10 folds.
    - I’d think L is affected by the vocabulary size of the tokenizer. Did the authors study the effect of L under different tokenizers? 
    - The kernel checks for token equivalence but an important piece to capture might be around semantic equivalence. Have the authors considered other kernels that can capture semantic equivalence?
- Evaluations:
    - Figure 3 lower left. The power of detecting Llama3.1 8B from Llama3.1 8B (int8) and (watermark) are 0.07 and 0.32, respectively, which are very low. I’d love to hear the author's thoughts on this as one of the premises of the method is to be able to detect such changes in the model. 
    - Consider adding an analysis that shows that the method does not detect any changes if the two samples come from the same model. It would be great to show the detection power using samples obtained at different temperatures.
    - Can you extend the analysis in Figure 3 lower left to other models? One thing I’m interested to understand is that the method is able to detect different model sizes and looking at Figure 3 on the right hand side, the models from the same family look very close in terms of distance.
- Nit: Mention that L is the number of tokens in the manuscript.
soundness: 2
strengths: The paper is overall well written, organized and easy to follow. The application of two sample testing using the Maximum Mean Discrepancy kernel is novel for studying distributions coming from different models.
summary: The paper proposes Model Equality Testing to understand the differences between models served by APIs. The proposed method is a two-sample test that uses various prompts along with multiple generations of a given prompt and tries to infer if two instances of generations come from the same distributions using the Maximum Mean Discrepancy kernel. The proposed method is studied empirically to understand its power in identifying the differences for models served by many APIs.
weaknesses: - I’m not convinced of the significance of this problem. Can this problem be solved through policy if the users ask from providers to disclose such changes? The API provides can be even motivated to charge users differently based on the optimizations done to models. 
- Evaluations can be stronger:
    - The authors claim that the proposed method works using an average of 10 samples per prompt across 20-25 prompts. Since the paper relies on empirical analysis, I would love to see more analysis backing this claim/more guidelines around these requirements. For instance, do I still need to provide 10 samples per prompt if I generate the samples using 0 temperature? What about if my prompts are all from a niche topic versus if I’m testing across various different unrelated topics? Does one still obtain high power using 20-25 prompts?
    - The evaluations are done using samples from English, German, Spanish, French, and Russian Wikipedia. I’d love to see more diversity in the evaluation tasks. For instance, consider showing the results for coding tasks, which can help with the broader applicability of the proposed method.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. MMD performance declines with small sample sizes, which is likely the case in real-world scenarios. Although the paper discusses sample efficiency, additional experiments with smaller sample sizes could provide further insights into this limitation.

2. In practice, API services may not use the same model but rather similar models(like MOE). I suggest the authors provide some discussion on this point.

3. How does MMD perform in terms of efficiency in large-scale testing scenarios?
soundness: 3
strengths: 1. This paper is well-organized and written, making it easy to understand.
  
2. The problem addressed is valuable, as APIs have indeed become one of the mainstream forms of LLM applications.

3. The method design is reasonable and concise, and the experiments appear to be effective.
summary: In this paper, the authors study the equivalence problem of LLMs in an API service environment. They formulate it as a two-sample test to assess the consistency of output distributions from different APIs. Ultimately, they use Maximum Mean Discrepancy (MMD) and achieve effective differentiation.
weaknesses: 1. The study primarily focuses on the LLaMA series models. Although this aligns with the paper’s emphasis, I recommend that the authors verify the generalizability of MMD across more models.

2. The computation cost of MMD appears to be somewhat high.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: - In Figure 4, you compare absolute task accuracy difference with MMD, but this is slightly less informative than showing non-absolute task difference. You imply on lines 415-418 that task accuracy difference are usually lower for the non-reference models, but this isn't necessarily obvious that a provider's interventions are always harmful to accuracy / quality. Could you provide some information about MMD and absolute accuracy?
soundness: 2
strengths: - The overall idea of providing more analytic tools for black-box models / APIs seems important and interesting. I think many people reliant on these black-box APIs are left at their mercy and have little means of understanding how different service providers could affect their system performance.
- The public release of generated samples offers valuable data for community use and future API change tracking.
- Providing a framework for analyzing APIs on non-classification based (i.e. difficult to evaluate) tasks seems fairly novel and is a good contribution.
summary: The authors propose a method for determining whether a model black-box (i.e. an LM served through an API) differs from a reference model; usually a canonical model version or simply an earlier version of the API.
The proposed method relies on sampling texts from different models / APIs or the same one at different times and estimating their relation against a distance metric under some kernel. The authors propose a Hamming distance kernel as an empirically reliable approach for this estimation. They provide analysis of different foundation models and APIs over time gleaning insight into how inference service providers may affect inference reliability.

### Typos
- Line 081: extra article 'a' inserted before 'an API'
weaknesses: - One immediate problem I see is that acquiring the reference distribution on a user-defined task requires setting up the reference LM anyways, which was something that is acknowledged to be inconvenient or infeasible in many cases. That would make this method impossible in certain scenarios.
- The hamming kernel's effectiveness across different tasks raises concerns, particularly for open-ended tasks like creative generation where diverse outputs may be desirable. The significance tests may struggle with high-variance outputs, potentially missing quality differences. Further analysis is needed on how task semantics and natural distribution affect the method's applicability. Additionally, using Wikipedia-based language modeling as the primary test task may underestimate output diversity due to the highly factual nature of the content and potential data contamination. This is partially addressed by the inclusion of HumanEval, but additional comparisons and/or analysis could strengthen the results.
- I didn't see any mention of two model customizations that I suspect may be popular: system prompt customization and generation-time safety interventions. It's not clear to me how the proposed method would behave under these customizations.

---

confidence: 4
contribution: 3
initial_rating: 8
presentation: 3
questions: Q1. My intuition is that Hamming distance would be too sensitive to positions of tokens in the sentence by its construction, e.g. they don't capture the similarity between two shifted chunks of text. I would have started with Levenshtein distance, have you maybe tried it in your experiments?

Q2. How do you compute the kernels if a language model generates a shorter sentence compared to the maximal length?

Other questions:
- eq.4, should there z and z' be different?
- 162: what does accurate completion means?
- 210: is the only difference in the weights of the models, can generation algorithms also be different?
- Figure 2: a not well-readable sentence: "Curves first median power ..."
- 312: you write 1.1M unicode characters, but as far as I can check in Unicode 16.0 there are 155,063 unique characters, how it was computed?
- 326: "significantly different training data": do you use word significantly in statistical meaning here?
- 337: semantic caching, quantization: is it possible to put citations here?
- Figure 4 (right): what are the colours here?
soundness: 4
strengths: This paper touches upon a very timely problem of verifying the integrity of the black-box APIs and model versioning in the cases when full detailed for a particular API are not open. Namely, the important question this paper formulates and proposes a test for is whether two different versions of the same model are the same. This is an important test given that sometimes models are changed, compressed for efficiency or safety reasons, and it is important for practitioners to understand what to expect from a given API.

Overall, the paper is easy to follow, and methodology is sound. Experimental setup is sound enough to support the claims. Proposed method is simple and intuitive and builds upon the older works on string kernel based tests. I expect this paper will have a high impact within the language modelling community.
summary: The authors formulate the problem of testing whether two versions of a language models (either fully accessible, or accessible via API) are the same given two sets of samples from one model and another. They propose a simple but effective statistical test to distinguish between two sets of samples based on the Maximum Mean Discrepancy (MMD) string kernel method with the Hamming distance. They compare the chosen Hamming distance to other simple string distances and find that Hamming distance is well-suitable kernel to compare the samples from prompted language models. To evaluate the proposed MMD test, they verify that it shows high power for distinguishing different version of the same model or different models with a reasonably small number of samples, and degrades less w.r.t. the sequence length. Finally, they use the test to audit the API endpoints and find that a large portion of LLaMa 3 API providers give access to a different version of the model.
weaknesses: There are no critical weaknesses, in my opinion, only minor ones.

The choice of baselines is limited to different choices of string kernels.
I was a bit surprised to see Hamming distance to perform the best, and in general felt that discussion and intuition why it works well is not present.

Meta-review: In this paper, the authors utilize Maximum Mean Discrepancy (MMD) to evaluate whether large language models (LLMs) offered by different cloud vendors are identical or have been modified in significant or subtle ways. From a technical perspective, the paper offers limited innovation. However, for practitioners deploying models across various cloud service providers, the findings are important. They highlight that the models can differ and may exhibit varying behaviors depending on the server or vendor, which is a crucial consideration for real-world applications. 

The reviewers have a positive opinion of this paper and believe it would be a valuable addition to the conference. However, the Area Chair has reservations about the paper's long-term relevance, as it primarily addresses a transitory problem that is unlikely to impact real-world applications in the future.

Decision: Accept (Poster)