ICML 2025

TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference
Jack Min Ong 1 Matthew Di Ferrante 1 Aaron Pazdera 1 Ryan Garner 1 Sami Jaghouar 1 Manveer Basra 1
Max Ryabinin 2 Johannes Hagemann 1
Abstract
Large language models (LLMs) have proven to
be very capable, but access to frontier models
currently relies on inference providers. This intro-
duces trust challenges: how can we be sure that
the provider is using the model configuration they
claim? We propose TOPLOC, a novel method for
verifiable inference that addresses this problem.
TOPLOC leverages a compact locality-sensitive
hashing mechanism for intermediate activations,
which can detect unauthorized modifications to
models, prompts, or precision with 100% accu-
racy, achieving no false positives or negatives in
our empirical evaluations. Our approach is ro-
bust across diverse hardware configurations, GPU
types, and algebraic reorderings, which allows
for validation speeds significantly faster than the
original inference. By introducing a polynomial
encoding scheme, TOPLOC minimizes the mem-
ory overhead of the generated proofs by 1000×,
requiring only 258 bytes of storage per 32 new
tokens, compared to the 262 KB requirement of
storing the token embeddings directly for Llama
3.1-8B-Instruct. Our method empowers users to
verify LLM inference computations efficiently,
fostering greater trust and transparency in open
ecosystems and laying a foundation for decentral-
ized, verifiable and trustless AI services.
1. Introduction
In recent years, large language models (LLMs) have trans-
formed natural language processing, enabling capabilities
such as high-quality text generation, advanced dialogue
systems, and improved reasoning (Grattafiori et al., 2024;
Gemma Team et al., 2024). Inference providers, entities that
run open-weights LLMs on their own hardware and expose
1Prime Intellect 2Together AI. Correspondence to: Johannes
Hagemann <johannes@primeintellect.ai>.
Proceedings of the 42 nd Intern...