Title: FLiPS: Few-Shot Fingerprinting of LLMs via Pseudorandom Sequences
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Model Fingerprinting, LLM, Random Generation

Abstract: Identifying online Large Language Models (LLMs) via black-box queries, or fingerprinting, is now an active research problem. The state-of-the-art schemes require substantial amounts of queries to a model for building their fingerprints, often implying having it at hand. This precludes a swift fingerprinting of new models or variants freshly deployed online.
In this paper, we propose FLiPS, a principled approach to LLM fingerprinting, which enables building a fingerprint using a trace number of queries (which we coin few-shot).
FLiPS exploits bias discrepancies in the generation of random binary sequences by LLMs for model identification. It employs the classical NIST cryptographic test suite to detect salient and interpretable differences in LLM outputs.
We demonstrate that FLiPS achieves nearly 99\% accuracy on a pool of 35 LLMs using as few as 40 queries to establish the fingerprint and 8 for its later identification. Furthermore, we propose an open-set environment where some models are unseen and must be labeled as such, and achieve 92.5\% accuracy (with 67.6\% on unseen models). This demonstrates that FLiPS achieves the novel task of the swift few-shot integration of new models in its operation.

TL;DR: FLiPS is a fingerprinting method of online LLMs, leveraging random generations of LLMs, reaching ~99% accuracy with as few as 40 queries to build the fingerprint (coin Few-shot) and 8 to read it.