cbx-secrets-detector-17M

Finds secrets and credentials (passwords, API keys, tokens, private keys, cloud and database credentials) in code, config files, logs, tickets and chat, and returns the exact position of each value with one of 12 labels. 17M parameters; about 6 ms per 1,000 tokens on a GPU and 37 ms on a CPU.

This is the fastest size, and the one to use on a CPU. Fine-tuned from jhu-clsp/ettin-encoder-17m. Version v0.1, the first release (use --revision v0.1 to pin it). Also available in 32M.

Unlike rule-based scanners, it reads the context: it finds passwords and keys that have no fixed prefix, and it leaves alone placeholders, variable references, hashes and obviously fake values.

Labels

Label	Covers
PASSWORD	Passwords and passphrases: logins, service accounts, keystores, key derivation
DATABASE_CREDENTIAL	A database connection URI that holds a password (the whole URI), or a database password
API_KEY	Keys for calling an API or service (sk_live_…, hex service keys)
ACCESS_TOKEN	Bearer and access tokens that are not JWTs: OAuth tokens, personal access tokens (ghp_…)
JWT_TOKEN	Well-formed JWTs (eyJ….….…)
SESSION_SECRET	Secrets that sign or encrypt sessions or cookies
WEBHOOK_SECRET	Secrets that sign or verify webhook deliveries (whsec_…)
CLOUD_CREDENTIAL	Cloud provider secrets: AWS secret access keys, cloud client secrets
PRIVATE_KEY	Private keys (PEM, OpenSSH, service-account private_key), the whole block
CERTIFICATE_SECRET	Certificate bundles that contain a private key (PKCS#12 / PFX)
ENCRYPTION_KEY	Symmetric encryption keys (AES keys, data keys, master keys)
GENERIC_SECRET	A secret whose type the text does not say

Not secrets (not labelled): references such as ${API_KEY} or os.environ["X"]; placeholders such as <your-api-key> or CHANGEME; masked values; identifiers (AWS access key IDs AKIA…, UUIDs, account and request ids, usernames); public keys and certificates; hashes and digests, including git commit SHAs; documented example values such as AKIAIOSFODNN7EXAMPLE. A realistic value is labelled even when its name says "test" or "example": the model cannot know whether it is live.

How to use

1. Download and install

pip install -U huggingface_hub
hf download CobuildX-AI/cbx-secrets-detector-17M --revision v0.1 --local-dir cbx-secrets-detector-17M
cd cbx-secrets-detector-17M
pip install -r requirements.txt

2. Use it in your code

SecretDetector (in cbx_secrets.py, part of this repo) loads the model once:

from cbx_secrets import SecretDetector

det = SecretDetector()                 # uses a GPU if one is available
text = 'STRIPE_SECRET_KEY=sk_live_4eC39HqLyjWDarjtT1zdp7dcQw8vXk2Z'
det.extract(text)
# [{'label': 'API_KEY', 'text': 'sk_live_4eC39HqLyjWDarjtT1zdp7dcQw8vXk2Z', 'start': 18, 'end': 58}]

det.redact(text)
# 'STRIPE_SECRET_KEY=[API_KEY]'

det.extract(text, labels=["PASSWORD", "PRIVATE_KEY"])   # keep only some labels

• Context length is 7,999 tokens, read in a single pass. Longer texts are split into overlapping pieces automatically.
• redact() replaces each value found with its label, or with your own template.
• The values in these examples are invented.

3. Optional: check your setup

python example.py runs four sample texts (all values invented) and prints what it finds.

Results

Span F1, label-agnostic: a found value counts if it overlaps a true secret, whatever its label (the benchmarks and the scanners use different type names). Precision and recall in brackets. "False alarms" is the share of labelled non-secret lines or candidates on which something is flagged.

Model	CredData test	false alarms	Issue reports test	false alarms	CBX secrets test
cbx-secrets-detector-17M	0.805 (0.80 / 0.81)	0.9%	0.836 (0.90 / 0.78)	0.9%	0.983 (0.99 / 0.98)
cbx-secrets-detector-32M	0.820 (0.77 / 0.87)	1.3%	0.861 (0.86 / 0.86)	1.4%	0.998 (1.00 / 1.00)
Gitleaks 8.30.1	0.442 (0.90 / 0.29)	0.6%	0.678 (0.93 / 0.54)	0.5%	0.757 (0.99 / 0.61)
detect-secrets 1.5.0	0.460 (0.47 / 0.45)	9.8%	0.290 (0.40 / 0.23)	4.0%	0.626 (0.90 / 0.48)
TruffleHog 3.99.2	0.042 (0.38 / 0.02)	0.8%	0.228 (0.66 / 0.14)	0.9%	0.215 (1.00 / 0.12)

• CredData test: Samsung CredData (secrets obfuscated by Samsung): the test half of the repositories, 168 repositories with 3,493 secrets and 17,961 labelled non-secret lines. No test repository was used for training or for choosing the model.
• Issue reports test: the test split of the issue-report dataset of Ahmed et al. (see Training data): 949 secrets and 8,132 non-secret candidates, scored per candidate. Texts that also appear in its training split were removed.
• CBX secrets test: our own held-out test data, generated by CobuildX and checked against our annotation guide: 3,000 texts with 3,136 secrets across all 12 labels. With the label required to match exactly as well, this model scores 0.961 (per label: CERTIFICATE_SECRET 1.00, ENCRYPTION_KEY 1.00, JWT_TOKEN 1.00, PRIVATE_KEY 1.00, API_KEY 1.00, WEBHOOK_SECRET 0.99, CLOUD_CREDENTIAL 0.99, ACCESS_TOKEN 0.99, SESSION_SECRET 0.97, GENERIC_SECRET 0.94, DATABASE_CREDENTIAL 0.90, PASSWORD 0.89). It comes from the same generators as part of the training data, so it overstates accuracy on real text; the two benchmarks above are the better guide.
• Scanners were run offline with their default rules and no live verification; TruffleHog normally relies on verification, so its numbers here understate it.

Speed. Median 6 ms (p95 6 ms) per text of 1,000 tokens on an NVIDIA TITAN RTX (fp16), 37 ms (p95 38 ms) on an Intel i7-10700 CPU with 8 threads; texts processed one at a time, end to end (text in, spans out).

Limitations

• English only. Trained for code, config files, logs, tickets and chat.
• Long encoded data: on long runs of random-looking characters (an inline base64 source map, a minified bundle, a log full of hex session ids) the model can flag many short pieces as secrets. Skip minified and generated files where you can.
• Recall is lower for tokens with no known prefix and no telling name nearby (key, token, password).
• Labels are less certain than detection: a password in a database context can be labelled PASSWORD or DATABASE_CREDENTIAL (exact F1 on our test set 0.89 for the weaker of the two), and a database URI is sometimes returned as two spans split at the password.
• It does not check whether a secret is live, and it can miss secrets: use it as a detection aid alongside rotation and review, not as a guarantee.

Training data

• CBX secrets train data: generated by CobuildX and checked against our annotation guide: code, configs, logs, chat and documents with invented secrets and hard negatives (placeholders, references, hashes, fake values).
• Samsung CredData (lines and labels from 116 repositories with MIT, Apache-2.0 or BSD licences; values already obfuscated by Samsung). The repositories and their licences are listed in training_data_sources.csv.
• Issue reports: the training split of Secret Leak Detection in Software Issue Reports using LLMs, Ahmed et al., arXiv:2410.23657, dataset 10.5281/zenodo.19622962, licensed CC BY 4.0. Changed: every real secret was replaced by an invented value of the same shape before training, and texts were cut into windows. No real secret from this dataset is in the model's training data or in this repository.

Licence

Apache-2.0. Base model: Ettin (MIT, arXiv:2507.11412).