lang-spot

Flags non-English words in mixed-language text. Linear heads on a frozen encoder: one for non-English words, one for likely names.

Usage

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("goldreal-ai/lang-spot")
sys.path.insert(0, path)
from span_probes import load

probe = load(path)
probe.analyze("Vamos a la playa this weekend")

Each word has character offsets, p_non_english, non_english_raw and name_review. Optional arguments: allowlist=[...] (words to keep as English) and threshold= (logit; default -3.68).

License

Heads: CC BY-SA 4.0. Encoder files: MIT.