DotCheck/valla-text-v12

Valla@12.2 is a text detector. It maps a UTF-8 string and an explicit lang code to (p \in [0,1]), an estimate of (P(\mathrm{AI})) for the writing, not for the truth of the claim.

There are two frozen encoder groups and eight language-specific logistic heads:

Latin group (en, es, pt, fr, it, de, nl): frozen TMR + Fakespot features, one logistic head per language.
Chinese group (zh): frozen MacBERT cls_emb, head zh_v1.

Unsupported lang fails closed (unsupported_language). There is no silent English fallback. Traditional-Chinese-dominant input is not scored (und).

Field	Value
Hub id	`DotCheck/valla-text-v12`
English wire	`inhouse-text@12`
Public token	`Valla@12.2`
English artifact	`text_stack_head_v12.npz`
Latin bases	`Oxidane/tmr-ai-text-detector` (MIT), `fakespot-ai/roberta-base-ai-text-detection-v1` (Apache-2.0)
Chinese base	`hfl/chinese-macbert-base` (Apache-2.0)
Heads	`text_stack_head_v12.npz` + `zh_v1` / `es_v3` / `pt_BR_v3` / `fr_v3` / `it_v3` / `de_v3` / `nl_v3`
`lang`	`en\|zh\|es\|pt\|fr\|it\|de\|nl` (`pt` uses the `pt_BR` head; `zh` = Simplified Chinese)
Output	(p \in [0,1]) — estimated (P(\mathrm{AI}))
Serve	CPU FastAPI `POST /v1/analyze-text` with required `lang`

Model description

Text is cleaned (TC1) before hashing and scoring: wiki-style citations stripped, whitespace collapsed, #category tokens dropped. The same cleaner is used in Express and the extension.

Latin languages share one frozen TMR+Fakespot forward. Chinese loads MacBERT only when text_stack_head_zh_v1.npz is present. Each head is a logistic classifier on that frozen vector. Train = serve for the Chinese cls_emb path.

In this repo: README.md, LICENSE, NOTICE, CITATION.cff, and the eight .npz heads listed above. Upstream encoder checkpoints are not redistributed here.

Architecture

text + lang
  → TC1 clean
  → if lang in {en, es, pt, fr, it, de, nl}:
        frozen TMR + Fakespot → shared Latin features
  → if lang == zh:
        frozen MacBERT cls_emb
  → else: unsupported_language (no score)
  → language-specific logistic head (.npz)
  → p

Inference

Open weights: the live .npz heads in this repository (Apache-2.0), used with the frozen upstream bases named above. This is not a transformers AutoModel package.

Product scoring: Check or Pro API.

Minimum length and fair-use caps are product policy at Express, not properties of the heads.

Training data

English fit AI is commercial-clean self-generation (Qwen2.5-7B, Mistral-7B, and Grok-4.5). Holdout AI is a smaller disjoint generator (Qwen2.5-1.5B; encyclopedic-heavy). English humans are a warehouse sample: Gutenberg.org windows 40%, Wikipedia 35%, WikiText 25%, sliced to 520–1100 characters. Other Latin languages keep their prior wiki / WikiText / Gutenberg mixes. Chinese humans are Wikipedia lead prose with OpenCC t2s. No NC datasets. No published holdout scraped from live ChatGPT or Claude.

	English	Other Latin	Chinese (zh-Hans)
Fit AI	Qwen2.5-7B + Mistral-7B + Grok-4.5	Qwen2.5-7B + Mistral-7B	Qwen2.5-7B + Mistral; wiki-style prompts
Holdout AI	Qwen2.5-1.5B	Qwen2.5-1.5B	Qwen2.5-1.5B (~200/200)
Human	Gut 40 / wiki 35 / WT 25	wiki / WikiText / Gutenberg	Wikipedia leads (OpenCC t2s)

Evidence files: text_gates_v12.json, text_gates_{lang}.json.

Evaluation

Absolute floors on every language head: human mean (P(\mathrm{AI}) \le 0.12), AI mean (\ge 0.85), balanced accuracy (\ge 0.90), all at threshold (0.5).

English (public claim)

Metric	Floor	Measured
mean (P(\mathrm{AI})) \| human	(\le 0.12)	0.012
mean (P(\mathrm{AI})) \| AI	(\ge 0.85)	1.000
balanced accuracy	(\ge 0.90)	0.995

English also beats live @11 rank-first on holdout and meets product-cut RAID-lite (ABS-windowed 520–1100) plus canary/style v2 floors (TEXT_GATES_OK, BEAT_V11_HOLDOUT_OK, TX26_PRODUCT_FLOORS_OK).

Other language heads (same floors)

Lang	Wire	Human	AI	bal_acc
zh	`inhouse-text-zh_v1`	0.033	0.973	0.972
es	`inhouse-text-es_v3`	0.029	0.993	0.982
fr	`inhouse-text-fr_v3`	0.021	0.990	0.978
pt	`inhouse-text-pt_BR_v3`	0.042	0.964	0.958
de	`inhouse-text-de_v3`	0.034	0.966	0.978
it	`inhouse-text-it_v3`	0.016	0.943	0.972
nl	`inhouse-text-nl_v3`	0.056	0.963	0.950

Portuguese lang=pt uses the Brazilian-trained head.

Intended use

Reproduce the eight heads and the holdout tables.
Research on synthetic-text detection in the eight supported languages.

Out of scope: languages outside {en, zh, es, pt, fr, it, de, nl}; Traditional Chinese; plagiarism, factual verification, or authorship; a product SLA from the Hub files alone.

Limitations

Holdout AI is Qwen2.5-1.5B, disjoint from fit. Live ChatGPT, Claude, and other closed generators are not a published holdout row.
Short, heavily edited, or machine-translated text has higher variance on this stack.
zh is Simplified Chinese (MacBERT cls_emb). Other scripts and Traditional-Chinese-dominant text are out of domain: no score, no English fallback.