Deckard618/textdet

AI-generated text detectors (ONNX)

Two binary classifiers (human / AI-generated) for English text, exported to ONNX for client-side inference with ONNX Runtime (Web or Python). In a browser, texts are analyzed on the device and never uploaded. They run in the web app frontend-deepfake-detection.

Folder	Base model	Parameters	File	Input
ettin-68m	Ettin encoder 68M (Johns Hopkins)	68M	77 MB, int8	first 512 tokens
ettin-150m	Ettin encoder 150M (Johns Hopkins)	149M	168 MB, int8	first 512 tokens

Each folder holds model.onnx, the tokenizer.json and tokenizer_config.json to use with it, a config.json for the web app, and parity.json (the int8 export checked against the PyTorch model: same decision on 40 of 40 texts).

• Output: 2 logits, index 0 = human, index 1 = AI-generated. Apply a softmax to get probabilities.
• Calibrated: the output is shifted so that P(AI) = 50% flags 5% of human texts (2,000 MAGE test texts).
• Tokenizer: use tokenizer.json as is. It normalizes the text the way training did (NFKC, zero-width characters removed, whitespace collapsed) and adds [CLS] / [SEP]. Truncate to 512 tokens.

Results

Held-out test sets, AUROC (1 = perfect, 0.5 = chance), then the share of AI texts caught when 1% of human texts are flagged:

Test set	ettin-68m	ettin-150m
DetectRL-X rewritten: polished, paraphrased, back-translated, character edits (attacks unseen in training)	0.980 / 0.77	0.964 / 0.75
MAGE GPT-4, unseen domains	0.946 / 0.39	0.933 / 0.10
MAGE GPT-4, paraphrased	0.822 / 0.13	0.803 / 0.02
TextSight 2026, the newest models (197 texts)	0.898 / 0.50	0.937 / 0.54
Beemo, edited chatbot answers	0.752 / 0.11	0.765 / 0.06
DetectRL-X (generators and domains seen in training)	1.000 / 1.00	1.000 / 1.00

TextSight 2026 holds only 28 AI texts: differences of a few points there are noise. ettin-68m is half the size of ettin-150m and catches more AI text at 1% false positives on most sets.

Usage (Python)

import numpy as np
import onnxruntime as ort
from tokenizers import Tokenizer

LABELS = ["human", "AI-generated"]

def predict(model_dir, text):
    tokenizer = Tokenizer.from_file(f"{model_dir}/tokenizer.json")
    tokenizer.enable_truncation(max_length=512)
    ids = np.array([tokenizer.encode(text).ids], dtype=np.int64)
    session = ort.InferenceSession(f"{model_dir}/model.onnx")
    logits = session.run(None, {"input_ids": ids, "attention_mask": np.ones_like(ids)})[0][0]
    probs = np.exp(logits - logits.max())
    probs /= probs.sum()
    return dict(zip(LABELS, probs.tolist()))

print(predict("ettin-68m", "Paste a paragraph of English text here."))

The web app (onnxruntime-web and tokenizers.js) gives the same token ids and probabilities within 1e-6.

Intended use and limitations

These models are meant for education, research and personal checks of texts. A score is a probability, not proof.

• English only: other languages get inconsistent scores.
• Edited and assistant-style human text is the weak spot. On Beemo, about half of the human answers are flagged at 50%.
• Paraphrased AI text is often missed (AUROC about 0.8 on paraphrased GPT-4 texts).
• Short texts are less reliable: every model loses about 0.04 AUROC when it reads only the first 100 words.
• The 5% calibration was measured on MAGE human texts. Human texts from other domains can be flagged more often.
• Do not use a score as the only basis for a decision about a person (academic, legal, disciplinary, professional or other).

Training

All weights of the Ettin encoders were fine-tuned, with a linear head on a learned mix of every layer's mean-pooled hidden states: 5 epochs of 100,000 texts, with EMA weights. Each training set weighs the same, and so does each class within it.

Training set	Texts	Human	AI generators	License
RAID (train)	164,303	66,714	11 (GPT-2/3/4, ChatGPT, Cohere, Llama-chat, Mistral, MPT), with 11 attacks	MIT
MAGE (train)	69,484	28,984	27 (OPT, GPT-J, NeoX, LLaMA, FLAN-T5, BLOOM, GLM-130B, text-davinci, GPT-3.5…)	Apache-2.0
DetectRL-X (train, English)	9,599	4,799	4 (GPT-4o, Gemini 2.5 Flash, DeepSeek-V3, Qwen-Max)	MIT

• Formatting is never a shortcut: typography, line breaks, typos, homoglyphs and case changes are applied to both classes.
• Half of the training texts are random excerpts, so length and openings are not shortcuts either.
• The models were selected on a validation split, never on the test sets above.

License

MIT, the license of the Ettin encoders the models are built on (LICENSE, Copyright (c) 2025 Johns Hopkins University Center for Language and Speech Processing). NOTICE lists the changes made to the original models. The models are provided as is, without any warranty.

Sources: Ettin code and paper.

Citation

If you use these models, please also cite Ettin:

@misc{weller2025seqvsseq,
  title={Seq vs Seq: An Open Suite of Paired Encoders and Decoders},
  author={Orion Weller and Kathryn Ricci and Marc Marone and Antoine Chaffin and Dawn Lawrie and Benjamin Van Durme},
  year={2025},
  eprint={2507.11412},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}