ai-detector-dev-11-14
A development checkpoint of a token-level AI-text detector. It labels every token of an English text as human, ai_assisted or ai_generated.
- Backbone: jhu-clsp/ettin-encoder-400m (395M parameters), fine-tuned.
- Heads: a 3-way token head, a segment head (how much of a window is AI-written) and a mixed-authorship head.
- Inference: windows of 512 tokens with stride 256; logits are averaged over overlapping windows. A calibrated linear-chain CRF then gives one label per token.
Files
| File | Content |
|---|---|
backbone/ |
Fine-tuned encoder weights and config |
head.safetensors |
The three heads |
tokenizer/ |
Tokenizer |
model.json |
Classes, heads, window and stride |
crf.json |
Calibration (W, alpha, delta) and CRF parameters (lambda, gamma, l_min) |
inference.py |
Standalone inference script |
Usage
inference.py needs torch, transformers, safetensors, numpy and huggingface_hub.
python inference.py essay.txt # verdict and labelled segments
python inference.py essay.txt --json # full result, with per-sentence probabilities
cat essay.txt | python inference.py -
from inference import Detector
detector = Detector() # or Detector("path/to/this/folder", device="cuda:0")
result = detector.detect(text)
result["verdict"] # "Human", "Mixed" or "AI"
result["segments"] # [{"start", "end", "label"}], character positions in result["text"]
result["sentences"] # the same per sentence, with the probability of each label
The verdict is Human if at least 90% of the text is labelled human, AI if the AI fraction (AI-generated plus half of AI-assisted) is at least 80%, and Mixed otherwise.
Input
The detector was trained on normalised text: straight quotes, -- for em dashes, ... for the ellipsis character, and plain text without Markdown. inference.py applies this normalisation itself, and the positions it returns refer to the normalised text. Feeding raw typographic punctuation without it raises the false-positive rate on human text.
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support
Model tree for CoderBak/ai-detector-dev-11-14
Base model
jhu-clsp/ettin-encoder-400m