bsgcasa/ai-text-detector-distilbert

AI Text Detector (DistilBERT, ONNX INT8)

A fine-tuned DistilBERT model that classifies text as human-written or ChatGPT-generated. Exported to ONNX and quantized to INT8 for fast, lightweight CPU inference.

Labels

Label ID	Meaning
0	human
1	chatgpt

Performance (held-out test set)

Metric	Score
Accuracy	99.4%
F1	0.991
Precision	0.983
Recall	0.999

Evaluated across multiple domains (medicine, Reddit ELI5, finance, open QA, wiki/CS-AI) and text lengths (0–500+ words), with consistently high accuracy across all buckets. Wiki/CS-AI was the weakest domain at ~89% accuracy, worth keeping in mind for edge cases.

Files

• model_int8.onnx — INT8-quantized ONNX model (~64 MB), recommended for deployment
• tokenizer.json, tokenizer_config.json — fast tokenizer files
• label_order.json — maps output indices to label names

Usage (Google Colab)

Install the required dependencies:

!pip install -q onnxruntime transformers huggingface_hub numpy

Then run:

from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer
import onnxruntime as ort
import numpy as np
import json, os

REPO_ID = "bsgcasa/ai-text-detector-distilbert"

tokenizer_path = hf_hub_download(REPO_ID, "tokenizer.json")
hf_hub_download(REPO_ID, "tokenizer_config.json")
model_path = hf_hub_download(REPO_ID, "model_int8.onnx")
label_order_path = hf_hub_download(REPO_ID, "label_order.json")

tokenizer = AutoTokenizer.from_pretrained(os.path.dirname(tokenizer_path))

with open(label_order_path) as f:
    label_order = json.load(f)

session = ort.InferenceSession(model_path)

def predict(text, max_length=256):
    enc = tokenizer(
        text,
        return_tensors="np",
        padding="max_length",
        truncation=True,
        max_length=max_length
    )

    inputs = {
        "input_ids": enc["input_ids"].astype(np.int64),
        "attention_mask": enc["attention_mask"].astype(np.int64),
    }

    logits = session.run(["logits"], inputs)[0]
    probs = np.exp(logits) / np.exp(logits).sum(axis=-1, keepdims=True)

    return {
        label_order[str(i)]: float(p)
        for i, p in enumerate(probs[0])
    }

print(predict(
    "honestly idk man, my car just broke down again lol third time this month"
))

# {'human': 0.9999..., 'chatgpt': 0.0000...}

Training Details

• Base model: distilbert-base-uncased
• Task: binary sequence classification (human vs. ChatGPT-generated text)
• Max sequence length: 256 tokens
• Class imbalance handling: weighted cross-entropy loss (train set was ~67% human / 33% ChatGPT)
• Epochs: 2
• Optimizer: AdamW, learning rate 2e-5

Limitations

• Trained on a specific mix of domains (medicine, finance, Reddit, open QA, wiki/CS-AI); performance may degrade on out-of-distribution text styles or newer LLM outputs not represented in training data.
• Detects patterns associated with ChatGPT specifically as trained — may not generalize equally well to other LLMs' outputs without further evaluation.
• Like all AI-text detectors, this should be treated as a supporting signal, not a definitive judgment — false positives/negatives are possible, especially on short or heavily edited text.