deberta-v3-base-hatbench-AB-curriculum-seed0

Sentence-level AI-text detector for HAT-Bench — variant A+B (hard BCE + easy-to-hard curriculum, full-FT), seed 0.

Part of the HAT-Baselines detector suite. This model predicts a per-sentence y_score ∈ [0, 1] indicating whether each sentence has been AI-modified.

Test metrics (HAT-Bench pooled test set)

Metric	Value
Headline macro-F1 (pooled)	0.8804
Human F1	0.8803
AI F1	0.8805
v1–v7 macro-F1 (partial-AI only)	0.8671
Accuracy	0.8804
AUROC	0.9535

Training recipe

• base_model: microsoft/deberta-v3-base
• max_seq_len: 512
• fine_tuning: full fine-tune (no LoRA)
• loss: BCE (hard labels)
• sampler: VersionCurriculumSampler (easy → hard, across versions v0..v8)
• batch_size: 2
• grad_accum: 4
• effective_batch_size: 8
• epochs: 1
• lr: 2e-5
• weight_decay: 0.01
• warmup_frac: 0.06
• bf16: yes
• seed: 0
• best-ckpt selection: dev macro_f1

Reproduction command (from the sentence-trajectory research worktree):

conda run -n omni-text python research/exp/03_curriculum.py --seed 0 --epochs 1 --direction easy_to_hard --batch-size 2 --grad-accum 4 --max-train-rows-per-file 0 --max-eval-rows-per-file 0

W&B run: https://wandb.ai/jiacheng-liu-19-mbzuai/hat_bench/runs/r4w88f00

Loading

import torch
from transformers import AutoTokenizer, AutoModel
from huggingface_hub import hf_hub_download

REPO = "HAT-Baselines/deberta-v3-base-hatbench-AB-curriculum-seed0"
tok = AutoTokenizer.from_pretrained(REPO, subfolder="tokenizer")
encoder = AutoModel.from_pretrained(REPO, subfolder="encoder")

head = torch.nn.Sequential(
    torch.nn.Linear(768, 256), torch.nn.GELU(),
    torch.nn.Dropout(0.1), torch.nn.Linear(256, 1),
)
head_path = hf_hub_download(REPO, "head.pt")
head.load_state_dict(torch.load(head_path, map_location="cpu"))

Apply per-sentence: mean-pool encoder hidden states over each sentence's token span, feed the pooled vector through head, and sigmoid the logit to get y_score. See research/utils/data.py :: segment_mean in the research worktree for the exact pooling.

Citation

If you use this model, please cite HAT-Bench (TBD).