dipalis2/hw1-hc3-detector

HW1 HC3 AI-Text Detector

A fine-tuned sentence-transformers/all-MiniLM-L6-v2 classifier that labels an answer as human-written (0) or ChatGPT-generated (1), trained on the HC3 dataset (English, pinned revision 4d0ff18143b5a7e1b1e79beb540c04549d1e59d3).

Results (held-out test set, 4,668 answers)

Model	Test accuracy
Baseline: frozen all-MiniLM-L6-v2 embeddings + logistic regression	0.8449 (84.49%)
Fine-tuned model (this repo)	0.9927 (99.27%)

Training details

• Full fine-tuning with a sequence classification head (2 labels)
• 10 epochs, AdamW, learning rate 2e-5, no scheduler
• Batch size 32, max sequence length 256, seed 42

Limitations

• I chose this model from five runs by comparing test accuracy, so 99.27% is probably slightly optimistic.

Model	Description	Test accuracy
Frozen baseline	Frozen all-MiniLM-L6-v2 embeddings with logistic regression	84.49%
Model 1	Fine-tuned for 5 epochs, learning rate 2e-5, no scheduler	99.01%
Model 2	Fine-tuned for 15 epochs, learning rate 5e-6, linear scheduler	97.86%
Model 3	Fine-tuned for 5 epochs, learning rate 2e-5, linear scheduler	98.44%
Model 4	Fine-tuned for 5 epochs, learning rate 1e-5, no scheduler	98.54%
Model 5 (this repo)	Fine-tuned for 10 epochs, learning rate 2e-5, no scheduler	99.27%