HongjiP/hw1-hc3-detector

HW1 — HC3 AI-Generated Text Detector

Fine-tuned sentence-transformers/all-MiniLM-L6-v2 for binary classification of HC3 answers: 0 = human, 1 = ChatGPT.

Results (HC3 test split, 80/10/10 question-level split, seed 42)

Model	Test accuracy
Frozen MiniLM embeddings + logistic regression (baseline)	0.8449
Fine-tuned MiniLM (5 epochs, AdamW, lr 2e-5, max_len 256)	0.9919

Training setup

• Base model: sentence-transformers/all-MiniLM-L6-v2
• Optimizer: AdamW, lr 2e-5, batch size 32, 5 epochs, max sequence length 256
• Data: Hello-SimpleAI/HC3 (English), revision 4d0ff18143b5a7e1b1e79beb540c04549d1e59d3, one human and one ChatGPT answer per question, split 80/10/10 by question so answers to the same question never cross splits.

Limitations

HC3 is a historical benchmark: the ChatGPT answers come from one model in late 2022 / early 2023 and the human answers from web forums, so much of the separable signal is style and formatting specific to those sources. This model is not a reliable detector for current student work or for text from newer or differently-prompted models.