zb210/hw1-hc3-detector

HW1 HC3 Detector

Fine-tuned for CS546 Homework 1 to classify English HC3 answers.

Labels

• 0: Human
• 1: ChatGPT

Data

Uses the provided HC3 preparation with seed 42 and an 80/10/10 question-level split. Only answer text is used as model input.

• Training: 37,334 answers
• Validation: 4,666 answers
• Test: 4,668 answers

Training

• Base model: sentence-transformers/all-MiniLM-L6-v2
• Optimizer: AdamW
• Learning rate: 2e-5
• Epochs: 5
• Batch size: 32
• Maximum sequence length: 256 tokens

Test Results

Model	Accuracy
Frozen embeddings + logistic regression	84.49%
Fine-tuned transformer	98.71%

Limitations

This model was evaluated on the historical HC3 dataset. Its performance may not generalize to newer AI models or other text sources. It is not a reliable detector for current student work.