pangboo/hw1-hc3-detector

HW1 HC3 Detector

This model classifies English answer text as human-written or ChatGPT-generated. It was fine-tuned from sentence-transformers/all-MiniLM-L6-v2 with a two-class sequence classification head for CS546 Homework 1.

Repository: https://huggingface.co/pangboo/hw1-hc3-detector

Labels

Label ID	Meaning
0	Human-written
1	ChatGPT-generated

Only answer text is used as input; questions are excluded from the model input.

Evaluation results

Both models were evaluated on the same held-out HC3 test split of 4,668 answers.

Model	Test accuracy
Baseline: frozen MiniLM sentence embeddings + logistic regression	0.8449 (84.49%)
Fine-tuned MiniLM sequence classifier	0.9931 (99.31%)

Dataset and preprocessing

The retained 23,334 question pairs are sorted by normalized question text, shuffled with seed 42, and split by question before answers are flattened. Answers associated with the same question therefore remain in the same split. Each split contains equal numbers of human and ChatGPT answers.

Split	Answer count
Training	37,334
Validation	4,666
Test	4,668

Training

The baseline uses frozen 384-dimensional SentenceTransformer embeddings and scikit-learn LogisticRegression(random_state=42) with its remaining defaults.

The fine-tuned model updates all encoder parameters and the classification head using the following settings:

Setting	Value
Base model	sentence-transformers/all-MiniLM-L6-v2
Model class	AutoModelForSequenceClassification
Number of labels	2
Random seed	42
Epochs	5
Training batch size	128
Optimizer	PyTorch AdamW
Learning rate	2e-5
Loss	Cross-entropy
Maximum input length	256 tokens
Padding	Dynamic padding within each batch
Test batch size	32

The final model is the checkpoint after five epochs. The training loop does not use early stopping, a learning-rate scheduler, or validation-based checkpoint selection.