hw1-hc3-detector

Fine-tuned sentence-transformers/all-MiniLM-L6-v2 for detecting whether an answer was written by a human or generated by ChatGPT, using the HC3 (Human ChatGPT Comparison Corpus) English dataset. This was made for Homework 1.

Results

Model	Test accuracy
Baseline: frozen all-MiniLM-L6-v2 embeddings + logistic regression	0.8449
Fine-tuned model (this repo)	0.9884

Test set: 4,668 answers (2,334 human, 2,334 ChatGPT).

Fine-tuned confusion matrix: 2,281 human correct, 53 human predicted as ChatGPT, 1 ChatGPT predicted as human, 2,333 ChatGPT correct. Total errors dropped from 724 (baseline) to 54.

Labels

• LABEL_0 = human-written
• LABEL_1 = ChatGPT-generated

Training details

• Data: HC3 (split 80/10/10 by question (37,334 train / 4,666 validation / 4,668 test answers). Only the answer text is used as input.
• Model: AutoModelForSequenceClassification with 2 labels, all layers fine-tuned
• Optimizer: AdamW, learning rate 2e-5
• Epochs: 5, batch size 32, max sequence length 256 (dynamic padding)
• Seed: 42
• Trained on a Google Colab GPU

Usage

from transformers import pipeline

clf = pipeline("text-classification", model="geethuj2/hw1-hc3-detector")
print(clf("Your answer text here."))