zhenhan7/hw1-hc3-detector

HW1 HC3 Text Classifier

This repository contains the fine-tuned classifier and matching tokenizer for CS546 Homework 1. The model is based on sentence-transformers/all-MiniLM-L6-v2 and predicts whether an HC3 answer was written by a human or ChatGPT.

Labels

• 0: human
• 1: ChatGPT

Evaluation results

These results come from the completed homework run on its 4,668-example test split.

Model	Test accuracy	Macro F1
Frozen embeddings + Logistic Regression baseline	0.844901 (84.49%)	0.844882
Fine-tuned classifier in this repository	0.995930 (99.59%)	0.995930

The split sizes were 37,334 training examples, 4,666 validation examples, and 4,668 test examples. The run did not compute validation loss. The baseline made 724 test errors; the fine-tuned classifier made 19.

Load the model

from transformers import AutoModelForSequenceClassification, AutoTokenizer

repo_id = "zhenhan7/hw1-hc3-detector"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id)
model.eval()

The reported accuracy measures performance on this homework's HC3 test split. It does not establish accuracy on other datasets or prove the authorship of an individual document.