DuJiahui/hw1-hc3-detector

HC3 Human vs. ChatGPT Text Classifier

This model was fine-tuned for binary classification of human-written and ChatGPT-generated answers using the HC3 dataset.

Model

Base model: sentence-transformers/all-MiniLM-L6-v2

Labels:

• 0: Human
• 1: ChatGPT

Results

• Baseline accuracy: 0.8449
• Fine-tuned test accuracy: 0.9908

The baseline used frozen sentence embeddings with logistic regression. After fine-tuning the transformer for five epochs, the test accuracy increased from 0.8449 to 0.9908.

Dataset

The model was trained and evaluated on the Human ChatGPT Comparison Corpus (HC3).

Limitations

The high performance on HC3 does not necessarily mean that the model will generalize to newer AI-generated text or text from other domains. The model may learn dataset-specific patterns rather than universal indicators of AI-generated text.