How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection
Paper • 2301.07597 • Published • 1
HC3 AI-Generated Text Detector
Fine-tuned sentence-transformers/all-MiniLM-L6-v2 for binary classification of
human-written vs. ChatGPT-generated answers, trained on the HC3 dataset
(Guo et al., 2023, arXiv:2301.07597).
- Labels:
0 = human,1 = ChatGPT - Base model: sentence-transformers/all-MiniLM-L6-v2
- Fine-tuned for: Homework 1 — Sentence Transformers for AI-Generated Text Detection
Results
| Model | Test Accuracy |
|---|---|
| Baseline (frozen embeddings + Logistic Regression) model | 84.49% |
| Fine-tuned model | 98.93% |
Training Details
- Epochs: 5
- Learning rate: 2e-5
- Optimizer: AdamW
- Batch size: 32
- Max sequence length: 256
Limitations
This is a historical benchmark trained on one snapshot of ChatGPT and a fixed HC3 split. It is not a reliable detector for current AI-generated or student-written text, and may reflect dataset-specific stylistic artifacts rather than general markers of AI authorship.
- Downloads last month
- 9