adityakp15/hw1-hc3-detector

HC3 AI-Generated Text Detector

This repository contains a fine-tuned transformer classifier for distinguishing human-written text from ChatGPT-generated text in the HC3 dataset.

Results

Model	Test accuracy
Frozen sentence-transformer baseline	0.8449
Fine-tuned classifier	0.9871

The fine-tuned model was trained for five epochs with AdamW and a learning rate of 2e-5. The test split was held out during training. Results may vary slightly with hardware or library versions.

Dataset

The model was trained and evaluated on the HC3 dataset, using its human and ChatGPT answer labels. Splits were kept disjoint by question ID to reduce question overlap between training, validation, and test data.

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

repo_id = "adityakp15/hw1-hc3-detector"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id)

text = "Text to classify"
inputs = tokenizer(text, return_tensors="pt", truncation=True)

with torch.no_grad():
    prediction = model(**inputs).logits.argmax(dim=-1).item()

label = model.config.id2label.get(prediction, str(prediction))
print(label)

The classifier labels are 0 for human-written text and 1 for ChatGPT- generated text.

Training Details

• Optimizer: AdamW
• Learning rate: 2e-5
• Epochs: 5
• Batch size: 32
• Random seed: 42
• Evaluation metric: accuracy