Hello-SimpleAI/HC3
Viewer • Updated • 48.6k • 5.84k • 219
How to use lilhuang15/bert-ai-text-detector-reddit-lora with PEFT:
from peft import PeftModel
from transformers import AutoModelForSequenceClassification
base_model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased")
model = PeftModel.from_pretrained(base_model, "lilhuang15/bert-ai-text-detector-reddit-lora")LoRA adapter (r=8 on Q/V attention; 296K trainable params = 0.27% of 110M) for
bert-base-uncased, trained on the same HC3 reddit_eli5 data as the full fine-tune
bert-ai-text-detector-reddit.
Held-out test macro-F1 0.9877 vs the full fine-tune's 0.9935 - the parameter-efficiency comparison study of the project below. The full fine-tune is the deployed model; this adapter exists to validate the PEFT tradeoff empirically.
Full project: https://github.com/lilhuang15/ai-generated-text-detector
Base model
google-bert/bert-base-uncased