Dataset procedure
- Dataset used: /tweets_hate_speech_detection
- size: 3196 (only 10% of dataset used)
- batch_size = 32
- num_epochs = 20
- learning_rate = 3e-4
- num_warmup_steps = 0.06 * (3196 * num_epochs)
- num_training_steps = (3196 * num_epochs)
Training procedure
LoraConfig procedure
r=8, #attention heads
lora_alpha=16, #alpha scaling
lora_dropout=0.1,
bias="none",
task_type="SEQ_CLS" # set this for CLM or Seq2Seq
Framework versions
- PEFT 0.6.0.dev0
- Downloads last month
- 0
Unable to determine this model’s pipeline type. Check the
docs
.