File size: 521 Bytes
d7c8144 4e7a0e0 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
---
license: mit
---
LoRA weights for `LLaMA-7B`
Trained on 'taesiri/webnlg-triplets-explanation-v1' for 4 epochs.
Command:
```
WORLD_SIZE=2 CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node=2 --master_port=1234 finetune.py --base_model='decapoda-research/llama-7b-hf' --data_path 'taesiri/webnlg-triplets-explanation-v1' --num_epochs=4 --cutoff_len=512 --group_by_length --lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' --lora_r=8 --micro_batch_size=8 --batch_size=32
```
|