Update README.md

README.md

@@ -1,3 +1,30 @@
 ---
 license: apache-2.0
+pipeline_tag: text2text-generation
+tags:
+- llama
+- llm
 ---
+
+This is LoRA checkpoint fine-tuned with the following CLI. The fine-tuning process is logged in [W&B dashboard](https://wandb.ai/chansung18/alpaca_lora/runs/iguhmy31?workspace=user-chansung18). I have used DGX workstation with 8 x A100(40G).
+
+```console
+python finetune.py \
+    --base_model='elinas/llama-13b-hf-transformers-4.29' \
+    --data_path='alpaca_data.json' \
+    --num_epochs=10 \
+    --cutoff_len=1024 \
+    --group_by_length \
+    --output_dir='./lora-alpaca-13b-elinas' \
+    --lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' \
+    --lora_r=16 \
+    --lora_alpha=32 \ 
+    --batch_size=1024 \
+    --micro_batch_size=28
+```
+
+This LoRA checkpoint is recommended to be used with `transformers >= 4.29` which should be installed with the following command currently(4/30/2023).
+```console
+pip install git+https://github.com/huggingface/transformers.git
+```
+