Update README.md

README.md

@@ -3,12 +3,12 @@ license: mit
 datasets:
 - yahma/alpaca-cleaned
 ---
-This repo contains a low-rank adapter for LLaMA-7b fit on the Cleaned Alpaca dataset.
+This repo contains a low-rank adapter for LLaMA-7b fit on the Cleaned Alpaca dataset (with the new GPT-4 training data).
 
 This version of the weights was trained with the following hyperparameters:
 
-    Cleaned dataset: Snapshot April 2, 2023
-    Epochs: 3
+    Cleaned dataset: Snapshot April 8, 2023
+    Epochs: 6 (Checkpoint with lowest eval loss at 3.6 epochs uploaded here)
     Validation set size: 1500
     Batch size: 128
     Micro batch size: 8
@@ -22,7 +22,7 @@ That is:
 python finetune.py \
     --base_model='decapoda-research/llama-7b-hf' \
     --data_path 'yahma/alpaca-cleaned' \
-    --num_epochs=3 \
+    --num_epochs=6 \
     --cutoff_len=512 \
     --output_dir='./lora-alpaca' \
     --lora_target_modules='[q_proj,k_proj, v_proj, o_proj]' \