Zangs3011 commited on
Commit
8f7fe65
1 Parent(s): 7aa88de

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -7
README.md CHANGED
@@ -3,31 +3,34 @@ datasets:
3
  - sahil2801/CodeAlpaca-20k
4
  library_name: peft
5
  tags:
6
- - codellama7b
7
  - code
8
  - instruct
9
  - instruct-code
10
  - code-alpaca
11
  - alpaca-instruct
12
  - alpaca
13
- - codellama7b
14
  - gpt2
15
  ---
16
 
17
- We finetuned CodeLlama7B on Code-Alpaca-Instruct Dataset (sahil2801/CodeAlpaca-20k) for 5 epochs or ~ 25,000 steps using [MonsterAPI](https://monsterapi.ai) no-code [LLM finetuner](https://docs.monsterapi.ai/fine-tune-a-large-language-model-llm).
18
 
19
- This dataset is HuggingFaceH4/CodeAlpaca_20K unfiltered, removing 36 instances of blatant alignment.
20
 
21
- The finetuning session got completed in 4 hours and costed us only `$16` for the entire finetuning run!
22
 
23
  #### Hyperparameters & Run details:
24
- - Model Path: meta-llama/CodeLlama7B
25
  - Dataset: sahil2801/CodeAlpaca-20k
26
  - Learning rate: 0.0003
27
  - Number of epochs: 5
28
  - Data split: Training: 90% / Validation: 10%
29
  - Gradient accumulation steps: 1
30
 
 
 
 
31
  ---
32
  license: apache-2.0
33
- ---
 
3
  - sahil2801/CodeAlpaca-20k
4
  library_name: peft
5
  tags:
6
+ - llama2-7b
7
  - code
8
  - instruct
9
  - instruct-code
10
  - code-alpaca
11
  - alpaca-instruct
12
  - alpaca
13
+ - llama7b
14
  - gpt2
15
  ---
16
 
17
+ We finetuned Llama2-7B on Code-Alpaca-Instruct Dataset (sahil2801/CodeAlpaca-20k) for 5 epochs or ~ 25,000 steps using [MonsterAPI](https://monsterapi.ai) no-code [LLM finetuner](https://docs.monsterapi.ai/fine-tune-a-large-language-model-llm).
18
 
19
+ This dataset is HuggingFaceH4/CodeAlpaca_20K unfiltered, removing 36 instances of blatant alignment.
20
 
21
+ The finetuning session got completed in 4 hours and costed us only `$16` for the entire finetuning run!
22
 
23
  #### Hyperparameters & Run details:
24
+ - Model Path: meta-llama/Llama-2-7b
25
  - Dataset: sahil2801/CodeAlpaca-20k
26
  - Learning rate: 0.0003
27
  - Number of epochs: 5
28
  - Data split: Training: 90% / Validation: 10%
29
  - Gradient accumulation steps: 1
30
 
31
+ Loss metrics:
32
+ ![training loss](train-loss.png "Training loss")
33
+
34
  ---
35
  license: apache-2.0
36
+ ---