nongfuyulang commited on
Commit
904cb93
·
verified ·
1 Parent(s): 2c96883

Model save

Browse files
Files changed (4) hide show
  1. README.md +6 -15
  2. all_results.json +5 -10
  3. train_results.json +5 -5
  4. trainer_state.json +0 -0
README.md CHANGED
@@ -1,20 +1,11 @@
1
  ---
2
  library_name: transformers
3
- license: llama3.2
4
- base_model: meta-llama/Llama-3.2-1B-Instruct
5
  tags:
6
- - alignment-handbook
7
  - trl
8
  - sft
9
  - generated_from_trainer
10
- - trl
11
- - sft
12
- - generated_from_trainer
13
- datasets:
14
- - barc0/induction_heavy_100k_jsonl
15
- - barc0/induction_heavy_suggestfunction_100k_jsonl
16
- - barc0/induction_100k_gpt4o-mini_generated_problems_seed100.jsonl_messages_format_0.3
17
- - barc0/induction_100k-gpt4-description-gpt4omini-code_generated_problems_messages_format_0.3
18
  model-index:
19
  - name: engineer-heavy-500k-barc-llama3.1-8b-ins-fft-induction_lr1e-5_epoch3
20
  results: []
@@ -25,9 +16,9 @@ should probably proofread and complete it, then remove this comment. -->
25
 
26
  # engineer-heavy-500k-barc-llama3.1-8b-ins-fft-induction_lr1e-5_epoch3
27
 
28
- This model is a fine-tuned version of [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) on the barc0/induction_heavy_100k_jsonl, the barc0/induction_heavy_suggestfunction_100k_jsonl, the barc0/induction_100k_gpt4o-mini_generated_problems_seed100.jsonl_messages_format_0.3 and the barc0/induction_100k-gpt4-description-gpt4omini-code_generated_problems_messages_format_0.3 datasets.
29
  It achieves the following results on the evaluation set:
30
- - Loss: 0.2990
31
 
32
  ## Model description
33
 
@@ -63,8 +54,8 @@ The following hyperparameters were used during training:
63
 
64
  | Training Loss | Epoch | Step | Validation Loss |
65
  |:-------------:|:-----:|:----:|:---------------:|
66
- | 0.3106 | 1.0 | 2995 | 0.3114 |
67
- | 0.2792 | 2.0 | 5990 | 0.2990 |
68
 
69
 
70
  ### Framework versions
 
1
  ---
2
  library_name: transformers
3
+ license: llama3.1
4
+ base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
5
  tags:
 
6
  - trl
7
  - sft
8
  - generated_from_trainer
 
 
 
 
 
 
 
 
9
  model-index:
10
  - name: engineer-heavy-500k-barc-llama3.1-8b-ins-fft-induction_lr1e-5_epoch3
11
  results: []
 
16
 
17
  # engineer-heavy-500k-barc-llama3.1-8b-ins-fft-induction_lr1e-5_epoch3
18
 
19
+ This model is a fine-tuned version of [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) on the None dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 0.2710
22
 
23
  ## Model description
24
 
 
54
 
55
  | Training Loss | Epoch | Step | Validation Loss |
56
  |:-------------:|:-----:|:----:|:---------------:|
57
+ | 0.2797 | 1.0 | 2995 | 0.2817 |
58
+ | 0.2389 | 2.0 | 5990 | 0.2710 |
59
 
60
 
61
  ### Framework versions
all_results.json CHANGED
@@ -1,14 +1,9 @@
1
  {
2
  "epoch": 2.0,
3
- "eval_loss": 0.29904305934906006,
4
- "eval_runtime": 87.7661,
5
- "eval_samples": 20178,
6
- "eval_samples_per_second": 229.907,
7
- "eval_steps_per_second": 1.8,
8
- "total_flos": 525762547179520.0,
9
- "train_loss": 0.32578817207447075,
10
- "train_runtime": 11617.078,
11
  "train_samples": 383339,
12
- "train_samples_per_second": 65.996,
13
- "train_steps_per_second": 0.516
14
  }
 
1
  {
2
  "epoch": 2.0,
3
+ "total_flos": 2071185792860160.0,
4
+ "train_loss": 0.2831706563921922,
5
+ "train_runtime": 60134.8665,
 
 
 
 
 
6
  "train_samples": 383339,
7
+ "train_samples_per_second": 12.749,
8
+ "train_steps_per_second": 0.1
9
  }
train_results.json CHANGED
@@ -1,9 +1,9 @@
1
  {
2
  "epoch": 2.0,
3
- "total_flos": 525762547179520.0,
4
- "train_loss": 0.32578817207447075,
5
- "train_runtime": 11617.078,
6
  "train_samples": 383339,
7
- "train_samples_per_second": 65.996,
8
- "train_steps_per_second": 0.516
9
  }
 
1
  {
2
  "epoch": 2.0,
3
+ "total_flos": 2071185792860160.0,
4
+ "train_loss": 0.2831706563921922,
5
+ "train_runtime": 60134.8665,
6
  "train_samples": 383339,
7
+ "train_samples_per_second": 12.749,
8
+ "train_steps_per_second": 0.1
9
  }
trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff