brthor commited on
Commit
ab06fd9
·
verified ·
1 Parent(s): f1f5ca8

Upload full trainer checkpoint step=220 (profile=llama3_1_8b_base_lessdata_grpo, run_type=grpo)

Browse files
.gitattributes CHANGED
@@ -167,3 +167,4 @@ meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-5856/tokenizer.json fil
167
  meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-77/tokenizer.json filter=lfs diff=lfs merge=lfs -text
168
  meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-154/tokenizer.json filter=lfs diff=lfs merge=lfs -text
169
  meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-231/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 
 
167
  meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-77/tokenizer.json filter=lfs diff=lfs merge=lfs -text
168
  meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-154/tokenizer.json filter=lfs diff=lfs merge=lfs -text
169
  meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-231/tokenizer.json filter=lfs diff=lfs merge=lfs -text
170
+ meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-220/tokenizer.json filter=lfs diff=lfs merge=lfs -text
meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-220/README.md ADDED
@@ -0,0 +1,210 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: unsloth/Meta-Llama-3.1-8B
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:unsloth/Meta-Llama-3.1-8B
7
+ - grpo
8
+ - lora
9
+ - transformers
10
+ - trl
11
+ - unsloth
12
+ ---
13
+
14
+ # Model Card for Model ID
15
+
16
+ <!-- Provide a quick summary of what the model is/does. -->
17
+
18
+
19
+
20
+ ## Model Details
21
+
22
+ ### Model Description
23
+
24
+ <!-- Provide a longer summary of what this model is. -->
25
+
26
+
27
+
28
+ - **Developed by:** [More Information Needed]
29
+ - **Funded by [optional]:** [More Information Needed]
30
+ - **Shared by [optional]:** [More Information Needed]
31
+ - **Model type:** [More Information Needed]
32
+ - **Language(s) (NLP):** [More Information Needed]
33
+ - **License:** [More Information Needed]
34
+ - **Finetuned from model [optional]:** [More Information Needed]
35
+
36
+ ### Model Sources [optional]
37
+
38
+ <!-- Provide the basic links for the model. -->
39
+
40
+ - **Repository:** [More Information Needed]
41
+ - **Paper [optional]:** [More Information Needed]
42
+ - **Demo [optional]:** [More Information Needed]
43
+
44
+ ## Uses
45
+
46
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
47
+
48
+ ### Direct Use
49
+
50
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
51
+
52
+ [More Information Needed]
53
+
54
+ ### Downstream Use [optional]
55
+
56
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
57
+
58
+ [More Information Needed]
59
+
60
+ ### Out-of-Scope Use
61
+
62
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
63
+
64
+ [More Information Needed]
65
+
66
+ ## Bias, Risks, and Limitations
67
+
68
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
69
+
70
+ [More Information Needed]
71
+
72
+ ### Recommendations
73
+
74
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
75
+
76
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
77
+
78
+ ## How to Get Started with the Model
79
+
80
+ Use the code below to get started with the model.
81
+
82
+ [More Information Needed]
83
+
84
+ ## Training Details
85
+
86
+ ### Training Data
87
+
88
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
89
+
90
+ [More Information Needed]
91
+
92
+ ### Training Procedure
93
+
94
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
95
+
96
+ #### Preprocessing [optional]
97
+
98
+ [More Information Needed]
99
+
100
+
101
+ #### Training Hyperparameters
102
+
103
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
104
+
105
+ #### Speeds, Sizes, Times [optional]
106
+
107
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
108
+
109
+ [More Information Needed]
110
+
111
+ ## Evaluation
112
+
113
+ <!-- This section describes the evaluation protocols and provides the results. -->
114
+
115
+ ### Testing Data, Factors & Metrics
116
+
117
+ #### Testing Data
118
+
119
+ <!-- This should link to a Dataset Card if possible. -->
120
+
121
+ [More Information Needed]
122
+
123
+ #### Factors
124
+
125
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
126
+
127
+ [More Information Needed]
128
+
129
+ #### Metrics
130
+
131
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
132
+
133
+ [More Information Needed]
134
+
135
+ ### Results
136
+
137
+ [More Information Needed]
138
+
139
+ #### Summary
140
+
141
+
142
+
143
+ ## Model Examination [optional]
144
+
145
+ <!-- Relevant interpretability work for the model goes here -->
146
+
147
+ [More Information Needed]
148
+
149
+ ## Environmental Impact
150
+
151
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
152
+
153
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
154
+
155
+ - **Hardware Type:** [More Information Needed]
156
+ - **Hours used:** [More Information Needed]
157
+ - **Cloud Provider:** [More Information Needed]
158
+ - **Compute Region:** [More Information Needed]
159
+ - **Carbon Emitted:** [More Information Needed]
160
+
161
+ ## Technical Specifications [optional]
162
+
163
+ ### Model Architecture and Objective
164
+
165
+ [More Information Needed]
166
+
167
+ ### Compute Infrastructure
168
+
169
+ [More Information Needed]
170
+
171
+ #### Hardware
172
+
173
+ [More Information Needed]
174
+
175
+ #### Software
176
+
177
+ [More Information Needed]
178
+
179
+ ## Citation [optional]
180
+
181
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
182
+
183
+ **BibTeX:**
184
+
185
+ [More Information Needed]
186
+
187
+ **APA:**
188
+
189
+ [More Information Needed]
190
+
191
+ ## Glossary [optional]
192
+
193
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
194
+
195
+ [More Information Needed]
196
+
197
+ ## More Information [optional]
198
+
199
+ [More Information Needed]
200
+
201
+ ## Model Card Authors [optional]
202
+
203
+ [More Information Needed]
204
+
205
+ ## Model Card Contact
206
+
207
+ [More Information Needed]
208
+ ### Framework versions
209
+
210
+ - PEFT 0.18.1
meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-220/adapter_config.json ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": {
6
+ "base_model_class": "LlamaForCausalLM",
7
+ "parent_library": "transformers.models.llama.modeling_llama",
8
+ "unsloth_fixed": true
9
+ },
10
+ "base_model_name_or_path": "unsloth/Meta-Llama-3.1-8B",
11
+ "bias": "none",
12
+ "corda_config": null,
13
+ "ensure_weight_tying": false,
14
+ "eva_config": null,
15
+ "exclude_modules": null,
16
+ "fan_in_fan_out": false,
17
+ "inference_mode": true,
18
+ "init_lora_weights": true,
19
+ "layer_replication": null,
20
+ "layers_pattern": null,
21
+ "layers_to_transform": null,
22
+ "loftq_config": {},
23
+ "lora_alpha": 16,
24
+ "lora_bias": false,
25
+ "lora_dropout": 0.0,
26
+ "megatron_config": null,
27
+ "megatron_core": "megatron.core",
28
+ "modules_to_save": null,
29
+ "peft_type": "LORA",
30
+ "peft_version": "0.18.1",
31
+ "qalora_group_size": 16,
32
+ "r": 8,
33
+ "rank_pattern": {},
34
+ "revision": null,
35
+ "target_modules": [
36
+ "o_proj",
37
+ "q_proj",
38
+ "down_proj",
39
+ "up_proj",
40
+ "v_proj",
41
+ "k_proj",
42
+ "gate_proj"
43
+ ],
44
+ "target_parameters": null,
45
+ "task_type": "CAUSAL_LM",
46
+ "trainable_token_indices": null,
47
+ "use_dora": false,
48
+ "use_qalora": false,
49
+ "use_rslora": false
50
+ }
meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-220/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ba5fe45dd76ba2d63f36843c5ef14d45a725fa248266772d2d818f0d8cbb3e52
3
+ size 83945296
meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-220/chat_template.jinja ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ messages[0]['content'] + '
2
+
3
+ ' }}{% set loop_messages = messages[1:] %}{% else %}{{ '' + '
4
+
5
+ ' }}{% set loop_messages = messages %}{% endif %}{% for message in loop_messages %}{% if message['role'] == 'user' %}{{ '### Instruction:
6
+ ' + message['content'] + '
7
+
8
+ ' }}{% elif message['role'] == 'assistant' %}{{ '### Response:
9
+ ' + message['content'] + eos_token + '
10
+
11
+ ' }}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '### Response:
12
+ ' }}{% endif %}
meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-220/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d35db5d42d017fc2aef06e381e2a3e16698c00993db064c3030c14561a60328f
3
+ size 43127525
meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-220/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8c95696854e3fe97ec080b60a38ee1613b968c78f33054560734dd3020bcc4fc
3
+ size 14645
meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-220/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c2d5c3dd3aaaae891b57fa0720c44854d12d865a62197aa80e5715094bb26ed
3
+ size 1465
meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-220/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a65c6c5f9764771aa485e6a1f5e63d7d9af8477fe0777148c17476ecb2e09a05
3
+ size 17210099
meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-220/tokenizer_config.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "bos_token": "<|begin_of_text|>",
4
+ "clean_up_tokenization_spaces": true,
5
+ "eos_token": "<|end_of_text|>",
6
+ "from_slow": true,
7
+ "is_local": false,
8
+ "legacy": false,
9
+ "model_input_names": [
10
+ "input_ids",
11
+ "attention_mask"
12
+ ],
13
+ "model_max_length": 131072,
14
+ "pad_token": "<|finetune_right_pad_id|>",
15
+ "padding_side": "right",
16
+ "tokenizer_class": "TokenizersBackend",
17
+ "unk_token": null
18
+ }
meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-220/trainer_state.json ADDED
@@ -0,0 +1,259 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 0.02501136880400182,
6
+ "eval_steps": 500,
7
+ "global_step": 220,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "clip_ratio/high_max": 0.0,
14
+ "clip_ratio/high_mean": 0.0,
15
+ "clip_ratio/low_mean": 0.0,
16
+ "clip_ratio/low_min": 0.0,
17
+ "clip_ratio/region_mean": 0.0,
18
+ "completion_length": 40.52,
19
+ "completions/clipped_ratio": 0.0,
20
+ "completions/max_length": 41.69230769230769,
21
+ "completions/max_terminated_length": 41.69230769230769,
22
+ "completions/mean_length": 41.5,
23
+ "completions/mean_terminated_length": 41.5,
24
+ "completions/min_length": 41.38461538461539,
25
+ "completions/min_terminated_length": 41.38461538461539,
26
+ "epoch": 0.002842201000454752,
27
+ "frac_reward_zero_std": 0.6153846153846154,
28
+ "grad_norm": 0.3078555464744568,
29
+ "kl": 0.4130884063243866,
30
+ "learning_rate": 6.818181818181817e-08,
31
+ "loss": 0.009083011150360108,
32
+ "num_tokens": 16252.0,
33
+ "reward": 6.017776635976938e-09,
34
+ "reward_std": 0.4111711337016179,
35
+ "rewards/TranscriptCorrectionGrpoReward/mean": 6.017776635976938e-09,
36
+ "rewards/TranscriptCorrectionGrpoReward/std": 0.4111711337016179,
37
+ "step": 25
38
+ },
39
+ {
40
+ "clip_ratio/high_max": 0.0,
41
+ "clip_ratio/high_mean": 0.0,
42
+ "clip_ratio/low_mean": 0.0,
43
+ "clip_ratio/low_min": 0.0,
44
+ "clip_ratio/region_mean": 0.0,
45
+ "completion_length": 33.35,
46
+ "completions/clipped_ratio": 0.0,
47
+ "completions/max_length": 32.5,
48
+ "completions/max_terminated_length": 32.5,
49
+ "completions/mean_length": 31.989583333333332,
50
+ "completions/mean_terminated_length": 31.989583333333332,
51
+ "completions/min_length": 31.166666666666668,
52
+ "completions/min_terminated_length": 31.166666666666668,
53
+ "epoch": 0.005684402000909504,
54
+ "frac_reward_zero_std": 0.5,
55
+ "grad_norm": 0.00011760747293010354,
56
+ "kl": 0.5057662864029407,
57
+ "learning_rate": 1.3920454545454543e-07,
58
+ "loss": -0.008437809944152832,
59
+ "num_tokens": 28507.0,
60
+ "reward": 2.483526865641276e-09,
61
+ "reward_std": 0.5345224638779958,
62
+ "rewards/TranscriptCorrectionGrpoReward/mean": 2.483526865641276e-09,
63
+ "rewards/TranscriptCorrectionGrpoReward/std": 0.5345224738121033,
64
+ "step": 50
65
+ },
66
+ {
67
+ "clip_ratio/high_max": 0.0,
68
+ "clip_ratio/high_mean": 0.0,
69
+ "clip_ratio/low_mean": 0.0,
70
+ "clip_ratio/low_min": 0.0,
71
+ "clip_ratio/region_mean": 0.0,
72
+ "completion_length": 28.39,
73
+ "completions/clipped_ratio": 0.0,
74
+ "completions/max_length": 29.76923076923077,
75
+ "completions/max_terminated_length": 29.76923076923077,
76
+ "completions/mean_length": 29.682692307692307,
77
+ "completions/mean_terminated_length": 29.682692307692307,
78
+ "completions/min_length": 29.53846153846154,
79
+ "completions/min_terminated_length": 29.53846153846154,
80
+ "epoch": 0.008526603001364257,
81
+ "frac_reward_zero_std": 0.6923076923076923,
82
+ "grad_norm": 0.1779666692018509,
83
+ "kl": 0.5825921362638473,
84
+ "learning_rate": 2.1022727272727273e-07,
85
+ "loss": 0.005986777544021607,
86
+ "num_tokens": 40810.0,
87
+ "reward": 4.584972675030048e-09,
88
+ "reward_std": 0.32893691613123965,
89
+ "rewards/TranscriptCorrectionGrpoReward/mean": 4.584972675030048e-09,
90
+ "rewards/TranscriptCorrectionGrpoReward/std": 0.3289369069612943,
91
+ "step": 75
92
+ },
93
+ {
94
+ "clip_ratio/high_max": 0.0,
95
+ "clip_ratio/high_mean": 0.0,
96
+ "clip_ratio/low_mean": 0.0,
97
+ "clip_ratio/low_min": 0.0,
98
+ "clip_ratio/region_mean": 0.0,
99
+ "completion_length": 25.68,
100
+ "completions/clipped_ratio": 0.0,
101
+ "completions/max_length": 24.25,
102
+ "completions/max_terminated_length": 24.25,
103
+ "completions/mean_length": 24.166666666666668,
104
+ "completions/mean_terminated_length": 24.166666666666668,
105
+ "completions/min_length": 24.083333333333332,
106
+ "completions/min_terminated_length": 24.083333333333332,
107
+ "epoch": 0.011368804001819008,
108
+ "frac_reward_zero_std": 0.75,
109
+ "grad_norm": 0.0001858328323578462,
110
+ "kl": 0.5760334812104702,
111
+ "learning_rate": 2.8125e-07,
112
+ "loss": -0.0015538424253463746,
113
+ "num_tokens": 49954.0,
114
+ "reward": 2.483526865641276e-09,
115
+ "reward_std": 0.26726123690605164,
116
+ "rewards/TranscriptCorrectionGrpoReward/mean": 2.483526865641276e-09,
117
+ "rewards/TranscriptCorrectionGrpoReward/std": 0.26726123690605164,
118
+ "step": 100
119
+ },
120
+ {
121
+ "clip_ratio/high_max": 0.0,
122
+ "clip_ratio/high_mean": 0.0,
123
+ "clip_ratio/low_mean": 0.0,
124
+ "clip_ratio/low_min": 0.0,
125
+ "clip_ratio/region_mean": 0.0,
126
+ "completion_length": 23.0,
127
+ "completions/clipped_ratio": 0.0,
128
+ "completions/max_length": 24.076923076923077,
129
+ "completions/max_terminated_length": 24.076923076923077,
130
+ "completions/mean_length": 23.73076923076923,
131
+ "completions/mean_terminated_length": 23.73076923076923,
132
+ "completions/min_length": 23.307692307692307,
133
+ "completions/min_terminated_length": 23.307692307692307,
134
+ "epoch": 0.014211005002273762,
135
+ "frac_reward_zero_std": 0.6153846153846154,
136
+ "grad_norm": 0.00013943837257102132,
137
+ "kl": 0.6416070720553398,
138
+ "learning_rate": 3.5227272727272725e-07,
139
+ "loss": 0.0062805676460266115,
140
+ "num_tokens": 60550.0,
141
+ "reward": 5.7312158437875604e-09,
142
+ "reward_std": 0.4111711337016179,
143
+ "rewards/TranscriptCorrectionGrpoReward/mean": 5.7312158437875604e-09,
144
+ "rewards/TranscriptCorrectionGrpoReward/std": 0.4111711337016179,
145
+ "step": 125
146
+ },
147
+ {
148
+ "clip_ratio/high_max": 0.0,
149
+ "clip_ratio/high_mean": 0.0,
150
+ "clip_ratio/low_mean": 0.0,
151
+ "clip_ratio/low_min": 0.0,
152
+ "clip_ratio/region_mean": 0.0,
153
+ "completion_length": 33.42,
154
+ "completions/clipped_ratio": 0.0,
155
+ "completions/max_length": 33.25,
156
+ "completions/max_terminated_length": 33.25,
157
+ "completions/mean_length": 33.0625,
158
+ "completions/mean_terminated_length": 33.0625,
159
+ "completions/min_length": 32.583333333333336,
160
+ "completions/min_terminated_length": 32.583333333333336,
161
+ "epoch": 0.017053206002728513,
162
+ "frac_reward_zero_std": 0.6666666666666666,
163
+ "grad_norm": 0.00035186903551220894,
164
+ "kl": 0.47411411479115484,
165
+ "learning_rate": 4.232954545454545e-07,
166
+ "loss": 0.0006420502066612244,
167
+ "num_tokens": 72636.0,
168
+ "reward": -6.20881716410319e-10,
169
+ "reward_std": 0.35634831587473553,
170
+ "rewards/TranscriptCorrectionGrpoReward/mean": -6.20881716410319e-10,
171
+ "rewards/TranscriptCorrectionGrpoReward/std": 0.35634831587473553,
172
+ "step": 150
173
+ },
174
+ {
175
+ "clip_ratio/high_max": 0.0003773584961891174,
176
+ "clip_ratio/high_mean": 0.0003773584961891174,
177
+ "clip_ratio/low_mean": 0.0001886792480945587,
178
+ "clip_ratio/low_min": 0.0001886792480945587,
179
+ "clip_ratio/region_mean": 0.0005660377442836762,
180
+ "completion_length": 47.65,
181
+ "completions/clipped_ratio": 0.0,
182
+ "completions/max_length": 46.92307692307692,
183
+ "completions/max_terminated_length": 46.92307692307692,
184
+ "completions/mean_length": 46.70192307692308,
185
+ "completions/mean_terminated_length": 46.70192307692308,
186
+ "completions/min_length": 46.30769230769231,
187
+ "completions/min_terminated_length": 46.30769230769231,
188
+ "epoch": 0.019895407003183267,
189
+ "frac_reward_zero_std": 0.5384615384615384,
190
+ "grad_norm": 0.00017194119573105127,
191
+ "kl": 0.37818876296281817,
192
+ "learning_rate": 4.943181818181818e-07,
193
+ "loss": 0.0006822419166564942,
194
+ "num_tokens": 90573.0,
195
+ "reward": 2.292486337515024e-09,
196
+ "reward_std": 0.49340536961188686,
197
+ "rewards/TranscriptCorrectionGrpoReward/mean": 2.292486337515024e-09,
198
+ "rewards/TranscriptCorrectionGrpoReward/std": 0.4934053604419415,
199
+ "step": 175
200
+ },
201
+ {
202
+ "clip_ratio/high_max": 0.0,
203
+ "clip_ratio/high_mean": 0.0,
204
+ "clip_ratio/low_mean": 0.0,
205
+ "clip_ratio/low_min": 0.0,
206
+ "clip_ratio/region_mean": 0.0,
207
+ "completion_length": 25.62,
208
+ "completions/clipped_ratio": 0.0,
209
+ "completions/max_length": 25.75,
210
+ "completions/max_terminated_length": 25.75,
211
+ "completions/mean_length": 25.729166666666668,
212
+ "completions/mean_terminated_length": 25.729166666666668,
213
+ "completions/min_length": 25.666666666666668,
214
+ "completions/min_terminated_length": 25.666666666666668,
215
+ "epoch": 0.022737608003638016,
216
+ "frac_reward_zero_std": 0.9166666666666666,
217
+ "grad_norm": 0.00034925329964607954,
218
+ "kl": 0.5583685766160488,
219
+ "learning_rate": 5.65340909090909e-07,
220
+ "loss": 0.0031090858578681946,
221
+ "num_tokens": 100355.0,
222
+ "reward": 0.0,
223
+ "reward_std": 0.08908707896868388,
224
+ "rewards/TranscriptCorrectionGrpoReward/mean": 0.0,
225
+ "rewards/TranscriptCorrectionGrpoReward/std": 0.08908707896868388,
226
+ "step": 200
227
+ },
228
+ {
229
+ "eval_cer_subset": 0.016214743153018123,
230
+ "eval_cer_subset_edit_distance": 119,
231
+ "eval_cer_subset_groups": 75,
232
+ "eval_cer_subset_items": 75,
233
+ "eval_cer_subset_ref_chars": 7339,
234
+ "eval_loss": 0.05158098271155596,
235
+ "step": 220
236
+ }
237
+ ],
238
+ "logging_steps": 25,
239
+ "max_steps": 35184,
240
+ "num_input_tokens_seen": 108278,
241
+ "num_train_epochs": 4,
242
+ "save_steps": 220,
243
+ "stateful_callbacks": {
244
+ "TrainerControl": {
245
+ "args": {
246
+ "should_epoch_stop": false,
247
+ "should_evaluate": false,
248
+ "should_log": false,
249
+ "should_save": true,
250
+ "should_training_stop": false
251
+ },
252
+ "attributes": {}
253
+ }
254
+ },
255
+ "total_flos": 0.0,
256
+ "train_batch_size": 1,
257
+ "trial_name": null,
258
+ "trial_params": null
259
+ }
meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-220/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:129648314bf4204d2156894abc87bea3f3a4a10e3de86dd342e9703104e2b8e7
3
+ size 6801
meta-llama__meta-llama-3.1-8b/grpo/run_manifest.json CHANGED
@@ -1,37 +1,25 @@
1
  {
2
  "schema_version": 1,
3
- "run_id": "2f808075-59c7-4825-afda-13ec73ace137",
4
  "run_type": "grpo",
5
  "profile_name": "llama3_1_8b_base_lessdata_grpo",
6
  "model_name": "meta-llama/Meta-Llama-3.1-8B",
7
  "status": "running",
8
  "best_metric_name": "eval_cer_subset",
9
  "best_metric_value": 0.016214743153018123,
10
- "best_step": 77,
11
- "best_checkpoint_step": 77,
12
- "latest_checkpoint_step": 231,
13
  "checkpoints": [
14
  {
15
- "step": 77,
16
- "epoch": 0.008753979081400637,
17
- "repo_path": "meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-77",
18
- "timestamp": "2026-04-09T23:23:53.593935+00:00"
19
- },
20
- {
21
- "step": 154,
22
- "epoch": 0.017507958162801273,
23
- "repo_path": "meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-154",
24
- "timestamp": "2026-04-09T23:25:43.129048+00:00"
25
- },
26
- {
27
- "step": 231,
28
- "epoch": 0.02626193724420191,
29
- "repo_path": "meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-231",
30
- "timestamp": "2026-04-09T23:27:40.426505+00:00"
31
  }
32
  ],
33
  "epoch_artifacts": [],
34
  "resume_history": [],
35
- "created_at": "2026-04-09T23:20:32.422192+00:00",
36
- "updated_at": "2026-04-09T23:27:40.426520+00:00"
37
  }
 
1
  {
2
  "schema_version": 1,
3
+ "run_id": "aa5ad4e1-1030-4baf-82ab-6949c69a6216",
4
  "run_type": "grpo",
5
  "profile_name": "llama3_1_8b_base_lessdata_grpo",
6
  "model_name": "meta-llama/Meta-Llama-3.1-8B",
7
  "status": "running",
8
  "best_metric_name": "eval_cer_subset",
9
  "best_metric_value": 0.016214743153018123,
10
+ "best_step": 220,
11
+ "best_checkpoint_step": 220,
12
+ "latest_checkpoint_step": 220,
13
  "checkpoints": [
14
  {
15
+ "step": 220,
16
+ "epoch": 0.02501136880400182,
17
+ "repo_path": "meta-llama__meta-llama-3.1-8b/grpo/checkpoints/checkpoint-220",
18
+ "timestamp": "2026-04-09T23:34:20.553650+00:00"
 
 
 
 
 
 
 
 
 
 
 
 
19
  }
20
  ],
21
  "epoch_artifacts": [],
22
  "resume_history": [],
23
+ "created_at": "2026-04-09T23:28:11.374630+00:00",
24
+ "updated_at": "2026-04-09T23:34:20.553666+00:00"
25
  }