End of training
Browse files- README.md +8 -9
- generation_config.json +3 -3
README.md
CHANGED
|
@@ -1,7 +1,6 @@
|
|
| 1 |
---
|
| 2 |
library_name: transformers
|
| 3 |
-
|
| 4 |
-
base_model: facebook/opt-125m
|
| 5 |
tags:
|
| 6 |
- axolotl
|
| 7 |
- generated_from_trainer
|
|
@@ -20,7 +19,7 @@ should probably proofread and complete it, then remove this comment. -->
|
|
| 20 |
|
| 21 |
axolotl version: `0.6.0`
|
| 22 |
```yaml
|
| 23 |
-
base_model:
|
| 24 |
batch_size: 256
|
| 25 |
bf16: true
|
| 26 |
chat_template: tokenizer_default_fallback_alpaca
|
|
@@ -58,13 +57,13 @@ sample_packing: true
|
|
| 58 |
save_steps: 40
|
| 59 |
save_total_limit: 2
|
| 60 |
sequence_len: 2048
|
| 61 |
-
tokenizer_type:
|
| 62 |
torch_dtype: bf16
|
| 63 |
trust_remote_code: true
|
| 64 |
val_set_size: 0.1
|
| 65 |
wandb_entity: ''
|
| 66 |
wandb_mode: online
|
| 67 |
-
wandb_name:
|
| 68 |
wandb_project: Gradients-On-Demand
|
| 69 |
wandb_run: your_name
|
| 70 |
wandb_runid: default
|
|
@@ -76,9 +75,9 @@ warmup_ratio: 0.05
|
|
| 76 |
|
| 77 |
# test-repo
|
| 78 |
|
| 79 |
-
This model is a fine-tuned version of [
|
| 80 |
It achieves the following results on the evaluation set:
|
| 81 |
-
- Loss:
|
| 82 |
|
| 83 |
## Model description
|
| 84 |
|
|
@@ -113,8 +112,8 @@ The following hyperparameters were used during training:
|
|
| 113 |
|
| 114 |
| Training Loss | Epoch | Step | Validation Loss |
|
| 115 |
|:-------------:|:-------:|:----:|:---------------:|
|
| 116 |
-
| No log | 0.3333 | 1 |
|
| 117 |
-
|
|
| 118 |
|
| 119 |
|
| 120 |
### Framework versions
|
|
|
|
| 1 |
---
|
| 2 |
library_name: transformers
|
| 3 |
+
base_model: trl-internal-testing/tiny-random-LlamaForCausalLM
|
|
|
|
| 4 |
tags:
|
| 5 |
- axolotl
|
| 6 |
- generated_from_trainer
|
|
|
|
| 19 |
|
| 20 |
axolotl version: `0.6.0`
|
| 21 |
```yaml
|
| 22 |
+
base_model: trl-internal-testing/tiny-random-LlamaForCausalLM
|
| 23 |
batch_size: 256
|
| 24 |
bf16: true
|
| 25 |
chat_template: tokenizer_default_fallback_alpaca
|
|
|
|
| 57 |
save_steps: 40
|
| 58 |
save_total_limit: 2
|
| 59 |
sequence_len: 2048
|
| 60 |
+
tokenizer_type: LlamaTokenizerFast
|
| 61 |
torch_dtype: bf16
|
| 62 |
trust_remote_code: true
|
| 63 |
val_set_size: 0.1
|
| 64 |
wandb_entity: ''
|
| 65 |
wandb_mode: online
|
| 66 |
+
wandb_name: trl-internal-testing/tiny-random-LlamaForCausalLM-argilla/databricks-dolly-15k-curated-en
|
| 67 |
wandb_project: Gradients-On-Demand
|
| 68 |
wandb_run: your_name
|
| 69 |
wandb_runid: default
|
|
|
|
| 75 |
|
| 76 |
# test-repo
|
| 77 |
|
| 78 |
+
This model is a fine-tuned version of [trl-internal-testing/tiny-random-LlamaForCausalLM](https://huggingface.co/trl-internal-testing/tiny-random-LlamaForCausalLM) on the argilla/databricks-dolly-15k-curated-en dataset.
|
| 79 |
It achieves the following results on the evaluation set:
|
| 80 |
+
- Loss: 9.6817
|
| 81 |
|
| 82 |
## Model description
|
| 83 |
|
|
|
|
| 112 |
|
| 113 |
| Training Loss | Epoch | Step | Validation Loss |
|
| 114 |
|:-------------:|:-------:|:----:|:---------------:|
|
| 115 |
+
| No log | 0.3333 | 1 | 10.3756 |
|
| 116 |
+
| 9.6746 | 66.6667 | 200 | 9.6817 |
|
| 117 |
|
| 118 |
|
| 119 |
### Framework versions
|
generation_config.json
CHANGED
|
@@ -1,8 +1,8 @@
|
|
| 1 |
{
|
| 2 |
"_from_model_config": true,
|
| 3 |
-
"bos_token_id":
|
| 4 |
"do_sample": true,
|
| 5 |
-
"eos_token_id":
|
| 6 |
-
"pad_token_id":
|
| 7 |
"transformers_version": "4.48.1"
|
| 8 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"_from_model_config": true,
|
| 3 |
+
"bos_token_id": 0,
|
| 4 |
"do_sample": true,
|
| 5 |
+
"eos_token_id": 1,
|
| 6 |
+
"pad_token_id": 2,
|
| 7 |
"transformers_version": "4.48.1"
|
| 8 |
}
|