Hugging Face's logo

fernandals
/
results

Text2Text Generation
PEFT
Safetensors
Portuguese
trl
sft
Generated from Trainer
Model card Files Community
fernandals commited on
Commit
589ce47
•
1 Parent(s): 726d552

fernandals/llama-2-13b-news-gen

Browse files
Files changed (5) hide show
  1. README.md +41 -41
  2. adapter_config.json +6 -6
  3. adapter_model.safetensors +2 -2
  4. tokenizer_config.json +1 -0
  5. training_args.bin +2 -2
README.md CHANGED
@@ -4,7 +4,7 @@ tags:
4
  - trl
5
  - sft
6
  - generated_from_trainer
7
- base_model: NousResearch/Llama-2-7b-hf
8
  model-index:
9
  - name: results
10
  results: []
@@ -15,9 +15,9 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  # results
17
 
18
- This model is a fine-tuned version of [NousResearch/Llama-2-7b-hf](https://huggingface.co/NousResearch/Llama-2-7b-hf) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
- - Loss: 0.9343
21
 
22
  ## Model description
23
 
@@ -51,44 +51,44 @@ The following hyperparameters were used during training:
51
 
52
  | Training Loss | Epoch | Step | Validation Loss |
53
  |:-------------:|:-----:|:----:|:---------------:|
54
- | 1.7949 | 0.1 | 25 | 1.7390 |
55
- | 1.7268 | 0.21 | 50 | 1.6898 |
56
- | 1.7214 | 0.31 | 75 | 1.6547 |
57
- | 1.6573 | 0.41 | 100 | 1.6260 |
58
- | 1.6663 | 0.52 | 125 | 1.5999 |
59
- | 1.6499 | 0.62 | 150 | 1.5735 |
60
- | 1.61 | 0.72 | 175 | 1.5483 |
61
- | 1.6491 | 0.82 | 200 | 1.5288 |
62
- | 1.6441 | 0.93 | 225 | 1.5034 |
63
- | 1.5055 | 1.03 | 250 | 1.4753 |
64
- | 1.4955 | 1.13 | 275 | 1.4409 |
65
- | 1.4345 | 1.24 | 300 | 1.4159 |
66
- | 1.449 | 1.34 | 325 | 1.3918 |
67
- | 1.4762 | 1.44 | 350 | 1.3701 |
68
- | 1.4562 | 1.55 | 375 | 1.3449 |
69
- | 1.4092 | 1.65 | 400 | 1.3205 |
70
- | 1.4173 | 1.75 | 425 | 1.3000 |
71
- | 1.4225 | 1.86 | 450 | 1.2798 |
72
- | 1.4358 | 1.96 | 475 | 1.2600 |
73
- | 1.2387 | 2.06 | 500 | 1.2198 |
74
- | 1.232 | 2.16 | 525 | 1.1936 |
75
- | 1.2212 | 2.27 | 550 | 1.1720 |
76
- | 1.2066 | 2.37 | 575 | 1.1491 |
77
- | 1.199 | 2.47 | 600 | 1.1301 |
78
- | 1.2365 | 2.58 | 625 | 1.1124 |
79
- | 1.2124 | 2.68 | 650 | 1.0934 |
80
- | 1.2113 | 2.78 | 675 | 1.0766 |
81
- | 1.1947 | 2.89 | 700 | 1.0620 |
82
- | 1.2041 | 2.99 | 725 | 1.0441 |
83
- | 1.0437 | 3.09 | 750 | 1.0133 |
84
- | 1.0239 | 3.2 | 775 | 0.9972 |
85
- | 1.0417 | 3.3 | 800 | 0.9839 |
86
- | 0.9909 | 3.4 | 825 | 0.9712 |
87
- | 1.0275 | 3.51 | 850 | 0.9600 |
88
- | 1.0466 | 3.61 | 875 | 0.9516 |
89
- | 1.0036 | 3.71 | 900 | 0.9423 |
90
- | 1.0039 | 3.81 | 925 | 0.9363 |
91
- | 0.9786 | 3.92 | 950 | 0.9343 |
92
 
93
 
94
  ### Framework versions
 
4
  - trl
5
  - sft
6
  - generated_from_trainer
7
+ base_model: NousResearch/Llama-2-13b-hf
8
  model-index:
9
  - name: results
10
  results: []
 
15
 
16
  # results
17
 
18
+ This model is a fine-tuned version of [NousResearch/Llama-2-13b-hf](https://huggingface.co/NousResearch/Llama-2-13b-hf) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 0.7605
21
 
22
  ## Model description
23
 
 
51
 
52
  | Training Loss | Epoch | Step | Validation Loss |
53
  |:-------------:|:-----:|:----:|:---------------:|
54
+ | 1.6794 | 0.1 | 25 | 1.6358 |
55
+ | 1.6218 | 0.21 | 50 | 1.5895 |
56
+ | 1.6211 | 0.31 | 75 | 1.5559 |
57
+ | 1.5658 | 0.41 | 100 | 1.5291 |
58
+ | 1.5722 | 0.52 | 125 | 1.5006 |
59
+ | 1.5555 | 0.62 | 150 | 1.4762 |
60
+ | 1.5135 | 0.72 | 175 | 1.4504 |
61
+ | 1.5607 | 0.82 | 200 | 1.4317 |
62
+ | 1.5449 | 0.93 | 225 | 1.4058 |
63
+ | 1.4032 | 1.03 | 250 | 1.3746 |
64
+ | 1.3898 | 1.13 | 275 | 1.3365 |
65
+ | 1.3305 | 1.24 | 300 | 1.3098 |
66
+ | 1.3502 | 1.34 | 325 | 1.2836 |
67
+ | 1.3796 | 1.44 | 350 | 1.2635 |
68
+ | 1.3519 | 1.55 | 375 | 1.2330 |
69
+ | 1.3051 | 1.65 | 400 | 1.2084 |
70
+ | 1.3192 | 1.75 | 425 | 1.1841 |
71
+ | 1.3232 | 1.86 | 450 | 1.1621 |
72
+ | 1.3408 | 1.96 | 475 | 1.1412 |
73
+ | 1.1116 | 2.06 | 500 | 1.0925 |
74
+ | 1.107 | 2.16 | 525 | 1.0628 |
75
+ | 1.0926 | 2.27 | 550 | 1.0387 |
76
+ | 1.0772 | 2.37 | 575 | 1.0121 |
77
+ | 1.0742 | 2.47 | 600 | 0.9918 |
78
+ | 1.109 | 2.58 | 625 | 0.9706 |
79
+ | 1.0826 | 2.68 | 650 | 0.9479 |
80
+ | 1.0857 | 2.78 | 675 | 0.9281 |
81
+ | 1.0705 | 2.89 | 700 | 0.9112 |
82
+ | 1.0742 | 2.99 | 725 | 0.8928 |
83
+ | 0.8945 | 3.09 | 750 | 0.8542 |
84
+ | 0.861 | 3.2 | 775 | 0.8344 |
85
+ | 0.8869 | 3.3 | 800 | 0.8198 |
86
+ | 0.8357 | 3.4 | 825 | 0.8039 |
87
+ | 0.8726 | 3.51 | 850 | 0.7924 |
88
+ | 0.8857 | 3.61 | 875 | 0.7809 |
89
+ | 0.8437 | 3.71 | 900 | 0.7702 |
90
+ | 0.8384 | 3.81 | 925 | 0.7637 |
91
+ | 0.819 | 3.92 | 950 | 0.7605 |
92
 
93
 
94
  ### Framework versions
adapter_config.json CHANGED
@@ -1,7 +1,7 @@
1
  {
2
  "alpha_pattern": {},
3
  "auto_mapping": null,
4
- "base_model_name_or_path": "NousResearch/Llama-2-7b-hf",
5
  "bias": "none",
6
  "fan_in_fan_out": false,
7
  "inference_mode": true,
@@ -19,13 +19,13 @@
19
  "rank_pattern": {},
20
  "revision": null,
21
  "target_modules": [
22
- "o_proj",
 
23
  "v_proj",
24
  "down_proj",
25
- "q_proj",
26
- "gate_proj",
27
- "up_proj",
28
- "k_proj"
29
  ],
30
  "task_type": "CAUSAL_LM",
31
  "use_rslora": false
 
1
  {
2
  "alpha_pattern": {},
3
  "auto_mapping": null,
4
+ "base_model_name_or_path": "NousResearch/Llama-2-13b-hf",
5
  "bias": "none",
6
  "fan_in_fan_out": false,
7
  "inference_mode": true,
 
19
  "rank_pattern": {},
20
  "revision": null,
21
  "target_modules": [
22
+ "gate_proj",
23
+ "q_proj",
24
  "v_proj",
25
  "down_proj",
26
+ "o_proj",
27
+ "k_proj",
28
+ "up_proj"
 
29
  ],
30
  "task_type": "CAUSAL_LM",
31
  "use_rslora": false
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5df1bc65e154b644b1a56fcf37cb7942e679546004264bc460bb5f87af55f6a2
3
- size 159967880
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:295c2d38f57332b2a47d5d7a04ba482b1eab8463c43187035cc8609a1451337f
3
+ size 250422888
tokenizer_config.json CHANGED
@@ -34,6 +34,7 @@
34
  "model_max_length": 1000000000000000019884624838656,
35
  "pad_token": "<unk>",
36
  "sp_model_kwargs": {},
 
37
  "tokenizer_class": "LlamaTokenizer",
38
  "unk_token": "<unk>",
39
  "use_default_system_prompt": false
 
34
  "model_max_length": 1000000000000000019884624838656,
35
  "pad_token": "<unk>",
36
  "sp_model_kwargs": {},
37
+ "spaces_between_special_tokens": false,
38
  "tokenizer_class": "LlamaTokenizer",
39
  "unk_token": "<unk>",
40
  "use_default_system_prompt": false
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:dddf22dc36607eaa1d9fd5d263b08aee52ded9a1d78ccf73d7c13bb63284cff4
3
- size 4664
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9a2e468e01326146efec19475305d6455a46aa47a0db955057ae929a355c378a
3
+ size 4728