Hugging Face's logo

NowaBwagel0
/
llama-68m-finetuned-wikitext2

Text Generation
Transformers
TensorBoard
Safetensors
llama
Generated from Trainer
Inference Endpoints
text-generation-inference
Model card Files Metrics Community
NowaBwagel0 commited on
Commit
3bb5343
•
1 Parent(s): 5b74238

End of training

Browse files
Files changed (4) hide show
  1. README.md +18 -6
  2. model.safetensors +1 -1
  3. runs/Jan01_17-58-54_ad169494b3b2/events.out.tfevents.1704131940.ad169494b3b2.942.3 +2 -2
  4. runs/Jan01_17-58-54_ad169494b3b2/events.out.tfevents.1704136101.ad169494b3b2.942.4 +3 -0
README.md CHANGED
@@ -15,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [JackFram/llama-68m](https://huggingface.co/JackFram/llama-68m) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 3.0545
19
 
20
  ## Model description
21
 
@@ -34,21 +34,33 @@ More information needed
34
  ### Training hyperparameters
35
 
36
  The following hyperparameters were used during training:
37
- - learning_rate: 2e-05
38
  - train_batch_size: 32
39
  - eval_batch_size: 32
40
  - seed: 42
41
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
  - lr_scheduler_type: linear
43
- - num_epochs: 3
44
 
45
  ### Training results
46
 
47
  | Training Loss | Epoch | Step | Validation Loss |
48
  |:-------------:|:-----:|:----:|:---------------:|
49
- | No log | 1.0 | 175 | 3.0891 |
50
- | No log | 2.0 | 350 | 3.0605 |
51
- | 3.0721 | 3.0 | 525 | 3.0545 |
 
 
 
 
 
 
 
 
 
 
 
 
52
 
53
 
54
  ### Framework versions
 
15
 
16
  This model is a fine-tuned version of [JackFram/llama-68m](https://huggingface.co/JackFram/llama-68m) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 5.9290
19
 
20
  ## Model description
21
 
 
34
  ### Training hyperparameters
35
 
36
  The following hyperparameters were used during training:
37
+ - learning_rate: 0.0003
38
  - train_batch_size: 32
39
  - eval_batch_size: 32
40
  - seed: 42
41
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
  - lr_scheduler_type: linear
43
+ - num_epochs: 15
44
 
45
  ### Training results
46
 
47
  | Training Loss | Epoch | Step | Validation Loss |
48
  |:-------------:|:-----:|:----:|:---------------:|
49
+ | No log | 1.0 | 175 | 3.1817 |
50
+ | No log | 2.0 | 350 | 3.2699 |
51
+ | 2.5169 | 3.0 | 525 | 3.4555 |
52
+ | 2.5169 | 4.0 | 700 | 3.6923 |
53
+ | 2.5169 | 5.0 | 875 | 3.9542 |
54
+ | 1.5624 | 6.0 | 1050 | 4.2089 |
55
+ | 1.5624 | 7.0 | 1225 | 4.4777 |
56
+ | 1.5624 | 8.0 | 1400 | 4.7426 |
57
+ | 0.841 | 9.0 | 1575 | 4.9898 |
58
+ | 0.841 | 10.0 | 1750 | 5.2385 |
59
+ | 0.841 | 11.0 | 1925 | 5.4648 |
60
+ | 0.3788 | 12.0 | 2100 | 5.6607 |
61
+ | 0.3788 | 13.0 | 2275 | 5.8074 |
62
+ | 0.3788 | 14.0 | 2450 | 5.8933 |
63
+ | 0.1474 | 15.0 | 2625 | 5.9290 |
64
 
65
 
66
  ### Framework versions
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5942e0869c86a469b3f48dd6f9d216164bcd1c95092b5270817732170f85fc25
3
  size 272123144
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1cf86b5601fb087555bd7ea7d7e717e591a2c5a3695c085cbb6c9204e8d3b1b6
3
  size 272123144
runs/Jan01_17-58-54_ad169494b3b2/events.out.tfevents.1704131940.ad169494b3b2.942.3 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e5c4a79abe38f477bb06a8562a71a343d4f27da44ee5d419ce028d49212859db
3
- size 8855
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f63fae7a45cbdf63c8f21eca4405f1057146eb63661dfd1438116677a7bfccc
3
+ size 9480
runs/Jan01_17-58-54_ad169494b3b2/events.out.tfevents.1704136101.ad169494b3b2.942.4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8fe1a6e53641dde76234d1bff21191198336c45d5c5c64ddffd96c2f53ae352c
3
+ size 359