Hugging Face's logo

ayousanz
/
japanese-mistral-0.3b-base

Text Generation
Transformers
Safetensors
mistral
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Community
ayousanz commited on
Commit
3a7c007
•
1 Parent(s): 583bab4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +116 -0
README.md CHANGED
@@ -1,3 +1,119 @@
1
  ---
 
 
 
 
 
 
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ base_model: None
3
+ tags:
4
+ - generated_from_trainer
5
+ model-index:
6
+ - name: checkpoints-mistral-0.3b
7
+ results: []
8
  license: apache-2.0
9
  ---
10
+
11
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
+ should probably proofread and complete it, then remove this comment. -->
13
+
14
+ # checkpoints-mistral-300M
15
+
16
+ This model is a fine-tuned version of [None](https://huggingface.co/None) on an unknown dataset.
17
+ It achieves the following results on the evaluation set:
18
+ - Loss: 2.4867
19
+
20
+ ## Model description
21
+
22
+ More information needed
23
+
24
+ ## Intended uses & limitations
25
+
26
+ More information needed
27
+
28
+ ## Training and evaluation data
29
+
30
+ More information needed
31
+
32
+ ## Training procedure
33
+
34
+ ### Training hyperparameters
35
+
36
+ The following hyperparameters were used during training:
37
+ - learning_rate: 0.0003
38
+ - train_batch_size: 6
39
+ - eval_batch_size: 6
40
+ - seed: 42
41
+ - distributed_type: multi-GPU
42
+ - num_devices: 2
43
+ - gradient_accumulation_steps: 16
44
+ - total_train_batch_size: 192
45
+ - total_eval_batch_size: 12
46
+ - optimizer: Adam with betas=(0.9,0.95) and epsilon=0.0001
47
+ - lr_scheduler_type: cosine
48
+ - lr_scheduler_warmup_steps: 4
49
+ - num_epochs: 6
50
+ - mixed_precision_training: Native AMP
51
+
52
+ ### Training results
53
+
54
+ | Training Loss | Epoch | Step | Validation Loss |
55
+ |:-------------:|:-----:|:-----:|:---------------:|
56
+ | 4.5141 | 0.09 | 1000 | 4.5160 |
57
+ | 3.7879 | 0.18 | 2000 | 3.8531 |
58
+ | 3.5484 | 0.27 | 3000 | 3.5881 |
59
+ | 3.3734 | 0.36 | 4000 | 3.4287 |
60
+ | 3.2722 | 0.45 | 5000 | 3.3144 |
61
+ | 3.2276 | 0.54 | 6000 | 3.2299 |
62
+ | 3.1809 | 0.63 | 7000 | 3.1597 |
63
+ | 3.0706 | 0.72 | 8000 | 3.1043 |
64
+ | 3.0185 | 0.81 | 9000 | 3.0578 |
65
+ | 2.9496 | 0.9 | 10000 | 3.0157 |
66
+ | 2.9374 | 0.99 | 11000 | 2.9815 |
67
+ | 2.8794 | 1.08 | 12000 | 2.9487 |
68
+ | 2.8407 | 1.17 | 13000 | 2.9229 |
69
+ | 2.8818 | 1.26 | 14000 | 2.8973 |
70
+ | 2.8167 | 1.35 | 15000 | 2.8730 |
71
+ | 2.7941 | 1.44 | 16000 | 2.8515 |
72
+ | 2.7878 | 1.53 | 17000 | 2.8311 |
73
+ | 2.7894 | 1.62 | 18000 | 2.8113 |
74
+ | 2.7158 | 1.71 | 19000 | 2.7935 |
75
+ | 2.7409 | 1.8 | 20000 | 2.7765 |
76
+ | 2.7349 | 1.89 | 21000 | 2.7613 |
77
+ | 2.6631 | 1.98 | 22000 | 2.7451 |
78
+ | 2.6766 | 2.07 | 23000 | 2.7353 |
79
+ | 2.6405 | 2.16 | 24000 | 2.7231 |
80
+ | 2.6707 | 2.25 | 25000 | 2.7121 |
81
+ | 2.6362 | 2.34 | 26000 | 2.7005 |
82
+ | 2.5997 | 2.43 | 27000 | 2.6904 |
83
+ | 2.6549 | 2.52 | 28000 | 2.6798 |
84
+ | 2.6056 | 2.61 | 29000 | 2.6688 |
85
+ | 2.5722 | 2.7 | 30000 | 2.6594 |
86
+ | 2.6179 | 2.79 | 31000 | 2.6509 |
87
+ | 2.6064 | 2.88 | 32000 | 2.6423 |
88
+ | 2.5836 | 2.97 | 33000 | 2.6340 |
89
+ | 2.5502 | 3.06 | 34000 | 2.6285 |
90
+ | 2.5428 | 3.15 | 35000 | 2.6218 |
91
+ | 2.5342 | 3.24 | 36000 | 2.6160 |
92
+ | 2.5152 | 3.33 | 37000 | 2.6090 |
93
+ | 2.5138 | 5.13 | 38000 | 2.5766 |
94
+ | 2.5032 | 5.27 | 39000 | 2.5683 |
95
+ | 2.4783 | 5.4 | 40000 | 2.5609 |
96
+ | 2.4519 | 5.54 | 41000 | 2.5545 |
97
+ | 2.4918 | 5.67 | 42000 | 2.5472 |
98
+ | 2.4591 | 5.81 | 43000 | 2.5411 |
99
+ | 2.4756 | 5.94 | 44000 | 2.5354 |
100
+ | 2.4434 | 6.08 | 45000 | 2.5345 |
101
+ | 2.4312 | 6.21 | 46000 | 2.5301 |
102
+ | 2.4576 | 6.35 | 47000 | 2.5242 |
103
+ | 2.4343 | 6.48 | 48000 | 2.5192 |
104
+ | 2.426 | 6.62 | 49000 | 2.5139 |
105
+ | 2.4136 | 6.75 | 50000 | 2.5084 |
106
+ | 2.4463 | 6.89 | 51000 | 2.5037 |
107
+ | 2.345 | 7.02 | 52000 | 2.5016 |
108
+ | 2.3736 | 7.16 | 53000 | 2.4990 |
109
+ | 2.4092 | 7.29 | 54000 | 2.4955 |
110
+ | 2.3689 | 7.43 | 55000 | 2.4917 |
111
+ | 2.3797 | 7.56 | 56000 | 2.4867 |
112
+
113
+
114
+ ### Framework versions
115
+
116
+ - Transformers 4.35.2
117
+ - Pytorch 2.1.2+cu121
118
+ - Datasets 2.14.5
119
+ - Tokenizers 0.14.1