Heralax commited on
Commit
ade8e62
1 Parent(s): d9b0908

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -45
README.md CHANGED
@@ -86,48 +86,4 @@ special_tokens:
86
 
87
  </details><br>
88
 
89
- # army-pretraining
90
-
91
- This model is a fine-tuned version of [alpindale/Mistral-7B-v0.2-hf](https://huggingface.co/alpindale/Mistral-7B-v0.2-hf) on the None dataset.
92
-
93
- ## Model description
94
-
95
- More information needed
96
-
97
- ## Intended uses & limitations
98
-
99
- More information needed
100
-
101
- ## Training and evaluation data
102
-
103
- More information needed
104
-
105
- ## Training procedure
106
-
107
- ### Training hyperparameters
108
-
109
- The following hyperparameters were used during training:
110
- - learning_rate: 2e-05
111
- - train_batch_size: 2
112
- - eval_batch_size: 1
113
- - seed: 42
114
- - distributed_type: multi-GPU
115
- - num_devices: 5
116
- - gradient_accumulation_steps: 6
117
- - total_train_batch_size: 60
118
- - total_eval_batch_size: 5
119
- - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
120
- - lr_scheduler_type: cosine
121
- - lr_scheduler_warmup_steps: 21
122
- - num_epochs: 11
123
-
124
- ### Training results
125
-
126
-
127
-
128
- ### Framework versions
129
-
130
- - Transformers 4.45.1
131
- - Pytorch 2.3.1+cu121
132
- - Datasets 2.21.0
133
- - Tokenizers 0.20.0
 
86
 
87
  </details><br>
88
 
89
+ pretrained base for Mannerstral 7b, only use if you are finetuning something on top of it.