Update README.md
Browse files
README.md
CHANGED
@@ -17,8 +17,8 @@ tags: []
|
|
17 |
|
18 |
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
|
19 |
|
20 |
-
- **Developed by:**
|
21 |
-
- **Finetuned from model
|
22 |
|
23 |
|
24 |
## Uses
|
@@ -36,18 +36,16 @@ This is the model card of a 🤗 transformers model that has been pushed on the
|
|
36 |
|
37 |
<!-- We used https://huggingface.co/datasets/arcee-ai/sec-data-full which consists of 2.5B tokens -->
|
38 |
|
39 |
-
[More Information Needed]
|
40 |
-
|
41 |
### Training Procedure
|
42 |
|
43 |
<!-- We used the CPT method in the hugging face alignment book -->
|
44 |
|
45 |
-
#### Preprocessing
|
46 |
|
47 |
-
|
48 |
|
49 |
|
50 |
#### Training Hyperparameters
|
51 |
|
52 |
-
- **Training regime:**
|
53 |
|
|
|
17 |
|
18 |
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
|
19 |
|
20 |
+
- **Developed by:** arcee.ai
|
21 |
+
- **Finetuned from model:** mistralai/Mistral-7B-Instruct-v0.2
|
22 |
|
23 |
|
24 |
## Uses
|
|
|
36 |
|
37 |
<!-- We used https://huggingface.co/datasets/arcee-ai/sec-data-full which consists of 2.5B tokens -->
|
38 |
|
|
|
|
|
39 |
### Training Procedure
|
40 |
|
41 |
<!-- We used the CPT method in the hugging face alignment book -->
|
42 |
|
43 |
+
#### Preprocessing
|
44 |
|
45 |
+
Tokenized and packed into 4096 chunks with the SFT trainer
|
46 |
|
47 |
|
48 |
#### Training Hyperparameters
|
49 |
|
50 |
+
- **Training regime:** Used SFT trainer <!-- bf16 mixed precision -->
|
51 |
|