jeff31415 commited on
Commit
d14f4ed
1 Parent(s): 97062b2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -8
README.md CHANGED
@@ -2,16 +2,23 @@
2
  license: apache-2.0
3
  datasets:
4
  - Open-Orca/OpenOrca
5
- - tiiuae/falcon-refinedweb
6
  - bigcode/starcoderdata
 
7
  language:
8
  - en
9
  ---
10
- Base model from PY007/TinyLlama-1.1B-intermediate-step-480k-1T
11
- Fine tuned on OpenOrca GPT4 subset for 1 epoch.
12
- Using CHATML format
13
- Model License: Apache 2.0, following the TinyLlama base model.
14
 
15
- Hardware and training spec:
16
- Hardware: 1*RTX A5000, ~16 hours to complete 1 epoch. GPU from autodl.com, cost around $3 for this finetuning.
17
- Training details:https://wandb.ai/jeff200402/TinyLlama-Orca?workspace=
 
 
 
 
 
 
 
 
 
 
 
 
2
  license: apache-2.0
3
  datasets:
4
  - Open-Orca/OpenOrca
 
5
  - bigcode/starcoderdata
6
+ - cerebras/SlimPajama-627B
7
  language:
8
  - en
9
  ---
 
 
 
 
10
 
11
+ [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
12
+
13
+ #### Base model:
14
+ PY007/TinyLlama-1.1B-intermediate-step-480k-1T
15
+
16
+ #### Dataset:
17
+ Fine tuned on OpenOrca GPT4 subset for 1 epoch,Using CHATML format
18
+
19
+ #### Model License:
20
+ Apache 2.0, following the TinyLlama base model.
21
+
22
+ #### Hardware and training details:
23
+ Hardware: 1*RTX A5000, ~16 hours to complete 1 epoch. GPU from autodl.com, cost around $3 for this finetuning.
24
+ https://wandb.ai/jeff200402/TinyLlama-Orca?workspace= for more details.