pszemraj commited on
Commit
7d50bcf
1 Parent(s): cde0db1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -12,9 +12,9 @@ model-index:
12
  # stablelm-4e1t-2b-v0.1
13
 
14
 
15
- This is a layer pruning experiment based off of the original llama-3-8b:
16
 
17
- - 8 layers pruned with [PruneMe](https://github.com/pszemraj/PruneMe/tree/upgrades)/MergeKit
18
  - layers selected using [BEE-spoke-data/fineweb-100k_en-med](https://hf.co/datasets/BEE-spoke-data/fineweb-100k_en-med)
19
  - brief subsequent continued pretraining @ ctx 4096
20
  - data: 10k rows of FineWeb (different than pruning data) + some curated data
 
12
  # stablelm-4e1t-2b-v0.1
13
 
14
 
15
+ This is a layer pruning experiment based off of [stablelm-3b-4e1t](https://huggingface.co/stabilityai/stablelm-3b-4e1t):
16
 
17
+ - 10 layers pruned with [PruneMe](https://github.com/pszemraj/PruneMe/tree/upgrades)/MergeKit
18
  - layers selected using [BEE-spoke-data/fineweb-100k_en-med](https://hf.co/datasets/BEE-spoke-data/fineweb-100k_en-med)
19
  - brief subsequent continued pretraining @ ctx 4096
20
  - data: 10k rows of FineWeb (different than pruning data) + some curated data