mgoin commited on
Commit
a5dd409
1 Parent(s): 1af5f10

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -13,6 +13,8 @@ tags:
13
 
14
  This repo contains model files for a [Llama 2 7B](https://huggingface.co/meta-llama/Llama-2-7b-hf) model that has had 50% of the parameters pruned in one-shot with [SparseGPT](https://arxiv.org/abs/2301.00774), then retrained by [Cerebras](https://huggingface.co/cerebras) with 50B tokens from SlimPajama while maintaining sparsity. It was then one-shot pruned to 70% sparsity and trained for another 100B tokens.
15
 
 
 
16
  **Authors**: Neural Magic, Cerebras
17
 
18
  ## Usage
 
13
 
14
  This repo contains model files for a [Llama 2 7B](https://huggingface.co/meta-llama/Llama-2-7b-hf) model that has had 50% of the parameters pruned in one-shot with [SparseGPT](https://arxiv.org/abs/2301.00774), then retrained by [Cerebras](https://huggingface.co/cerebras) with 50B tokens from SlimPajama while maintaining sparsity. It was then one-shot pruned to 70% sparsity and trained for another 100B tokens.
15
 
16
+ Official model weights from [Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment](https://arxiv.org/abs/2405.03594).
17
+
18
  **Authors**: Neural Magic, Cerebras
19
 
20
  ## Usage