princeton-nlp commited on
Commit
8f5cd82
1 Parent(s): 3bc5cf3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -0
README.md CHANGED
@@ -14,6 +14,10 @@ Sheared-LLaMA-1.3B is a model pruned and further pre-trained from [meta-llama/Ll
14
  model = AutoModelForCausalLM.from_pretrained("princeton-nlp/Sheared-LLaMA-1.3B")
15
  ```
16
 
 
 
 
 
17
  ## Downstream Tasks
18
 
19
  We evaluate on an extensive set of downstream tasks including reasoning, reading comprehension, language modeling and knowledge intensive tasks. Our Sheared-LLaMA models outperform existing large language models.
 
14
  model = AutoModelForCausalLM.from_pretrained("princeton-nlp/Sheared-LLaMA-1.3B")
15
  ```
16
 
17
+ - Smaller-scale
18
+ - Same vocabulary as LLaMA1 and LLaMA2
19
+ - Derived with 50B tokens by utilizing existing strong LLMs
20
+
21
  ## Downstream Tasks
22
 
23
  We evaluate on an extensive set of downstream tasks including reasoning, reading comprehension, language modeling and knowledge intensive tasks. Our Sheared-LLaMA models outperform existing large language models.