YX-Cerebras
commited on
Commit
•
37bc97b
1
Parent(s):
0ea9b34
Update README.md
Browse files
README.md
CHANGED
@@ -17,6 +17,10 @@ pipeline_tag: text-generation
|
|
17 |
|
18 |
**For final model checkpoint, please see:** https://huggingface.co/cerebras/Cerebras-GPT-13B
|
19 |
|
|
|
|
|
|
|
|
|
20 |
## Uses and Limitations
|
21 |
|
22 |
### Intended Use
|
|
|
17 |
|
18 |
**For final model checkpoint, please see:** https://huggingface.co/cerebras/Cerebras-GPT-13B
|
19 |
|
20 |
+
## Usage of muP checkpoints
|
21 |
+
|
22 |
+
Note: Transformers does not support muP for all models, so we need a custom model class (BTLM-3B-8k-base). This causes a situation where users must either (1) enable `trust_remote_code=True` when loading the model or (2) acknowledge the warning about code execution upon loading the model.
|
23 |
+
|
24 |
## Uses and Limitations
|
25 |
|
26 |
### Intended Use
|