ybelkada HF staff commited on
Commit
d586a2f
1 Parent(s): 8ad61ef

Slightly modify model card (#1)

Browse files

- add small modifs readme (2fd4bd96c1270ff50e922f8bec3fdd110b41f2ec)

Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -122,11 +122,11 @@ Please see [the BLOOM training README](https://github.com/bigscience-workshop/bi
122
 
123
  * ALiBI positional encodings (see [paper](https://arxiv.org/pdf/2108.12409.pdf)), with GeLU activation functions
124
 
125
- * 176 billion parameters:
126
 
127
- * 70 layers, 112 attention heads
128
 
129
- * Hidden layers are 14336-dimensional
130
 
131
  * Sequence length of 2048 tokens used (see [BLOOM tokenizer](https://huggingface.co/bigscience/tokenizer), [tokenizer description](#tokenization))
132
 
122
 
123
  * ALiBI positional encodings (see [paper](https://arxiv.org/pdf/2108.12409.pdf)), with GeLU activation functions
124
 
125
+ * 350 million parameters:
126
 
127
+ * 24 layers, 16 attention heads
128
 
129
+ * Hidden layers are 1024-dimensional
130
 
131
  * Sequence length of 2048 tokens used (see [BLOOM tokenizer](https://huggingface.co/bigscience/tokenizer), [tokenizer description](#tokenization))
132