bigscience
/

bloom

Text Generation

Carbon Emissions

Inference Endpoints

text-generation-inference

Model card Files Files and versions Metrics Training metrics Community

Add exact param counts

#106

by Muennighoff - opened Sep 8, 2022

base: refs/heads/main

←

from: refs/pr/106

Discussion Files changed

Files changed (1) hide show

README.md +3 -1

README.md CHANGED Viewed

@@ -265,7 +265,9 @@ Please see [the BLOOM training README](https://github.com/bigscience-workshop/bi
 * ALiBI positional encodings (see [paper](https://arxiv.org/pdf/2108.12409.pdf)), with GeLU activation functions
-* 176 billion parameters:
     * 70 layers, 112 attention heads

 * ALiBI positional encodings (see [paper](https://arxiv.org/pdf/2108.12409.pdf)), with GeLU activation functions
+* 176,247,271,424 parameters:
+    * 3,596,615,680 embedding parameters
     * 70 layers, 112 attention heads