Muennighoff
HF staff
commited on
Commit
9c4abbe
1 Parent(s): 889e0ff

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -3
README.md CHANGED
@@ -71,7 +71,7 @@ Version 1.0 / 26.May.2022
71
  8. [More Information](#more-information)
72
  9. [Model Card Authors](#model-card-authors)
73
 
74
- ## Model Details
75
 
76
  ### Basics
77
  *This section provides information for anyone who wants to know about the model.*
@@ -123,7 +123,9 @@ Please see [the BLOOM training README](https://github.com/bigscience-workshop/bi
123
 
124
  * ALiBI positional encodings (see [paper](https://arxiv.org/pdf/2108.12409.pdf)), with GeLU activation functions
125
 
126
- * 1.3 billion parameters:
 
 
127
 
128
  * 24 layers, 16 attention heads
129
 
@@ -559,4 +561,4 @@ Initial prompting experiments using interim checkpoints: https://huggingface.co/
559
  ## Model Card Authors
560
  *Ordered roughly chronologically and by amount of time spent.*
561
 
562
- Margaret Mitchell, Giada Pistilli, Yacine Jernite, Ezinwanne Ozoani, Marissa Gerchick, Nazneen Rajani, Sasha Luccioni, Irene Solaiman, Maraim Masoud, Somaieh Nikpoor, Carlos Muñoz Ferrandis, Stas Bekman, Christopher Akiki, Danish Contractor, David Lansky, Angelina McMillan-Major, Tristan Thrush, Suzana Ilić, Gérard Dupont, Shayne Longpre, Manan Dey, Stella Biderman, Douwe Kiela, Emi Baylor, Teven Le Scao, Aaron Gokaslan, Julien Launay
71
  8. [More Information](#more-information)
72
  9. [Model Card Authors](#model-card-authors)
73
 
74
+ ## Model Details
75
 
76
  ### Basics
77
  *This section provides information for anyone who wants to know about the model.*
123
 
124
  * ALiBI positional encodings (see [paper](https://arxiv.org/pdf/2108.12409.pdf)), with GeLU activation functions
125
 
126
+ * 1,722,408,960 parameters:
127
+
128
+ * 513,802,240 embedding parameters
129
 
130
  * 24 layers, 16 attention heads
131
 
561
  ## Model Card Authors
562
  *Ordered roughly chronologically and by amount of time spent.*
563
 
564
+ Margaret Mitchell, Giada Pistilli, Yacine Jernite, Ezinwanne Ozoani, Marissa Gerchick, Nazneen Rajani, Sasha Luccioni, Irene Solaiman, Maraim Masoud, Somaieh Nikpoor, Carlos Muñoz Ferrandis, Stas Bekman, Christopher Akiki, Danish Contractor, David Lansky, Angelina McMillan-Major, Tristan Thrush, Suzana Ilić, Gérard Dupont, Shayne Longpre, Manan Dey, Stella Biderman, Douwe Kiela, Emi Baylor, Teven Le Scao, Aaron Gokaslan, Julien Launay, Niklas Muennighoff