genbio-ai
/

AIDO.Protein-16B

Model card Files Files and versions Community

xingyi2 commited on 9 days ago

Commit

ead4174

•

1 Parent(s): 7441644

Update README.md

Add FFN Hidden SIZE.

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -18,7 +18,8 @@ More architecture details are shown below:
 | ------------- |:-------------:|
 | Num Attention Head |36 |
 | Num Hidden Layer |36 |
-| Hidden Size |2304 |
 | Num MoE Layer per Block| 8|
 | Num MoE Layer per Token| 2|
 |Vocab Size|44 |

 | ------------- |:-------------:|
 | Num Attention Head |36 |
 | Num Hidden Layer |36 |
+| Hidden Size |2304 |
+| FFN Hidden Size |7680 |
 | Num MoE Layer per Block| 8|
 | Num MoE Layer per Token| 2|
 |Vocab Size|44 |