BEE-spoke-data
/

smol_llama-81M-tied

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

pszemraj commited on Oct 26, 2023

Commit

aed721c

•

1 Parent(s): 366be30

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -62,7 +62,7 @@ datasets:
 # smol_llama-81M-tied
-A small 101M param (total) decoder model. This is the first version of the model.
 - 768 hidden size, 6 layers
 - standard multi-head attention (24 heads), context length 1024

 # smol_llama-81M-tied
+A small 81M param (total) decoder model, enabled through tying the input/output embeddings. This is the first version of the model.
 - 768 hidden size, 6 layers
 - standard multi-head attention (24 heads), context length 1024