Hugging Face's logo

EleutherAI
/
polyglot-ko-3.8b

Text Generation
Transformers
PyTorch
Safetensors
Korean
gpt_neox
causal-lm
text-generation-inference
Inference Endpoints
Model card Files Community
1
bzantium commited on
Commit
8df93b8
•
1 Parent(s): 7504230

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -16,11 +16,11 @@ Polyglot-Ko is a series of large-scale Korean autoregressive language models mad
16
  |----------------------|----------------------------------------------------------------------------------------------------------------------------------------|
17
  | \\(n_{parameters}\\) | 3,809,974,272 |
18
  | \\(n_{layers}\\) | 32 |
19
- | \\(d_{model}\\) | 3072 |
20
  | \\(d_{ff}\\) | 12,288 |
21
  | \\(n_{heads}\\) | 24 |
22
  | \\(d_{head}\\) | 128 |
23
- | \\(n_{ctx}\\) | 2048 |
24
  | \\(n_{vocab}\\) | 30,003 / 30,080 |
25
  | Positional Encoding | [Rotary Position Embedding (RoPE)](https://arxiv.org/abs/2104.09864) |
26
  | RoPE Dimensions | [64](https://github.com/kingoflolz/mesh-transformer-jax/blob/f2aa66e0925de6593dcbb70e72399b97b4130482/mesh_transformer/layers.py#L223) |
 
16
  |----------------------|----------------------------------------------------------------------------------------------------------------------------------------|
17
  | \\(n_{parameters}\\) | 3,809,974,272 |
18
  | \\(n_{layers}\\) | 32 |
19
+ | \\(d_{model}\\) | 3,072 |
20
  | \\(d_{ff}\\) | 12,288 |
21
  | \\(n_{heads}\\) | 24 |
22
  | \\(d_{head}\\) | 128 |
23
+ | \\(n_{ctx}\\) | 2,048 |
24
  | \\(n_{vocab}\\) | 30,003 / 30,080 |
25
  | Positional Encoding | [Rotary Position Embedding (RoPE)](https://arxiv.org/abs/2104.09864) |
26
  | RoPE Dimensions | [64](https://github.com/kingoflolz/mesh-transformer-jax/blob/f2aa66e0925de6593dcbb70e72399b97b4130482/mesh_transformer/layers.py#L223) |