---
language:
- fi
pipeline_tag: text-generation
---
Generative Pretrained Transformer with 8B parameteres for Finnish.

TurkuNLP Finnish GPT-3-models are a model family of pretrained monolingual GPT-style language models that are based on BLOOM-architecture.
Note that the models are pure language models, meaning that they are not [instruction finetuned](https://arxiv.org/abs/2203.02155) for dialogue
or answering questions.

These models are intended to be used as foundational models that can be e.g. instruction finetuned to serve as modern chat-models.



**Parameters**
| Model | Layers | Dim  | Heads | Params |
|--------|--------|------|-------|--------|
| Small  | 12     | 768  | 12    | 186M   |
| Medium | 24     | 1024 | 16    | 437M   |
| Large  | 24     | 1536 | 16    | 881M   |
| XL     | 24     | 2064 | 24    | 1.5B   |
| ”3B”   | 32     | 2560 | 32    | 2.8B   |
| ”8B”   | 32     | 4096 | 32    | 7.5B   |
| "13B"  | 40     | 5120 | 40    | 13.3B  |