File size: 2,179 Bytes
6f754ce 3d78d3b db05c3e 3d78d3b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
---
license: apache-2.0
language:
- fi
- nn
- en
- 'no'
- da
- sv
- is
---
This is a pre-release checkpoint for a Nordic generative language model currently in training.
This preliminary release is provided for HPLT (https://hplt-project.org/) deliverable 4.1 (“First language models trained”)(https://hplt-project.org/deliverables). Consult the HPLT website for further details.
More documentation will be provided soon.
UPDATE: our Nordic model is now called Viking!
-------
# Viking 7B, 13B and 33B
_**NOTE:** These are **research checkpoint** of a model for which **training has not been completed.** It is being provided in its current state for research and testing purposes. **Care should be taken when using the outputs of the model.** Once pretraining has completed we intend to release additional instruction-tuned and chat-tuned varieties._
Viking 7B, 13B and 13B are a 7B, 13B and 33B parameter decoder-only transformers pretrained on Finnish,
English, Swedish, Danish, Norwegian, Icelandic and code. They are being trained
on 2 trillion tokens (1.3 trillion as of this release).
Viking is a fully open source model and is made available under the Apache 2.0 License.
Viking was created in a collaboration between the [TurkuNLP group](https://turkunlp.org/) of the University of Turku, [SiloGen](https://www.silo.ai/silogen) from [Silo AI](https://www.silo.ai/), and [High Performance Language Technologies](https://hplt-project.org/) (HPLT). Training was conducted on the [LUMI supercomputer](https://www.lumi-supercomputer.eu/), using compute resources generously provided by [CSC](https://csc.fi/) - IT Center for Science, Finland.
This project is part of an ongoing effort to create open source large language models for non-English and especially low resource languages like Finnish. The mode is fluent in Finnish, English, the Scandinavian languages and capable of basic translation between them. It is also able to understand and generate code.
More info available at:
[Viking 7B](https://huggingface.co/LumiOpen/Viking-7B)
[Viking 13B](https://huggingface.co/LumiOpen/Viking-13B)
[Viking 33B](https://huggingface.co/LumiOpen/Viking-33B)
|