  We present the ongoing NorLM initiative to support the creation and use of
very large contextualised language models for Norwegian (and in principle other
Nordic languages), including a ready-to-use software environment, as well as an
experience report for data preparation and training. This paper introduces the
first large-scale monolingual language models for Norwegian, based on both the
ELMo and BERT frameworks. In addition to detailing the training process, we
present contrastive benchmark results on a suite of NLP tasks for Norwegian.
For additional background and access to the data, models, and software, please
see http://norlm.nlpl.eu
