ai-forever commited on
Commit
f397ab4
1 Parent(s): 2ba65e8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -0
README.md CHANGED
@@ -1,3 +1,16 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ language:
4
+ - ru, en
5
+ pipeline_tag: text-generation
6
+ tags:
7
+ - PyTorch
8
+ - Transformers
9
+ thumbnail: "https://github.com/sberbank-ai/mgpt"
10
  ---
11
+
12
+ # Multilingual GPT model
13
+
14
+ Model was trained with sequence length 1024 using transformers lib by [SberDevices](https://sberdevices.ru/) team on 80B tokens for 3 epochs. After that model was finetuned 1 epoch with sequence length 2048.
15
+
16
+ Total training time was around n days on n GPUs for n context and few days on n GPUs for n context.