kazzand commited on
Commit
4eaa725
1 Parent(s): 9142173

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -3
README.md CHANGED
@@ -3,9 +3,14 @@ language:
3
  - ru
4
  ---
5
 
6
- This is a large version of Russian Longformer model created from [ai-forever/ruRoberta-large](https://huggingface.co/ai-forever/ruRoberta-large) weights with the length of context expanded to 4096 tokens.
7
- The model was fine-tuned on russian books dataset but also supports English as its source model.
8
- For a more comprehensive overview, please refer to this Habr post, which is available in Russian.
 
 
 
 
 
9
 
10
  The model can be used as-is to produce text embeddings or it can be further fine-tuned for a specific downstream task.
11
 
 
3
  - ru
4
  ---
5
 
6
+ This is a large Longformer model designed for Russian language.
7
+ It was initialized from [ai-forever/ruRoberta-large](https://huggingface.co/ai-forever/ruRoberta-large) weights and has been modified to support a context length of up to 4096 tokens.
8
+ We fine-tuned it on a dataset of Russian books. For a detailed information check out our post on Habr.
9
+
10
+ Model attributes:
11
+ * 16 attention heads
12
+ * 24 hidden layers
13
+ * 4096 tokens length of context
14
 
15
  The model can be used as-is to produce text embeddings or it can be further fine-tuned for a specific downstream task.
16