Yura Kuratov commited on
Commit
c89e312
1 Parent(s): e86a0b4

upload model with MLM and NSP heads

Browse files
Files changed (3) hide show
  1. README.md +1 -0
  2. flax_model.msgpack +2 -2
  3. pytorch_model.bin +2 -2
README.md CHANGED
@@ -7,5 +7,6 @@ language:
7
 
8
  RuBERT \(Russian, cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) was trained on the Russian part of Wikipedia and news data. We used this training data to build a vocabulary of Russian subtokens and took a multilingual version of BERT‑base as an initialization for RuBERT\[1\].
9
 
 
10
 
11
  \[1\]: Kuratov, Y., Arkhipov, M. \(2019\). Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language. arXiv preprint [arXiv:1905.07213](https://arxiv.org/abs/1905.07213).
 
7
 
8
  RuBERT \(Russian, cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) was trained on the Russian part of Wikipedia and news data. We used this training data to build a vocabulary of Russian subtokens and took a multilingual version of BERT‑base as an initialization for RuBERT\[1\].
9
 
10
+ 08.11.2021: upload model with MLM and NSP heads
11
 
12
  \[1\]: Kuratov, Y., Arkhipov, M. \(2019\). Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language. arXiv preprint [arXiv:1905.07213](https://arxiv.org/abs/1905.07213).
flax_model.msgpack CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ff01c0908f8f9987eb4bd777564be972a5412f4e882beb9d050e37ae67e13030
3
- size 711420911
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8da346601df87881d568b074d00dd9346ef528b3b77edcf57f2d5ed682256902
3
+ size 714274024
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:222a0ce2298b4d0cf4ce2f30761835805534125449efe2c92e00a7d0bde8c728
3
- size 711456796
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e5691c9cb13c98d396ae2e584f4d01a098fc9416a882425ece4359b807730890
3
+ size 714355318