upload model with MLM and NSP heads

Files changed (3) hide show

README.md CHANGED Viewed

@@ -7,5 +7,6 @@ language:
 RuBERT \(Russian, cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) was trained on the Russian part of Wikipedia and news data. We used this training data to build a vocabulary of Russian subtokens and took a multilingual version of BERT‑base as an initialization for RuBERT\[1\].
 \[1\]: Kuratov, Y., Arkhipov, M. \(2019\). Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language. arXiv preprint [arXiv:1905.07213](https://arxiv.org/abs/1905.07213).

 RuBERT \(Russian, cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) was trained on the Russian part of Wikipedia and news data. We used this training data to build a vocabulary of Russian subtokens and took a multilingual version of BERT‑base as an initialization for RuBERT\[1\].
+08.11.2021: upload model with MLM and NSP heads
 \[1\]: Kuratov, Y., Arkhipov, M. \(2019\). Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language. arXiv preprint [arXiv:1905.07213](https://arxiv.org/abs/1905.07213).

flax_model.msgpack CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ff01c0908f8f9987eb4bd777564be972a5412f4e882beb9d050e37ae67e13030
-size 711420911

 version https://git-lfs.github.com/spec/v1
+oid sha256:8da346601df87881d568b074d00dd9346ef528b3b77edcf57f2d5ed682256902
+size 714274024

pytorch_model.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:222a0ce2298b4d0cf4ce2f30761835805534125449efe2c92e00a7d0bde8c728
-size 711456796

 version https://git-lfs.github.com/spec/v1
+oid sha256:e5691c9cb13c98d396ae2e584f4d01a098fc9416a882425ece4359b807730890
+size 714355318