Model release

.gitattributes

@@ -25,3 +25,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zstandard filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,40 @@
+# oBERT-12-upstream-pruned-unstructured-97-finetuned-mnli-v2
+
+This model is obtained with [The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models](https://arxiv.org/abs/2203.07259).
+
+
+It corresponds to the model presented in the `Table 2 - oBERT - MNLI 97%` (in the upcoming updated version of the paper).
+
+```
+Pruning method: oBERT upstream unstructured + sparse-transfer to downstream
+Paper: https://arxiv.org/abs/2203.07259
+Dataset: MNLI
+Sparsity: 97%
+Number of layers: 12
+```
+
+The dev-set performance reported in the paper is averaged over four seeds, and we release the best model (marked with `(*)`):
+
+```
+| oBERT 97%     | m-acc | mm-acc|
+| ------------- | ----- | ----- |
+| seed=42       | 80.86 | 80.88 |
+| seed=3407     | 80.83 | 81.65 |
+| seed=123   (*)| 81.18 | 81.06 |
+| seed=12345    | 80.79 | 80.95 |
+| ------------- | ----- | ----- |
+| mean          | 80.91 | 81.13 |
+| stdev         | 0.178 | 0.351 |
+```
+
+Code: _coming soon_
+
+## BibTeX entry and citation info
+```bibtex
+@article{kurtic2022optimal,
+  title={The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models},
+  author={Kurtic, Eldar and Campos, Daniel and Nguyen, Tuan and Frantar, Elias and Kurtz, Mark and Fineran, Benjamin and Goin, Michael and Alistarh, Dan},
+  journal={arXiv preprint arXiv:2203.07259},
+  year={2022}
+}
+```

all_results.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:129775b61b3b1d44878161d5e98c658a63a03618966b2460a3d39b7deeccfe91
+size 657

config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8eaeb7d693e98ba0031a91263d6dcb17eabd9aeab210560f1c05734326765968
+size 1028

eval_results.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3dc4989c3a5ed7d577778a3dbbfa21e8bacdc72f30276ca138263f0902ba4348
+size 479

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:469c97bd0037cad2d9716c140f281dc21bdf32f8a6a8cc106e9d43d48bc37591
+size 438025197

special_tokens_map.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:303df45a03609e4ead04bc3dc1536d0ab19b5358db685b6f3da123d05ec200e3
+size 112

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da0e79933b9ed51798a3ae27893d3c5fa4a201126cef75586296df9b4d2c62a0
+size 711661

tokenizer_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c6863d5eead8a7621fbfc9f434b9ad08693b40f7e17da33922dfb395a8645ca
+size 392

train_results.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16ef4261c51becc4d18930b04949f7f15e3be762361d37063fd20993644869da
+size 198

trainer_state.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:adc464875aee69fd04fd45164b20cdc50963d4fc8ddd420795e72d4b37ffef92
+size 26496

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f71299b9048fe5f2c1b32267c10f33629a3505089438fec47f14997f17a74321
+size 3375