Model release

.gitattributes

@@ -25,3 +25,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zstandard filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,40 @@
+# oBERT-12-upstream-pruned-unstructured-97-finetuned-squadv1-v2
+
+This model is obtained with [The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models](https://arxiv.org/abs/2203.07259).
+
+
+It corresponds to the model presented in the `Table 2 - oBERT - SQuADv1 97%` (in the upcoming updated version of the paper).
+
+```
+Pruning method: oBERT upstream unstructured + sparse-transfer to downstream
+Paper: https://arxiv.org/abs/2203.07259
+Dataset: SQuADv1
+Sparsity: 97%
+Number of layers: 12
+```
+
+The dev-set performance reported in the paper is averaged over four seeds, and we release the best model (marked with `(*)`):
+
+```
+| oBERT 97%     | F1    | EM    |
+| ------------- | ----- | ----- |
+| seed=42       | 84.92 | 76.94 |
+| seed=3407     | 84.87 | 76.71 |
+| seed=123      | 84.95 | 77.06 |
+| seed=12345 (*)| 84.95 | 76.90 |
+| ------------- | ----- | ----- |
+| mean          | 84.92 | 76.90 |
+| stdev         | 0.037 | 0.145 |
+```
+
+Code: _coming soon_
+
+## BibTeX entry and citation info
+```bibtex
+@article{kurtic2022optimal,
+  title={The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models},
+  author={Kurtic, Eldar and Campos, Daniel and Nguyen, Tuan and Frantar, Elias and Kurtz, Mark and Fineran, Benjamin and Goin, Michael and Alistarh, Dan},
+  journal={arXiv preprint arXiv:2203.07259},
+  year={2022}
+}
+```

all_results.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63f7493be1e34268fcfffb915725111bd22c911df381aa1f39fcc719ace55059
+size 298

config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:380169abe557124a472ea17ca3aa889ac28e1ee16e22e059b040e21a1baff592
+size 773

eval_nbest_predictions.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a855a7f678ac8c6f5326651aab25ea3c8771860b2bedf2cddec61061ca2ef4c1
+size 45702613

eval_predictions.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c776888ecc10cf6c6a9c802f60c5320eb84316390b9358f4ae1c0961d1b03a83
+size 589310

eval_results.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6dab5105be37e79d7624fdd08613d6d7bec5a39edb7247aacd8f0d98e11607e2
+size 124

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0c26f79c4bbd2ba97e80149252b9e8e052355a4944cb73b0ebe0350d347c591
+size 435658993

special_tokens_map.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:303df45a03609e4ead04bc3dc1536d0ab19b5358db685b6f3da123d05ec200e3
+size 112

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d241a60d5e8f04cc1b2b3e9ef7a4921b27bf526d9f6050ab90f9267a1f9e5c66
+size 711396

tokenizer_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2995bd0620bd945c6fedbfb8ec229865428cb1fb08d690e79e3d7715755af2e
+size 441

train_results.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67228c69072a369dad24defb885bd438794532fea561de26b96eb3d995962e11
+size 194

trainer_state.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4702db99461754b85bccdafa1be5a7602bd918c735a0d8eaa377f028e96ac23
+size 12402

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2ad7267334c8e1d3f0033f765f27d7dbdd8ae2b70592901dcd45e5788a5e6b6
+size 3375