Update README.md

README.md

@@ -1,3 +1,43 @@
 ---
 license: apache-2.0
+language:
+- en
+library_name: fairseq
+pipeline_tag: automatic-speech-recognition
+inference: false
 ---
+
+
+<br>
+<br>
+
+# ARMHuBERT Model Card
+
+This repo contains the models from our paper [**Recycle-and-Distill: Universal Compression Strategy for Transformer-based Speech SSL Models with Attention Map Reusing and Masking Distillation**](https://arxiv.org/abs/2305.11685), INTERSPEECH 2023.
+
+
+## Model details
+
+**Model type:**
+ARMHuBERT is an open-source speech SSL model distilled from HuBERT-Base, by attention map reusing and masking distillation. 
+We also provide the model checkpoints of MaskHuBERT (without attention map reusing) and ARMwavLM (wavLM-Base teacher). 
+
+- Attention Map Reusing: Reuse previous layer's attention map to remove key & query parameters in Transformer.
+- Masking Distillation: Masking distillation treating masked frames and unmasked frames separately.
+
+**License:**
+Apache 2.0 License
+
+**Where to send questions or comments about the model:**
+https://github.com/sungnyun/ARMHuBERT/issues
+
+
+## Training dataset
+Pretraining data: [LibriSpeech](https://www.openslr.org/12)
+- ``[ModelName]-100h.ckpt``: train-clean-100
+- ``[ModelName]-960h.ckpt``: train-clean-100 + train-clean-360 + train-other-500
+
+
+<br>
+
+More detials are in our github, https://github.com/sungnyun/ARMHuBERT.