Update README.md

README.md

@@ -1,9 +1,58 @@
 ---
 license: apache-2.0
+datasets:
+- mrqa
+language:
+- en
+metrics:
+- exact_match
+- f1
+
+
+model-index:
+- name: VMware/TinyRoBERTa-MRQA
+  results:
+  - task:
+      type: Extractive Question-Answering            
+    dataset:
+      type: mrqa       # Required. Example: common_voice. Use dataset id from https://hf.co/datasets
+      name: mrqa       # Required. A pretty name for the dataset. Example: Common Voice (French)
+
+    metrics:
+      - type: exact_match       # Required. Example: wer. Use metric id from https://hf.co/metrics
+        value: 69.21      # Required. Example: 20.90
+        name: Eval EM         # Optional. Example: Test WER
+      - type: f1         # Required. Example: wer. Use metric id from https://hf.co/metrics
+        value: 79.65     # Required. Example: 20.90
+        name: Eval F1         # Optional. Example: Test WER
+      - type: exact_match         # Required. Example: wer. Use metric id from https://hf.co/metrics
+        value: 52.8       # Required. Example: 20.90
+        name: Test EM         # Optional. Example: Test WER
+      - type: f1        # Required. Example: wer. Use metric id from https://hf.co/metrics
+        value: 63.4    # Required. Example: 20.90
+        name: Test F1         # Optional. Example: Test WER
 ---
-Int-8 dynamic quantized version of (VMware/tinyroberta-mrqa)[https://huggingface.co/VMware/tinyroberta-mrqa].
 
-```
+# VMware/TinyRoBERTa-quantized-mrqa
+
+Int-8 dynamic quantized version of [VMware/tinyroberta-mrqa](https://huggingface.co/VMware/tinyroberta-mrqa).
+
+
+## Overview
+- **Model name:** tinyroberta-quantized-mrqa 
+- **Model type:** Extractive Question Answering
+- **Teacher Model:** [VMware/roberta-large-mrqa](https://huggingface.co/VMware/roberta-large-mrqa)
+- **Full Precision Model:** [VMware/tinyroberta-mrqa](https://huggingface.co/VMware/tinyroberta-mrqa)
+- **Training dataset:** [MRQA](https://huggingface.co/datasets/mrqa) (Machine Reading for Question Answering)
+- **Training data size:** 516,819 examples
+- **Language:** English
+- **Framework:** ONNX
+- **Model version:** 1.0 
+
+## Usage
+
+### In Transformers
+```python
 from optimum.onnxruntime import ORTModelForQuestionAnswering
 from transformers import pipeline, AutoTokenizer
 
@@ -13,11 +62,23 @@ quantized_model = ORTModelForQuestionAnswering.from_pretrained(model_name, file_
 
 qa_model = pipeline('question-answering', model=quantized_model, tokenizer=tokenizer)
 
-qa_input = {
-    'question': '',
-    'context': ''
+QA_input = {
+context = "We present the results of the Machine Reading for Question Answering (MRQA) 2019 shared task on evaluating the generalization capabilities of reading comprehension systems. In this task, we adapted and unified 18 distinct question answering datasets into the same format. Among them, six datasets were made available for training, six datasets were made available for development, and the final six were hidden for final evaluation. Ten teams submitted systems, which explored various ideas including data sampling, multi-task learning, adversarial training and ensembling. The best system achieved an average F1 score of 72.5 on the 12 held-out datasets, 10.7 absolute points higher than our initial baseline based on BERT."
+question = "What is MRQA?"
 }
 
 qa_answers = qa_model(qa_input)
 
 ```
+
+# Limitations and Bias
+
+The model is based on a large and diverse dataset, but it may still have limitations and biases in certain areas. Some limitations include:
+
+- Language: The model is designed to work with English text only and may not perform as well on other languages.
+
+- Domain-specific knowledge: The model has been trained on a general dataset and may not perform well on questions that require domain-specific knowledge.
+
+- Out-of-distribution questions: The model may struggle with questions that are outside the scope of the MRQA dataset.  This is best demonstrated by the delta between its scores on the eval vs test datasets.
+
+In addition, the model may have some bias in terms of the data it was trained on. The dataset includes questions from a variety of sources, but it may not be representative of all populations or perspectives. As a result, the model may perform better or worse for certain types of questions or on certain types of texts.