metadata

tags:
  - generated_from_trainer
datasets:
  - squad
model-index:
  - name: test.mobilebert-uncased-squadv1
    results:
      - task:
          name: Question Answering
          type: question-answering
        dataset:
          name: Squadv1
          type: squadv1
          split: validation
        metrics:
          - name: F1
            type: f1
            value: 90.9013
          - name: EM
            type: exact_match
            value: 83.9735

test.mobilebert-uncased-squadv1

This model takes the same weights as neuralmagic/mobilebert-uncased-finetuned-squadv1 but is converted to OpenVINO IR format.

Eval results: (pad_to_max_length=True, max_length=384)

torch model eval_exact_match: 83.9735
torch model eval_f1: 90.9013

Framework versions

Transformers 4.26.0
Pytorch 1.13.1+cu116
Datasets 2.8.0
Tokenizers 0.13.2
Optimum-intel 1.7.0
NNCF 2.4.0
Openvino-dev 2023.0.0.dev20230217