---
license: apache-2.0
language:
- en
library_name: transformers
---

# TinyBERT_L-4_H-312_v2 ONNX Model

This repository provides an ONNX version of the `TinyBERT_L-4_H-312_v2` model, originally developed by the team at [Huawei Noah's Ark Lab](https://arxiv.org/abs/1909.10351)
and ported to Transformers by [Nils Reimers](https://huggingface.co/nreimers). 
The model is a compact version of BERT, designed for efficient inference and reduced memory footprint. The ONNX version includes mean pooling of the last hidden layer for convenient feature extraction.

## Model Overview

TinyBERT is a smaller version of BERT that maintains competitive performance while significantly reducing the number of parameters and computational cost. This makes it ideal for deployment in resource-constrained environments. The model is based on the work presented in the paper ["TinyBERT: Distilling BERT for Natural Language Understanding"](https://arxiv.org/abs/1909.10351).

## License

This model is distributed under the Apache 2.0 License. For more details, please refer to the [license file](https://github.com/huawei-noah/Pretrained-Language-Model/blob/master/TinyBERT/LICENSE) in the original repository.

## Model Details

- **Model:** TinyBERT_L-4_H-312_v2
- **Layers:** 4
- **Hidden Size:** 312
- **Pooling:** Mean pooling of the last hidden layer
- **Format:** ONNX

## Usage

To use this model, you will need to have `onnxruntime` installed. You can install it via pip:

```bash
pip install onnxruntime, transformers
```

Below is a Python code snippet demonstrating how to run inference using this ONNX model:

```python
import onnxruntime as ort
from transformers import AutoTokenizer

model_path="TinyBERT_L-4_H-312_v2-onnx/"
tokenizer = AutoTokenizer.from_pretrained(model_path)
ort_sess = ort.InferenceSession(model_path + "/tinybert_mean_embeddings.onnx")

features = tokenizer(['How many people live in Berlin?','Berlin has a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.', 'New York City is famous for the Metropolitan Museum of Art.'],  padding=True, truncation=True, return_tensors="np")
onnx_inputs = {k: v for k, v in features.items() if k != 'token_type_ids'}
ort_outs = ort_sess.run(None, onnx_inputs)
print(ort_outs)

print("Mean pooled output:", mean_pooled_output)
```

Make sure to replace `'model_path'` with the actual path to your ONNX model file.

## Training Details

For detailed information on the training process of TinyBERT, please refer to the [original paper](https://arxiv.org/abs/1909.10351) by Huawei Noah's Ark Lab.

## Acknowledgements

This model is based on the work by the team at Huawei Noah's Ark Lab and by Nils Reimers. Special thanks to the developers for providing the pre-trained model and making it accessible to the community.