Failed to find input with name: attention_mask in the model input def list
Author: imVParashar
Pull request: no

There's something wrong while using ONNX Runtime for the `TFDistilBertForSequenceClassification` model.

It throws the following error while using the TensorFlow model, `Failed to find input with the name: attention_mask in the model input def list`.

However, if I only pass `input_ids`, then it works.

For Pytorch models, both the `input_ids` & `attention_mask` works.

### Steps to reproduce the error:

- Finetune a `TFDistilBertForSequenceClassification` model.
- Export the model using `transformers.convert_graph_to_onnx.convert()`.
- Use ONNX Runtime to load the model and create an inference session.
- Pass a `dict` of `input_ids` and `attention_mask` to `session.run()`.

It does not accept the `attention_mask` during inference, this is what the error says. But for PyTorch models, it works completely fine.

### Here's the code snippet that I've used.
```
from transformers.convert_graph_to_onnx import convert
from pathlib import Path
import onnxruntime

model_ckpt = "model/"
onnx_model_path = Path("onnx/model.onnx")
convert(framework="tf", model=model_ckpt, tokenizer=tokenizer, 
        output=onnx_model_path, opset=12, pipeline_name="text-classification")

sess = onnxruntime.InferenceSession("onnx/model.quant.onnx")

inputs = {'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1]],
 'input_ids': [[101, 2129, 2052, 2017, 2360, 4875, 1999, 3059, 102]]}

logits_onnx = onnx_model.run(None, inputs)[0]
logits_onnx.shape
```
### It throws the following error:
```
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-79-722e62f57d27> in <module>
----> 1 logits_onnx = onnx_model.run(None, inputs)[0]
      2 logits_onnx.shape

~/.local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py in run(self, output_names, input_feed, run_options)
    190             output_names = [output.name for output in self._outputs_meta]
    191         try:
--> 192             return self._sess.run(output_names, input_feed, run_options)
    193         except C.EPFail as err:
    194             if self._enable_fallback:

RuntimeError: Failed to find input with name: attention_mask in the model input def list
```
Also, is it important to pass 'attention_mask' during the inference time? How does it impact the results?


Comments:
cc @lewtun 

Hey @imVParashar thanks for raising the issue. We recently had to fix the (deprecated) TensorFlow exporter in #15856, so if you install from `master` the following should work:

```python
from pathlib import Path

import numpy as np
import onnxruntime

from transformers import AutoTokenizer
from transformers.convert_graph_to_onnx import convert

model_ckpt = "distilbert-base-uncased-finetuned-sst-2-english"
onnx_model_path = Path("onnx/model.onnx")
tokenizer = AutoTokenizer.from_pretrained(model_ckpt)
convert(
    framework="tf",
    model=model_ckpt,
    tokenizer=tokenizer,
    output=onnx_model_path,
    opset=12,
    pipeline_name="text-classification",
)

sess = onnxruntime.InferenceSession("onnx/model.onnx")

inputs = tokenizer("Running TensorFlow ONNX", return_tensors="tf")
# ONNX Runtime expects NumPy arrays
inputs = {k: v.numpy() for k, v in inputs.items()}

logits_onnx = sess.run(None, inputs)[0]
logits_onnx.shape
```

**Warning:** As far as I know, ONNX models that are exported from TensorFlow / Keras only work for a _fixed input shape_. Although my example above works, you'll find it throws an error if you change the sequence length / batch size. I need to dig deeper into this issue, but if it's a problem for you, I suggest loading your TensorFlow model in PyTorch via the `from_tf=True` argument in the `from_pretrained()` method. If you save and export that model, you should be able to provide dynamic shapes via the `transformers.onnx` exporter.

Incidentally, the `convert_graph_to_onnx` module is deprecated and so the recommended way to do this is via the `transformers.onnx` module, e.g.

```
python -m transformers.onnx --model=distilbert-base-uncased-finetuned-sst-2-english --feature=sequence-classification onnx/
```



Thank you so much. The approach you mentioned, worked for me. :)