MarianMT Doesn't export to ONNX correctly
Author: calebdkofahl
Pull request: no

I am running Python FastAPI in a docker container via VSCode's devcontainer setup. I am using the pre-trained MarianMT model for translation of chinese_simple to english and running into an error when trying to use the exported model from the transformers.onnx module. This is the error I am receiving:
```sh
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 366, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/usr/local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 208, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 580, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 241, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 52, in app
    response = await func(request)
  File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 226, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 159, in run_endpoint_function
    return await dependant.call(**values)
  File "./app/main.py", line 300, in run_onnx_inference
    onnx_output = onnx_session.run(output_names=["last_hidden_state"], input_feed=dict(onnx_inputs))
  File "/usr/local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 188, in run
    raise ValueError("Model requires {} inputs. Input Feed contains {}".format(num_required_inputs, num_inputs))
ValueError: Model requires 4 inputs. Input Feed contains 2
```

Dockerfile:
```Docker
FROM python:3.8

# Install required binaries and data
RUN apt-get update
RUN apt install wget tesseract-ocr -y
RUN apt install tesseract-ocr-chi-sim -y
RUN apt install python3-opencv -y

# Install python requirements
COPY ./requirements.txt .
RUN pip3 install -r requirements.txt

# Copy all source code and data
COPY ./README.md /README.md
COPY ./app /app
COPY ./data /data

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80"]
```

Package Versions:
onnx                       1.11.0
onnxmltools           1.11.0
onnxruntime          1.11.1
transformers         4.19.2

Code causing the error:
```python
import onnx
import onnxruntime as onnxrt  # pylint: disable=import-error

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, pipeline

@app.post("/onnx")
async def run_onnx_inference(text="不要把乐趣搞得一团糟"):
    onnx_model_filename = f"{MODELS_DIR}/model.onnx"
    mt_tokenizer = AutoTokenizer.from_pretrained(f"{MODELS_DIR}/marian")
    
    onnx_session = onnxrt.InferenceSession(onnx_model_filename)
    onnx_inputs = mt_tokenizer(str(text), return_tensors="np")
    onnx_output = onnx_session.run(output_names=["last_hidden_state"], input_feed=dict(onnx_inputs))
    
    return onnx_output
```

Command used to export the model to onnx:
```sh
python -m transformers.onnx --model=app/models/marian --atol=2e-04 --feature=seq2seq-lm app/models
```

I have also tried exporting with the following, but I receive the same error:
```sh
python -m transformers.onnx --model=app/models/marian --atol=2e-04 app/models
```

It appears that, even though https://huggingface.co/docs/transformers/serialization clearly lists Marian as a valid model to export to ONNX, it doesn't export the model in a way that allows ONNX to run inference on the model.

Any help would be greatly appreciated.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @calebdkofahl, thanks for sharing this issue!

The reason you see the error

```
ValueError: Model requires 4 inputs. Input Feed contains 2
```

is because MarianMT is a seq2seq model, so we need to pass the decoder input IDs and attention masks in addition to the encoder's ones. 

The simplest way to generate inference with ONNX Runtime would be to use the new inference pipelines in our `optimum` library: https://huggingface.co/docs/optimum/onnxruntime/modeling_ort#optimum.onnxruntime.ORTModelForSeq2SeqLM.forward.example

This will allows you to skip the annoying step of needing to manually create the decoder inputs - hope that helps!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.