Add support for exporting summarization models to ONNX
Author: sagarreddypatil
Pull request: no

# 🚀 Feature request
Add support for exporting summarization models to ONNX.

## Motivation

I want to serve summarization models on edge, through an ONNX runtime. However, I am unable to convert facebook/bart-large-cnn(using class BartModelForConditionalGeneration) to ONNX as the provided script doesn't support the summarization pipeline, due to PyTorch not being able to export the triu operator to ONNX. However, there are workarounds listed at https://github.com/pytorch/pytorch/issues/32968, and could be used to make this possible.

## Your contribution
I don't really know the internals of PyTorch that well, so I don't think I can make any direct contributions.


Comments:
I just realized that the triu operation was fixed recently, sorry about that. I will create a PR to add summarization to the ONNX conversion script.

@sagarreddypatil Did you need to make any other changes to get summarization working with ONNX Runtime? It only appears to work for text with five tokens for me, which I believe it due to <strike>the [dummy inputs](https://huggingface.co/transformers/serialization.html#dummy-inputs-and-standard-lengths)</strike> (edit: looks to be due to the `infer_shapes` method)

```python
from transformers import AutoTokenizer
import onnxruntime as rt
import numpy as np

text = "one two three four five"
tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-cnn")
tokens = tokenizer(text)

input = {
  'input_ids': np.array([tokens['input_ids']]),
  'attention_mask': np.array([tokens['attention_mask']])
}

sess = rt.InferenceSession("bart-large-cnn.onnx")
output = sess.run(None, input)
print(output)
```

Other token counts fail with:

```text
2020-10-05 16:44:25.884944 [E:onnxruntime:, sequential_executor.cc:318 Execute] Non-zero status code returned while running Reshape node. Name:'Reshape_62' Status Message: /Users/runner/work/1/s/onnxruntime/core/providers/cpu/tensor/reshape_helper.h:43 onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape &, std::vector<int64_t> &) gsl::narrow_cast<int64_t>(input_shape.Size()) == size was false. The input tensor cannot be reshaped to the requested shape. Input shape:{1,7}, requested shape:{6}
```

I actually did encounter that issue. In my "hotfix", I simply added the summarization option to the list, but I believe the implementation of how the ONNX model is made needs to be changed. But yes, it does not work for more than 5 tokens. I am not really sure how to fix that, but you seem to be better with this than I am.

Any chance someone was able to solve this issue?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I'm receiving this error 

`Error while converting the model: The type of axis index is expected to be an integer
`

when trying to convert bart-large-cnn

`python3 -m transformers.convert_graph_to_onnx --model facebook/bart-large-cnn --framework pt bart-large-cnn.onnx
`

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

> I'm receiving this error
> 
> `Error while converting the model: The type of axis index is expected to be an integer `
> 
> when trying to convert bart-large-cnn
> 
> `python3 -m transformers.convert_graph_to_onnx --model facebook/bart-large-cnn --framework pt bart-large-cnn.onnx `

I'm encountering the same issue when exporting a gpt2 model using --pipeline text-generation.

We're currently working on a rework of the ONNX implementation within Transformers, which is available here: https://github.com/huggingface/transformers/pull/11786

Instead of offering a script to enable conversions for all models (which was not kept up to date with recent model releases), we're opting for a case-by-case approach, while offering the tools to convert models manually in a straightforward and simple manner; by creating `OnnxConfig` configuration objects to specify the input and output types of each model.

Please take a look at the PR and give us your feedback.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.