[Swin] Replace hard-coded batch size to enable dynamic ONNX export
Author: lewtun
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR tweaks the modeling code of `swin` to enable dynamic batch sizes with the ONNX export. With this fix, the ONNX slow tests for this model now pass, including the slow tests for the original PyTorch model:

```
This passes
RUN_SLOW=1 pytest -x -sv tests/models/swin/test_modeling_swin.py
This also passes
RUN_SLOW=1 pytest -x -sv tests/onnx/test_onnx_v2.py -k "swin"
```

Since this change also impacts other models, I've also checked the modeling slow tests pass for:

- [x] `maskformer`
- [x] `donut_swin`
- [x] `swin_v2`

Related to https://github.com/huggingface/transformers/issues/17476




Comments:
_The documentation is not available anymore as the PR was closed or merged._

(a bit off-topic, but still related question)

Viewing the issue and the fix provided this PR, I was thinking we would have a lot of the same errors due to this `hard-coded batch size`. However, when I check `bert`:
https://github.com/huggingface/transformers/blob/10100979ed0594d4cfe1982cdfac9642a68e473e/src/transformers/models/bert/modeling_bert.py#L962
and
https://github.com/huggingface/transformers/blob/10100979ed0594d4cfe1982cdfac9642a68e473e/src/transformers/models/bert/modeling_bert.py#L968

but the onnx tests still pass for `bert`. Are `batch_size` and `seq_length` not hard-coded here? Just wondering if @lewtun has already some insight regarding this.



> Is the issue caused by the changes in #19255? More precisely, from (newly added code) https://github.com/dwyatte/transformers/blob/949683675d83cc38620106626822279cd45b076b/src/transformers/onnx/convert.py#L368
> 
> The error shows `Outputs values doesn't match between reference model and ONNX exported model` - it must be non-trivi al to figure out this is coming from the shape things! How are you able to find out 💯 ? Is there some tool we can use to check things (tensor values/shape) when running onnx inference?

Yes, this issue was surfaced by #19255, which implemented a stronger validation test on exported ONNX models. Basically, it generates the ONNX graph using dummy data with one batch size `b`, and then validates the forward pass with a different `b'`. 

The reason it can be non-trivial to figure out when an export fails to have agreement between the PyTorch / ONNX models is that ONNX traces a graph based on dummy data, and this tracing can be incorrect if there are data-dependent flow statements (Swin in particular has a lot of these if/else statements). Currently , the best tool I know of is to visualise the graph with [Netron](https://netron.app/) and manually inspect for discrepancies.

> Viewing the issue and the fix provided this PR, I was thinking we would have a lot of the same errors due to this `hard-coded batch size`. However, when I check `bert`:

I think in those cases we don't hit a problem because `batch_size` is only used to create the attention mask when none is provided. Since our dummy input provides an attention mask, this flow in the graph is never traced AFAICT