Fx support for multiple model architectures
Author: michaelbenayoun
Pull request: yes

# What does this PR do?

This PR adds support torch.fx tracing for the following model architectures:

- BART
- mBART
- Marian
- M2M100
- Blenderbot
- Blenderbot Small
- Pegasus
- PLBart
- XGLM
- Speech2Text
- Speech2Text2
- OPT
- CLIP
- TrOCR

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@michaelbenayoun 
```
import inspect

import transformers.utils.fx as fx
from transformers import *

model = LayoutLMForMaskedLM(LayoutLMConfig())

input_names = model.dummy_inputs.keys()
sig = inspect.signature(model.forward)
concrete_args = {p.name: p.default for p in sig.parameters.values() if p.name not in input_names}

hf_tracer = fx.HFTracer()

hf_tracer.trace(model, concrete_args=concrete_args)
```

```
Traceback (most recent call last):
  File "/Users/pbelevich/PycharmProjects/PiPPy/test/hf_test3.py", line 14, in <module>
    hf_tracer.trace(model, concrete_args=concrete_args)
  File "/Users/pbelevich/PycharmProjects/pbelevich-transformers/src/transformers/utils/fx.py", line 877, in trace
    self.graph = super().trace(root, concrete_args=concrete_args)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 587, in trace
    self.create_node('output', 'output', (self.create_arg(fn(*args)),), {},
  File "/Users/pbelevich/PycharmProjects/pbelevich-transformers/src/transformers/models/layoutlm/modeling_layoutlm.py", line 935, in forward
    outputs = self.layoutlm(
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 577, in module_call_wrapper
    return self.call_module(mod, forward, args, kwargs)
  File "/Users/pbelevich/PycharmProjects/pbelevich-transformers/src/transformers/utils/fx.py", line 834, in call_module
    return super().call_module(m, forward, args, kwargs)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 372, in call_module
    return forward(*args, **kwargs)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 573, in forward
    return _orig_module_call(mod, *args, **kwargs)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/pbelevich/PycharmProjects/pbelevich-transformers/src/transformers/models/layoutlm/modeling_layoutlm.py", line 803, in forward
    bbox = torch.zeros(tuple(list(input_shape) + [4]), dtype=torch.long, device=device)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/proxy.py", line 260, in __iter__
    return self.tracer.iter(self)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/proxy.py", line 169, in iter
    raise TraceError('Proxy object cannot be iterated. This can be '
torch.fx.proxy.TraceError: Proxy object cannot be iterated. This can be attempted when the Proxy is used in a loop or as a *args or **kwargs function argument. See the torch.fx docs on pytorch.org for a more detailed explanation of what types of control flow can be traced, and check out the Proxy docstring for help troubleshooting Proxy iteration errors

Process finished with exit code 1
```

@michaelbenayoun 
```
import inspect

import transformers.utils.fx as fx
from transformers import *

model = Speech2TextForConditionalGeneration(Speech2TextConfig())

input_names = model.dummy_inputs.keys()
sig = inspect.signature(model.forward)
concrete_args = {p.name: p.default for p in sig.parameters.values() if p.name not in input_names}

hf_tracer = fx.HFTracer()

hf_tracer.trace(model, concrete_args=concrete_args)
```
```
Traceback (most recent call last):
  File "/Users/pbelevich/PycharmProjects/PiPPy/test/hf_test4.py", line 14, in <module>
    hf_tracer.trace(model, concrete_args=concrete_args)
  File "/Users/pbelevich/PycharmProjects/pbelevich-transformers/src/transformers/utils/fx.py", line 877, in trace
    self.graph = super().trace(root, concrete_args=concrete_args)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 587, in trace
    self.create_node('output', 'output', (self.create_arg(fn(*args)),), {},
  File "/Users/pbelevich/PycharmProjects/pbelevich-transformers/src/transformers/models/speech_to_text/modeling_speech_to_text.py", line 1349, in forward
    outputs = self.model(
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 577, in module_call_wrapper
    return self.call_module(mod, forward, args, kwargs)
  File "/Users/pbelevich/PycharmProjects/pbelevich-transformers/src/transformers/utils/fx.py", line 834, in call_module
    return super().call_module(m, forward, args, kwargs)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 372, in call_module
    return forward(*args, **kwargs)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 573, in forward
    return _orig_module_call(mod, *args, **kwargs)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/pbelevich/PycharmProjects/pbelevich-transformers/src/transformers/models/speech_to_text/modeling_speech_to_text.py", line 1193, in forward
    encoder_outputs = self.encoder(
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 577, in module_call_wrapper
    return self.call_module(mod, forward, args, kwargs)
  File "/Users/pbelevich/PycharmProjects/pbelevich-transformers/src/transformers/utils/fx.py", line 834, in call_module
    return super().call_module(m, forward, args, kwargs)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 372, in call_module
    return forward(*args, **kwargs)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 573, in forward
    return _orig_module_call(mod, *args, **kwargs)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/pbelevich/PycharmProjects/pbelevich-transformers/src/transformers/models/speech_to_text/modeling_speech_to_text.py", line 770, in forward
    inputs_embeds = self.conv(input_features)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 577, in module_call_wrapper
    return self.call_module(mod, forward, args, kwargs)
  File "/Users/pbelevich/PycharmProjects/pbelevich-transformers/src/transformers/utils/fx.py", line 834, in call_module
    return super().call_module(m, forward, args, kwargs)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 372, in call_module
    return forward(*args, **kwargs)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/fx/_symbolic_trace.py", line 573, in forward
    return _orig_module_call(mod, *args, **kwargs)
  File "/Users/pbelevich/miniconda3/envs/PiPPy/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/pbelevich/PycharmProjects/pbelevich-transformers/src/transformers/models/speech_to_text/modeling_speech_to_text.py", line 124, in forward
    hidden_states = input_features.transpose(1, 2).contiguous()  # -> B x (C x D) x T
AttributeError: 'NoneType' object has no attribute 'transpose'

Process finished with exit code 1
```

@pbelevich Thanks for raising those issues!

- About LayoutLM, I just pushed a fix that should solve the issue.
- About Speech2Text, I don't think that this is an issue, it's just that the dummy inputs for this model are wrong... it creates `input_ids` but it should create something else since this model does not have `input_ids` as inputs... I added a check in the `symbolic_trace` function (and not `HFTracer.trace`), that will test if the `input_names` passed are correct for the model we want to trace.