torch.jit.script for BertEmbeddings works on MacOS but fails on Linux
Author: priyamtejaswin
Pull request: no

### System Info

transformers : 4.23.1
torch : 1.13.0
python : 3.9.13

**Versions are the same on MacOS and Linux.**

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
###### On MacOS ######
import torch
from transformers.models.bert.modeling_bert import BertConfig, BertEmbeddings

config = BertConfig()
embed = BertEmbeddings(config)

scripted = torch.jit.script(embed)
print(scripted)

OUTPUT = """
/opt/anaconda3/envs/mobi/lib/python3.9/site-packages/torch/jit/annotations.py:299: UserWarning: TorchScript will treat type annotations of Tensor dtype-specific subtypes as if they are normal Tensors. dtype constraints are not enforced in compilation either.
  warnings.warn("TorchScript will treat type annotations of Tensor "
RecursiveScriptModule(
  original_name=BertEmbeddings
  (word_embeddings): RecursiveScriptModule(original_name=Embedding)
  (position_embeddings): RecursiveScriptModule(original_name=Embedding)
  (token_type_embeddings): RecursiveScriptModule(original_name=Embedding)
  (LayerNorm): RecursiveScriptModule(original_name=LayerNorm)
  (dropout): RecursiveScriptModule(original_name=Dropout)
)
"""
```

```python
###### On Linux ######
import torch
from transformers.models.bert.modeling_bert import BertConfig, BertEmbeddings

config = BertConfig()
embed = BertEmbeddings(config)

scripted = torch.jit.script(embed)
print(scripted)

OUTPUT  =  """
/miniconda3/envs/mobile/lib/python3.9/site-packages/torch/jit/annotations.py:299: UserWarning: TorchScript will treat type annotations of Tensor dtype-specific subtypes as if they are normal Tensors. dtype constraints are not enforced in compilation either.
  warnings.warn("TorchScript will treat type annotations of Tensor "
Traceback (most recent call last):
  File "/ocean/projects/tra220029p/tejaswin/ViLT/script_bertembeddings.py", line 7, in <module>
    scripted = torch.jit.script(embed)
  File "/ocean/projects/tra220029p/tejaswin/miniconda3/envs/mobile/lib/python3.9/site-packages/torch/jit/_script.py", line 1286, in script
    return torch.jit._recursive.create_script_module(
  File "/ocean/projects/tra220029p/tejaswin/miniconda3/envs/mobile/lib/python3.9/site-packages/torch/jit/_recursive.py", line 476, in create_script_module
    return create_script_module_impl(nn_module, concrete_type, stubs_fn)
  File "/ocean/projects/tra220029p/tejaswin/miniconda3/envs/mobile/lib/python3.9/site-packages/torch/jit/_recursive.py", line 542, in create_script_module_impl
    create_methods_and_properties_from_stubs(concrete_type, method_stubs, property_stubs)
  File "/ocean/projects/tra220029p/tejaswin/miniconda3/envs/mobile/lib/python3.9/site-packages/torch/jit/_recursive.py", line 393, in create_methods_and_properties_from_stubs
    concrete_type._create_methods_and_properties(property_defs, property_rcbs, method_defs, method_rcbs, method_defaults)
RuntimeError: 
'Optional[Tensor]' object has no attribute or method 'size'.:
  File "/ocean/projects/tra220029p/tejaswin/miniconda3/envs/mobile/lib/python3.9/site-packages/transformers/models/bert/modeling_bert.py", line 212
            input_shape = input_ids.size()
        else:
            input_shape = inputs_embeds.size()[:-1]
                          ~~~~~~~~~~~~~~~~~~ <--- HERE
    
        seq_length = input_shape[1]
"""
```

### Expected behavior

The code should work on both platforms.

Comments:
Looks like the inconsistency comes from PyTorch, if all versions are the same.

Thanks @sgugger . I have a temporary workaround on Linux. Will raise it on the PyTorch forums.

> Thanks @sgugger . I have a temporary workaround on Linux. Will raise it on the PyTorch forums.

How did you solve this problem? I also encountered the same problem. Can you ask me for experience?