Longformer model: tf.Tensor as a Python bool is not allowed
Author: rdisipio
Pull request: no

### System Info

Python 3.9.7
Tensorflow 2.13.0
Transformers 4.31.0
keras 2.13.1
huggingface-hub 0.16.4
CUDA 11.8


### Who can help?

@ArthurZucker, @gante and @Rocketknight1

I am upgrading pre-existing code (which worked perfectly) to the latest releases of TensorFlow and transformers. One of my tests now fails to save to disc a trained model which includes Longformer as the embedder. You can find a dump of the full error message below, however the main issue is the following:

```
Using a symbolic `tf.Tensor` as a Python `bool` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.
```

the offending line being:

```
if padding_len > 0:
         blablabla
```

see the code here: https://github.com/huggingface/transformers/blob/e42587f596181396e1c4b63660abf0c736b10dae/src/transformers/models/longformer/modeling_tf_longformer.py#L1821 

Digging into the repository, I realized that a [PR](https://github.com/huggingface/transformers/pull/9942) was actually opened by @jplu and merged in 2021. I don't understand why the updated lines of code are not part of the master branch. 
Have these changes been discarded for some reason, or is there a deeper issue?

Cheers,
Riccardo
 


```
Traceback (most recent call last):
  File "/actions-runner/_work/jd-parser/jd-parser/./scripts/train.py", line 128, in <module>
    trainer.save_model(model_path=output_file)
  File "/actions-runner/_work/jd-parser/jd-parser/jd_parser/trainer.py", line 418, in save_model
    self.model.save(model_path)
  File "/ml-data/jd_parser_data/venv/lib/python3.9/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/actions-runner/_work/_tool/pyenv_root/2.3.0/x64/versions/3.9.7/lib/python3.9/contextlib.py", line 126, in __exit__
    next(self.gen)
  File "/ml-data/jd_parser_data/venv/lib/python3.9/site-packages/transformers/modeling_tf_utils.py", line 426, in run_call_with_unpacked_inputs
    return func(self, **unpacked_inputs)
  File "/ml-data/jd_parser_data/venv/lib/python3.9/site-packages/transformers/models/longformer/modeling_tf_longformer.py", line 2037, in call
    outputs = self.longformer(
  File "/ml-data/jd_parser_data/venv/lib/python3.9/site-packages/transformers/modeling_tf_utils.py", line 426, in run_call_with_unpacked_inputs
    return func(self, **unpacked_inputs)
  File "/ml-data/jd_parser_data/venv/lib/python3.9/site-packages/transformers/models/longformer/modeling_tf_longformer.py", line 1735, in call
    ) = self._pad_to_window_size(
  File "/ml-data/jd_parser_data/venv/lib/python3.9/site-packages/transformers/models/longformer/modeling_tf_longformer.py", line 1821, in _pad_to_window_size
    if padding_len > 0:
tensorflow.python.framework.errors_impl.OperatorNotAllowedInGraphError: Exception encountered when calling layer 'longformer' (type TFLongformerMainLayer).

Using a symbolic `tf.Tensor` as a Python `bool` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.

Call arguments received by layer 'longformer' (type TFLongformerMainLayer):
  • self=tf.Tensor(shape=(None, None), dtype=int32)
  • input_ids=None
  • attention_mask=tf.Tensor(shape=(None, None), dtype=int32)
  • head_mask=None
  • global_attention_mask=tf.Tensor(shape=(None, None), dtype=int32)
  • token_type_ids=tf.Tensor(shape=(None, None), dtype=int32)
  • position_ids=None
  • inputs_embeds=None
  • output_attentions=False
  • output_hidden_states=False
  • return_dict=True
  • training=True
```

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
>>> from transformers import AutoConfig, TFAutoModel
>>> import tensorflow as tf
>>> 
>>> pretrained_bert_model = "allenai/longformer-base-4096"
>>> enc_config = AutoConfig.from_pretrained(pretrained_bert_model)
>>> encoder = TFAutoModel.from_pretrained(pretrained_bert_model, config=enc_config)
>>> token_ids = tf.keras.layers.Input(shape=(None,), name="token_ids", dtype=tf.int32)
>>> attn_mask = tf.keras.layers.Input(shape=(None,), name="attn_mask", dtype=tf.int32)
>>> inputs = {'input_ids': token_ids, 'attention_mask': attn_mask}
>>> embedding = encoder(**inputs)
>>> pooled_output = embedding[1]
>>> output = tf.keras.layers.Dense(1)(pooled_output)
>>> model = tf.keras.Model(inputs=inputs, outputs=output)
>>> model.build((None,1))
>>> model.save("mymodel")

WARNING:tensorflow:Compiled the loaded model, but the compiled metrics have yet to be built. `model.compile_metrics` will be empty until you train or evaluate the model.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/Riccardo.DiSipio/myproject/venv/lib/python3.9/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/contextlib.py", line 124, in __exit__
    next(self.gen)
  File "/Users/Riccardo.DiSipio/myproject/venv/lib/python3.9/site-packages/transformers/modeling_tf_utils.py", line 426, in run_call_with_unpacked_inputs
    return func(self, **unpacked_inputs)
  File "/Users/Riccardo.DiSipio/myproject/venv/lib/python3.9/site-packages/transformers/models/longformer/modeling_tf_longformer.py", line 2037, in call
    outputs = self.longformer(
  File "/Users/Riccardo.DiSipio/myproject/venv/lib/python3.9/site-packages/transformers/modeling_tf_utils.py", line 426, in run_call_with_unpacked_inputs
    return func(self, **unpacked_inputs)
  File "/Users/Riccardo.DiSipio/myproject/venv/lib/python3.9/site-packages/transformers/models/longformer/modeling_tf_longformer.py", line 1735, in call
    ) = self._pad_to_window_size(
  File "/Users/Riccardo.DiSipio/myproject/venv/lib/python3.9/site-packages/transformers/models/longformer/modeling_tf_longformer.py", line 1821, in _pad_to_window_size
    if padding_len > 0:
tensorflow.python.framework.errors_impl.OperatorNotAllowedInGraphError: Exception encountered when calling layer 'longformer' (type TFLongformerMainLayer).

Using a symbolic `tf.Tensor` as a Python `bool` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.

Call arguments received by layer 'longformer' (type TFLongformerMainLayer):
  • self=tf.Tensor(shape=(None, None), dtype=int32)
  • input_ids=None
  • attention_mask=tf.Tensor(shape=(None, None), dtype=int32)
  • head_mask=None
  • global_attention_mask=tf.Tensor(shape=(None, None), dtype=int32)
  • token_type_ids=tf.Tensor(shape=(None, None), dtype=int32)
  • position_ids=None
  • inputs_embeds=None
  • output_attentions=False
  • output_hidden_states=False
  • return_dict=True
  • training=True
```

### Expected behavior

It is supposed to save the model to a local folder.

Comments:
cc our TF compile expert @Rocketknight1 !

Looks like this change was (accidentaly?) reverted in this PR with no clear indication as to why https://github.com/huggingface/transformers/pull/10007/files#diff-782b222e9d393fe6750cf8e4cd870bcf3748a92ade5086e518b4d716a80080f8R1723, it likely reintroduced this bug

@rdisipio @arthurkok2 we've opened a PR to fix this issue at #25496.

@rdisipio @arthurkok2 PR has been merged. You can try it by installing from main with
`pip install --upgrade git+https://github.com/huggingface/transformers.git`

If you're still encountering issues afterwards, feel free to reopen the issue and let me know!