KerasTensor can't be used with TFBertTokenizer
Author: antoninoLorenzo
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.48.3
- Platform: Linux-6.1.85+-x86_64-with-glibc2.35
- Python version: 3.11.11
- Huggingface_hub version: 0.28.1
- Safetensors version: 0.5.2
- Accelerate version: 1.3.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.5.1+cu124 (False)
- Tensorflow version (GPU?): 2.18.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.10.3 (cpu)
- Jax version: 0.4.33
- JaxLib version: 0.4.33
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@gante and @Rocketknight1

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

I'm trying to make a Text Extraction model using BERT, to do this I'm trying to make a model using Keras following [this](https://keras.io/examples/nlp/text_extraction_with_bert/) tutorial.

```python
import os
import tensorflow as tf
from tensorflow import keras
from datasets import load_dataset
from tokenizers import BertWordPieceTokenizer
from transformers import BertTokenizer, TFBertTokenizer, TFBertModel, BertConfig


max_len = 512

input_ids = keras.layers.Input(shape=(max_len,), dtype=tf.int32)
token_type_ids = keras.layers.Input(shape=(max_len,), dtype=tf.int32)
attention_mask = keras.layers.Input(shape=(max_len,), dtype=tf.int32)
encoder = TFBertModel.from_pretrained("bert-base-uncased")

embedding = encoder(
    input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask
)[0]
```

When I try to run the code (on colab) I get a `ValueError` stating that `KerasLayer` isn't accepted as input. I even tried  using `keras.layers.Lambda(lambda x: tf.identity(x))(input_ids)` to make the layers treated as `Tensor` but I still get the same result; here is the full stack trace:
```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-34-24c5e536e6d2>](https://localhost:8080/#) in <cell line: 0>()
      2 token_type_ids = keras.layers.Input(shape=(max_len,), dtype=tf.int32)
      3 attention_mask = keras.layers.Input(shape=(max_len,), dtype=tf.int32)
----> 4 embedding = encoder(
      5     input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask
      6 )[0]

2 frames
[/usr/local/lib/python3.11/dist-packages/transformers/modeling_tf_utils.py](https://localhost:8080/#) in input_processing(func, config, **kwargs)
    511             output[k] = v
    512         else:
--> 513             raise ValueError(f"Data of type {type(v)} is not allowed only {allowed_types} is accepted for {k}.")
    514 
    515     if isinstance(main_input, (tuple, list)):

ValueError: Exception encountered when calling layer 'tf_bert_model_1' (type TFBertModel).

Data of type <class 'keras.src.backend.common.keras_tensor.KerasTensor'> is not allowed only (<class 'tensorflow.python.framework.tensor.Tensor'>, <class 'bool'>, <class 'int'>, <class 'transformers.utils.generic.ModelOutput'>, <class 'tuple'>, <class 'list'>, <class 'dict'>, <class 'numpy.ndarray'>) is accepted for token_type_ids.

Call arguments received by layer 'tf_bert_model_1' (type TFBertModel):
  • input_ids=<KerasTensor shape=(None, 512), dtype=int32, sparse=False, name=keras_tensor_33>
  • attention_mask=<KerasTensor shape=(None, 512), dtype=int32, sparse=False, name=keras_tensor_35>
  • token_type_ids=<KerasTensor shape=(None, 512), dtype=int32, sparse=False, name=keras_tensor_34>
  • position_ids=None
  • head_mask=None
  • inputs_embeds=None
  • encoder_hidden_states=None
  • encoder_attention_mask=None
  • past_key_values=None
  • use_cache=None
  • output_attentions=None
  • output_hidden_states=None
  • return_dict=None
  • training=False
```

### Expected behavior

Since the `TFBertModel` is a wrapper to `TFBertMainLayer` that itself is a subclass of a keras layer, I don't know why it doesn't work; also I can't find where the function `input_processing` from `modeling_tf_utils.py` is actually called in the codebase, so I'm on a dead end.

However, the expected outcome is that I should be able somehow to extend BERT model in Keras using `TFBertModel`.

Comments:
Hi @antoninoLorenzo, Transformers only supports Keras 2. When you use `keras.layers`, those are Keras 3 objects. Passing Keras 3 objects to a Keras 2 model is not supported.

The best solution is to install the `tf-keras` compatibility package and then import from `tf_keras` instead. Alternatively, you can set the `TF_USE_LEGACY_KERAS` environment var and then import from `tf.keras`.

If you prefer to use Keras 3, it's still possible to load a lot of models! However, you should use the **keras-nlp** framework for this instead of Transformers. For supported architectures, Keras-NLP can load model checkpoints from the Hugging Face Hub the same way Transformers can: https://huggingface.co/blog/keras-hub-integration