TFSwinModel: AttributeError: module 'keras.utils' has no attribute 'unpack_x_y_sample_weight'
Author: adam-kozlowski-280
Pull request: no
Labels: name: bug

### System Info

python==3.8.10
transformers[tf]==4.38.2 (also tried 4.44.1)
tensorflow==2.12.1
keras==2.12.0

### Who can help?

 @gante and @Rocketknight1

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I've tried to use `TFSwinModel` for segmentation training, however I've got an `AttributeError: module 'keras.utils' has no attribute 'unpack_x_y_sample_weight'` during fit.. The code below should reproduce the error:
```
import tensorflow as tf
from transformers import SwinConfig, TFSwinModel

image_size = 224

# create dummy dataset
X = tf.random.normal((10, 3, image_size, image_size))
Y = tf.random.normal((10, 3, image_size, image_size))
dataset = tf.data.Dataset.from_tensor_slices((X, Y))
dataset = dataset.batch(4)

# create model
config = SwinConfig(image_size=image_size)
model = TFSwinModel(config)
model.compile(optimizer="adam", loss="mse")

# train model
model.fit(dataset, epochs=1)
```
Error message:
```
Traceback (most recent call last):
  File "swin_dev.py", line 16, in <module>
    model.fit(dataset, epochs=1)
  File "/mnt/md0/workspace/home/akozlowski/venvs/troy3-8/lib/python3.8/site-packages/transformers/modeling_tf_utils.py", line 1161, in fit
    return super().fit(*args, **kwargs)
[...]
  File "/mnt/md0/workspace/home/akozlowski/venvs/troy3-8/lib/python3.8/site-packages/transformers/modeling_tf_utils.py", line 1562, in train_step
    x, y, sample_weight = keras.utils.unpack_x_y_sample_weight(data)
AttributeError: in user code:

    File "/mnt/md0/workspace/home/akozlowski/venvs/troy3-8/lib/python3.8/site-packages/keras/engine/training.py", line 1284, in train_function  *
        return step_function(self, iterator)
    File "/mnt/md0/workspace/home/akozlowski/venvs/troy3-8/lib/python3.8/site-packages/keras/engine/training.py", line 1268, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/mnt/md0/workspace/home/akozlowski/venvs/troy3-8/lib/python3.8/site-packages/keras/engine/training.py", line 1249, in run_step  **
        outputs = model.train_step(data)
    File "/mnt/md0/workspace/home/akozlowski/venvs/troy3-8/lib/python3.8/site-packages/transformers/modeling_tf_utils.py", line 1562, in train_step
        x, y, sample_weight = keras.utils.unpack_x_y_sample_weight(data)

    AttributeError: module 'keras.utils' has no attribute 'unpack_x_y_sample_weight'
```

### Expected behavior

The fit method runs training loop

Comments:
Hi @adam-kozlowski-280, I don't get the same error! Can you try a newer version of TF, maybe? Make sure to also install the `tf-keras` package.

@Rocketknight1 Are you sure you have environment with these specific requirements? Just now I have rebuilt the environment with these steps:
```
python3.8 -V # gives 3.8.10
python3.8 -m venv venvs/transformers
source venvs/transformers/bin/activate
pip install tensorflow==2.12.1 
pip install transformers[tf]==4.38.2
```
This results in the environemnt mentioned above + I have `tf-keras==2.15.1` (installed by `transformers[tf]`). The running script I've provided above should raise mentioned error. 

Recently there was update to python 3.8.19 which no longer supports tensorflow 2.12.1, I've tried to reproduce on different computer but it didn't work.

Upgrading both `tensorflow`to 2.13  and `transformes[tf]` to 4.44.2 (latest) solved the issue. We'll soon migrate to newer python and tensorflow version, but still this bug does not allow to use SWIN model on older version of both tensorflow and python

btw, this problem exsits also for `SegFormer `model in this particular environment setup

Hi @adam-kozlowski-280 - yes, this probably indicates we need to update our minimum TF version! We probably won't rewrite the code to support older TF versions, because there will likely be other bugs as well.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.