Repository: tensorflow
Issue:  Test_on_batch() gives the same loss output on different batches in a single run
Labels: stat:awaiting tensorflower,type:bug,comp:keras,TF2.14

### Issue type

Bug

### Have you reproduced the bug with TensorFlow Nightly?

Yes


### TensorFlow version

tf 2.10.0

### Custom code

Yes

### OS platform and distribution

windows 10

### Python version

3.10.12

### Current behavior?

I noticed the problem when I got a straight horizontal line on plotting the test results on the trained network. I used the sequential models.

I use train_on_batch(), which gives converging losses. When I switch to test_on_batch(), the losses remain the same for different batches. When I restart the test with different test files, it will give a different loss value, which remains the same for all the batches. In other words, the loss from test_on_batch() remains the some for all batches in a single run.

It's a sequential model.

Here is the code of the section:

    print('mfccs3 value = ', tf.keras.backend.eval(mfccs3[1,:]) )               
    #logs = vadModel.train_on_batch(mfccs3, vadLabel)
    logs = vadModel.test_on_batch(mfccs3, vadLabel) 
    print('string logs = ', str(logs))
The result is:
index = 1

```
mfccs3 value = [[-8.2793800e+01 -5.9538417e+00 9.8302096e-01 -3.5255635e-01
3.0392697e-01 -6.4597696e-01 2.2358397e-02 2.5344249e-02
-6.8171650e-01 -3.7053981e-01 -3.4044239e-01 -8.1056818e-02]]
```

string logs = 0.2398043

index = 2

```
mfccs3 value = [[-69.159195 -2.2269542 4.2501264 -1.3486748 0.62957734
-3.2606528 -3.253118 -3.5308673 -1.1313365 -1.1839466
-2.330786 -1.6313086 ]]
```

string logs = 0.2398043

index = 3

```
mfccs3 value = [[-64.894104 -1.892648 0.11392474 -0.81098145 -1.4640433
-1.1901256 -1.7744782 -0.85753983 -0.9694403 -0.8149232
-1.0680746 -1.0442001 ]]
```

string logs = 0.2398043

You can see that the inputs for test_on_batch() have changed. However, the loss remains the same. I use the same code for train_on_batch(), which gives converging losses.



### Standalone code to reproduce the issue

```shell
logs = vadModel.train_on_batch(mfccs3, vadLabel)
""" vs.  """
    logs = vadModel.test_on_batch(mfccs3, vadLabel) 

it's just these two lines for a sequential model.
```


### Relevant log output

```shell
I even tried latest Tensorflow version. It has the same problem.

tensorflow version: 2.15.0-dev20230817
listOfFiles 1681
2023-08-17 15:42:12.560549: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE SSE2 SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
model length =  7
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 lstm (LSTM)                 multiple                  2640      
                                                                 
 dense (Dense)               multiple                  210       
                                                                 
 dense_1 (Dense)             multiple                  11        
                                                                 
=================================================================
Total params: 2861 (11.18 KB)
Trainable params: 2861 (11.18 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
```