Repository: keras-team/keras (python)

Issue:
Cannot call AttentionLayer with symbolic tensors and return_attention_scores=True in Keras 3
In Keras 3 I cannot call the call method of AttentionLayer (or other attention layer classes) with a symbolic KerasTensor input and return_attention_scores=True.

I am using Python 3.12.1 and Keras 3.6.0

```
>>> in1 = keras.Input(shape=(10, 7))
>>> in2 = keras.Input(shape=(8, 7))
>>> attLayer = keras.layers.Attention()     
>>> out1, out2 = attLayer([in1, in2], return_attention_scores=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<venv_root>/lib/python3.12/site-packages/keras/src/backend/common/keras_tensor.py", line 167, in __iter__
    raise NotImplementedError(
NotImplementedError: Iterating over a symbolic KerasTensor is not supported.
```

The layer call method works when replacing the symbolic inputs with np.ndarray inputs.

Pull request:
fix attention output with symbolic tensors and attention scores
fix attention layer output with symbolic tensors and return_attention_scores = True .

Fixes #20621

Patch:
diff --git a/keras/src/layers/attention/attention.py b/keras/src/layers/attention/attention.py
index 15ff906e5..d336781c8 100644
--- a/keras/src/layers/attention/attention.py
+++ b/keras/src/layers/attention/attention.py
@@ -280,7 +280,7 @@ class Attention(Layer):
         output_spec = KerasTensor(output_shape, dtype=self.compute_dtype)
 
         # Handle attention scores if requested
-        if self._return_attention_scores:
+        if self._return_attention_scores or return_attention_scores:
             scores_shape = (
                 query.shape[0],
                 query.shape[1],


Test patch:
diff --git a/keras/src/layers/attention/attention_test.py b/keras/src/layers/attention/attention_test.py
index eab40b2a0..88598d721 100644
--- a/keras/src/layers/attention/attention_test.py
+++ b/keras/src/layers/attention/attention_test.py
@@ -417,3 +417,15 @@ class AttentionTest(testing.TestCase):
         self.assertEqual(
             attention_scores.shape, (2, 8, 4)
         )  # Attention scores shape
+
+    def test_return_attention_scores_with_symbolic_tensors(self):
+        """Test to check outputs with symbolic tensors with
+        return_attention_scores = True"""
+        attention = layers.Attention()
+        x = layers.Input(shape=(3, 5))
+        y = layers.Input(shape=(4, 5))
+        output, attention_scores = attention(
+            [x, y], return_attention_scores=True
+        )
+        self.assertEqual(output.shape, (None, 3, 5))  # Output shape
+        self.assertEqual(attention_scores.shape, (None, 3, 4))