internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: keras-team/keras

Issue:
EfficientNetV2 does not match google implementation
Hi,
I may be wrong, but checking the efficientnetv2 implementation I think there is a difference with the [google one](https://github.com/google/automl/blob/387d5ddb92bb8fbbec4b012e5636a81ea65fffda/efficientnetv2/effnetv2_model.py)
The  ["survival_probability"](https://github.com/keras-team/keras/blob/d8fcb9d4d4dad45080ecfdd575483653028f8eda/keras/applications/efficientnet_v2.py#L990) is defined as `survival_probability=drop_connect_rate * b / blocks` but b is set to zero, while according to the [google implementation](https://github.com/google/automl/blob/387d5ddb92bb8fbbec4b012e5636a81ea65fffda/efficientnetv2/effnetv2_model.py#L619) it should increase with the "number" of the block.

I think that line should be replaced with:
`survival_probability=drop_connect_rate * i / blocks` 
where i is the counter of the for loop



Discussion:
cc. @sebastian-sz 
thanks for cc @innat.

Yes, I think you are correct @SergioG-M. 
I mostly followed the Efficientnet V1 implementation, which indeed increases `b` in the [end of the loop](https://github.com/keras-team/keras/blob/d8fcb9d4d4dad45080ecfdd575483653028f8eda/keras/applications/efficientnet.py#L360) and it looks like i missed this step.

Without it, the value of `drop_connect_rate * b / blocks` is always 0.0.

>I think that line should be replaced with:
survival_probability=drop_connect_rate * i / blocks
where i is the counter of the for loop

EDIT: I don't think this will work, as b is increased also in the second for loop, as opposed to i, which increases only when second for loop finishes. 
One can, similar to Efficientnet V1 implementation, add `b+=1` in line 995 of `efficientnet_v2.py`.

I can submit the PR with the fix, unless you want to do it? Please, let me know.
> I don't think this will work, as b is increased also in the second for loop, as opposed to i, which increases only when second for loop finishes.

You're right, I was following the google implementation, but their blocks already include the second for loop in the keras one, so it should be` b += 1`

Go ahead with the PR please.

Resolving patch:
diff --git a/keras/applications/efficientnet_v2.py b/keras/applications/efficientnet_v2.py
--- a/keras/applications/efficientnet_v2.py
+++ b/keras/applications/efficientnet_v2.py
@@ -991,6 +991,7 @@ def EfficientNetV2(
           name="block{}{}_".format(i + 1, chr(j + 97)),
           **args,
       )(x)
+      b += 1
 
   # Build top
   top_filters = round_filters(


Test patch:
Repository: keras-team/keras

Issue:
EfficentNets no longer train on non-float32 inputs
**System information**
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
- TensorFlow installed from (source or binary): Pip `tf-nightly`
- TensorFlow version (use command below): 2.10.0-dev20220409
- Python version: 3.7.3
- GPU model and memory: N/A - reproducible on CPU, GPU, and TPU

**Describe the problem**.
[This PR](https://github.com/keras-team/keras/pull/16032/files) merged to solve [an issue with efficientnet normalisation](https://github.com/tensorflow/tensorflow/issues/49930) hardcoded a set of float values into the efficientnet architecture.

These magic numbers are fixed to float32 when running in graph mode, and will raise an exception as soon as the network is called on float16 or bfloat16 inputs. [bfloat16 inputs are recommended when training on TPUs](https://cloud.google.com/tpu/docs/bfloat16#improving_performance_with_bfloat16) 

**Describe the current behavior**.
An exception is raised whenever a keras efficientnet is trained on a non-float32 input

**Describe the expected behavior**.
An exception is not raised whenever a keras efficientnet is trained on a non-float32 input



- Do you want to contribute a PR? (yes/no): yes
- Briefly describe your candidate solution(if contributing):  Replace the hardcoded graph constants with:

`x = layers.Rescaling(1. / tf.math.sqrt(IMAGENET_STDDEV_RGB))(x)`

**Standalone code to reproduce the issue**.

```python
import tensorflow as tf
from tensorflow.keras.applications import efficientnet as efn

tf.keras.mixed_precision.set_global_policy("mixed_bfloat16")

ds = (
    tf.data.Dataset.from_tensor_slices(
        (
            tf.ones([1, 224, 224, 3], dtype=tf.bfloat16),
            tf.ones([1, 1000], dtype=tf.float32),
        )
    )
    .repeat()
    .batch(batch_size=1)
)

e = efn.EfficientNetB0()
e.compile(loss="binary_crossentropy")
e.fit(ds, steps_per_epoch=1, epochs=1)
```


Discussion:

Resolving patch:
diff --git a/keras/applications/efficientnet.py b/keras/applications/efficientnet.py
--- a/keras/applications/efficientnet.py
+++ b/keras/applications/efficientnet.py
@@ -331,7 +331,7 @@ def round_repeats(repeats):
     # normalize the input, we need to divide another sqrt(var) to match the
     # original implementation.
     # See https://github.com/tensorflow/tensorflow/issues/49930 for more details
-    x = x / tf.math.sqrt(IMAGENET_STDDEV_RGB)
+    x = layers.Rescaling(1. / tf.math.sqrt(IMAGENET_STDDEV_RGB))(x)
 
   x = layers.ZeroPadding2D(
       padding=imagenet_utils.correct_pad(x, 3),


Test patch:
Repository: keras-team/keras

Issue:
The vocabulary_size method of preprocessing layers does not work in graph mode
**System information**.
- Have I written custom code (as opposed to using a stock example script provided in Keras): **yes**
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): **Colab**
- TensorFlow installed from (source or binary): **binary**
- TensorFlow version (use command below): **TF 2.8, nightly 2.10.0-dev20220422**

**Describe the problem**.

Using the `vocabulary_size()` method of preprocessing layers (like `tf.keras.layers.StringLookup`) fails in graph mode, because the implementation looks like this: https://github.com/keras-team/keras/blob/aea9728313bcaa8262774699c21976288171b209/keras/layers/preprocessing/index_lookup.py#L346-L352

**Describe the current behavior**.

Calling `vocabulary_size()` in graph mode fails.

**Describe the expected behavior**.

Calling `vocabulary_size()` in graph mode succeeds and returns a `tf.Tensor` with the size.

**Standalone code to reproduce the issue**.

Colab notebook showing the issue is at https://colab.research.google.com/drive/1Mq9G8eUvNLw6ykk4ARKf6jurLREAvRhu?usp=sharing both for TF 2.8 and TF nightly.

**Source code / logs**.

An obvious fix is to avoid using the `numpy()` method when in graph mode, i.e., instead of
```python
return int(self.lookup_table.size().numpy()) + self._token_start_index()
```
use just
```python
return self.lookup_table.size() + self._token_start_index()
```
Note that the above notebook also shows that this implementation works in the graph mode.


Discussion:

Resolving patch:
diff --git a/keras/layers/preprocessing/index_lookup.py b/keras/layers/preprocessing/index_lookup.py
--- a/keras/layers/preprocessing/index_lookup.py
+++ b/keras/layers/preprocessing/index_lookup.py
@@ -349,7 +349,10 @@ def vocabulary_size(self):
     Returns:
       The integer size of the voculary, including optional mask and oov indices.
     """
-    return int(self.lookup_table.size().numpy()) + self._token_start_index()
+    if tf.executing_eagerly():
+      return int(self.lookup_table.size().numpy()) + self._token_start_index()
+    else:
+      return self.lookup_table.size() + self._token_start_index()
 
   def vocab_size(self):
     logging.warning("vocab_size is deprecated, please use vocabulary_size.")


Test patch:
Repository: keras-team/keras

Issue:
Function _create_seed() in keras.backend causes DeprecationWarning in Python 3.10
In [line 2032 of keras.backend.py](https://github.com/keras-team/keras/blob/af1408d3255e3db9067522762e22a6c454c56654/keras/backend.py#L2032) we call random.randint(1, 1e9), which causes DeprecationWarning in Python 3.10 and will raise an error in some future version of Python. I suggest to replace it with `random.randint(1, int(1e9))` or `random.randint(1, 1000000000)`.


Discussion:
What would you think of adding support for `border_mode = same` instead?

I'd love to, but don't know how to do it. Can you help me figure it out?

I believe Lasagne supports it. Looking at the source, this seems to be the relevant bit: https://github.com/Lasagne/Lasagne/blob/master/lasagne/layers/conv.py#L441

Basically, setting border_mode to full and resizing the output after the fact. Very simple and straightforward.

Ok, thanks for the help. This is basically what the CropImage function did. Let me know what you think now and if you would do anything differently

Looks good, I updated the doc and the standardized the style. It's merged.

Thanks for the change!
Wouldn't it be less computationally intensive to pad with zeros before
calling conv2d and then use a _valid_ border_mode (perhaps by using the
newly ZeroPadding2D Layer?

On Thu, Jul 16, 2015 at 8:20 AM, François Chollet notifications@github.com
wrote:

> Looks good, I updated the doc and the standardized the style. It's merged.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/fchollet/keras/pull/396#issuecomment-121831833.


Resolving patch:
diff --git a/keras/backend.py b/keras/backend.py
--- a/keras/backend.py
+++ b/keras/backend.py
@@ -2029,7 +2029,7 @@ def _create_seed(self, user_specified_seed):
         elif getattr(_SEED_GENERATOR, "generator", None):
             return _SEED_GENERATOR.generator.randint(1, 1e9)
         else:
-            return random.randint(1, 1e9)
+            return random.randint(1, int(1e9))
 
     def random_normal(
         self, shape, mean=0.0, stddev=1.0, dtype=None, nonce=None


Test patch:
Repository: keras-team/keras

Issue:
Masking layer doesn´t work on GPU
Using a Masking layer in an LSTM model and Tensorflow as backend when training on CPU runs without problem, howerver, when using GPU the following error is shown:

```
Using a symbolic `tf.Tensor` as a Python `bool` is not allowed: AutoGraph is disabled in this function. Try decorating it directly with @tf.function.

Arguments received by LSTM.call():
  • sequences=tf.Tensor(shape=(None, 30, 5), dtype=float32)
  • initial_state=None
  • mask=tf.Tensor(shape=(None, 30), dtype=bool)
  • training=None
```

Here is a simple example in Google Colab where you can reproduce the exact behavior. Just make sure the GPU option is selected in Runtime:
https://colab.research.google.com/drive/1Kv90SJo1kcYye1Yl0nRHihRX3De1fBDd?usp=sharing

Otherwise, here is the code:

```
samples, timesteps, features = 32, 30, 8
x_train = np.random.random([samples, timesteps, features]).astype(np.float32)
x_train[:, 3, :] = 0.
x_train[:, 5, :] = 0.

y_train = np.random.random([samples, 1]).astype(np.float32)
inputs = keras_core.layers.Input(shape=(timesteps, features,))
mask = keras_core.layers.Masking(mask_value=0.)(inputs)
lstm = keras_core.layers.LSTM(32)(mask)
dense = keras_core.layers.Dense(1)(lstm)
model = keras_core.Model(inputs, dense)

model.compile(optimizer=keras_core.optimizers.Adam(learning_rate=0.001), loss=keras_core.losses.MeanSquaredError())
model.fit(x_train, y_train, epochs=10, batch_size=32)
```


Thanks in advance


Discussion:
For anyone interested in taking this up: it's coming from [here](https://github.com/keras-team/keras-core/blob/6383d8a81182b69e0319d78c4bac79fcbf5f7331/keras_core/backend/tensorflow/rnn.py#L823), since [_do_rnn_inputs_support_cudnn](https://github.com/keras-team/keras-core/blob/6383d8a81182b69e0319d78c4bac79fcbf5f7331/keras_core/backend/tensorflow/rnn.py#L537) emits either a bool or a bool-typed tensor.

Resolving patch:
diff --git a/keras/backend/tensorflow/rnn.py b/keras/backend/tensorflow/rnn.py
--- a/keras/backend/tensorflow/rnn.py
+++ b/keras/backend/tensorflow/rnn.py
@@ -441,6 +441,7 @@ def _step(time, output_ta_t, *states):
     return last_output, outputs, new_states
 
 
+@tf.function
 def gru(
     inputs,
     initial_state,
@@ -801,6 +802,7 @@ def cudnn_ok(
     return args_supported and _is_gpu_available()
 
 
+@tf.function
 def lstm(
     inputs,
     initial_state_h,
@@ -853,6 +855,7 @@ def lstm(
         raise NotImplementedError
 
 
+@tf.function(autograph=False)
 def _cudnn_lstm(
     inputs,
     initial_state_h,


Test patch:
Repository: keras-team/keras

Issue:
Doc: AdamW docstring is different from documentation on keras and tensorflow website
From [Keras](https://keras.io/api/optimizers/adamw/) documentation on AdamW:

> weight_decay: Float, defaults to None. If set, weight decay is applied.

However, the AdamW constructor raises a ValueError if `weight_decay` is None

```
if self.weight_decay is None:
            raise ValueError(
                "Missing value of `weight_decay` which is required and"
                " must be a float value.")
```




Discussion:
Hi @AjaniStewart ,

It seems documentation bug. It is copying the arguments from the Parent class arguments. Will look into it. Thanks!

Resolving patch:
diff --git a/keras/optimizers/base_optimizer.py b/keras/optimizers/base_optimizer.py
--- a/keras/optimizers/base_optimizer.py
+++ b/keras/optimizers/base_optimizer.py
@@ -698,7 +698,7 @@ def __setattr__(self, name, value):
 base_optimizer_keyword_args = """name: String. The name to use
           for momentum accumulator weights created by
           the optimizer.
-        weight_decay: Float, defaults to None. If set, weight decay is applied.
+        weight_decay: Float. If set, weight decay is applied.
         clipnorm: Float. If set, the gradient of each weight is individually
           clipped so that its norm is no higher than this value.
         clipvalue: Float. If set, the gradient of each weight is clipped to be


Test patch:
Repository: keras-team/keras

Issue:
Error in Documentation
The documentation confuses the ‘channels_last’ input format. It needs to be changed to match the correct format ==> (batch_size, height, width, channels)

https://github.com/keras-team/keras/blob/037ec9f5fc61a53c6e1f4c02b7bf1443429dcd45/keras/layers/convolutional/conv2d_transpose.py#L35


Discussion:

Resolving patch:
diff --git a/keras/layers/convolutional/conv2d_transpose.py b/keras/layers/convolutional/conv2d_transpose.py
--- a/keras/layers/convolutional/conv2d_transpose.py
+++ b/keras/layers/convolutional/conv2d_transpose.py
@@ -32,7 +32,7 @@ class Conv2DTranspose(BaseConvTranspose):
         data_format: string, either `"channels_last"` or `"channels_first"`.
             The ordering of the dimensions in the inputs. `"channels_last"`
             corresponds to inputs with shape
-            `(batch_size, channels, height, width)`
+            `(batch_size, height, width, channels)`
             while `"channels_first"` corresponds to inputs with shape
             `(batch_size, channels, height, width)`. It defaults to the
             `image_data_format` value found in your Keras config file at


Test patch:
Repository: keras-team/keras

Issue:
`keras.ops.softmax` errors out when used in a TensorFlow compiled function
## MRE

```python
import keras
from keras import ops

class SoftmaxLayer(keras.Layer):
  def call(self, x):
    return ops.softmax(x, axis=-1)

class Model(keras.Model):
  def __init__(self):
    x = keras.Input(shape=(None,))
    y = SoftmaxLayer()(x)
    super().__init__(inputs=x, outputs=y)

model = Model()  # Error
```

## Additional Details

The regression was introduced in [d5a4521](https://github.com/keras-team/keras/commit/d5a452155a415d5fcbe568eb2a8441f64e57aa90)

Discovered by KerasCV test run with `keras-nightly`: https://github.com/keras-team/keras-cv/actions/runs/8259807616/job/22594330565


Discussion:

Resolving patch:
diff --git a/keras/ops/nn.py b/keras/ops/nn.py
--- a/keras/ops/nn.py
+++ b/keras/ops/nn.py
@@ -538,10 +538,13 @@ def softmax(x, axis=-1):
     array([0.09003057, 0.24472847, 0.66524096], shape=(3,), dtype=float64)
 
     """
-    if isinstance(axis, int) and backend.shape(x)[axis] == 1:
+    # Don't use `backend.shape` since TensorFlow returns
+    # symbolic tensors for unknown shape which can trigger
+    # an error in TensorFlow graph execution.
+    if isinstance(axis, int) and x.shape[axis] == 1:
         warnings.warn(
             f"You are using a softmax over axis {axis} "
-            f"of a tensor of shape {backend.shape(x)}. This axis "
+            f"of a tensor of shape {x.shape}. This axis "
             "has size 1. The softmax operation will always return "
             "the value 1, which is likely not what you intended. "
             "Did you mean to use a sigmoid instead?"


Test patch:
diff --git a/keras/ops/nn_test.py b/keras/ops/nn_test.py
--- a/keras/ops/nn_test.py
+++ b/keras/ops/nn_test.py
@@ -2,10 +2,12 @@
 import pytest
 from absl.testing import parameterized
 
+import keras
 from keras import backend
 from keras import layers
 from keras import losses
 from keras import models
+from keras import ops
 from keras import testing
 from keras.backend.common import standardize_dtype
 from keras.backend.common.keras_tensor import KerasTensor
@@ -84,6 +86,22 @@ def test_softmax(self):
         self.assertEqual(knn.softmax(x, axis=1).shape, (None, 2, 3))
         self.assertEqual(knn.softmax(x, axis=-1).shape, (None, 2, 3))
 
+    def test_softmax_in_graph(self):
+        class SoftmaxLayer(keras.Layer):
+            def call(self, x):
+                return ops.softmax(x, axis=-1)
+
+        class Model(keras.Model):
+            def __init__(self):
+                x = keras.Input(shape=(None,))
+                y = SoftmaxLayer()(x)
+                super().__init__(inputs=x, outputs=y)
+
+        # Make sure Keras is able to compile the model graph
+        model = Model()
+        x = ops.array([[1.0, 2.0, 3.0, 4.0]])
+        model.predict(x)
+
     def test_log_softmax(self):
         x = KerasTensor([None, 2, 3])
         self.assertEqual(knn.log_softmax(x).shape, (None, 2, 3))
Repository: keras-team/keras

Issue:
CenterCrop.compute_output_shape() does not consider the list compatibility
When the input is a list of tensor, CenterCrop can correctly generate the output, while` CenterCrop.compute_output_shape()` will generate a wrong shape. Please refer to this [gist](https://colab.research.google.com/drive/1JLuEr0-dNCmawTb2dRX7kaSxOKGZ6Obv?usp=sharing).

Interestingly, I found the test code in [center_crop_test.py, ](https://github.com/keras-team/keras/blob/master/keras/layers/preprocessing/center_crop_test.py#L178) has tested the list compatibility, while its `compute_output_shape` function missed it.


Discussion:

Resolving patch:
diff --git a/keras/layers/preprocessing/center_crop.py b/keras/layers/preprocessing/center_crop.py
--- a/keras/layers/preprocessing/center_crop.py
+++ b/keras/layers/preprocessing/center_crop.py
@@ -111,6 +111,13 @@ def call(self, inputs):
 
     def compute_output_shape(self, input_shape):
         input_shape = list(input_shape)
+        if isinstance(input_shape[0], (list, tuple)) or len(
+            input_shape
+        ) not in (3, 4):
+            raise ValueError(
+                "`input_shape` must be a non-nested tuple or list "
+                "of rank-1 with size 3 (unbatched) or 4 (batched). "
+            )
         if len(input_shape) == 4:
             if self.data_format == "channels_last":
                 input_shape[1] = self.height


Test patch:
Repository: keras-team/keras

Issue:
Numpy Ops function nonzero(x) appers to be missing check for symbolic tensors
In updating  code from Keras 2 to 3, we noticed that nonzero function continues to throw errors for use of KerasTensor in TF functions, even when run though tf.keras.ops

Digging into the source, it appears that this function does not receive the check for any_symbolic_tensors(), and thus no instantiation of the NonZero() class. In turn failing when used with a KerasTensor

https://github.com/keras-team/keras/blob/42a1535ed7d3d75711a11d295f58a2dc9a59fdae/keras/ops/numpy.py#L3976


Discussion:

Resolving patch:
diff --git a/keras/ops/numpy.py b/keras/ops/numpy.py
--- a/keras/ops/numpy.py
+++ b/keras/ops/numpy.py
@@ -3992,6 +3992,9 @@ class Nonzero(Operation):
     def call(self, x):
         return backend.numpy.nonzero(x)
 
+    def compute_output_spec(self, x):
+        return KerasTensor([None] * len(x.shape))
+
 
 @keras_export(["keras.ops.nonzero", "keras.ops.numpy.nonzero"])
 def nonzero(x):
@@ -4003,6 +4006,8 @@ def nonzero(x):
     Returns:
         Indices of elements that are non-zero.
     """
+    if any_symbolic_tensors((x,)):
+        return Nonzero().symbolic_call(x)
     return backend.numpy.nonzero(x)
 
 


Test patch:
diff --git a/keras/ops/numpy_test.py b/keras/ops/numpy_test.py
--- a/keras/ops/numpy_test.py
+++ b/keras/ops/numpy_test.py
@@ -1311,6 +1311,10 @@ def test_ndim(self):
         x = KerasTensor((None, 3))
         self.assertEqual(knp.ndim(x).shape, (2,))
 
+    def test_nonzero(self):
+        x = KerasTensor((None, 5, 6))
+        self.assertEqual(knp.nonzero(x).shape, (None, None, None))
+
     def test_ones_like(self):
         x = KerasTensor((None, 3))
         self.assertEqual(knp.ones_like(x).shape, (None, 3))
Repository: keras-team/keras

Issue:
imdb.load_data function returns a python list instead of ndarray object
In Keras 3.2.1 the imdb.load_data function returns a Python list instead of a ndarray object as described in the function documentation. 
In Keras 2.15 the train and test data are converted to a ndarray using the following statements before returning the tuple.
x_train, y_train = np.array(xs[:idx], dtype="object"), labels[:idx]
x_test, y_test = np.array(xs[idx:], dtype="object"), labels[idx:]

In Keras 3.2.1 the conversion is not applied, i.e., 
x_train, y_train = xs[:idx], labels[:idx]
x_test, y_test = xs[idx:], labels[idx:]



Discussion:

Resolving patch:
diff --git a/keras/src/datasets/imdb.py b/keras/src/datasets/imdb.py
--- a/keras/src/datasets/imdb.py
+++ b/keras/src/datasets/imdb.py
@@ -135,8 +135,8 @@ def load_data(
         xs = [[w for w in x if skip_top <= w < num_words] for x in xs]
 
     idx = len(x_train)
-    x_train, y_train = xs[:idx], labels[:idx]
-    x_test, y_test = xs[idx:], labels[idx:]
+    x_train, y_train = np.array(xs[:idx], dtype="object"), labels[:idx]
+    x_test, y_test = np.array(xs[idx:], dtype="object"), labels[idx:]
     return (x_train, y_train), (x_test, y_test)
 
 


Test patch: