Repository: keras-team/keras (Python)
Date: 2024-06-12 08:33:53+00:00
Task type: Bug Fix

Issue:
Keras 3.0 load h5 model with Orthogonal initializer fails
Hi guys, 

I'm trying to load an h5 model that was working in earlier versions.

* This is a small part of the h5 file, where you can see (last part of the snippet) a recurrent initializer with a classname of **Orthogonal**.

```
{"name": "decoder_gru0", "class_name": "GRU", "config": {"name": "decoder_gru0", "trainable": true, "return_sequences": true, "return_state": false, "go_backwards": false, "stateful": false, "unroll": false, "implementation": 0, "units": 488, "activation": "tanh", "recurrent_activation": "hard_sigmoid", "use_bias": true, "kernel_initializer": {"class_name": "VarianceScaling", "config": {"scale": 1.0, "mode": "fan_avg", "distribution": "uniform", "seed": null}}, "recurrent_initializer": {"class_name": "Orthogonal", "config": {"gain": 1.0, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}
```


* The error returned is:


```bash
  File "..../keras/src/initializers/__init__.py", line 118, in get
    raise ValueError(
ValueError: Could not interpret initializer identifier: {'class_name': 'Orthogonal', 'config': {'gain': 1.0, 'seed': None}}
```

## Addition

I then added the Orthogonal initializer to the custom objects, and it seems to go further, but gets stuck here:

```bash
    raise ValueError(
ValueError: Unrecognized keyword arguments passed to GRU: {'implementation': 0}
```

Any ideas on how to fix this @mehtamansi29 ?


Issue discussion:
Hi @mahnehsilla -

Thanks for raising the issue. Can you share the code snippet and h5 model with me where you are getting this error ? So I can reproduce it and try to help you on this. 


Solution patch:
diff --git a/keras/src/initializers/__init__.py b/keras/src/initializers/__init__.py
--- a/keras/src/initializers/__init__.py
+++ b/keras/src/initializers/__init__.py
@@ -49,6 +49,7 @@
         "uniform": RandomUniform,
         "normal": RandomNormal,
         "orthogonal": OrthogonalInitializer,
+        "Orthogonal": OrthogonalInitializer,  # Legacy
         "one": Ones,
         "zero": Zeros,
     }
diff --git a/keras/src/layers/rnn/gru.py b/keras/src/layers/rnn/gru.py
--- a/keras/src/layers/rnn/gru.py
+++ b/keras/src/layers/rnn/gru.py
@@ -500,6 +500,7 @@ def __init__(
             trainable=kwargs.get("trainable", True),
             name="gru_cell",
             seed=seed,
+            implementation=kwargs.pop("implementation", 2),
         )
         super().__init__(
             cell,


Test patch:
diff --git a/keras/src/initializers/random_initializers_test.py b/keras/src/initializers/random_initializers_test.py
--- a/keras/src/initializers/random_initializers_test.py
+++ b/keras/src/initializers/random_initializers_test.py
@@ -147,6 +147,10 @@ def test_orthogonal_initializer(self):
 
         self.run_class_serialization_test(initializer)
 
+        # Test legacy class_name
+        initializer = initializers.get("Orthogonal")
+        self.assertIsInstance(initializer, initializers.OrthogonalInitializer)
+
     def test_get_method(self):
         obj = initializers.get("glorot_normal")
         self.assertTrue(obj, initializers.GlorotNormal)
diff --git a/keras/src/layers/rnn/gru_test.py b/keras/src/layers/rnn/gru_test.py
--- a/keras/src/layers/rnn/gru_test.py
+++ b/keras/src/layers/rnn/gru_test.py
@@ -286,3 +286,26 @@ def test_masking(self):
             np.array([[0.11669192, 0.11669192], [0.28380975, 0.28380975]]),
             output,
         )
+
+    def test_legacy_implementation_argument(self):
+        sequence = np.arange(72).reshape((3, 6, 4)).astype("float32")
+        layer = layers.GRU(
+            3,
+            kernel_initializer=initializers.Constant(0.01),
+            recurrent_initializer=initializers.Constant(0.02),
+            bias_initializer=initializers.Constant(0.03),
+        )
+        config = layer.get_config()
+        config["implementation"] = 0  # Add legacy argument
+        layer = layers.GRU.from_config(config)
+        output = layer(sequence)
+        self.assertAllClose(
+            np.array(
+                [
+                    [0.5217289, 0.5217289, 0.5217289],
+                    [0.6371659, 0.6371659, 0.6371659],
+                    [0.39384964, 0.39384964, 0.3938496],
+                ]
+            ),
+            output,
+        )