Repository: keras-team/autokeras

Issue:
Specify hyperparameter ranges for blocks
### Feature Description
We want to enable the users to specify the value ranges for any argument in the blocks.
The following code example shows a typical use case.
The users can specify the number of units in a DenseBlock to be either 10 or 20.

### Code Example
<!---
Please provide a code example for using that feature
given the proposed feature is implemented.
-->
```python
import autokeras as ak
from kerastuner.engine.hyperparameters import Choice

input_node = ak.ImageInput()
output_node = ak.DenseBlock(num_units=Choice("num_units", [10, 20]))(input_node)
output_node = ak.ClassificationHead()(output_node)
model = ak.AutoModel(input_node, output_node)
```

### Note
<!---
Why do we need the feature?
-->
Each pull request should only change one hyperparameter in one of the blocks.

### Solution
<!---
Please tell us how to implement the feature,
if you have one in mind.
-->
Example pull requests are #1419 #1425 .
Here are the steps to follow:
1. You can just change any other argument in any other block supported by AutoKeras, as shown [here](https://github.com/keras-team/autokeras/blob/master/autokeras/__init__.py#L16-L38).
2. Change the docstring. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL60-R61)
3. Make sure you imported the module. `from kerastuner.engine import hyperparameters`.
4. Change the typing of the argument. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL73-R74)
5. Change the saving mechanism to serialized objects. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL95-R100)
6. Change the loading mechanism to deserialized objects. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cR110)
7. Change how we initialize the hyperparameter to self. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL80-R85) Copy from where it is originally defined. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL115)
8. Change how we use it. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL124-R129)


Discussion:
@haifeng-jin I would be happy to take this one
@lc0 Great! Thank you. Let me know if you have any questions.
@lc0 Are you planning to take all the blocks? Can you share something with me?
@dineshkumarsarangapani we can use this issue for coordination, so if you are taking any of the blocks, feel free to just mention it here
@lc0 Thank you. I will work on BertBlock and ConvBlock today and raise a PR in a day or 2.
@haifeng-jin we need to re-open this one again. Also how do you feel about adding a checklist of blocks to do?

Resolving patch:
diff --git a/autokeras/blocks/basic.py b/autokeras/blocks/basic.py
--- a/autokeras/blocks/basic.py
+++ b/autokeras/blocks/basic.py
@@ -853,32 +853,47 @@ class BertBlock(block_module.Block):
         clf = ak.AutoModel(inputs=input_node, outputs=output_node, max_trials=10)
     ```
     # Arguments
-        max_sequence_length: Int. The maximum length of a sequence that is
-            used to train the model.
+        max_sequence_length: Int or kerastuner.engine.hyperparameters.Choice.
+            The maximum length of a sequence that is used to train the model.
     """
 
     def __init__(
         self,
-        max_sequence_length: Optional[int] = None,
+        max_sequence_length: Optional[Union[int, hyperparameters.Choice]] = None,
         **kwargs,
     ):
         super().__init__(**kwargs)
-        self.max_sequence_length = max_sequence_length
+        self.max_sequence_length = utils.get_hyperparameter(
+            max_sequence_length,
+            hyperparameters.Choice(
+                "max_sequence_length", [128, 256, 512], default=128
+            ),
+            int,
+        )
 
     def get_config(self):
         config = super().get_config()
-        config.update({"max_sequence_length": self.max_sequence_length})
+        config.update(
+            {
+                "max_sequence_length": hyperparameters.serialize(
+                    self.max_sequence_length
+                )
+            }
+        )
         return config
 
+    @classmethod
+    def from_config(cls, config):
+        config["max_sequence_length"] = hyperparameters.deserialize(
+            config["max_sequence_length"]
+        )
+        return cls(**config)
+
     def build(self, hp, inputs=None):
         input_tensor = nest.flatten(inputs)[0]
 
-        max_sequence_length = self.max_sequence_length or hp.Choice(
-            "max_seq_len", [128, 256, 512], default=128
-        )
-
         tokenizer_layer = keras_layers.BertTokenizer(
-            max_sequence_length=max_sequence_length
+            max_sequence_length=utils.add_to_hp(self.max_sequence_length, hp)
         )
         output_node = tokenizer_layer(input_tensor)
 
diff --git a/autokeras/tuners/task_specific.py b/autokeras/tuners/task_specific.py
--- a/autokeras/tuners/task_specific.py
+++ b/autokeras/tuners/task_specific.py
@@ -118,7 +118,7 @@
         "classification_head_1/dropout": 0,
         "optimizer": "adam_weight_decay",
         "learning_rate": 2e-5,
-        "text_block_1/bert_block_1/max_seq_len": 512,
+        "text_block_1/bert_block_1/max_sequence_length": 512,
         "text_block_1/max_tokens": 20000,
     },
 ]


Test patch: