Repository: keras-team/autokeras

Issue:
Specify hyperparameter ranges for blocks
### Feature Description
We want to enable the users to specify the value ranges for any argument in the blocks.
The following code example shows a typical use case.
The users can specify the number of units in a DenseBlock to be either 10 or 20.

### Code Example
<!---
Please provide a code example for using that feature
given the proposed feature is implemented.
-->
```python
import autokeras as ak
from kerastuner.engine.hyperparameters import Choice

input_node = ak.ImageInput()
output_node = ak.DenseBlock(num_units=Choice("num_units", [10, 20]))(input_node)
output_node = ak.ClassificationHead()(output_node)
model = ak.AutoModel(input_node, output_node)
```

### Note
<!---
Why do we need the feature?
-->
Each pull request should only change one hyperparameter in one of the blocks.

### Solution
<!---
Please tell us how to implement the feature,
if you have one in mind.
-->
Example pull requests are #1419 #1425 .
Here are the steps to follow:
1. You can just change any other argument in any other block supported by AutoKeras, as shown [here](https://github.com/keras-team/autokeras/blob/master/autokeras/__init__.py#L16-L38).
2. Change the docstring. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL60-R61)
3. Make sure you imported the module. `from kerastuner.engine import hyperparameters`.
4. Change the typing of the argument. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL73-R74)
5. Change the saving mechanism to serialized objects. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL95-R100)
6. Change the loading mechanism to deserialized objects. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cR110)
7. Change how we initialize the hyperparameter to self. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL80-R85) Copy from where it is originally defined. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL115)
8. Change how we use it. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL124-R129)


Discussion:
@haifeng-jin I would be happy to take this one
@lc0 Great! Thank you. Let me know if you have any questions.
@lc0 Are you planning to take all the blocks? Can you share something with me?
@dineshkumarsarangapani we can use this issue for coordination, so if you are taking any of the blocks, feel free to just mention it here
@lc0 Thank you. I will work on BertBlock and ConvBlock today and raise a PR in a day or 2.

Resolving patch:
diff --git a/autokeras/blocks/basic.py b/autokeras/blocks/basic.py
--- a/autokeras/blocks/basic.py
+++ b/autokeras/blocks/basic.py
@@ -227,7 +227,8 @@ class ConvBlock(block_module.Block):
         kernel_size: Int or kerastuner.engine.hyperparameters.Choice.
             The size of the kernel.
             If left unspecified, it will be tuned automatically.
-        num_blocks: Int. The number of conv blocks, each of which may contain
+        num_blocks: Int or kerastuner.engine.hyperparameters.Choice.
+            The number of conv blocks, each of which may contain
             convolutional, max pooling, dropout, and activation. If left unspecified,
             it will be tuned automatically.
         num_layers: Int. The number of convolutional layers in each block. If left
@@ -244,7 +245,7 @@ class ConvBlock(block_module.Block):
     def __init__(
         self,
         kernel_size: Optional[Union[int, hyperparameters.Choice]] = None,
-        num_blocks: Optional[int] = None,
+        num_blocks: Optional[Union[int, hyperparameters.Choice]] = None,
         num_layers: Optional[int] = None,
         max_pooling: Optional[bool] = None,
         separable: Optional[bool] = None,
@@ -257,7 +258,11 @@ def __init__(
             hyperparameters.Choice("kernel_size", [3, 5, 7], default=3),
             int,
         )
-        self.num_blocks = num_blocks
+        self.num_blocks = utils.get_hyperparameter(
+            num_blocks,
+            hyperparameters.Choice("num_blocks", [1, 2, 3], default=2),
+            int,
+        )
         self.num_layers = num_layers
         self.max_pooling = max_pooling
         self.separable = separable
@@ -268,7 +273,7 @@ def get_config(self):
         config.update(
             {
                 "kernel_size": hyperparameters.serialize(self.kernel_size),
-                "num_blocks": self.num_blocks,
+                "num_blocks": hyperparameters.serialize(self.num_blocks),
                 "num_layers": self.num_layers,
                 "max_pooling": self.max_pooling,
                 "separable": self.separable,
@@ -280,6 +285,7 @@ def get_config(self):
     @classmethod
     def from_config(cls, config):
         config["kernel_size"] = hyperparameters.deserialize(config["kernel_size"])
+        config["num_blocks"] = hyperparameters.deserialize(config["num_blocks"])
         return cls(**config)
 
     def build(self, hp, inputs=None):
@@ -289,7 +295,6 @@ def build(self, hp, inputs=None):
         output_node = input_node
 
         kernel_size = utils.add_to_hp(self.kernel_size, hp)
-        num_blocks = self.num_blocks or hp.Choice("num_blocks", [1, 2, 3], default=2)
         num_layers = self.num_layers or hp.Choice("num_layers", [1, 2], default=2)
         separable = self.separable
         if separable is None:
@@ -310,7 +315,7 @@ def build(self, hp, inputs=None):
         else:
             dropout = hp.Choice("dropout", [0.0, 0.25, 0.5], default=0)
 
-        for i in range(num_blocks):
+        for i in range(utils.add_to_hp(self.num_blocks, hp)):
             for j in range(num_layers):
                 output_node = conv(
                     hp.Choice(


Test patch: