Repository: keras-team/autokeras

Issue:
Specify hyperparameter ranges for blocks
### Feature Description
We want to enable the users to specify the value ranges for any argument in the blocks.
The following code example shows a typical use case.
The users can specify the number of units in a DenseBlock to be either 10 or 20.

### Code Example
<!---
Please provide a code example for using that feature
given the proposed feature is implemented.
-->
```python
import autokeras as ak
from kerastuner.engine.hyperparameters import Choice

input_node = ak.ImageInput()
output_node = ak.DenseBlock(num_units=Choice("num_units", [10, 20]))(input_node)
output_node = ak.ClassificationHead()(output_node)
model = ak.AutoModel(input_node, output_node)
```

### Note
<!---
Why do we need the feature?
-->
Each pull request should only change one hyperparameter in one of the blocks.

### Solution
<!---
Please tell us how to implement the feature,
if you have one in mind.
-->
Example pull requests are #1419 #1425 .
Here are the steps to follow:
1. You can just change any other argument in any other block supported by AutoKeras, as shown [here](https://github.com/keras-team/autokeras/blob/master/autokeras/__init__.py#L16-L38).
2. Change the docstring. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL60-R61)
3. Make sure you imported the module. `from kerastuner.engine import hyperparameters`.
4. Change the typing of the argument. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL73-R74)
5. Change the saving mechanism to serialized objects. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL95-R100)
6. Change the loading mechanism to deserialized objects. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cR110)
7. Change how we initialize the hyperparameter to self. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL80-R85) Copy from where it is originally defined. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL115)
8. Change how we use it. [example](https://github.com/keras-team/autokeras/pull/1419/files#diff-7b757ddf51e45c0e02ad8de148d02fd2abd59d670f91d35cac71157e6684c13cL124-R129)


Discussion:
@haifeng-jin I would be happy to take this one
@lc0 Great! Thank you. Let me know if you have any questions.
@lc0 Are you planning to take all the blocks? Can you share something with me?
@dineshkumarsarangapani we can use this issue for coordination, so if you are taking any of the blocks, feel free to just mention it here
@lc0 Thank you. I will work on BertBlock and ConvBlock today and raise a PR in a day or 2.
@haifeng-jin we need to re-open this one again. Also how do you feel about adding a checklist of blocks to do?

Resolving patch:
diff --git a/autokeras/blocks/basic.py b/autokeras/blocks/basic.py
--- a/autokeras/blocks/basic.py
+++ b/autokeras/blocks/basic.py
@@ -231,7 +231,8 @@ class ConvBlock(block_module.Block):
             The number of conv blocks, each of which may contain
             convolutional, max pooling, dropout, and activation. If left unspecified,
             it will be tuned automatically.
-        num_layers: Int. The number of convolutional layers in each block. If left
+        num_layers: Int or hyperparameters.Choice.
+            The number of convolutional layers in each block. If left
             unspecified, it will be tuned automatically.
         filters: Int or kerastuner.engine.hyperparameters.Choice. The number of
             filters in the convolutional layers. If left unspecified, it will
@@ -249,7 +250,7 @@ def __init__(
         self,
         kernel_size: Optional[Union[int, hyperparameters.Choice]] = None,
         num_blocks: Optional[Union[int, hyperparameters.Choice]] = None,
-        num_layers: Optional[int] = None,
+        num_layers: Optional[Union[int, hyperparameters.Choice]] = None,
         filters: Optional[Union[int, hyperparameters.Choice]] = None,
         max_pooling: Optional[bool] = None,
         separable: Optional[bool] = None,
@@ -267,7 +268,11 @@ def __init__(
             hyperparameters.Choice("num_blocks", [1, 2, 3], default=2),
             int,
         )
-        self.num_layers = num_layers
+        self.num_layers = utils.get_hyperparameter(
+            num_layers,
+            hyperparameters.Choice("num_layers", [1, 2], default=2),
+            int,
+        )
         self.filters = utils.get_hyperparameter(
             filters,
             hyperparameters.Choice(
@@ -285,7 +290,7 @@ def get_config(self):
             {
                 "kernel_size": hyperparameters.serialize(self.kernel_size),
                 "num_blocks": hyperparameters.serialize(self.num_blocks),
-                "num_layers": self.num_layers,
+                "num_layers": hyperparameters.serialize(self.num_layers),
                 "filters": hyperparameters.serialize(self.filters),
                 "max_pooling": self.max_pooling,
                 "separable": self.separable,
@@ -298,6 +303,7 @@ def get_config(self):
     def from_config(cls, config):
         config["kernel_size"] = hyperparameters.deserialize(config["kernel_size"])
         config["num_blocks"] = hyperparameters.deserialize(config["num_blocks"])
+        config["num_layers"] = hyperparameters.deserialize(config["num_layers"])
         config["filters"] = hyperparameters.deserialize(config["filters"])
         return cls(**config)
 
@@ -308,7 +314,7 @@ def build(self, hp, inputs=None):
         output_node = input_node
 
         kernel_size = utils.add_to_hp(self.kernel_size, hp)
-        num_layers = self.num_layers or hp.Choice("num_layers", [1, 2], default=2)
+
         separable = self.separable
         if separable is None:
             separable = hp.Boolean("separable", default=False)
@@ -329,7 +335,7 @@ def build(self, hp, inputs=None):
             dropout = hp.Choice("dropout", [0.0, 0.25, 0.5], default=0)
 
         for i in range(utils.add_to_hp(self.num_blocks, hp)):
-            for j in range(num_layers):
+            for j in range(utils.add_to_hp(self.num_layers, hp)):
                 output_node = conv(
                     utils.add_to_hp(
                         self.filters, hp, "filters_{i}_{j}".format(i=i, j=j)


Test patch: