Repository: keras-team/keras-tuner

Issue:
search_space_summary() to search for all the hps based on conditions
Currently, the summary only build the model for once, which  may not cover all the hps.
We can use the conditions to fetch all the hps to build multiple times.

Discussion:
codecov-commenter: # [Codecov](https://codecov.io/gh/keras-team/keras-tuner/pull/584?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team) Report
> Merging [#584](https://codecov.io/gh/keras-team/keras-tuner/pull/584?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team) (5c631fd) into [master](https://codecov.io/gh/keras-team/keras-tuner/commit/bb27e674d99d5ec235c257f521fde7c35e51488a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team) (bb27e67) will **decrease** coverage by `3.48%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/keras-team/keras-tuner/pull/584/graphs/tree.svg?width=650&height=150&src=pr&token=XiR9QNcEiW&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team)](https://codecov.io/gh/keras-team/keras-tuner/pull/584?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team)

```diff
@@            Coverage Diff             @@
##           master     #584      +/-   ##
==========================================
- Coverage   92.10%   88.61%   -3.49%     
==========================================
  Files          33       33              
  Lines        2875     2899      +24     
==========================================
- Hits         2648     2569      -79     
- Misses        227      330     +103     
```


| [Impacted Files](https://codecov.io/gh/keras-team/keras-tuner/pull/584?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team) | Coverage Δ | |
|---|---|---|
| [keras\_tuner/engine/base\_tuner.py](https://codecov.io/gh/keras-team/keras-tuner/pull/584/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-a2VyYXNfdHVuZXIvZW5naW5lL2Jhc2VfdHVuZXIucHk=) | `98.37% <100.00%> (ø)` | |
| [keras\_tuner/engine/hyperparameters.py](https://codecov.io/gh/keras-team/keras-tuner/pull/584/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-a2VyYXNfdHVuZXIvZW5naW5lL2h5cGVycGFyYW1ldGVycy5weQ==) | `92.03% <100.00%> (+0.38%)` | :arrow_up: |
| [keras\_tuner/applications/augment.py](https://codecov.io/gh/keras-team/keras-tuner/pull/584/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-a2VyYXNfdHVuZXIvYXBwbGljYXRpb25zL2F1Z21lbnQucHk=) | `22.72% <0.00%> (-70.46%)` | :arrow_down: |
| [keras\_tuner/applications/efficientnet.py](https://codecov.io/gh/keras-team/keras-tuner/pull/584/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-a2VyYXNfdHVuZXIvYXBwbGljYXRpb25zL2VmZmljaWVudG5ldC5weQ==) | `33.33% <0.00%> (-60.32%)` | :arrow_down: |
| [keras\_tuner/engine/tuner.py](https://codecov.io/gh/keras-team/keras-tuner/pull/584/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-a2VyYXNfdHVuZXIvZW5naW5lL3R1bmVyLnB5) | `88.04% <0.00%> (-3.27%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/keras-team/keras-tuner/pull/584?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/keras-team/keras-tuner/pull/584?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team). Last update [bb27e67...5c631fd](https://codecov.io/gh/keras-team/keras-tuner/pull/584?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team).


Patch:
diff --git a/keras_tuner/engine/base_tuner.py b/keras_tuner/engine/base_tuner.py
index fa61deb..a37b818 100644
--- a/keras_tuner/engine/base_tuner.py
+++ b/keras_tuner/engine/base_tuner.py
@@ -17,6 +17,7 @@ from __future__ import absolute_import
 from __future__ import division
 from __future__ import print_function
 
+import copy
 import os
 
 import tensorflow as tf
@@ -110,12 +111,43 @@ class BaseTuner(stateful.Stateful):
     def _populate_initial_space(self):
         """Populate initial search space for oracle.
 
-        Keep this function as a subroutine for AutoKeras to override. The space may
-        not be ready at the initialization of the tuner, but after seeing the
-        training data.
+        Keep this function as a subroutine for AutoKeras to override. The space
+        may not be ready at the initialization of the tuner, but after seeing
+        the training data.
+
+        Build hypermodel multiple times to find all conditional hps. It
+        generates hp values based on the not activated `conditional_scopes`
+        found in the builds.
         """
         hp = self.oracle.get_space()
-        self.hypermodel.build(hp)
+
+        # Lists of stacks of conditions used during `explore_space()`.
+        scopes_never_active = []
+        scopes_once_active = []
+
+        while True:
+            self.hypermodel.build(hp)
+
+            # Update the recored scopes.
+            for conditions in hp.active_scopes:
+                if conditions not in scopes_once_active:
+                    scopes_once_active.append(copy.deepcopy(conditions))
+                if conditions in scopes_never_active:
+                    scopes_never_active.remove(conditions)
+
+            for conditions in hp.inactive_scopes:
+                if conditions not in scopes_once_active:
+                    scopes_never_active.append(copy.deepcopy(conditions))
+
+            # All conditional scopes are activated.
+            if len(scopes_never_active) == 0:
+                break
+
+            # Generate new values to activate new conditions.
+            conditions = scopes_never_active[0]
+            for condition in conditions:
+                hp.values[condition.name] = condition.values[0]
+
         self.oracle.update_space(hp)
 
     def search(self, *fit_args, **fit_kwargs):
diff --git a/keras_tuner/engine/hyperparameters.py b/keras_tuner/engine/hyperparameters.py
index 32fd644..0f352ba 100644
--- a/keras_tuner/engine/hyperparameters.py
+++ b/keras_tuner/engine/hyperparameters.py
@@ -19,6 +19,7 @@ from __future__ import print_function
 
 import collections
 import contextlib
+import copy
 import math
 import random
 
@@ -70,6 +71,11 @@ def _check_int(val, arg):
 class HyperParameter(object):
     """Hyperparameter base class.
 
+    A `HyperParameter` instance is uniquely identified by its `name` and
+    `conditions` attributes. `HyperParameter`s with the same `name` but with
+    different `conditions` are considered as different `HyperParameter`s by
+    the `HyperParameters` instance.
+
     Args:
         name: A string. the name of parameter. Must be unique for each
             `HyperParameter` instance in the search space.
@@ -557,6 +563,12 @@ class HyperParameters(object):
         # Active values for this `Trial`.
         self.values = {}
 
+        # A list of active `conditional_scope`s in a build,
+        # each of which is a list of condtions.
+        self.active_scopes = []
+        # Similar for inactive `conditional_scope`s.
+        self.inactive_scopes = []
+
     @contextlib.contextmanager
     def name_scope(self, name):
         self._name_scopes.append(name)
@@ -594,12 +606,12 @@ class HyperParameters(object):
                 model = Sequential()
                 model.add(Input(shape=(32, 32, 3)))
                 model_type = hp.Choice("model_type", ["mlp", "cnn"])
-                if model_type == "mlp":
-                    with hp.conditional_scope("model_type", ["mlp"]):
+                with hp.conditional_scope("model_type", ["mlp"]):
+                    if model_type == "mlp":
                         model.add(Flatten())
                         model.add(Dense(32, activation='relu'))
-                if model_type == "cnn":
-                    with hp.conditional_scope("model_type", ["cnn"]):
+                with hp.conditional_scope("model_type", ["cnn"]):
+                    if model_type == "cnn":
                         model.add(Conv2D(64, 3, activation='relu'))
                         model.add(GlobalAveragePooling2D())
                 model.add(Dense(10, activation='softmax'))
@@ -620,7 +632,14 @@ class HyperParameters(object):
                 "`HyperParameter` named: " + parent_name + " " "not defined."
             )
 
-        self._conditions.append(conditions_mod.Parent(parent_name, parent_values))
+        condition = conditions_mod.Parent(parent_name, parent_values)
+        self._conditions.append(condition)
+
+        if condition.is_active(self.values):
+            self.active_scopes.append(copy.deepcopy(self._conditions))
+        else:
+            self.inactive_scopes.append(copy.deepcopy(self._conditions))
+
         try:
             yield
         finally:


Test patch:
diff --git a/tests/keras_tuner/engine/tuner_correctness_test.py b/tests/keras_tuner/engine/tuner_correctness_test.py
index 4c2c47e..d25d3d4 100644
--- a/tests/keras_tuner/engine/tuner_correctness_test.py
+++ b/tests/keras_tuner/engine/tuner_correctness_test.py
@@ -400,3 +400,57 @@ def test_save_model_delete_called(tmp_dir):
     tuner = save_model_setup_tuner(tmp_dir)
     tuner.save_model("a", None, step=16)
     assert tuner.was_called
+
+
+def test_init_build_all_hps_in_all_conditions(tmp_dir):
+    class ConditionalHyperModel(MockHyperModel):
+        def build(self, hp):
+            model_type = hp.Choice("model_type", ["cnn", "mlp"])
+            with hp.conditional_scope("model_type", ["cnn"]):
+                if model_type == "cnn":
+                    sub_cnn = hp.Choice("sub_cnn", ["a", "b"])
+                    with hp.conditional_scope("sub_cnn", ["a"]):
+                        if sub_cnn == "a":
+                            hp.Int("n_filters_a", 2, 4)
+                    with hp.conditional_scope("sub_cnn", ["b"]):
+                        if sub_cnn == "b":
+                            hp.Int("n_filters_b", 6, 8)
+            with hp.conditional_scope("model_type", ["mlp"]):
+                if model_type == "mlp":
+                    sub_mlp = hp.Choice("sub_mlp", ["a", "b"])
+                    with hp.conditional_scope("sub_mlp", ["a"]):
+                        if sub_mlp == "a":
+                            hp.Int("n_units_a", 2, 4)
+                    with hp.conditional_scope("sub_mlp", ["b"]):
+                        if sub_mlp == "b":
+                            hp.Int("n_units_b", 6, 8)
+            more_block = hp.Boolean("more_block", default=False)
+            with hp.conditional_scope("more_block", [True]):
+                if more_block:
+                    hp.Int("new_block_hp", 1, 3)
+            return super().build(hp)
+
+    def name_in_hp(name, hp):
+        return any([name == single_hp.name for single_hp in hp.space])
+
+    class MyTuner(tuner_module.Tuner):
+        def _populate_initial_space(self):
+            super()._populate_initial_space()
+            hp = self.oracle.hyperparameters
+            assert name_in_hp("model_type", hp)
+            assert name_in_hp("sub_cnn", hp)
+            assert name_in_hp("n_filters_a", hp)
+            assert name_in_hp("n_filters_b", hp)
+            assert name_in_hp("sub_mlp", hp)
+            assert name_in_hp("n_units_a", hp)
+            assert name_in_hp("n_units_b", hp)
+            assert name_in_hp("more_block", hp)
+            assert name_in_hp("new_block_hp", hp)
+
+    MyTuner(
+        oracle=keras_tuner.tuners.randomsearch.RandomSearchOracle(
+            objective="loss", max_trials=2, seed=1337
+        ),
+        hypermodel=ConditionalHyperModel(),
+        directory=tmp_dir,
+    )