Repository: keras-team/keras-tuner

Issue:
Support multi-objective optimization
Line 160 [here](https://github.com/keras-team/keras-tuner/blob/64c547621230f43b9d68e5ef95537d9efbc0b293/keras_tuner/engine/oracle.py#L150) suggests that :

"# Assumes single objective, subclasses can override."

Please provide a working example so we can see how to do this.

Discussion:
codecov-commenter: # [Codecov](https://codecov.io/gh/keras-team/keras-tuner/pull/641?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team) Report
> Merging [#641](https://codecov.io/gh/keras-team/keras-tuner/pull/641?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team) (64d98c4) into [master](https://codecov.io/gh/keras-team/keras-tuner/commit/003559cc82bc82b01480c85e51568cd4d3e5ca86?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team) (003559c) will **decrease** coverage by `0.06%`.
> The diff coverage is `86.30%`.

[![Impacted file tree graph](https://codecov.io/gh/keras-team/keras-tuner/pull/641/graphs/tree.svg?width=650&height=150&src=pr&token=XiR9QNcEiW&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team)](https://codecov.io/gh/keras-team/keras-tuner/pull/641?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team)

```diff
@@            Coverage Diff             @@
##           master     #641      +/-   ##
==========================================
- Coverage   92.12%   92.06%   -0.07%     
==========================================
  Files          35       36       +1     
  Lines        2833     2861      +28     
==========================================
+ Hits         2610     2634      +24     
- Misses        223      227       +4     
```


| [Impacted Files](https://codecov.io/gh/keras-team/keras-tuner/pull/641?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team) | Coverage Δ | |
|---|---|---|
| [keras\_tuner/engine/base\_tuner.py](https://codecov.io/gh/keras-team/keras-tuner/pull/641/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-a2VyYXNfdHVuZXIvZW5naW5lL2Jhc2VfdHVuZXIucHk=) | `98.60% <ø> (ø)` | |
| [keras\_tuner/engine/objective.py](https://codecov.io/gh/keras-team/keras-tuner/pull/641/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-a2VyYXNfdHVuZXIvZW5naW5lL29iamVjdGl2ZS5weQ==) | `75.67% <75.67%> (ø)` | |
| [keras\_tuner/engine/tuner\_utils.py](https://codecov.io/gh/keras-team/keras-tuner/pull/641/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-a2VyYXNfdHVuZXIvZW5naW5lL3R1bmVyX3V0aWxzLnB5) | `85.63% <96.42%> (+1.26%)` | :arrow_up: |
| [keras\_tuner/\_\_init\_\_.py](https://codecov.io/gh/keras-team/keras-tuner/pull/641/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-a2VyYXNfdHVuZXIvX19pbml0X18ucHk=) | `100.00% <100.00%> (ø)` | |
| [keras\_tuner/engine/oracle.py](https://codecov.io/gh/keras-team/keras-tuner/pull/641/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-a2VyYXNfdHVuZXIvZW5naW5lL29yYWNsZS5weQ==) | `95.00% <100.00%> (+0.06%)` | :arrow_up: |
| [keras\_tuner/engine/tuner.py](https://codecov.io/gh/keras-team/keras-tuner/pull/641/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-a2VyYXNfdHVuZXIvZW5naW5lL3R1bmVyLnB5) | `91.53% <100.00%> (+2.72%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/keras-team/keras-tuner/pull/641?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/keras-team/keras-tuner/pull/641?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team). Last update [003559c...64d98c4](https://codecov.io/gh/keras-team/keras-tuner/pull/641?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team).

fchollet: > Create a custom callback to collect the metric values and saves the model at its best value. Because the objective is not one of the metrics, we cannot use the built-in ModelCheckPoint callback. This one is much more straight-forward to implement. It also allows advanced objective like FLOPs or model size in the future.

Is this the approach you prefer? Since we have control of the callbacks list, it seems like a good solution.

Do we intend to support objective contraints? E.g. best accuracy but keep FLOPS in a certain range.
haifeng-jin: > > Create a custom callback to collect the metric values and saves the model at its best value. Because the objective is not one of the metrics, we cannot use the built-in ModelCheckPoint callback. This one is much more straight-forward to implement. It also allows advanced objective like FLOPs or model size in the future.
> 
> Is this the approach you prefer? Since we have control of the callbacks list, it seems like a good solution.

Yes, this PR is based on this approach.

> Do we intend to support objective contraints? E.g. best accuracy but keep FLOPS in a certain range.

Yes, we may want to do this in the future since it is a widely requested.


Patch:
diff --git a/keras_tuner/__init__.py b/keras_tuner/__init__.py
index 4c6c519..41dbf9b 100755
--- a/keras_tuner/__init__.py
+++ b/keras_tuner/__init__.py
@@ -21,7 +21,7 @@ from keras_tuner.engine.hyperparameters import HyperParameter
 from keras_tuner.engine.hyperparameters import HyperParameters
 from keras_tuner.engine.logger import CloudLogger
 from keras_tuner.engine.logger import Logger
-from keras_tuner.engine.oracle import Objective
+from keras_tuner.engine.objective import Objective
 from keras_tuner.engine.oracle import Oracle
 from keras_tuner.engine.tuner import Tuner
 from keras_tuner.tuners import BayesianOptimization
diff --git a/keras_tuner/engine/base_tuner.py b/keras_tuner/engine/base_tuner.py
index fec9d37..2b802c1 100644
--- a/keras_tuner/engine/base_tuner.py
+++ b/keras_tuner/engine/base_tuner.py
@@ -191,6 +191,8 @@ class BaseTuner(stateful.Stateful):
             else:
                 self.oracle.update_trial(
                     trial.trial_id,
+                    # Convert to dictionary before calling `update_trial()`
+                    # to pass it from gRPC.
                     tuner_utils.convert_to_metrics_dict(
                         results, self.oracle.objective
                     ),
diff --git a/keras_tuner/engine/objective.py b/keras_tuner/engine/objective.py
new file mode 100644
index 0000000..bc03d59
--- /dev/null
+++ b/keras_tuner/engine/objective.py
@@ -0,0 +1,110 @@
+# Copyright 2022 The KerasTuner Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from keras_tuner.engine import metrics_tracking
+from keras_tuner.engine import objective as obj_module
+
+
+class Objective:
+    """The objective for optimization during tuning.
+
+    Args:
+        name: String. The name of the objective.
+        direction: String. The value should be "min" or "max" indicating
+            whether the objective value should be minimized or maximized.
+    """
+
+    def __init__(self, name, direction):
+        self.name = name
+        self.direction = direction
+
+    def get_value(self, logs):
+        """Get the objective value from the metrics logs.
+
+        Args:
+            logs: A dictionary with the metric names as the keys and the metric
+                values as the values, which is in the same format as the `logs`
+                argument for `Callback.on_epoch_end()`.
+
+        Returns:
+            The objective value.
+        """
+        return logs[self.name]
+
+    def better_than(self, a, b):
+        """Whether the first objective value is better than the second.
+
+        Args:
+            a: A float, an objective value to compare.
+            b: A float, another objective value to compare.
+
+        Returns:
+            Boolean, whether the first objective value is better than the
+            second.
+        """
+        return (a > b and self.direction == "max") or (
+            a < b and self.direction == "min"
+        )
+
+
+class MultiObjective(Objective):
+    """A container for a list of objectives.
+
+    Args:
+        objectives: A list of `Objective`s.
+    """
+
+    def __init__(self, objectives):
+        super().__init__(name="multi_objective", direction="min")
+        self.objectives = objectives
+        self.name_to_direction = {
+            objective.name: objective.direction for objective in self.objectives
+        }
+
+    def get_value(self, logs):
+        obj_value = 0
+        for metric_name, metric_value in logs.items():
+            if metric_name not in self.name_to_direction:
+                continue
+            if self.name_to_direction[metric_name] == "min":
+                obj_value += metric_value
+            else:
+                obj_value -= metric_value
+        return obj_value
+
+
+def create_objective(objective):
+    if objective is None:
+        return obj_module.Objective("default_objective", "min")
+    if isinstance(objective, list):
+        return MultiObjective([create_objective(obj) for obj in objective])
+    if isinstance(objective, obj_module.Objective):
+        return objective
+    if isinstance(objective, str):
+        direction = metrics_tracking.infer_metric_direction(objective)
+        if direction is None:
+            error_msg = (
+                'Could not infer optimization direction ("min" or "max") '
+                'for unknown metric "{obj}". Please specify the objective  as'
+                "a `keras_tuner.Objective`, for example `keras_tuner.Objective("
+                '"{obj}", direction="min")`.'
+            )
+            error_msg = error_msg.format(obj=objective)
+            raise ValueError(error_msg)
+        return obj_module.Objective(name=objective, direction=direction)
+    else:
+        raise ValueError(
+            "`objective` not understood, expected str or "
+            "`Objective` object, found: {}".format(objective)
+        )
diff --git a/keras_tuner/engine/oracle.py b/keras_tuner/engine/oracle.py
index 0d73550..788c2a7 100644
--- a/keras_tuner/engine/oracle.py
+++ b/keras_tuner/engine/oracle.py
@@ -14,7 +14,6 @@
 "Oracle base class."
 
 
-import collections
 import hashlib
 import json
 import os
@@ -25,21 +24,23 @@ import tensorflow as tf
 
 from keras_tuner import utils
 from keras_tuner.engine import hyperparameters as hp_module
-from keras_tuner.engine import metrics_tracking
+from keras_tuner.engine import objective as obj_module
 from keras_tuner.engine import stateful
 from keras_tuner.engine import trial as trial_lib
 
-Objective = collections.namedtuple("Objective", "name direction")
-
 
 class Oracle(stateful.Stateful):
     """Implements a hyperparameter optimization algorithm.
 
     Args:
-        objective: A string or `keras_tuner.Objective` instance. If a string,
-            the direction of the optimization (min or max) will be inferred.
-            It is optional when `Tuner.run_trial()` or `HyperModel.fit()`
-            returns a single float as the objective to minimize.
+        objective: A string, `keras_tuner.Objective` instance, or a list of
+            `keras_tuner.Objective`s and strings. If a string, the direction of
+            the optimization (min or max) will be inferred. If a list of
+            `keras_tuner.Objective`, we will minimize the sum of all the
+            objectives to minimize subtracting the sum of all the objectives to
+            maximize. The `objective` argument is optional when
+            `Tuner.run_trial()` or `HyperModel.fit()` returns a single float as
+            the objective to minimize.
         max_trials: Integer, the total number of trials (model configurations)
             to test at most. Note that the oracle may interrupt the search
             before `max_trial` models have been tested if the search space has
@@ -67,7 +68,7 @@ class Oracle(stateful.Stateful):
         tune_new_entries=True,
         seed=None,
     ):
-        self.objective = _format_objective(objective)
+        self.objective = obj_module.create_objective(objective)
         self.max_trials = max_trials
         if not hyperparameters:
             if not tune_new_entries:
@@ -157,7 +158,6 @@ class Oracle(stateful.Stateful):
         Args:
             trial: A completed `Trial` object.
         """
-        # Assumes single objective, subclasses can override.
         trial.score = trial.metrics.get_best_value(self.objective.name)
         trial.best_step = trial.metrics.get_best_step(self.objective.name)
 
@@ -300,7 +300,6 @@ class Oracle(stateful.Stateful):
         sorted_trials = sorted(
             trials,
             key=lambda trial: trial.score,
-            # Assumes single objective, subclasses can override.
             reverse=self.objective.direction == "max",
         )
         return sorted_trials[:num_trials]
@@ -392,7 +391,7 @@ class Oracle(stateful.Stateful):
         return hashlib.sha256(s.encode("utf-8")).hexdigest()[:32]
 
     def _check_objective_found(self, metrics):
-        if isinstance(self.objective, Objective):
+        if isinstance(self.objective, obj_module.Objective):
             objective_names = [self.objective.name]
         else:
             objective_names = [obj.name for obj in self.objective]
@@ -446,34 +445,8 @@ class Oracle(stateful.Stateful):
         return values
 
 
-def _format_objective(objective):
-    if objective is None:
-        return Objective("default_objective", "min")
-    if isinstance(objective, list):
-        return [_format_objective(obj) for obj in objective]
-    if isinstance(objective, Objective):
-        return objective
-    if isinstance(objective, str):
-        direction = metrics_tracking.infer_metric_direction(objective)
-        if direction is None:
-            error_msg = (
-                'Could not infer optimization direction ("min" or "max") '
-                'for unknown metric "{obj}". Please specify the objective  as'
-                "a `keras_tuner.Objective`, for example `keras_tuner.Objective("
-                '"{obj}", direction="min")`.'
-            )
-            error_msg = error_msg.format(obj=objective)
-            raise ValueError(error_msg)
-        return Objective(name=objective, direction=direction)
-    else:
-        raise ValueError(
-            "`objective` not understood, expected str or "
-            "`Objective` object, found: {}".format(objective)
-        )
-
-
 def _maybe_infer_direction_from_objective(objective, metric_name):
-    if isinstance(objective, Objective):
+    if isinstance(objective, obj_module.Objective):
         objective = [objective]
     for obj in objective:
         if obj.name == metric_name:
diff --git a/keras_tuner/engine/tuner.py b/keras_tuner/engine/tuner.py
index 5bed1c0..22dc73f 100644
--- a/keras_tuner/engine/tuner.py
+++ b/keras_tuner/engine/tuner.py
@@ -21,7 +21,6 @@ import os
 import traceback
 
 import numpy as np
-import tensorflow as tf
 from tensorboard.plugins.hparams import api as hparams_api
 from tensorflow import keras
 
@@ -117,13 +116,6 @@ class Tuner(base_tuner.BaseTuner):
             overwrite=overwrite,
         )
 
-        if isinstance(oracle.objective, list) and len(oracle.objective) > 1:
-            raise ValueError(
-                "Multi-objective is not supported, found: {}".format(
-                    oracle.objective
-                )
-            )
-
         self.max_model_size = max_model_size
         self.optimizer = optimizer
         self.loss = loss
@@ -148,10 +140,6 @@ class Tuner(base_tuner.BaseTuner):
         self.tuner_id = tuner_id or self.tuner_id
 
         self.executions_per_trial = executions_per_trial
-        # This is the `step` that will be reported to the Oracle at the end
-        # of the Trial. Since intermediate results are not used, this is set
-        # to 0.
-        self._reported_step = 0
 
     def _build_hypermodel(self, hp):
         with maybe_distribute(self.distribution_strategy):
@@ -282,12 +270,11 @@ class Tuner(base_tuner.BaseTuner):
             of results of any of the types above. The final objective value is
             the average of the results in the list.
         """
-        model_checkpoint = keras.callbacks.ModelCheckpoint(
-            filepath=self._get_checkpoint_fname(trial.trial_id, self._reported_step),
-            monitor=self.oracle.objective.name,
-            mode=self.oracle.objective.direction,
-            save_best_only=True,
-            save_weights_only=True,
+        # Not using `ModelCheckpoint` to support MultiObjective.
+        # It can only track one of the metrics to save the best model.
+        model_checkpoint = tuner_utils.SaveBestEpoch(
+            objective=self.oracle.objective,
+            filepath=self._get_checkpoint_fname(trial.trial_id),
         )
         original_callbacks = kwargs.pop("callbacks", [])
 
@@ -321,14 +308,10 @@ class Tuner(base_tuner.BaseTuner):
 
     def load_model(self, trial):
         model = self._try_build(trial.hyperparameters)
-        # Reload best checkpoint. The Oracle scores the Trial and also
-        # indicates at what epoch the best value of the objective was
-        # obtained.
-        best_epoch = trial.best_step
+        # Reload best checkpoint.
+        # Only load weights to avoid loading `custom_objects`.
         with maybe_distribute(self.distribution_strategy):
-            model.load_weights(
-                self._get_checkpoint_fname(trial.trial_id, best_epoch)
-            )
+            model.load_weights(self._get_checkpoint_fname(trial.trial_id))
         return model
 
     def on_batch_begin(self, trial, model, batch, logs):
@@ -366,7 +349,7 @@ class Tuner(base_tuner.BaseTuner):
 
     def on_epoch_end(self, trial, model, epoch, logs=None):
         # Intermediate results are not passed to the Oracle, and
-        # checkpointing is handled via a `ModelCheckpoint` callback.
+        # checkpointing is handled via a `SaveBestEpoch` callback.
         pass
 
     def get_best_models(self, num_models=1):
@@ -422,27 +405,13 @@ class Tuner(base_tuner.BaseTuner):
     def _get_tensorboard_dir(self, logdir, trial_id, execution):
         return os.path.join(str(logdir), str(trial_id), "execution" + str(execution))
 
-    def _get_checkpoint_dir(self, trial_id, epoch):
-        return os.path.join(
-            self.get_trial_dir(trial_id), "checkpoints", "epoch_" + str(epoch)
-        )
-
-    def _get_checkpoint_fname(self, trial_id, epoch):
+    def _get_checkpoint_fname(self, trial_id):
         return os.path.join(
             # Each checkpoint is saved in its own directory.
-            self._get_checkpoint_dir(trial_id, epoch),
+            self.get_trial_dir(trial_id),
             "checkpoint",
         )
 
-    def _checkpoint_model(self, model, trial_id, epoch):
-        fname = self._get_checkpoint_fname(trial_id, epoch)
-        # Save in TF format.
-        model.save_weights(fname)
-        return fname
-
-    def _delete_checkpoint(self, trial_id, epoch):
-        tf.io.gfile.rmtree(self._get_checkpoint_dir(trial_id, epoch))
-
 
 def maybe_compute_model_size(model):
     """Compute the size of a given model, if it has been built."""
diff --git a/keras_tuner/engine/tuner_utils.py b/keras_tuner/engine/tuner_utils.py
index d1f89c4..945569e 100644
--- a/keras_tuner/engine/tuner_utils.py
+++ b/keras_tuner/engine/tuner_utils.py
@@ -178,6 +178,30 @@ class Display(object):
         return time.strftime("%Hh %Mm %Ss", time.gmtime(t))
 
 
+class SaveBestEpoch(keras.callbacks.Callback):
+    """A Keras callback to save the model weights at the best epoch.
+
+    Args:
+        objective: An `Objective` instance.
+        filepath: String. The file path to save the model weights.
+    """
+
+    def __init__(self, objective, filepath):
+        super().__init__()
+        self.objective = objective
+        self.filepath = filepath
+        if self.objective.direction == "max":
+            self.best_value = float("-inf")
+        else:
+            self.best_value = float("inf")
+
+    def on_epoch_end(self, epoch, logs=None):
+        current_value = self.objective.get_value(logs)
+        if self.objective.better_than(current_value, self.best_value):
+            self.best_value = current_value
+            self.model.save_weights(self.filepath)
+
+
 def average_metrics_dicts(metrics_dicts):
     """Averages the metrics dictionaries to one metrics dictionary."""
     metrics = collections.defaultdict(list)
@@ -202,14 +226,22 @@ def convert_to_metrics_dict(results, objective):
 
     # A History.
     if isinstance(results, keras.callbacks.History):
-        metrics = {}
-        for metric, epoch_values in results.history.items():
-            if objective.direction == "min":
-                best_value = np.min(epoch_values)
-            else:
-                best_value = np.max(epoch_values)
-            metrics[metric] = best_value
-        return metrics
+        # A dictionary to record the metric values through epochs.
+        # Usage: epoch_metric[epoch_number][metric_name] == metric_value
+        epoch_metrics = collections.defaultdict(dict)
+        for metric_name, epoch_values in results.history.items():
+            for epoch, value in enumerate(epoch_values):
+                epoch_metrics[epoch][metric_name] = value
+        best_epoch = 0
+        for epoch, metrics in epoch_metrics.items():
+            objective_value = objective.get_value(metrics)
+            # Support multi-objective.
+            if objective.name not in metrics:
+                metrics[objective.name] = objective_value
+            best_value = epoch_metrics[best_epoch][objective.name]
+            if objective.better_than(objective_value, best_value):
+                best_epoch = epoch
+        return epoch_metrics[best_epoch]
 
     # List to be averaged.
     if isinstance(results, list):
diff --git a/keras_tuner/tuners/bayesian.py b/keras_tuner/tuners/bayesian.py
index 17b9aa6..96f91f2 100644
--- a/keras_tuner/tuners/bayesian.py
+++ b/keras_tuner/tuners/bayesian.py
@@ -137,10 +137,14 @@ class BayesianOptimizationOracle(oracle_module.Oracle):
     https://www.cse.wustl.edu/~garnett/cse515t/spring_2015/files/lecture_notes/12.pdf).
 
     Args:
-        objective: A string or `keras_tuner.Objective` instance. If a string,
-            the direction of the optimization (min or max) will be inferred.
-            It is optional when `Tuner.run_trial()` or `HyperModel.fit()`
-            returns a single float as the objective to minimize.
+        objective: A string, `keras_tuner.Objective` instance, or a list of
+            `keras_tuner.Objective`s and strings. If a string, the direction of
+            the optimization (min or max) will be inferred. If a list of
+            `keras_tuner.Objective`, we will minimize the sum of all the
+            objectives to minimize subtracting the sum of all the objectives to
+            maximize. The `objective` argument is optional when
+            `Tuner.run_trial()` or `HyperModel.fit()` returns a single float as
+            the objective to minimize.
         max_trials: Integer, the total number of trials (model configurations)
             to test at most. Note that the oracle may interrupt the search
             before `max_trial` models have been tested if the search space has
@@ -366,10 +370,14 @@ class BayesianOptimization(tuner_module.Tuner):
             hyperparameters and returns a `Model` instance). It is optional
             when `Tuner.run_trial()` is overriden and does not use
             `self.hypermodel`.
-        objective: A string or `keras_tuner.Objective` instance. If a string,
-            the direction of the optimization (min or max) will be inferred.
-            It is optional when `Tuner.run_trial()` or `HyperModel.fit()`
-            returns a single float as the objective to minimize.
+        objective: A string, `keras_tuner.Objective` instance, or a list of
+            `keras_tuner.Objective`s and strings. If a string, the direction of
+            the optimization (min or max) will be inferred. If a list of
+            `keras_tuner.Objective`, we will minimize the sum of all the
+            objectives to minimize subtracting the sum of all the objectives to
+            maximize. The `objective` argument is optional when
+            `Tuner.run_trial()` or `HyperModel.fit()` returns a single float as
+            the objective to minimize.
         max_trials: Integer, the total number of trials (model configurations)
             to test at most. Note that the oracle may interrupt the search
             before `max_trial` models have been tested if the search space has
diff --git a/keras_tuner/tuners/hyperband.py b/keras_tuner/tuners/hyperband.py
index 9111594..ac613cd 100644
--- a/keras_tuner/tuners/hyperband.py
+++ b/keras_tuner/tuners/hyperband.py
@@ -57,10 +57,14 @@ class HyperbandOracle(oracle_module.Oracle):
     ```
 
     Args:
-        objective: A string or `keras_tuner.Objective` instance. If a string,
-            the direction of the optimization (min or max) will be inferred.
-            It is optional when `Tuner.run_trial()` or `HyperModel.fit()`
-            returns a single float as the objective to minimize.
+        objective: A string, `keras_tuner.Objective` instance, or a list of
+            `keras_tuner.Objective`s and strings. If a string, the direction of
+            the optimization (min or max) will be inferred. If a list of
+            `keras_tuner.Objective`, we will minimize the sum of all the
+            objectives to minimize subtracting the sum of all the objectives to
+            maximize. The `objective` argument is optional when
+            `Tuner.run_trial()` or `HyperModel.fit()` returns a single float as
+            the objective to minimize.
         max_epochs: Integer, the maximum number of epochs to train one model.
             It is recommended to set this to a value slightly higher than the
             expected epochs to convergence for your largest Model, and to use
@@ -308,10 +312,14 @@ class Hyperband(tuner_module.Tuner):
             hyperparameters and returns a `Model` instance). It is optional
             when `Tuner.run_trial()` is overriden and does not use
             `self.hypermodel`.
-        objective: A string or `keras_tuner.Objective` instance. If a string,
-            the direction of the optimization (min or max) will be inferred.
-            It is optional when `Tuner.run_trial()` or `HyperModel.fit()`
-            returns a single float as the objective to minimize.
+        objective: A string, `keras_tuner.Objective` instance, or a list of
+            `keras_tuner.Objective`s and strings. If a string, the direction of
+            the optimization (min or max) will be inferred. If a list of
+            `keras_tuner.Objective`, we will minimize the sum of all the
+            objectives to minimize subtracting the sum of all the objectives to
+            maximize. The `objective` argument is optional when
+            `Tuner.run_trial()` or `HyperModel.fit()` returns a single float as
+            the objective to minimize.
         max_epochs: Integer, the maximum number of epochs to train one model.
             It is recommended to set this to a value slightly higher than the
             expected epochs to convergence for your largest Model, and to use
diff --git a/keras_tuner/tuners/randomsearch.py b/keras_tuner/tuners/randomsearch.py
index a8a4365..f8ab131 100644
--- a/keras_tuner/tuners/randomsearch.py
+++ b/keras_tuner/tuners/randomsearch.py
@@ -24,10 +24,14 @@ class RandomSearchOracle(oracle_module.Oracle):
     """Random search oracle.
 
     Args:
-        objective: A string or `keras_tuner.Objective` instance. If a string,
-            the direction of the optimization (min or max) will be inferred.
-            It is optional when `Tuner.run_trial()` or `HyperModel.fit()`
-            returns a single float as the objective to minimize.
+        objective: A string, `keras_tuner.Objective` instance, or a list of
+            `keras_tuner.Objective`s and strings. If a string, the direction of
+            the optimization (min or max) will be inferred. If a list of
+            `keras_tuner.Objective`, we will minimize the sum of all the
+            objectives to minimize subtracting the sum of all the objectives to
+            maximize. The `objective` argument is optional when
+            `Tuner.run_trial()` or `HyperModel.fit()` returns a single float as
+            the objective to minimize.
         max_trials: Integer, the total number of trials (model configurations)
             to test at most. Note that the oracle may interrupt the search
             before `max_trial` models have been tested if the search space has
@@ -90,10 +94,14 @@ class RandomSearch(tuner_module.Tuner):
             hyperparameters and returns a Model instance). It is optional when
             `Tuner.run_trial()` is overriden and does not use
             `self.hypermodel`.
-        objective: A string or `keras_tuner.Objective` instance. If a string,
-            the direction of the optimization (min or max) will be inferred.
-            It is optional when `Tuner.run_trial()` or `HyperModel.fit()`
-            returns a single float as the objective to minimize.
+        objective: A string, `keras_tuner.Objective` instance, or a list of
+            `keras_tuner.Objective`s and strings. If a string, the direction of
+            the optimization (min or max) will be inferred. If a list of
+            `keras_tuner.Objective`, we will minimize the sum of all the
+            objectives to minimize subtracting the sum of all the objectives to
+            maximize. The `objective` argument is optional when
+            `Tuner.run_trial()` or `HyperModel.fit()` returns a single float as
+            the objective to minimize.
         max_trials: Integer, the total number of trials (model configurations)
             to test at most. Note that the oracle may interrupt the search
             before `max_trial` models have been tested if the search space has


Test patch:
diff --git a/tests/unit_tests/engine/objective_test.py b/tests/unit_tests/engine/objective_test.py
new file mode 100644
index 0000000..2754785
--- /dev/null
+++ b/tests/unit_tests/engine/objective_test.py
@@ -0,0 +1,69 @@
+# Copyright 2022 The KerasTuner Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import keras_tuner
+from keras_tuner.engine import objective
+
+
+def test_create_objective_with_str():
+    obj = objective.create_objective("accuracy")
+    assert obj.name == "accuracy" and obj.direction == "max"
+
+
+def test_create_objective_with_objective():
+    obj = objective.create_objective("accuracy")
+    obj = objective.create_objective(keras_tuner.Objective("score", "min"))
+    assert obj.name == "score" and obj.direction == "min"
+
+
+def test_create_objective_with_multi_objective():
+    obj = objective.create_objective(
+        [keras_tuner.Objective("score", "max"), keras_tuner.Objective("loss", "min")]
+    )
+    assert isinstance(obj, objective.MultiObjective)
+    assert obj.objectives[0].name == "score" and obj.objectives[0].direction == "max"
+    assert obj.objectives[1].name == "loss" and obj.objectives[1].direction == "min"
+
+
+def test_create_objective_with_multi_str():
+    obj = objective.create_objective(["accuracy", "loss"])
+    assert isinstance(obj, objective.MultiObjective)
+    assert (
+        obj.objectives[0].name == "accuracy" and obj.objectives[0].direction == "max"
+    )
+    assert obj.objectives[1].name == "loss" and obj.objectives[1].direction == "min"
+
+
+def test_objective_better_than_max():
+    obj = objective.create_objective("accuracy")
+    assert obj.better_than(1, 0)
+    assert not obj.better_than(0, 1)
+    assert not obj.better_than(0, 0)
+
+
+def test_objective_better_than_min():
+    obj = objective.create_objective("loss")
+    assert obj.better_than(0, 1)
+    assert not obj.better_than(1, 0)
+    assert not obj.better_than(0, 0)
+
+
+def test_objective_get_value():
+    obj = objective.create_objective("loss")
+    assert obj.get_value({"accuracy": 3.0, "loss": 2.0}) == 2.0
+
+
+def test_multi_objective_get_value():
+    obj = objective.create_objective(["accuracy", "loss"])
+    assert obj.get_value({"accuracy": 3.0, "loss": 2.0}) == -1.0
diff --git a/tests/unit_tests/engine/tuner_utils_test.py b/tests/unit_tests/engine/tuner_utils_test.py
new file mode 100644
index 0000000..e477152
--- /dev/null
+++ b/tests/unit_tests/engine/tuner_utils_test.py
@@ -0,0 +1,102 @@
+# Copyright 2022 The KerasTuner Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+
+import numpy as np
+from tensorflow import keras
+
+from keras_tuner.engine import objective as obj_module
+from keras_tuner.engine import tuner_utils
+
+
+def test_save_best_epoch_with_single_objective(tmp_path):
+    objective = obj_module.create_objective("val_loss")
+    filepath = os.path.join(tmp_path, "saved_weights")
+    callback = tuner_utils.SaveBestEpoch(objective, filepath)
+
+    model = keras.Sequential([keras.layers.Dense(1)])
+    model.compile(loss="mse")
+    val_x = np.random.rand(10, 10)
+    val_y = np.random.rand(10, 10)
+    history = model.fit(
+        x=np.random.rand(10, 10),
+        y=np.random.rand(10, 1),
+        validation_data=(val_x, val_y),
+        epochs=10,
+        callbacks=[callback],
+    )
+
+    model.load_weights(filepath)
+
+    assert min(history.history["val_loss"]) == model.evaluate(val_x, val_y)
+
+
+def test_save_best_epoch_with_multi_objective(tmp_path):
+    objective = obj_module.create_objective(["val_loss", "val_mae"])
+    filepath = os.path.join(tmp_path, "saved_weights")
+    callback = tuner_utils.SaveBestEpoch(objective, filepath)
+
+    model = keras.Sequential([keras.layers.Dense(1)])
+    model.compile(loss="mse", metrics=["mae"])
+    val_x = np.random.rand(10, 10)
+    val_y = np.random.rand(10, 10)
+    history = model.fit(
+        x=np.random.rand(10, 10),
+        y=np.random.rand(10, 1),
+        validation_data=(val_x, val_y),
+        epochs=10,
+        callbacks=[callback],
+    )
+
+    model.load_weights(filepath)
+
+    assert min(history.history["val_loss"]) + min(history.history["val_mae"]) == sum(
+        model.evaluate(val_x, val_y)
+    )
+
+
+def test_convert_to_metrics_with_history():
+    model = keras.Sequential([keras.layers.Dense(1)])
+    model.compile(loss="mse", metrics=["mae"])
+    val_x = np.random.rand(10, 10)
+    val_y = np.random.rand(10, 10)
+    history = model.fit(
+        x=np.random.rand(10, 10),
+        y=np.random.rand(10, 1),
+        validation_data=(val_x, val_y),
+    )
+
+    results = tuner_utils.convert_to_metrics_dict(
+        history, obj_module.Objective("val_loss", "min")
+    )
+    assert all([key in results for key in ["loss", "val_loss", "mae", "val_mae"]])
+
+
+def test_convert_to_metrics_with_float():
+    assert tuner_utils.convert_to_metrics_dict(
+        0.1, obj_module.Objective("val_loss", "min")
+    ) == {"val_loss": 0.1}
+
+
+def test_convert_to_metrics_with_dict():
+    assert tuner_utils.convert_to_metrics_dict(
+        {"loss": 0.2, "val_loss": 0.1}, obj_module.Objective("val_loss", "min")
+    ) == {"loss": 0.2, "val_loss": 0.1}
+
+
+def test_convert_to_metrics_with_list_of_floats():
+    assert tuner_utils.convert_to_metrics_dict(
+        [0.1, 0.2], obj_module.Objective("val_loss", "min")
+    ) == {"val_loss": (0.1 + 0.2) / 2}
diff --git a/tests/unit_tests/engine/tuner_workflows_test.py b/tests/unit_tests/engine/tuner_workflows_test.py
index 83e80ec..ff9ca16 100644
--- a/tests/unit_tests/engine/tuner_workflows_test.py
+++ b/tests/unit_tests/engine/tuner_workflows_test.py
@@ -141,6 +141,29 @@ def test_basic_tuner_attributes(tmp_dir):
     assert os.path.exists(os.path.join(str(tmp_dir), "untitled_project"))
 
 
+def test_multi_objective(tmp_dir):
+    tuner = keras_tuner.tuners.RandomSearch(
+        build_model,
+        objective=["val_accuracy", "val_loss"],
+        max_trials=2,
+        executions_per_trial=3,
+        directory=tmp_dir,
+    )
+
+    assert tuner.oracle.objective.name == "multi_objective"
+
+    tuner.search_space_summary()
+
+    tuner.search(
+        x=TRAIN_INPUTS,
+        y=TRAIN_TARGETS,
+        epochs=2,
+        validation_data=(VAL_INPUTS, VAL_TARGETS),
+    )
+
+    tuner.results_summary()
+
+
 def test_no_hypermodel_with_objective(tmp_dir):
     class MyTuner(keras_tuner.tuners.RandomSearch):
         def run_trial(self, trial, *args, **kwargs):
@@ -245,7 +268,7 @@ def test_callbacks_in_fit_kwargs(tmp_dir):
             "TensorBoard",
             "Callback",
             "TunerCallback",
-            "ModelCheckpoint",
+            "SaveBestEpoch",
         ]
 
 
@@ -653,30 +676,6 @@ def test_update_trial(tmp_dir):
         assert len(trial.metrics.get_history("val_accuracy")) == 1
 
 
-def test_objective_formats():
-    obj = keras_tuner.engine.oracle._format_objective("accuracy")
-    assert obj == keras_tuner.Objective("accuracy", "max")
-
-    obj = keras_tuner.engine.oracle._format_objective(
-        keras_tuner.Objective("score", "min")
-    )
-    assert obj == keras_tuner.Objective("score", "min")
-
-    obj = keras_tuner.engine.oracle._format_objective(
-        [keras_tuner.Objective("score", "max"), keras_tuner.Objective("loss", "min")]
-    )
-    assert obj == [
-        keras_tuner.Objective("score", "max"),
-        keras_tuner.Objective("loss", "min"),
-    ]
-
-    obj = keras_tuner.engine.oracle._format_objective(["accuracy", "loss"])
-    assert obj == [
-        keras_tuner.Objective("accuracy", "max"),
-        keras_tuner.Objective("loss", "min"),
-    ]
-
-
 def test_tunable_false_hypermodel(tmp_dir):
     def build_model(hp):
         input_shape = (256, 256, 3)
diff --git a/tests/unit_tests/legacy_import_test.py b/tests/unit_tests/legacy_import_test.py
index 869b701..ef126bf 100644
--- a/tests/unit_tests/legacy_import_test.py
+++ b/tests/unit_tests/legacy_import_test.py
@@ -35,7 +35,7 @@ def test_kerastuner_same_as_keras_tuner():
         from kerastuner.engine.metrics_tracking import (  # noqa: F401
             MetricObservation,
         )
-        from kerastuner.engine.oracle import Objective  # noqa: F401
+        from kerastuner import Objective  # noqa: F401
         from kerastuner.engine.oracle import Oracle  # noqa: F401
         from kerastuner.engine.tuner import Tuner  # noqa: F401
         from kerastuner.engine.stateful import Stateful  # noqa: F401