Repository: optuna/optuna

Issue:
Objective values of +/-inf not handled correctly with PostgreSQL
### Expected behavior

Hi Optuna devs, thanks for all the hardwork on an amazing library! We're trying to put Optuna to good use at a big corp to optimise our industrial processes. Recently we've hit an issue with distributed optimisation jobs.

When performing an optimisation which attempts to maximise/minimise an objective function which returns -/+inf in some cases, I expect the optimisation process to be robust against the presence of possible inf values. In the case that I choose to maximise and some trials return -inf whilst others return finite values, I expect the optimum trial to be that which returned the largest finite value; instead, the first trial returning -inf is found to be the optimum. The same occurs when minimising and the objective sometimes returns +inf.

### Environment

- Optuna version: 3.1.0
- Python version: 3.9.16
- OS: Ubuntu 20.04
- (Optional) Other libraries and their versions:
  - PostgreSQL 11


### Error messages, stack traces, or logs

```shell
[I 2023-02-14 16:20:48,737] A new study created in RDB with name: optuna-inf-bug-87376ba1-5ca6-4da4-bf7f-7050ea2d6df6
[I 2023-02-14 16:20:51,302] Trial 0 finished with value: 1.0 and parameters: {}. Best is trial 0 with value: 1.0.
[I 2023-02-14 16:20:53,541] Trial 1 finished with value: 2.0 and parameters: {}. Best is trial 1 with value: 2.0.
[I 2023-02-14 16:20:56,011] Trial 2 finished with value: 3.0 and parameters: {}. Best is trial 2 with value: 3.0.
[I 2023-02-14 16:20:58,538] Trial 3 finished with value: -inf and parameters: {}. Best is trial 3 with value: -inf.
[I 2023-02-14 16:21:00,780] Trial 4 finished with value: 5.0 and parameters: {}. Best is trial 3 with value: -inf.
```


### Steps to reproduce

1. Set `DATABASE_URL` in the environment to securely connect to a PostgreSQL database configured for use with Optuna.
2. Run the below script.
3. Observe spurious reported best trial info in the Optuna output.
```python
#!/usr/bin/env python3
"""Minimal reproducible example for Optuna PostgreSQL storage inf values bug."""

from collections import deque
import os
import typing as _t
from uuid import uuid4

import numpy as np
import optuna


# Values of np.inf are stored with TrialValueModel.value=null and value_type=INF_NEG causing incorrect min/max values.
DATA: _t.Deque[float] = deque([1.0, 2.0, 3.0, -np.inf, 5.0])
# Using largest system float instead is a workaround.
# DATA: _t.Deque[float] = deque([1.0, 2.0, 3.0, np.finfo(np.float64).min, 5.0])


def objective(trial: optuna.Trial) -> float:
    return DATA.popleft()


def main() -> None:
    study: optuna.Study = optuna.create_study(
        study_name=f"optuna-inf-bug-{uuid4()}",
        sampler=optuna.samplers.TPESampler(),
        storage=os.getenv("DATABASE_URL"),
        direction="maximize",
    )
    study.optimize(objective, n_trials=len(DATA), n_jobs=1)


if __name__ == "__main__":
    main()
```


### Additional context (optional)

From poking around in the debugger and looking at the data in my PostgreSQL instance I have found the root cause. When attempting to retrieve the best trial from the database in [`optuna.storages._rdb.models.TrialModel.find_max_value_trial`](https://github.com/optuna/optuna/blob/master/optuna/storages/_rdb/models.py#L182), there is an ORDER BY DESC TrialValueModel.value followed by LIMIT 1. This query does not account for TrialValueModel.value being stored as null in the database as it is for +/-inf. These null values appear at the top in the ordered query results and so get returned as optimal by the LIMIT statement.

Some thoughts... `order_by(nullslast(desc(TrialValueModel.value)))` could be used but this would preclude the possibility that +/-inf values constitute part of an optimal group. Results could be loaded into local memory without the LIMIT statement, converted to their python value with `TrialValueModel.stored_repr_to_value`, then passed to `max/min`; however, this could involve very large query sets being transferred to the host which is not desireable. I'd be happy to put in a PR for a fix but I'm not sure what the best approach would be here..


Discussion:

Resolving patch:
diff --git a/optuna/storages/_rdb/models.py b/optuna/storages/_rdb/models.py
--- a/optuna/storages/_rdb/models.py
+++ b/optuna/storages/_rdb/models.py
@@ -6,6 +6,7 @@
 from typing import Tuple
 
 from sqlalchemy import asc
+from sqlalchemy import case
 from sqlalchemy import CheckConstraint
 from sqlalchemy import Column
 from sqlalchemy import DateTime
@@ -188,7 +189,15 @@ def find_max_value_trial(
             .filter(cls.state == TrialState.COMPLETE)
             .join(TrialValueModel)
             .filter(TrialValueModel.objective == objective)
-            .order_by(desc(TrialValueModel.value))
+            .order_by(
+                desc(
+                    case(
+                        {"INF_NEG": -1, "FINITE": 0, "INF_POS": 1},
+                        value=TrialValueModel.value_type,
+                    )
+                ),
+                desc(TrialValueModel.value),
+            )
             .limit(1)
             .one_or_none()
         )
@@ -206,7 +215,15 @@ def find_min_value_trial(
             .filter(cls.state == TrialState.COMPLETE)
             .join(TrialValueModel)
             .filter(TrialValueModel.objective == objective)
-            .order_by(asc(TrialValueModel.value))
+            .order_by(
+                asc(
+                    case(
+                        {"INF_NEG": -1, "FINITE": 0, "INF_POS": 1},
+                        value=TrialValueModel.value_type,
+                    )
+                ),
+                asc(TrialValueModel.value),
+            )
             .limit(1)
             .one_or_none()
         )


Test patch:
diff --git a/tests/storages_tests/test_storages.py b/tests/storages_tests/test_storages.py
--- a/tests/storages_tests/test_storages.py
+++ b/tests/storages_tests/test_storages.py
@@ -865,9 +865,21 @@ def test_get_n_trials_state_option(storage_mode: str) -> None:
 
 
 @pytest.mark.parametrize("storage_mode", STORAGE_MODES)
-def test_get_best_trial(storage_mode: str) -> None:
+@pytest.mark.parametrize("direction", [StudyDirection.MAXIMIZE, StudyDirection.MINIMIZE])
+@pytest.mark.parametrize(
+    "values",
+    [
+        [0.0, 1.0, 2.0],
+        [0.0, float("inf"), 1.0],
+        [0.0, float("-inf"), 1.0],
+        [float("inf"), 0.0, 1.0, float("-inf")],
+        [float("inf")],
+        [float("-inf")],
+    ],
+)
+def test_get_best_trial(storage_mode: str, direction: StudyDirection, values: List[float]) -> None:
     with StorageSupplier(storage_mode) as storage:
-        study_id = storage.create_new_study(directions=[StudyDirection.MAXIMIZE])
+        study_id = storage.create_new_study(directions=[direction])
         with pytest.raises(ValueError):
             storage.get_best_trial(study_id)
 
@@ -875,12 +887,14 @@ def test_get_best_trial(storage_mode: str) -> None:
             storage.get_best_trial(study_id + 1)
 
         generator = random.Random(51)
-        for i in range(3):
+
+        for v in values:
             template_trial = _generate_trial(generator)
             template_trial.state = TrialState.COMPLETE
-            template_trial.value = float(i)
+            template_trial.value = v
             storage.create_new_trial(study_id, template_trial=template_trial)
-        assert storage.get_best_trial(study_id).number == i
+        expected_value = max(values) if direction == StudyDirection.MAXIMIZE else min(values)
+        assert storage.get_best_trial(study_id).value == expected_value
 
 
 def test_get_trials_excluded_trial_ids() -> None:
diff --git a/tests/storages_tests/test_with_server.py b/tests/storages_tests/test_with_server.py
--- a/tests/storages_tests/test_with_server.py
+++ b/tests/storages_tests/test_with_server.py
@@ -180,3 +180,25 @@ def test_pickle_storage() -> None:
     storage_system_attrs = storage.get_study_system_attrs(study_id)
     restored_storage_system_attrs = restored_storage.get_study_system_attrs(study_id)
     assert storage_system_attrs == restored_storage_system_attrs == {"key": "pickle"}
+
+
+@pytest.mark.parametrize("direction", [StudyDirection.MAXIMIZE, StudyDirection.MINIMIZE])
+@pytest.mark.parametrize(
+    "values",
+    [
+        [0.0, 1.0, 2.0],
+        [0.0, float("inf"), 1.0],
+        [0.0, float("-inf"), 1.0],
+        [float("inf"), 0.0, 1.0, float("-inf")],
+        [float("inf")],
+        [float("-inf")],
+    ],
+)
+def test_get_best_trial(direction: StudyDirection, values: Sequence[float]) -> None:
+    storage = get_storage()
+    study = optuna.create_study(direction=direction, storage=storage)
+    study.add_trials(
+        [optuna.create_trial(params={}, distributions={}, value=value) for value in values]
+    )
+    expected_value = max(values) if direction == StudyDirection.MAXIMIZE else min(values)
+    assert study.best_value == expected_value