Repository: optuna/optuna
Pull request: Validation for the parameter enqueued in `to_internal_repr` of `FloatDistribution` and `IntDistribution`
<!-- Thank you for creating a pull request! In general, we merge your pull request after it gets two or more approvals. To proceed to the review process by the maintainers, please make sure that the PR meets the following conditions: (1) it passes all CI checks, and (2) it is neither in the draft nor WIP state. If you wish to discuss the PR in the draft state or need any other help, please mention the Optuna development team in the PR. -->
Note: this PR depends on PR https://github.com/optuna/optuna/pull/4126. Please review and merge it first.

## Motivation
<!-- Describe your motivation why you will submit this PR. This is useful for reviewers to understand the context of PR. -->
Resolve https://github.com/optuna/optuna/issues/4070

## Description of the changes
<!-- Describe the changes in this PR. -->
Add validation to raise error when value <= 0.0 is enqueued with log=True.
In addition, I add to error handling when not a number is enqueued.
I also added test cases to validate this change.



Problem statement:
ValueError: math domain error
### Expected behavior

Getting a math domain error on what appears to be valid parameter suggestions (ie no 0 or negative log)


### Environment

- Python version:3.8.13
- Optuna version:3.0.2
- OS:Linux-3.10.0-1127.el7.x86_64-x86_64-with-glibc2.17

### Error messages, stack traces, or logs

```shell
Error Message:

Traceback (most recent call last):
  File "eras.py", line 47, in <module>
    pipeline = pipeline.fit(X_train, y_train, x_test=X_test, y_test=y_test)
  File "/home/jmrichardson/fml/fml/pipelines/standard.py", line 236, in fit
    model.optimize(features, test=test_features)
  File "/home/jmrichardson/fml/fml/models/lgbm.py", line 92, in optimize
    study.optimize(partial(objective, train, test), n_trials=self.n_trials, timeout=self.timeout, show_progress_bar=True)
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/study/study.py", line 419, in optimize
    _optimize(
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/study/_optimize.py", line 66, in _optimize
    _optimize_sequential(
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/study/_optimize.py", line 160, in _optimize_sequential
    frozen_trial = _run_trial(study, func, catch)
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/study/_optimize.py", line 187, in _run_trial
    trial = study.ask()
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/study/study.py", line 514, in ask
    trial = trial_module.Trial(self, trial_id)
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/trial/_trial.py", line 54, in __init__
    self._init_relative_params()
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/trial/_trial.py", line 63, in _init_relative_params
    self.relative_params = self.study.sampler.sample_relative(
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/samplers/nsgaii/_sampler.py", line 211, in sample_relative
    child_params = perform_crossover(
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/samplers/nsgaii/_crossover.py", line 104, in perform_crossover
    child_params = _try_crossover(
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/samplers/nsgaii/_crossover.py", line 60, in _try_crossover
    [
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/samplers/nsgaii/_crossover.py", line 61, in <listcomp>
    numerical_transform.transform(
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/_transform.py", line 117, in transform
    trans_params[bound_idx] = _transform_numerical_param(
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/_transform.py", line 239, in _transform_numerical_param
    trans_param = math.log(param) if transform_log else float(param)
ValueError: math domain error
```


### Steps to reproduce

Parameter suggest in objective function:

```
params = {'n_estimators': trial.suggest_int("n_estimators", 2000, 6000),
              'num_leaves': trial.suggest_int("num_leaves", 12, 256),
              'min_child_weight': trial.suggest_float('min_child_weight', 1e-5, 1024.0, log=True),
              'learning_rate': trial.suggest_float("learning_rate", .0001, 0.02, step=.001),
              'colsample_bytree': trial.suggest_float("colsample_bytree", 0.01, 1.0),
              'subsample': trial.suggest_float("subsample", 0.01, 1.0),
              'reg_alpha': trial.suggest_float("reg_alpha", 1e-5, 1024, log=True),
              'reg_lambda': trial.suggest_float("reg_lambda", 1e-5, 1024, log=True),
              'max_depth': trial.suggest_int("max_depth", 3, 32),
              "seed": 123,
              "objective": "regression",
              "verbose": -1,
              "force_row_wise": True,
              "metric": "None"}
```

### Additional context (optional)

_No response_

Hints from issue discussion:
Hi, could you share minimal reproducible codes with us?
Hi @nzw0301 

Unfortunately I don't have reproduceable code but I believe I was able to narrow it down to enqueueing a trial with parameters out of range from trial suggestion.  For example, if I enqueue reg_alpha as 0.0 while my trial suggestion is in range (1e-5, 1024).  The enqueue does work but future suggestions does appear to include reg_alpha as 0.0 which I believe causes the error because math.log(0.0) produces the math domain error.  
I'm afraid I cannot reproduce it:

```python
import optuna

def objective1(trial):
    trial.suggest_float("y", 1e-4, 10, log=True)
    return trial.suggest_float("x", 1e-4, 10, log=True)


study = optuna.create_study(sampler=optuna.samplers.NSGAIISampler(seed=2))
study.enqueue_trial({"x": 0., "y": 0.})
study.optimize(objective1, n_trials=20)
```
Hi @nzw0301 

I think it may be related to the dataset in terms of when this may occur.  My dataset has a low signal to noise ratio and 1M rows, 1200 features.  The first enqueue is the best parameter set I've found and I am thinking it's likely that Optuna may be trying to use the parameters of the enqueue because it has the best performance.  Also, the error happens around the 100 to 200 trials typically which takes a considerable amount of time.  Since I updated the enqueue to be 1e-5 instead of 0 the issue hasn't happened again.  I also put a try,except block around math.log and verified the parameter is 0 when the error occurred.
I may miss the point, but I found that the error occurred in the crossover of NSGAIISampler.
Could you tell us which crossover class you used, please?

```console
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/samplers/nsgaii/_sampler.py", line 211, in sample_relative
    child_params = perform_crossover(
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/samplers/nsgaii/_crossover.py", line 104, in perform_crossover
    child_params = _try_crossover(
  File "/home/jmrichardson/anaconda3/envs/fml/lib/python3.8/site-packages/optuna/samplers/nsgaii/_crossover.py", line 60, in _try_crossover
    [
```
I found the following code reproduces the error.

```python
import optuna

def objective1(trial):
    trial.suggest_float("y", 1e-4, 10, log=True)
    return trial.suggest_float("x", 1e-4, 10, log=True)


study = optuna.create_study(sampler=optuna.samplers.NSGAIISampler(seed=2, population_size=2))
study.enqueue_trial({"x": 0., "y": 0.})
study.optimize(objective1, n_trials=20)
```

Commit message: Add comment to _suggest method about validation of parameter in to_internal_repr method

Patch to review:
diff --git a/optuna/distributions.py b/optuna/distributions.py
index fd1d785fde3..7fdd4d99abf 100644
--- a/optuna/distributions.py
+++ b/optuna/distributions.py
@@ -191,13 +191,21 @@ def _contains(self, param_value_in_internal_repr: float) -> bool:
 
     def to_internal_repr(self, param_value_in_external_repr: float) -> float:
         try:
-            return float(param_value_in_external_repr)
+            internal_repr = float(param_value_in_external_repr)
         except (ValueError, TypeError) as e:
             raise ValueError(
                 f"'{param_value_in_external_repr}' is not a valid type. "
                 "float-castable value is expected."
             ) from e
 
+        if np.isnan(internal_repr):
+            raise ValueError(f"`{param_value_in_external_repr}` is invalid value.")
+        if self.log and internal_repr <= 0.0:
+            raise ValueError(
+                f"`{param_value_in_external_repr}` is invalid value for the case log=True."
+            )
+        return internal_repr
+
 
 @deprecated_class("3.0.0", "6.0.0", text=_float_distribution_deprecated_msg)
 class UniformDistribution(FloatDistribution):
@@ -375,13 +383,21 @@ def to_external_repr(self, param_value_in_internal_repr: float) -> int:
 
     def to_internal_repr(self, param_value_in_external_repr: int) -> float:
         try:
-            return float(param_value_in_external_repr)
+            internal_repr = float(param_value_in_external_repr)
         except (ValueError, TypeError) as e:
             raise ValueError(
                 f"'{param_value_in_external_repr}' is not a valid type. "
                 "float-castable value is expected."
             ) from e
 
+        if np.isnan(internal_repr):
+            raise ValueError(f"`{param_value_in_external_repr}` is invalid value.")
+        if self.log and internal_repr <= 0.0:
+            raise ValueError(
+                f"`{param_value_in_external_repr}` is invalid value for the case log=True."
+            )
+        return internal_repr
+
     def single(self) -> bool:
         if self.log:
             return self.low == self.high
diff --git a/optuna/trial/_trial.py b/optuna/trial/_trial.py
index 5579b7bd337..d20316b25c1 100644
--- a/optuna/trial/_trial.py
+++ b/optuna/trial/_trial.py
@@ -597,6 +597,7 @@ def _suggest(self, name: str, distribution: BaseDistribution) -> Any:
                     study, trial, name, distribution
                 )
 
+            # param_value is validated here (invalid value like np.nan raises ValueError)
             param_value_in_internal_repr = distribution.to_internal_repr(param_value)
             storage.set_trial_param(trial_id, name, param_value_in_internal_repr, distribution)
 
diff --git a/tests/test_distributions.py b/tests/test_distributions.py
index fea40a569de..be86c90492f 100644
--- a/tests/test_distributions.py
+++ b/tests/test_distributions.py
@@ -213,9 +213,21 @@ def test_int_internal_representation(value: Any) -> None:
     assert i.to_external_repr(i.to_internal_repr(value)) == expected_value
 
 
-@pytest.mark.parametrize("value", ("foo", (), [], {}, set(), np.ones(2)))
-def test_int_internal_representation_error(value: Any) -> None:
-    i = distributions.IntDistribution(low=1, high=10)
+@pytest.mark.parametrize(
+    "value, kwargs",
+    [
+        ("foo", {}),
+        ((), {}),
+        ([], {}),
+        ({}, {}),
+        (set(), {}),
+        (np.ones(2), {}),
+        (np.nan, {}),
+        (0, dict(log=True)),
+    ],
+)
+def test_int_internal_representation_error(value: Any, kwargs: Dict[str, Any]) -> None:
+    i = distributions.IntDistribution(low=1, high=10, **kwargs)
     with pytest.raises(ValueError):
         i.to_internal_repr(value)
 
@@ -234,9 +246,21 @@ def test_float_internal_representation(value: Any) -> None:
     assert f.to_external_repr(f.to_internal_repr(value)) == expected_value
 
 
-@pytest.mark.parametrize("value", ("foo", (), [], {}, set(), np.ones(2)))
-def test_float_internal_representation_error(value: Any) -> None:
-    f = distributions.FloatDistribution(low=2.0, high=7.0)
+@pytest.mark.parametrize(
+    "value, kwargs",
+    [
+        ("foo", {}),
+        ((), {}),
+        ([], {}),
+        ({}, {}),
+        (set(), {}),
+        (np.ones(2), {}),
+        (np.nan, {}),
+        (0, dict(log=True)),
+    ],
+)
+def test_float_internal_representation_error(value: Any, kwargs: Dict[str, Any]) -> None:
+    f = distributions.FloatDistribution(low=2.0, high=7.0, **kwargs)
     with pytest.raises(ValueError):
         f.to_internal_repr(value)
 


Code review comments:
diff_hunk: @@ -234,9 +246,21 @@ def test_float_internal_representation(value: Any) -> None:
     assert f.to_external_repr(f.to_internal_repr(value)) == expected_value
 
 
-@pytest.mark.parametrize("value", ("foo", (), [], {}, set(), np.ones(2)))
-def test_float_internal_representation_error(value: Any) -> None:
-    f = distributions.FloatDistribution(low=2.0, high=7.0)
+@pytest.mark.parametrize(
+    "value, kwargs",
+    [
+        ("foo", {}),
+        ((), {}),
+        ([], {}),
+        ({}, {}),
+        (set(), {}),
+        (np.ones(2), {}),
+        (np.nan, {}),
+        (0, dict(log=True)),
path: tests/test_distributions.py
text: @user1:
```suggestion
        (0.0, dict(log=True)),
        (-1.0, dict(log=True)),
```

@user1:
Ditto.