Repository: optuna/optuna

Issue:
TPESampler constant_liar=True with constrained optimization spams "Trial does not have constraint values. It will be treated as a lower priority than other trials."
### Expected behavior

When I don't pass in `constant_liar=True` this warning doesn't show up, and I don't think having `constant_liar=True` should be warning either. Also when I rerun the same code it no longer warns.

### Environment

- Optuna version:3.5.0
- Python version:3.10.13
- OS:Linux-5.15.102

### Error messages, stack traces, or logs

```shell
/usr/.../lib/python3.10/site-packages/optuna/samplers/_tpe/sampler.py:746: UserWarning:
Trial 10 does not have constraint values. It will be treated as a lower priority than other trials.
...
Trial 199 does not have constraint values. It will be treated as a lower priority than other trials.
```

Strangely, Trials 0-9 do not have any warnings.


### Steps to reproduce


```python
import optuna

def objective(trial):
    # Binh and Korn function with constraints.
    x = trial.suggest_float("x", -15, 30)
    y = trial.suggest_float("y", -15, 30)

    # Constraints which are considered feasible if less than or equal to zero.
    # The feasible region is basically the intersection of a circle centered at (x=5, y=0)
    # and the complement to a circle centered at (x=8, y=-3).
    c0 = (x - 5) ** 2 + y**2 - 25
    c1 = -((x - 8) ** 2) - (y + 3) ** 2 + 7.7

    # Store the constraints as user attributes so that they can be restored after optimization.
    trial.set_user_attr("constraint", (c0, c1))

    v0 = 4 * x**2 + 4 * y**2
    v1 = (x - 5) ** 2 + (y - 5) ** 2

    return v0, v1


def constraints(trial):
    return trial.user_attrs["constraint"]

sampler = optuna.samplers.TPESampler(constraints_func=constraints, constant_liar=True)
study = optuna.create_study(
    directions=["minimize", "minimize"],
    sampler=sampler,
)

# distributed optimization
batch_size = 10
rounds = 20
for round_idx in range(rounds):
    objs = []
    trials = []
    for batch_idx in range(batch_size):
        trials.append((trial := study.ask()))
        objs.append(objective(trial))

    for obj, trial in zip(objs, trials):
        study.tell(trial, obj)
```


Discussion:
> Strangely, Trials 0-9 do not have any warnings

The first 10 trials use random sampler instead of TPESampler with a default argument . At least this behaviour is expected.
Thank you for reporting it. As a workaround, could you use `trial.set_system_attr` rather than `set_user_attr`? 

```python
import optuna


def objective(trial):
    # Binh and Korn function with constraints.
    x = trial.suggest_float("x", -15, 30)
    y = trial.suggest_float("y", -15, 30)

    # Constraints which are considered feasible if less than or equal to zero.
    # The feasible region is basically the intersection of a circle centered at (x=5, y=0)
    # and the complement to a circle centered at (x=8, y=-3).
    c0 = (x - 5) ** 2 + y ** 2 - 25
    c1 = -((x - 8) ** 2) - (y + 3) ** 2 + 7.7

    # Store the constraints as user attributes so that they can be restored after optimization.
    # trial.set_user_attr("constraint", (c0, c1))
    trial.set_system_attr("constraint", (c0, c1))

    v0 = 4 * x ** 2 + 4 * y ** 2
    v1 = (x - 5) ** 2 + (y - 5) ** 2

    return v0, v1


def constraints(trial):
    # return trial.user_attrs["constraint"]
    return trial.system_attrs["constraint"]


sampler = optuna.samplers.TPESampler(constraints_func=constraints, constant_liar=True)
study = optuna.create_study(
    directions=["minimize", "minimize"],
    sampler=sampler,
)
study.optimize(objective, n_trials=32, timeout=600)
```

Anyways, I think this should be fixed.
Hi @nzw0301 thanks for getting back to me so quickly. I think using system attrs doesn't work either:

```
import optuna


def objective(trial):
    # Binh and Korn function with constraints.
    x = trial.suggest_float("x", -15, 30)
    y = trial.suggest_float("y", -15, 30)

    # Constraints which are considered feasible if less than or equal to zero.
    # The feasible region is basically the intersection of a circle centered at (x=5, y=0)
    # and the complement to a circle centered at (x=8, y=-3).
    c0 = (x - 5) ** 2 + y**2 - 25
    c1 = -((x - 8) ** 2) - (y + 3) ** 2 + 7.7

    # Store the constraints as user attributes so that they can be restored after optimization.
    trial.set_system_attr("constraint", (c0, c1))

    v0 = 4 * x**2 + 4 * y**2
    v1 = (x - 5) ** 2 + (y - 5) ** 2

    return v0, v1


def constraints(trial):
    return trial.system_attrs["constraint"]


sampler = optuna.samplers.TPESampler(constraints_func=constraints, constant_liar=True)
study = optuna.create_study(
    directions=["minimize", "minimize"],
    sampler=sampler,
)

# distributed optimization
batch_size = 10
rounds = 20
for round_idx in range(rounds):
    objs = []
    trials = []
    for batch_idx in range(batch_size):
        trials.append((trial := study.ask()))
        objs.append(objective(trial))

    for obj, trial in zip(objs, trials):
        study.tell(trial, obj)
```

Runs into the same problem. I don't think you can reproduce it if you use `study.optimize` instead of this batch optimization approach
Thank you for your check. You're right. Sorry for confusing you.
no problem. Is there any quick hacks I can do to get around this? I'm assuming this is showing up because constant_liar's implementation doesn't add any constraints during the period after ask() and before tell(). Does the fact that we don't have constraint values and it being "treated as a lower priority" actually cause a quantifiable difference in the optimization quality? If not, I'm happy to suppress this warning for now on my side.

Resolving patch:
diff --git a/optuna/samplers/_tpe/sampler.py b/optuna/samplers/_tpe/sampler.py
--- a/optuna/samplers/_tpe/sampler.py
+++ b/optuna/samplers/_tpe/sampler.py
@@ -620,14 +620,16 @@ def _split_trials(
     infeasible_trials = []
 
     for trial in trials:
-        if constraints_enabled and _get_infeasible_trial_score(trial) > 0:
+        if trial.state == TrialState.RUNNING:
+            # We should check if the trial is RUNNING before the feasibility check
+            # because its constraint values have not yet been set.
+            running_trials.append(trial)
+        elif constraints_enabled and _get_infeasible_trial_score(trial) > 0:
             infeasible_trials.append(trial)
         elif trial.state == TrialState.COMPLETE:
             complete_trials.append(trial)
         elif trial.state == TrialState.PRUNED:
             pruned_trials.append(trial)
-        elif trial.state == TrialState.RUNNING:
-            running_trials.append(trial)
         else:
             assert False
 


Test patch:
diff --git a/tests/samplers_tests/tpe_tests/test_sampler.py b/tests/samplers_tests/tpe_tests/test_sampler.py
--- a/tests/samplers_tests/tpe_tests/test_sampler.py
+++ b/tests/samplers_tests/tpe_tests/test_sampler.py
@@ -776,7 +776,6 @@ def test_split_trials(direction: str, constant_liar: bool, constraints: bool) ->
             state=optuna.trial.TrialState.RUNNING,
             params={"x": 0},
             distributions={"x": optuna.distributions.FloatDistribution(-1.0, 1.0)},
-            system_attrs={_CONSTRAINTS_KEY: [-1]},
         )
     )