Repository: optuna/optuna

Issue:
`NSGAIISampler` tries to evaluate `constraint_func` even when trails fail.
## Expected behavior

`NSGAIISampler` does not evaluate `constraints_func` when trials did not complete.

## Environment

- Optuna version: `2.9.0.dev`
- Python version: 3.8.6
- OS: ubuntu (WSL2)
- (Optional) Other libraries and their versions:

## Error messages, stack traces, or logs

```
[I 2021-07-07 12:44:56,751] A new study created in memory with name: no-name-be57671e-58b7-4219-a290-93f93adccdcd
[I 2021-07-07 12:44:56,752] Trial 0 finished with values: [0.1, 0.2] and parameters: {}.
[I 2021-07-07 12:44:56,752] Trial 1 pruned.
Traceback (most recent call last):
  File "tmp.py", line 11, in <module>
    optuna.create_study(sampler=sampler, directions=["maximize", "minimize"]).optimize(objective, n_trials=200)
  File "path/to/optuna/optuna/study/study.py", line 400, in optimize
    _optimize(
  File "path/to/optuna/optuna/study/_optimize.py", line 65, in _optimize
    _optimize_sequential(
  File "path/to/optuna/optuna/study/_optimize.py", line 162, in _optimize_sequential
    trial = _run_trial(study, func, catch)
  File "path/to/optuna/optuna/study/_optimize.py", line 243, in _run_trial
    study.tell(trial, values=values, state=state)
  File "path/to/optuna/optuna/study/study.py", line 657, in tell
    self.sampler.after_trial(study, frozen_trial, state, values)
  File "path/to/optuna/optuna/samplers/_nsga2.py", line 353, in after_trial
    con = self._constraints_func(trial)
  File "tmp.py", line 10, in <lambda>
    sampler = optuna.samplers.NSGAIISampler(constraints_func=lambda t: [t.user_attrs["constraint"]])
KeyError: 'constraint'
```

## Steps to reproduce

1. Optimize an objective that failes w/ `NSGAIISampler` and `constraint_func`.

## Reproducible examples (optional)

```python
import optuna

def objective(trial):
    if trial.number % 2:
        raise optuna.TrialPruned
    trial.set_user_attr("constraint", 0.2 - 0.1)
    return 0.1, 0.2


sampler = optuna.samplers.NSGAIISampler(constraints_func=lambda t: [t.user_attrs["constraint"]])
optuna.create_study(sampler=sampler, directions=["maximize", "minimize"]).optimize(objective, n_trials=200)
```

## Additional context (optional)

<!-- Please add any other context or screenshots about the problem here. -->



Discussion:

Resolving patch:
diff --git a/optuna/samplers/_nsga2.py b/optuna/samplers/_nsga2.py
--- a/optuna/samplers/_nsga2.py
+++ b/optuna/samplers/_nsga2.py
@@ -71,6 +71,10 @@ class NSGAIISampler(BaseSampler):
             If constraints_func returns more than one value for a trial, that trial is considered
             feasible if and only if all values are equal to 0 or smaller.
 
+            The constraint_func will be evaluated after each successful trial.
+            The function won't be called when trials fail or they are pruned, but this behavior is
+            subject to change in the future releases.
+
             The constraints are handled by the constrained domination. A trial x is said to
             constrained-dominate a trial y, if any of the following conditions is true:
 
@@ -213,6 +217,7 @@ def _collect_parent_population(self, study: Study) -> Tuple[int, List[FrozenTria
                     generation_to_runnings[generation].append(trial)
                 continue
 
+            # Do not use trials whose states are not COMPLETE, or `constraint` will be unavailable.
             generation_to_population[generation].append(trial)
 
         hasher = hashlib.sha256()
@@ -347,7 +352,8 @@ def after_trial(
         state: TrialState,
         values: Optional[Sequence[float]],
     ) -> None:
-        if self._constraints_func is not None:
+        assert state in [TrialState.COMPLETE, TrialState.FAIL, TrialState.PRUNED]
+        if state == TrialState.COMPLETE and self._constraints_func is not None:
             constraints = None
             try:
                 con = self._constraints_func(trial)


Test patch: