Repository: optuna/optuna

Issue:
Default value of `verbose_eval` for LightGBMTuner/LightGBMTunerCV causes deprecated warning 
Since LightGBM 3.3.0, the following arguments are deprecated to use callbacks instead:
- `verbose_eval`
- `early_stopping_rounds` 
- `learning_rates`
- `eval_result`

https://github.com/microsoft/LightGBM/commit/86bda6f061138b666aa41f4dbea7285f8efdc5a7

Example arguments before LightGBM 3.3.0:

```
tuner = lgb.LightGBMTunerCV(params, dtrain, verbose_eval=100, early_stopping_rounds=100, folds=KFold(n_splits=3))
```

Equivalent arguments in LightGBM 3.3.0:

```
tuner = lgb.LightGBMTunerCV(params, dtrain, callbacks=[log_evaluation(100), early_stopping(100, False, True)], folds=KFold(n_splits=3))
```

However, the default value of `verbose_eval` for `LightGBMTuner` and `LightGBMTunerCV` is `True` so that we need to write extra `verbose_eval=None` to avoid blocking to use `log_evaluation` callback.

https://github.com/optuna/optuna/blob/563d0c7a50114a19f5cac2a502cdf7b0a68d2bab/optuna/integration/_lightgbm_tuner/optimize.py#L364

```
tuner = lgb.LightGBMTunerCV(params, dtrain, verbose_eval=None, callbacks=[log_evaluation(100), early_stopping(100, False, True)], folds=KFold(n_splits=3))
```


## Expected behavior

No need to write extra `verbose_eval=None` and output logs like:

```
[I 2021-10-17 02:22:20,543] A new study created in memory with name: no-name-1f095c7a-7f99-4b63-b18d-09d228fd3e16

  0%|          | 0/7 [00:00<?, ?it/s]
feature_fraction, val_score: inf:   0%|          | 0/7 [00:00<?, ?it/s]Training until validation scores don't improve for 100 rounds
[100]	cv_agg's l2: 1067.5 + 83.0349

feature_fraction, val_score: 1051.388339:   0%|          | 0/7 [00:00<?, ?it/s]
feature_fraction, val_score: 1051.388339:  14%|#4        | 1/7 [00:00<00:01,  3.21it/s][I 2021-10-17 02:22:20,880] Trial 0 finished with value: 1051.3883391380703 and parameters: {'feature_fraction': 0.4}. Best is trial 0 with value: 1051.3883391380703.

feature_fraction, val_score: 1051.388339:  14%|#4        | 1/7 [00:00<00:01,  3.21it/s]Early stopping, best iteration is:
[35]	cv_agg's l2: 1051.39 + 83.9186
```



## Environment

<!--
You can get this information by typing the following:
```
python -c 'import optuna; print(optuna.__version__)'
python -c 'import platform; print(platform.python_version())'
python -c 'import platform; print(platform.platform())'
```
-->

- Optuna version: 2.10.0
- Python version: 3.7.12
- OS: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- (Optional) Other libraries and their versions: LightGBM 3.3.0

## Error messages, stack traces, or logs

```
[I 2021-10-17 02:39:50,486] A new study created in memory with name: no-name-741b6e25-44ba-4f3b-bbd9-54547b04f3c9


  0%|          | 0/7 [00:00<?, ?it/s]

feature_fraction, val_score: inf:   0%|          | 0/7 [00:00<?, ?it/s]/usr/local/lib/python3.7/dist-packages/lightgbm/engine.py:621: UserWarning:

'verbose_eval' argument is deprecated and will be removed in a future release of LightGBM. Pass 'log_evaluation()' callback via 'callbacks' argument instead.

[1]	cv_agg's l2: 1218.99 + 85.3389
Training until validation scores don't improve for 100 rounds
[2]	cv_agg's l2: 1185.36 + 85.4061
[3]	cv_agg's l2: 1158.39 + 85.7244
[4]	cv_agg's l2: 1136.65 + 86.152
```




## Steps to reproduce

1. Create an instance of LightGBMTuner/LightGBMTunerCV without `verbose_eval=None` with `log_evaluation` callbacks
2. Run tuner

## Reproducible examples (optional)

```python
from sklearn.model_selection import KFold
from lightgbm import early_stopping, log_evaluation

params = {
    "objective": "regression",
    "metric": "l2",
    "verbosity": -1,
    "boosting_type": "gbdt",
    "random_state": 314
}

tuner = lgb.LightGBMTunerCV(params, dtrain, callbacks=[log_evaluation(100), early_stopping(100, False, True)], folds=KFold(n_splits=3))
tuner.run()
```

## Additional context (optional)

<!-- Please add any other context or screenshots about the problem here. -->



Discussion:

Resolving patch:
diff --git a/optuna/integration/_lightgbm_tuner/optimize.py b/optuna/integration/_lightgbm_tuner/optimize.py
--- a/optuna/integration/_lightgbm_tuner/optimize.py
+++ b/optuna/integration/_lightgbm_tuner/optimize.py
@@ -361,7 +361,7 @@ def __init__(
         feature_name: str = "auto",
         categorical_feature: str = "auto",
         early_stopping_rounds: Optional[int] = None,
-        verbose_eval: Optional[Union[bool, int]] = True,
+        verbose_eval: Optional[Union[bool, int, str]] = None,
         callbacks: Optional[List[Callable[..., Any]]] = None,
         time_budget: Optional[int] = None,
         sample_size: Optional[int] = None,
@@ -814,7 +814,7 @@ def __init__(
         categorical_feature: str = "auto",
         early_stopping_rounds: Optional[int] = None,
         evals_result: Optional[Dict[Any, Any]] = None,
-        verbose_eval: Optional[Union[bool, int]] = True,
+        verbose_eval: Optional[Union[bool, int, str]] = "warn",
         learning_rates: Optional[List[float]] = None,
         keep_training_booster: bool = False,
         callbacks: Optional[List[Callable[..., Any]]] = None,
@@ -993,7 +993,7 @@ def __init__(
         categorical_feature: str = "auto",
         early_stopping_rounds: Optional[int] = None,
         fpreproc: Optional[Callable[..., Any]] = None,
-        verbose_eval: Optional[Union[bool, int]] = True,
+        verbose_eval: Optional[Union[bool, int]] = None,
         show_stdv: bool = True,
         seed: int = 0,
         callbacks: Optional[List[Callable[..., Any]]] = None,


Test patch: