Repository: optuna/optuna
Pull request: Added `user_attrs` to print by optuna studies in `cli.py`
<!-- Thank you for creating a pull request! In general, we merge your pull request after it gets two or more approvals. To proceed to the review process by the maintainers, please make sure that the PR meets the following conditions: (1) it passes all CI checks, and (2) it is neither in the draft nor WIP state. If you wish to discuss the PR in the draft state or need any other help, please mention the Optuna development team in the PR. -->

## Motivation
<!-- Describe your motivation why you will submit this PR. This is useful for reviewers to understand the context of PR. -->
Resolve #3899 

## Description of the changes
<!-- Describe the changes in this PR. -->


Problem statement:
Print `study.user_attr` by `optuna studies` command
### Motivation

Using the cli to query studies for their user_attr would be quite useful.  There doesn't seem to be a way to do this, though there does seem to be a way to set a user_attr from the cli

### Description

optuna studies could have a longform option, with format json/yaml/table; or a new option, optuna study --study-name could be added.

### Alternatives (optional)

_No response_

### Additional context (optional)

_No response_

Hints from issue discussion:
Thank you for sharing the feature request!  You're right. The current `optuna studies` command doesn't show `study.user_attr`. The example is as follows:
 
```shell
STUDY_NAME=`optuna create-study --storage sqlite:///example.db`
optuna study optimize foo.py objective --n-trials=20 --storage sqlite:///example.db --study-name $STUDY_NAME
optuna study set-user-attr --key key --value value --study-name $STUDY_NAME --storage sqlite:///example.db

optuna studies --storage sqlite:///example.db -f yaml
- datetime_start: '2022-08-18 00:46:20'
  direction:
  - MINIMIZE
  n_trials: 20
  name: no-name-903cc7fb-34b5-45ff-b2f7-be1e0eea2dea
```


```python
# foo.py

def objective(trial):
    x = trial.suggest_float('x', -10, 10)
    return (x - 2) ** 2
```

If we implement this feature request, we add a few lines to https://github.com/optuna/optuna/blob/3247030717413b5097db10a26afca67322c91973/optuna/cli.py#L382-L387.


--- 

> optuna studies could have a longform option, with format json/yaml/table

I think the command already has the option as demonstrated above :)
ah, got it, yeah optuna studies shows a different help, which is why i missed it.

Do you think there should be an option, like --show-user-attrs?  or what would be a good option?  I'll take a stab at a pr with some further guidance.
> Do you think there should be an option, like `--show-user-attrs`? or what would be a good option? I'll take a stab at a pr with some further guidance.

I don't think we need to introduce a new option because similar command, `optuna trials`, shows the `user_attrs` by default. 

My concern might be that if we do it by default it will make the table view mostly unreadable.  Currently it's quite clean, at least for single objective trials.  An alternative might be --no-user-attrs
That concern is also applicable to `optuna trials`, so we might need to hear the other dev members' or community's opinions. Hence how about separating the concerns from the original motivation to reduce the complexity of forthcoming implementation; Firstly, we will implement the feature where `optuna studies` shows `user_attrs` by default as a PR. Secondly, introducing a new option as you proposed (more generally specify printed or hidden columns) after discussion. 
@blazespinnaker Let me confirm that you're working on this issue. If not, I'd like to add the label `contribution-welcome` to this issue.
That's fine.  I was also hoping for some more discussion.
Let me confirm the current target of this issue. We'll work on BOTH of the following changes; correct?

- Changing the output format of `optuna studies` so that `user_attrs` by default.
- Add new options to hide a part of output.

My concern is that they might cause a long-running discussion among developers, which is burdensome for a first-time contributor. Historically, the current output format has been carefully adjusted to make the output width reasonable.
@g-votte thank you for your comment. I thought it would be great to work on 

> Changing the output format of `optuna studies` so that `user_attrs` by default.

at least for consistency with `optuna trials` and the current width of the output of `optuna studies` is not too long compared with `optuna trials`.  So a first-time contributor would be able to work on adding this feature as a PR without discussion and we could create another issue to discuss the options. 

However, if we also care about the second one:

> Add new options to hide a part of output.

at the same time, it would require discussion among developers, so this issue is not for the first-time contributor.

@nzw0301 
Thanks for your clarification. This direction looks good to me.

> at least for consistency with optuna trials and the current width of the output of optuna studies is not too long compared with optuna trials.

So, let's target to make this change in this issue: `Changing the output format of optuna studies so that user_attrs by default.`

Commit message: Merge branch 'optuna:master' into add_print

Patch to review:
diff --git a/optuna/cli.py b/optuna/cli.py
index 8350eb68df4..f95869011c7 100644
--- a/optuna/cli.py
+++ b/optuna/cli.py
@@ -343,6 +343,7 @@ class _Studies(_BaseCommand):
 
     _study_list_header = [
         ("name", ""),
+        ("user_attrs", ""),
         ("direction", ""),
         ("n_trials", ""),
         ("datetime_start", ""),
@@ -381,6 +382,7 @@ def take_action(self, parsed_args: Namespace) -> None:
             )
             record: Dict[Tuple[str, str], Any] = {}
             record[("name", "")] = s.study_name
+            record[("user_attrs", "")] = s.user_attrs
             record[("direction", "")] = tuple(d.name for d in s.directions)
             record[("n_trials", "")] = s.n_trials
             record[("datetime_start", "")] = start
diff --git a/tests/test_cli.py b/tests/test_cli.py
index 62731c84d66..39451dbcd41 100644
--- a/tests/test_cli.py
+++ b/tests/test_cli.py
@@ -295,7 +295,7 @@ def test_studies_command(output_format: Optional[str]) -> None:
         output = str(subprocess.check_output(command).decode().strip())
         studies = _parse_output(output, output_format or "table")
 
-        expected_keys = ["name", "direction", "n_trials", "datetime_start"]
+        expected_keys = ["name", "user_attrs", "direction", "n_trials", "datetime_start"]
 
         assert len(studies) == 2
         for study in studies:
@@ -351,15 +351,17 @@ def test_studies_command_flatten(output_format: Optional[str]) -> None:
         if output_format is None or output_format == "table":
             expected_keys_1 = expected_keys_2 = [
                 "name",
+                "user_attrs",
                 "direction_0",
                 "direction_1",
                 "n_trials",
                 "datetime_start",
             ]
         else:
-            expected_keys_1 = ["name", "direction_0", "n_trials", "datetime_start"]
+            expected_keys_1 = ["name", "user_attrs", "direction_0", "n_trials", "datetime_start"]
             expected_keys_2 = [
                 "name",
+                "user_attrs",
                 "direction_0",
                 "direction_1",
                 "n_trials",


Code review comments:
diff_hunk: @@ -343,6 +343,7 @@ class _Studies(_BaseCommand):
 
     _study_list_header = [
         ("name", ""),
+        ("user_attrs", ""),
path: optuna/cli.py
text: @user3:
According to https://github.com/optuna/optuna/issues/3899#issuecomment-1232431835, the purpose of this PR is to make `optuna studies` and `optuna trial s` consistent in terms of `user_attrs`.
On the other hand, the current implementation is inconsistent when the `user_attrs` is empty.

*optuna trial*
`user_attrs` column is hidden when no trials have `user_attrs`.

```python
import optuna

study = optuna.create_study(
    storage="sqlite:///4129.db",
    study_name="4129",
)
```

```console
% study.optimize(objective, n_trials=2)
+--------+-------+---------------------+---------------------+----------------+----------+
| number | value | datetime_start      | datetime_complete   | duration       | state    |
+--------+-------+---------------------+---------------------+----------------+----------+
|      0 |   1.0 | 2022-11-06 14:25:54 | 2022-11-06 14:25:54 | 0:00:00.003276 | COMPLETE |
+--------+-------+---------------------+---------------------+----------------+----------+
```

*optuna studies*
`user_attrs` is always shown.

```console
% optuna create-study --storage sqlite:///4129-empty.db --study-name empty 
%  optuna studies --storage sqlite:///4129-empty.db                                                                                                                   [~/pfn/code/optuna]
+-------+------------+---------------+----------+----------------+
| name  | user_attrs | direction     | n_trials | datetime_start |
+-------+------------+---------------+----------+----------------+
| empty | {}         | ('MINIMIZE',) |        0 |                |
+-------+------------+---------------+----------+----------------+
```

Do we align the behavior when the `user_attrs` is empty?
@user1-votte @user2 

@user4:
As @user3 mentions, the goal is to make `optuna studies` and `optuna trials` consistent. It is the case even for the edge case where `user_attrs` is empty.

@author:
True I will fix this