Repository: optuna/optuna

Issue:
Print `study.user_attr` by `optuna studies` command
### Motivation

Using the cli to query studies for their user_attr would be quite useful.  There doesn't seem to be a way to do this, though there does seem to be a way to set a user_attr from the cli

### Description

optuna studies could have a longform option, with format json/yaml/table; or a new option, optuna study --study-name could be added.

### Alternatives (optional)

_No response_

### Additional context (optional)

_No response_


Discussion:
Thank you for sharing the feature request!  You're right. The current `optuna studies` command doesn't show `study.user_attr`. The example is as follows:
 
```shell
STUDY_NAME=`optuna create-study --storage sqlite:///example.db`
optuna study optimize foo.py objective --n-trials=20 --storage sqlite:///example.db --study-name $STUDY_NAME
optuna study set-user-attr --key key --value value --study-name $STUDY_NAME --storage sqlite:///example.db

optuna studies --storage sqlite:///example.db -f yaml
- datetime_start: '2022-08-18 00:46:20'
  direction:
  - MINIMIZE
  n_trials: 20
  name: no-name-903cc7fb-34b5-45ff-b2f7-be1e0eea2dea
```


```python
# foo.py

def objective(trial):
    x = trial.suggest_float('x', -10, 10)
    return (x - 2) ** 2
```

If we implement this feature request, we add a few lines to https://github.com/optuna/optuna/blob/3247030717413b5097db10a26afca67322c91973/optuna/cli.py#L382-L387.


--- 

> optuna studies could have a longform option, with format json/yaml/table

I think the command already has the option as demonstrated above :)
ah, got it, yeah optuna studies shows a different help, which is why i missed it.

Do you think there should be an option, like --show-user-attrs?  or what would be a good option?  I'll take a stab at a pr with some further guidance.
> Do you think there should be an option, like `--show-user-attrs`? or what would be a good option? I'll take a stab at a pr with some further guidance.

I don't think we need to introduce a new option because similar command, `optuna trials`, shows the `user_attrs` by default. 

My concern might be that if we do it by default it will make the table view mostly unreadable.  Currently it's quite clean, at least for single objective trials.  An alternative might be --no-user-attrs
That concern is also applicable to `optuna trials`, so we might need to hear the other dev members' or community's opinions. Hence how about separating the concerns from the original motivation to reduce the complexity of forthcoming implementation; Firstly, we will implement the feature where `optuna studies` shows `user_attrs` by default as a PR. Secondly, introducing a new option as you proposed (more generally specify printed or hidden columns) after discussion. 
@blazespinnaker Let me confirm that you're working on this issue. If not, I'd like to add the label `contribution-welcome` to this issue.
That's fine.  I was also hoping for some more discussion.
Let me confirm the current target of this issue. We'll work on BOTH of the following changes; correct?

- Changing the output format of `optuna studies` so that `user_attrs` by default.
- Add new options to hide a part of output.

My concern is that they might cause a long-running discussion among developers, which is burdensome for a first-time contributor. Historically, the current output format has been carefully adjusted to make the output width reasonable.
@g-votte thank you for your comment. I thought it would be great to work on 

> Changing the output format of `optuna studies` so that `user_attrs` by default.

at least for consistency with `optuna trials` and the current width of the output of `optuna studies` is not too long compared with `optuna trials`.  So a first-time contributor would be able to work on adding this feature as a PR without discussion and we could create another issue to discuss the options. 

However, if we also care about the second one:

> Add new options to hide a part of output.

at the same time, it would require discussion among developers, so this issue is not for the first-time contributor.

@nzw0301 
Thanks for your clarification. This direction looks good to me.

> at least for consistency with optuna trials and the current width of the output of optuna studies is not too long compared with optuna trials.

So, let's target to make this change in this issue: `Changing the output format of optuna studies so that user_attrs by default.`

This issue has not seen any recent activity.
@blazespinnaker can you send a PR? As summarised in https://github.com/optuna/optuna/issues/3899#issuecomment-1243350188, the issue has no-discussion to send a PR.
@nzw0301 Sure, though I'm a bit unsure by the above.  Does this mean the PR should include a show user attr flag?  Without it, I believe workflow will be broken for most folks (myself included)
> Does this mean the PR should include a show user attr flag? 

No, it doesn't. Adding a flag is unnecessary for the PR. 

Thanks @blazespinnaker for looking into this issue!

I removed `ontribution-welcome` label just to keep the [CW list](https://github.com/optuna/optuna/issues?q=is%3Aissue+is%3Aopen+label%3Acontribution-welcome) with open, unstarted issues.
This issue has not seen any recent activity.
This issue was closed automatically because it had not seen any recent activity. If you want to discuss it, you can reopen it freely.
I will like to work on this one, implementing as default behaviour as described in [comment](https://github.com/optuna/optuna/issues/3899#issuecomment-1243350188)

Resolving patch:
diff --git a/optuna/cli.py b/optuna/cli.py
--- a/optuna/cli.py
+++ b/optuna/cli.py
@@ -384,8 +384,11 @@ def take_action(self, parsed_args: Namespace) -> None:
             record[("direction", "")] = tuple(d.name for d in s.directions)
             record[("n_trials", "")] = s.n_trials
             record[("datetime_start", "")] = start
+            record[("user_attrs", "")] = s.user_attrs
             records.append(record)
 
+        if any(r[("user_attrs", "")] != {} for r in records):
+            self._study_list_header.append(("user_attrs", ""))
         print(
             _format_output(
                 records, self._study_list_header, parsed_args.format, parsed_args.flatten


Test patch:
diff --git a/tests/test_cli.py b/tests/test_cli.py
--- a/tests/test_cli.py
+++ b/tests/test_cli.py
@@ -281,12 +281,6 @@ def test_studies_command(output_format: Optional[str]) -> None:
         # First study.
         study_1 = optuna.create_study(storage=storage)
 
-        # Second study.
-        study_2 = optuna.create_study(
-            storage=storage, study_name="study_2", directions=["minimize", "maximize"]
-        )
-        study_2.optimize(objective_func_multi_objective, n_trials=10)
-
         # Run command.
         command = ["optuna", "studies", "--storage", storage_url]
         if output_format is not None:
@@ -297,6 +291,26 @@ def test_studies_command(output_format: Optional[str]) -> None:
 
         expected_keys = ["name", "direction", "n_trials", "datetime_start"]
 
+        # Check user_attrs are not printed.
+        if output_format is None or output_format == "table":
+            assert list(studies[0].keys()) == expected_keys
+        else:
+            assert set(studies[0].keys()) == set(expected_keys)
+
+        # Add a second study.
+        study_2 = optuna.create_study(
+            storage=storage, study_name="study_2", directions=["minimize", "maximize"]
+        )
+        study_2.optimize(objective_func_multi_objective, n_trials=10)
+        study_2.set_user_attr("key_1", "value_1")
+        study_2.set_user_attr("key_2", "value_2")
+
+        # Run command again to include second study.
+        output = str(subprocess.check_output(command).decode().strip())
+        studies = _parse_output(output, output_format or "table")
+
+        expected_keys = ["name", "direction", "n_trials", "datetime_start", "user_attrs"]
+
         assert len(studies) == 2
         for study in studies:
             if output_format is None or output_format == "table":
@@ -304,23 +318,27 @@ def test_studies_command(output_format: Optional[str]) -> None:
             else:
                 assert set(study.keys()) == set(expected_keys)
 
-        # Check study_name, direction, and n_trials for the first study.
+        # Check study_name, direction, n_trials and user_attrs for the first study.
         assert studies[0]["name"] == study_1.study_name
         if output_format is None or output_format == "table":
             assert studies[0]["n_trials"] == "0"
             assert eval(studies[0]["direction"]) == ("MINIMIZE",)
+            assert eval(studies[0]["user_attrs"]) == {}
         else:
             assert studies[0]["n_trials"] == 0
             assert studies[0]["direction"] == ["MINIMIZE"]
+            assert studies[0]["user_attrs"] == {}
 
-        # Check study_name, direction, and n_trials for the second study.
+        # Check study_name, direction, n_trials and user_attrs for the second study.
         assert studies[1]["name"] == study_2.study_name
         if output_format is None or output_format == "table":
             assert studies[1]["n_trials"] == "10"
             assert eval(studies[1]["direction"]) == ("MINIMIZE", "MAXIMIZE")
+            assert eval(studies[1]["user_attrs"]) == {"key_1": "value_1", "key_2": "value_2"}
         else:
             assert studies[1]["n_trials"] == 10
             assert studies[1]["direction"] == ["MINIMIZE", "MAXIMIZE"]
+            assert studies[1]["user_attrs"] == {"key_1": "value_1", "key_2": "value_2"}
 
 
 @pytest.mark.skip_coverage
@@ -334,12 +352,6 @@ def test_studies_command_flatten(output_format: Optional[str]) -> None:
         # First study.
         study_1 = optuna.create_study(storage=storage)
 
-        # Second study.
-        study_2 = optuna.create_study(
-            storage=storage, study_name="study_2", directions=["minimize", "maximize"]
-        )
-        study_2.optimize(objective_func_multi_objective, n_trials=10)
-
         # Run command.
         command = ["optuna", "studies", "--storage", storage_url, "--flatten"]
         if output_format is not None:
@@ -348,6 +360,31 @@ def test_studies_command_flatten(output_format: Optional[str]) -> None:
         output = str(subprocess.check_output(command).decode().strip())
         studies = _parse_output(output, output_format or "table")
 
+        expected_keys_1 = [
+            "name",
+            "direction_0",
+            "n_trials",
+            "datetime_start",
+        ]
+
+        # Check user_attrs are not printed.
+        if output_format is None or output_format == "table":
+            assert list(studies[0].keys()) == expected_keys_1
+        else:
+            assert set(studies[0].keys()) == set(expected_keys_1)
+
+        # Add a second study.
+        study_2 = optuna.create_study(
+            storage=storage, study_name="study_2", directions=["minimize", "maximize"]
+        )
+        study_2.optimize(objective_func_multi_objective, n_trials=10)
+        study_2.set_user_attr("key_1", "value_1")
+        study_2.set_user_attr("key_2", "value_2")
+
+        # Run command again to include second study.
+        output = str(subprocess.check_output(command).decode().strip())
+        studies = _parse_output(output, output_format or "table")
+
         if output_format is None or output_format == "table":
             expected_keys_1 = expected_keys_2 = [
                 "name",
@@ -355,15 +392,17 @@ def test_studies_command_flatten(output_format: Optional[str]) -> None:
                 "direction_1",
                 "n_trials",
                 "datetime_start",
+                "user_attrs",
             ]
         else:
-            expected_keys_1 = ["name", "direction_0", "n_trials", "datetime_start"]
+            expected_keys_1 = ["name", "direction_0", "n_trials", "datetime_start", "user_attrs"]
             expected_keys_2 = [
                 "name",
                 "direction_0",
                 "direction_1",
                 "n_trials",
                 "datetime_start",
+                "user_attrs",
             ]
 
         assert len(studies) == 2
@@ -374,20 +413,24 @@ def test_studies_command_flatten(output_format: Optional[str]) -> None:
             assert set(studies[0].keys()) == set(expected_keys_1)
             assert set(studies[1].keys()) == set(expected_keys_2)
 
-        # Check study_name, direction, and n_trials for the first study.
+        # Check study_name, direction, n_trials and user_attrs for the first study.
         assert studies[0]["name"] == study_1.study_name
         if output_format is None or output_format == "table":
             assert studies[0]["n_trials"] == "0"
+            assert studies[0]["user_attrs"] == "{}"
         else:
             assert studies[0]["n_trials"] == 0
+            assert studies[0]["user_attrs"] == {}
         assert studies[0]["direction_0"] == "MINIMIZE"
 
-        # Check study_name, direction, and n_trials for the second study.
+        # Check study_name, direction, n_trials and user_attrs for the second study.
         assert studies[1]["name"] == study_2.study_name
         if output_format is None or output_format == "table":
             assert studies[1]["n_trials"] == "10"
+            assert studies[1]["user_attrs"] == "{'key_1': 'value_1', 'key_2': 'value_2'}"
         else:
             assert studies[1]["n_trials"] == 10
+            assert studies[1]["user_attrs"] == {"key_1": "value_1", "key_2": "value_2"}
         assert studies[1]["direction_0"] == "MINIMIZE"
         assert studies[1]["direction_1"] == "MAXIMIZE"