Repository: ray-project/ray

Issue:
[Tune] status tables during training aren't showing values for tunable, nested config parameters
### What is the problem?

Ray 0.8.6. Python 3.7.7.

Running this script (actually run with ipython):

```python
from ray import tune
ray.init()
tune.run(
    "PPO",
    stop={"episode_reward_mean": 400},
    config={
        "env": "CartPole-v1",
        "num_gpus": 0,
        "num_workers": 1,
        # "lr": tune.grid_search([0.01, 0.001, 0.0001]),
        "model": {
            'fcnet_hiddens': [tune.grid_search([20, 40, 60, 80]), tune.grid_search([20, 40, 60, 80])]
        },
        "eager": False,
    },
)
```

It appears that Tune easily drives the 16 different combinations for the two elements of `fcnet_hidden` (number of weights for two NN layers). All combinations easily reach the reward of 400, but some do so with lower iterations and/or elapsed times, which is what I want to determine.

The bug is in the status table printed. It has two columns for the tuning parameters, `model/fcnet_hiddens/0` and `model/fcnet_hiddens/1`, but they are empty. I can determine the actual values by looking at the `experiment_tag`, e.g., ` experiment_tag: 8_fcnet_hiddens_0=20,fcnet_hiddens_1=60`



Discussion:
Taking a look at this this week. I can reproduce the issue locally.

```
== Status ==
Memory usage on this node: 10.5/16.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 8/8 CPUs, 0/0 GPUs, 0.0/4.49 GiB heap, 0.0/1.51 GiB objects
Number of trials: 16 (4 RUNNING, 12 TERMINATED)
+-----------------------------+------------+--------------------+-------------------------+-------------------------+--------+------------------+--------+----------+
| Trial name                  | status     | loc                | model/fcnet_hiddens/0   | model/fcnet_hiddens/1   |   iter |   total time (s) |     ts |   reward |
|-----------------------------+------------+--------------------+-------------------------+-------------------------+--------+------------------+--------+----------|
| PPO_CartPole-v1_e7acd_00000 | TERMINATED |                    |                         |                         |     18 |          220.755 |  72000 |   406.54 |
| PPO_CartPole-v1_e7acd_00001 | TERMINATED |                    |                         |                         |     31 |          371.998 | 124000 |   403.33 |
| PPO_CartPole-v1_e7acd_00002 | TERMINATED |                    |                         |                         |     18 |          220.732 |  72000 |   406.52 |
| PPO_CartPole-v1_e7acd_00003 | TERMINATED |                    |                         |                         |     15 |          189.082 |  60000 |   409.03 |
| PPO_CartPole-v1_e7acd_00004 | TERMINATED |                    |                         |                         |     19 |          233.262 |  76000 |   410.45 |
| PPO_CartPole-v1_e7acd_00005 | TERMINATED |                    |                         |                         |     22 |          260.052 |  88000 |   412.14 |
| PPO_CartPole-v1_e7acd_00006 | TERMINATED |                    |                         |                         |     30 |          354.216 | 120000 |   401.67 |
| PPO_CartPole-v1_e7acd_00007 | TERMINATED |                    |                         |                         |     18 |          211.58  |  72000 |   403.37 |
| PPO_CartPole-v1_e7acd_00008 | TERMINATED |                    |                         |                         |     19 |          222.288 |  76000 |   416.59 |
| PPO_CartPole-v1_e7acd_00009 | TERMINATED |                    |                         |                         |     19 |          218.809 |  76000 |   406.62 |
| PPO_CartPole-v1_e7acd_00010 | TERMINATED |                    |                         |                         |     16 |          531.601 |  64000 |   400.38 |
| PPO_CartPole-v1_e7acd_00011 | TERMINATED |                    |                         |                         |     21 |         4509.18  |  84000 |   415.27 |
| PPO_CartPole-v1_e7acd_00012 | RUNNING    | 192.168.2.66:49528 |                         |                         |     16 |         4449.35  |  64000 |   383.28 |
| PPO_CartPole-v1_e7acd_00013 | RUNNING    | 192.168.2.66:49553 |                         |                         |     12 |         4406.25  |  48000 |   292.06 |
| PPO_CartPole-v1_e7acd_00014 | RUNNING    | 192.168.2.66:49578 |                         |                         |      5 |         3502.22  |  20000 |   110.89 |
| PPO_CartPole-v1_e7acd_00015 | RUNNING    |                    |                         |                         |        |                  |        |          |
+-----------------------------+------------+--------------------+-------------------------+-------------------------+--------+------------------+--------+----------+
```
the trial config contains the `model/fcnet_hiddens` key e.g. `{'env': 'CartPole-v1', 'num_gpus': 0, 'num_workers': 1, 'eager': False, 'model/fcnet_hiddens': [20, 20]}` whereas `trial.evaluated_params` contains the flattened keys e.g. `{'model/fcnet_hiddens/0': 20, 'model/fcnet_hiddens/1': 20}` instead. 

`progress_reporter` uses `trial.evaluated_params` to generate the header and looks into `trial.config` for the config values


Resolving patch:
diff --git a/python/ray/tune/progress_reporter.py b/python/ray/tune/progress_reporter.py
--- a/python/ray/tune/progress_reporter.py
+++ b/python/ray/tune/progress_reporter.py
@@ -5,7 +5,7 @@
 
 from ray.tune.result import (EPISODE_REWARD_MEAN, MEAN_ACCURACY, MEAN_LOSS,
                              TRAINING_ITERATION, TIME_TOTAL_S, TIMESTEPS_TOTAL)
-from ray.tune.utils import flatten_dict
+from ray.tune.utils import unflattened_lookup
 
 try:
     from collections.abc import Mapping
@@ -466,9 +466,9 @@ def _get_trial_info(trial, parameters, metrics):
         parameters (list[str]): Names of trial parameters to include.
         metrics (list[str]): Names of metrics to include.
     """
-    result = flatten_dict(trial.last_result)
-    config = flatten_dict(trial.config)
+    result = trial.last_result
+    config = trial.config
     trial_info = [str(trial), trial.status, str(trial.location)]
-    trial_info += [config.get(param) for param in parameters]
-    trial_info += [result.get(metric) for metric in metrics]
+    trial_info += [unflattened_lookup(param, config) for param in parameters]
+    trial_info += [unflattened_lookup(metric, result) for metric in metrics]
     return trial_info
diff --git a/python/ray/tune/utils/__init__.py b/python/ray/tune/utils/__init__.py
--- a/python/ray/tune/utils/__init__.py
+++ b/python/ray/tune/utils/__init__.py
@@ -1,6 +1,6 @@
 from ray.tune.utils.util import deep_update, flatten_dict, get_pinned_object, \
-    merge_dicts, pin_in_object_store, UtilMonitor, validate_save_restore, \
-    warn_if_slow
+    merge_dicts, pin_in_object_store, unflattened_lookup, UtilMonitor, \
+    validate_save_restore, warn_if_slow
 
 __all__ = [
     "deep_update",
@@ -8,6 +8,7 @@
     "get_pinned_object",
     "merge_dicts",
     "pin_in_object_store",
+    "unflattened_lookup",
     "UtilMonitor",
     "validate_save_restore",
     "warn_if_slow",
diff --git a/python/ray/tune/utils/util.py b/python/ray/tune/utils/util.py
--- a/python/ray/tune/utils/util.py
+++ b/python/ray/tune/utils/util.py
@@ -2,7 +2,7 @@
 import logging
 import threading
 import time
-from collections import defaultdict
+from collections import defaultdict, deque, Mapping, Sequence
 from threading import Thread
 
 import numpy as np
@@ -216,6 +216,27 @@ def flatten_dict(dt, delimiter="/"):
     return dt
 
 
+def unflattened_lookup(flat_key, lookup, delimiter="/", default=None):
+    """
+    Unflatten `flat_key` and iteratively look up in `lookup`. E.g.
+    `flat_key="a/0/b"` will try to return `lookup["a"][0]["b"]`.
+    """
+    keys = deque(flat_key.split(delimiter))
+    base = lookup
+    while keys:
+        key = keys.popleft()
+        try:
+            if isinstance(base, Mapping):
+                base = base[key]
+            elif isinstance(base, Sequence):
+                base = base[int(key)]
+            else:
+                raise KeyError()
+        except KeyError:
+            return default
+    return base
+
+
 def _to_pinnable(obj):
     """Converts obj to a form that can be pinned in object store memory.
 


Test patch:
diff --git a/python/ray/tune/tests/test_progress_reporter.py b/python/ray/tune/tests/test_progress_reporter.py
--- a/python/ray/tune/tests/test_progress_reporter.py
+++ b/python/ray/tune/tests/test_progress_reporter.py
@@ -22,15 +22,15 @@
 
 EXPECTED_RESULT_2 = """Result logdir: /foo
 Number of trials: 5 (1 PENDING, 3 RUNNING, 1 TERMINATED)
-+--------------+------------+-------+-----+-----+
-|   Trial name | status     | loc   |   a |   b |
-|--------------+------------+-------+-----+-----|
-|        00000 | TERMINATED | here  |   0 |   0 |
-|        00001 | PENDING    | here  |   1 |   2 |
-|        00002 | RUNNING    | here  |   2 |   4 |
-|        00003 | RUNNING    | here  |   3 |   6 |
-|        00004 | RUNNING    | here  |   4 |   8 |
-+--------------+------------+-------+-----+-----+"""
++--------------+------------+-------+-----+-----+---------+---------+
+|   Trial name | status     | loc   |   a |   b |   n/k/0 |   n/k/1 |
+|--------------+------------+-------+-----+-----+---------+---------|
+|        00000 | TERMINATED | here  |   0 |   0 |       0 |       0 |
+|        00001 | PENDING    | here  |   1 |   2 |       1 |       2 |
+|        00002 | RUNNING    | here  |   2 |   4 |       2 |       4 |
+|        00003 | RUNNING    | here  |   3 |   6 |       3 |       6 |
+|        00004 | RUNNING    | here  |   4 |   8 |       4 |       8 |
++--------------+------------+-------+-----+-----+---------+---------+"""
 
 EXPECTED_RESULT_3 = """Result logdir: /foo
 Number of trials: 5 (1 PENDING, 3 RUNNING, 1 TERMINATED)
@@ -246,21 +246,29 @@ def testProgressStr(self):
             t.trial_id = "%05d" % i
             t.local_dir = "/foo"
             t.location = "here"
-            t.config = {"a": i, "b": i * 2}
-            t.evaluated_params = t.config
+            t.config = {"a": i, "b": i * 2, "n": {"k": [i, 2 * i]}}
+            t.evaluated_params = {
+                "a": i,
+                "b": i * 2,
+                "n/k/0": i,
+                "n/k/1": 2 * i
+            }
             t.last_result = {
                 "config": {
                     "a": i,
-                    "b": i * 2
+                    "b": i * 2,
+                    "n": {
+                        "k": [i, 2 * i]
+                    }
                 },
                 "metric_1": i / 2,
                 "metric_2": i / 4
             }
             t.__str__ = lambda self: self.trial_id
             trials.append(t)
-        # One metric, all parameters
+        # One metric, two parameters
         prog1 = trial_progress_str(
-            trials, ["metric_1"], None, fmt="psql", max_rows=3)
+            trials, ["metric_1"], ["a", "b"], fmt="psql", max_rows=3)
         print(prog1)
         assert prog1 == EXPECTED_RESULT_1