Repository: iterative/dvc

Issue:
DVC metrics show doesn't find metrics files
## Bug Report

After marking some outputs as metrics using the `-M` option to `dvc run`, `dvc metrics show` tells me there are no metrics files. 

### Please provide information about your setup

**Output of `dvc version`:**



```console
$ dvc version
DVC version: 1.4.0 (pip)
---------------------------------
Platform: Python 3.6.9 on Linux-4.15.0-111-generic-x86_64-with-Ubuntu-18.04-bionic
Supports: http, https, s3, ssh
Cache types: hardlink, symlink
Repo: dvc, git
```
as well as

```console
$dvc version
DVC version: 1.4.0 (pip)
---------------------------------
Platform: Python 3.7.7 on Linux-4.19.132-1.pvops.qubes.x86_64-x86_64-with-fedora-30-Thirty
Supports: http, https, s3, ssh
Workspace directory: ext4 on /dev/xvdb
Repo: dvc, git
```

**Additional Information (if any):**
Steps to reproduce:
```console
$ git clone --depth 1 --branch metrics-show-bug https://github.com/e3bo/random-walks.git
$ cd random-walks
$ dvc metrics show --verbose
2020-08-07 14:29:47,439 DEBUG: fetched: [(3,)]                                  
2020-08-07 14:29:47,459 DEBUG: fetched: [(0,)]
2020-08-07 14:29:47,466 ERROR: failed to show metrics - no metric files in this repository. Use `-m/-M` options for `dvc run` to mark stage outputs as  metrics.
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/venv/lib/python3.7/site-packages/dvc/command/metrics.py", line 49, in run
    recursive=self.args.recursive,
  File "/home/user/venv/lib/python3.7/site-packages/dvc/repo/metrics/__init__.py", line 8, in show
    return show(self.repo, *args, **kwargs)
  File "/home/user/venv/lib/python3.7/site-packages/dvc/repo/__init__.py", line 34, in wrapper
    ret = f(repo, *args, **kwargs)
  File "/home/user/venv/lib/python3.7/site-packages/dvc/repo/metrics/show.py", line 105, in show
    raise NoMetricsError()
dvc.exceptions.NoMetricsError: no metric files in this repository. Use `-m/-M` options for `dvc run` to mark stage outputs as  metrics.
------------------------------------------------------------

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2020-08-07 14:29:47,468 DEBUG: Analytics is enabled.
2020-08-07 14:29:47,522 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmps0gegceh']'
2020-08-07 14:29:47,524 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmps0gegceh']
```

The `dvc.yaml` has the paths to the metrics files, which are present in the repository. It was generated using a script similar to `forecast.sh`. I asked about this problem on discord chat and ivan suggested I open a bug report here.


Discussion:
Ok so metrics in this particular repo are structured as list:
`{"wall":[67.889]}`
And this is the reason why don't get any results, as we are expecting singular values.

I have to admit that this is unfortunate, as the error message is unclear what actually happened.
The main problem lies in [this line](https://github.com/iterative/dvc/blob/dea1950ba022ee0a1bfdde357f6a8435d827eb5e/dvc/repo/metrics/show.py#L44) 
AFAIK it is done that way to be able to show as many metrics as possible and skip the ones wrongly structured. 

I think we should print some warnings that the unknown metric type is present and we are skipping it.
@shcheklein @e3bo What do you think?
Thanks, @pared for this research! It's helpful.

I guess for the `metrics show` we should be able to support arrays as well as scalar types. I don't see any reasons against it (except may be some code simplification?). 

Error message is indeed misleading and hint is wrong in this situation. It can be improved to at least include both - mention of -m/-M and supported metrics formats. 

I would not print warnings though (they all look very ugly in DVC, and we have one per every wrong metric that will be disastrous for us I think). We can print debug messages though, so that we can see what's happening in the `-v` mode.
Thanks, @pared and @shcheklein ! I think the mention of supported metrics formats in the message would have helped me self-correct. I have no immediate need for arrays.

Resolving patch:
diff --git a/dvc/exceptions.py b/dvc/exceptions.py
--- a/dvc/exceptions.py
+++ b/dvc/exceptions.py
@@ -171,11 +171,7 @@ def __init__(self, paths):
 
 
 class NoMetricsError(DvcException):
-    def __init__(self):
-        super().__init__(
-            "no metric files in this repository. Use `-m/-M` options for "
-            "`dvc run` to mark stage outputs as  metrics."
-        )
+    pass
 
 
 class NoPlotsError(DvcException):
diff --git a/dvc/repo/metrics/show.py b/dvc/repo/metrics/show.py
--- a/dvc/repo/metrics/show.py
+++ b/dvc/repo/metrics/show.py
@@ -36,7 +36,7 @@ def _filter(path_info):
     return list(filter(_filter, metrics))
 
 
-def _extract_metrics(metrics):
+def _extract_metrics(metrics, path, rev):
     if isinstance(metrics, (int, float)):
         return metrics
 
@@ -45,9 +45,18 @@ def _extract_metrics(metrics):
 
     ret = {}
     for key, val in metrics.items():
-        m = _extract_metrics(val)
+        m = _extract_metrics(val, path, rev)
         if m:
             ret[key] = m
+        else:
+            logger.debug(
+                "Could not parse '%s' metric from '%s' at '%s' "
+                "due to its unsupported type: '%s'",
+                key,
+                path,
+                rev,
+                type(val).__name__,
+            )
 
     return ret
 
@@ -70,7 +79,7 @@ def _read_metrics(repo, metrics, rev):
             )
             continue
 
-        val = _extract_metrics(val)
+        val = _extract_metrics(val, metric, rev)
         if val:
             res[str(metric)] = val
 
@@ -88,6 +97,7 @@ def show(
     all_commits=False,
 ):
     res = {}
+    metrics_found = False
 
     for rev in repo.brancher(
         revs=revs,
@@ -96,13 +106,27 @@ def show(
         all_commits=all_commits,
     ):
         metrics = _collect_metrics(repo, targets, recursive)
+
+        if not metrics_found and metrics:
+            metrics_found = True
+
         vals = _read_metrics(repo, metrics, rev)
 
         if vals:
             res[rev] = vals
 
     if not res:
-        raise NoMetricsError()
+        if metrics_found:
+            msg = (
+                "Could not parse metric files. Use `-v` option to see more "
+                "details."
+            )
+        else:
+            msg = (
+                "no metric files in this repository. Use `-m/-M` options for "
+                "`dvc run` to mark stage outputs as  metrics."
+            )
+        raise NoMetricsError(msg)
 
     # Hide workspace metrics if they are the same as in the active branch
     try:


Test patch: