Repository: iterative/dvc

Issue:
status: support outputs as targets [qa]
## Bug Report

UPDATE: Jump to https://github.com/iterative/dvc/issues/4191#issuecomment-657106691

---

Looks like `dvc status` is recursive by default now (maybe always was) from some testing I just did:

```console
λ dvc status
foo.dvc:
        changed outs:
                modified:           foo
data\raw.dvc:
        changed outs:
                modified:           data\raw

λ dvc status -R data/
data\raw.dvc:
        changed outs:
                modified:           data\raw
```

1. So it's really just useful to limit the status to specific directories (not really about recursion).
2. Why not just support dirs AND FILES as `targets` of the command, like all other commands that take targets (I think)?
  > Relater to https://github.com/iterative/dvc.org/pull/1384#issuecomment-656878839
3. In fact this is already supported by `status`, but only in remote mode (`-r` or `-c` options), which is kind of confusing (and complicates the docs).

### Please provide information about your setup

```
DVC version: 1.1.2
Python version: 3.7.5
Platform: Windows-10-10.0.18362-SP0
Binary: True
Package: exe
Supported remotes: azure, gdrive, gs, hdfs, http, https, s3, ssh, oss
Cache: reflink - not supported, hardlink - supported, symlink - not supported
Filesystem type (cache directory): ('NTFS', 'C:\\')
Repo: dvc, git
Filesystem type (workspace): ('NTFS', 'C:\\')
```


Discussion:
@jorgeorpinel Non -c status just doesn't support granularity right now.
Closing in favor of https://github.com/iterative/dvc/issues/2180
To clarify:

> Why not just support dirs AND FILES as targets of the command, like all other commands that take targets (I think)?

It does support files as targets, but not files(or subdirs) in tracked directories (this is what -c/push/pull/etc support).
> It does support files as targets

@efiop so is this a bug? 👇 

```console
λ ls foo*
foo  foo.dvc
λ dvc status foo
ERROR: failed to obtain data status - 'dvc.yaml' does not exist.
```
@jorgeorpinel Yes, looks like a bug. Reopening. Thanks!
OK. Note on docs:

- ~~Move granularity note from **Remote comparisons** example to **Simple usage** when this is fixed.~~
Never mind. I'll make a separate **Specific targets** example now.
Another related question: `dvc repro` also has `targets` but only accepts stage and .dvc file names, right? (Otherwise, `-R` would also seem obsolete there.) If so, should it accept file/dir names (and support granularity)? Prob not.
@jorgeorpinel `-R` is not obsolete, it searches recursively for dvc files. But when you specify target explicitly, it finds 1 dvc file that it belongs too, so it is not the same.
OK thanks. So `dvc repro` also supports tracked files and thus granularity already? Will double check and include in #1384 then.

> What about run, (un)freeze, remove, unprotect, update, and metrics/plots show? Will ask in the appropriate issue...
@jorgeorpinel tracked files and granularity is not the same thing. When we were talking about push/pull we were talking about being able to pull specific file (or subdir) in a tracked dataset (`dvc add data_dir`). In case of repro we support addressing stages by outputs, which is a different thing.
Got it. So I think we need to clarify when file/dir targets support granularity and when they don't, after all. I'll add a note in all the sync-related ones (in #1384) both in the description and in **Specific target** examples.
> > It does support files as targets
> 
> @efiop so is this a bug? 👇
> 
> ```
> λ ls foo*
> foo  foo.dvc
> λ dvc status foo
> ERROR: failed to obtain data status - 'dvc.yaml' does not exist.
> ```

@jorgeorpinel , @efiop 
Supporting outputs as targets is a feature, not a bug I think?  

![image](https://user-images.githubusercontent.com/6745454/90505045-9336db80-e184-11ea-92c5-91a884122454.png)

Targets in `dvc status` are stages just the same as what in `dvc repo`. But the error message  `ERROR: failed to obtain data status - 'dvc.yaml' does not exist.`  is a bug. In a previous version (0.94.1), it used to be
![image](https://user-images.githubusercontent.com/6745454/90505533-5c14fa00-e185-11ea-89d8-1a95b015d370.png)

So the solution is either to support outputs as targets or to improve the message.
@karajan1001 Adjusted the labels, thanks! :slightly_smiling_face: 

Since 1.0 we've changed the defaults, hence why it looks for `dvc.yaml` first. `targets` are shared by `status` and `status -c`, hence the confusion.
@efiop 
In a local status mode, we use `Repo.collect` to collect stages 
https://github.com/iterative/dvc/blob/32b5b33329ef71586ae47b1b879bf90f79edab2f/dvc/repo/status.py#L28-L33

While in a cloud status mode we are using `Repo.collect_granular`

https://github.com/iterative/dvc/blob/32b5b33329ef71586ae47b1b879bf90f79edab2f/dvc/repo/__init__.py#L378-L382
https://github.com/iterative/dvc/blob/32b5b33329ef71586ae47b1b879bf90f79edab2f/dvc/repo/__init__.py#L340-L352
https://github.com/iterative/dvc/blob/32b5b33329ef71586ae47b1b879bf90f79edab2f/dvc/repo/status.py#L79-L89
https://github.com/iterative/dvc/blob/32b5b33329ef71586ae47b1b879bf90f79edab2f/dvc/repo/status.py#L40-L50

And if the one stage and one output have the same name, the stage would win. This will prevent the users from selecting those outputs which have the same name with stages. (Before version 1.0, stages name will always have a `.dvc` suffix which prevents this problem)

![image](https://user-images.githubusercontent.com/6745454/90585766-367d0480-e208-11ea-9404-f61f758ed474.png)

https://github.com/iterative/dvc/blob/32b5b33329ef71586ae47b1b879bf90f79edab2f/dvc/repo/__init__.py#L302-L304
Wow, had been considered before. `stage` is preferred to `output` 
@karajan1001, there's a workaround: `dvc status ./<filename>` for now. :slightly_smiling_face:

But, clearly, it's not implemented for `status`, only for `-c`.
And, we don't recommend to have a stage name same as outputs.

Resolving patch:
diff --git a/dvc/repo/status.py b/dvc/repo/status.py
--- a/dvc/repo/status.py
+++ b/dvc/repo/status.py
@@ -10,31 +10,32 @@
 logger = logging.getLogger(__name__)
 
 
-def _joint_status(stages):
+def _joint_status(pairs):
     status_info = {}
 
-    for stage in stages:
+    for stage, filter_info in pairs:
         if stage.frozen and not stage.is_repo_import:
             logger.warning(
                 "{} is frozen. Its dependencies are"
                 " not going to be shown in the status output.".format(stage)
             )
-
-        status_info.update(stage.status(check_updates=True))
+        if not filter_info:
+            status_info.update(stage.status(check_updates=True))
+        else:
+            for out in stage.filter_outs(filter_info):
+                status_info.update(out.status())
 
     return status_info
 
 
 def _local_status(self, targets=None, with_deps=False, recursive=False):
-    if targets:
-        stages = cat(
-            self.collect(t, with_deps=with_deps, recursive=recursive)
-            for t in targets
-        )
-    else:
-        stages = self.collect(None, with_deps=with_deps, recursive=recursive)
+    targets = targets or [None]
+    pairs = cat(
+        self.collect_granular(t, with_deps=with_deps, recursive=recursive)
+        for t in targets
+    )
 
-    return _joint_status(stages)
+    return _joint_status(pairs)
 
 
 def _cloud_status(
diff --git a/dvc/stage/__init__.py b/dvc/stage/__init__.py
--- a/dvc/stage/__init__.py
+++ b/dvc/stage/__init__.py
@@ -442,7 +442,7 @@ def run(self, dry=False, no_commit=False, force=False, run_cache=True):
             if not no_commit:
                 self.commit()
 
-    def _filter_outs(self, path_info):
+    def filter_outs(self, path_info):
         def _func(o):
             return path_info.isin_or_eq(o.path_info)
 
@@ -451,7 +451,7 @@ def _func(o):
     @rwlocked(write=["outs"])
     def checkout(self, **kwargs):
         stats = defaultdict(list)
-        for out in self._filter_outs(kwargs.get("filter_info")):
+        for out in self.filter_outs(kwargs.get("filter_info")):
             key, outs = self._checkout(out, **kwargs)
             if key:
                 stats[key].extend(outs)
@@ -526,14 +526,14 @@ def outs_cached(self):
     def get_all_files_number(self, filter_info=None):
         return sum(
             out.get_files_number(filter_info)
-            for out in self._filter_outs(filter_info)
+            for out in self.filter_outs(filter_info)
         )
 
     def get_used_cache(self, *args, **kwargs):
         from dvc.cache import NamedCache
 
         cache = NamedCache()
-        for out in self._filter_outs(kwargs.get("filter_info")):
+        for out in self.filter_outs(kwargs.get("filter_info")):
             cache.update(out.get_used_cache(*args, **kwargs))
 
         return cache


Test patch:
diff --git a/tests/func/test_status.py b/tests/func/test_status.py
--- a/tests/func/test_status.py
+++ b/tests/func/test_status.py
@@ -118,3 +118,22 @@ def test_status_recursive(tmp_dir, dvc):
             }
         ],
     }
+
+
+def test_status_outputs(tmp_dir, dvc):
+    tmp_dir.dvc_gen({"foo": "foo", "bar": "bar"})
+    dvc.run(
+        outs=["alice", "bob"],
+        deps=["foo", "bar"],
+        cmd="echo alice>alice && echo bob>bob",
+        name="alice_bob",
+    )
+    tmp_dir.gen({"alice": "new alice", "bob": "new bob"})
+
+    assert dvc.status(targets=["alice_bob"]) == {
+        "alice_bob": [
+            {"changed outs": {"alice": "modified", "bob": "modified"}}
+        ]
+    }
+
+    assert dvc.status(targets=["alice"]) == {"alice": "modified"}