Repository: iterative/dvc

Issue:
DVC repro is reproducing unchanged stages
Hey guys, I noticed that DVC repro reproduces unnecessary stages (version 0.9.7 from pip): 

```bash
# Initialize GIT and DVC repository
git init
dvc init
git add .dvc

# Create example folder
mkdir example
cd example

# Create hello.txt
dvc run -o hello.txt 'yes "Hello" | head -10 > hello.txt'
## Using 'hello.txt.dvc' as a stage file
## Reproducing 'hello.txt.dvc':
## 	yes "Hello" | head -10 > hello.txt

# Create world.txt
dvc run -o world.txt 'yes "World" | head -10 > world.txt'
## Using 'world.txt.dvc' as a stage file
## Reproducing 'world.txt.dvc':
## 	yes "World" | head -10 > world.txt

# Merge into hello_world.txt
dvc run -d hello.txt -d world.txt -o hello_world.txt 'paste hello.txt world.txt > hello_world.txt'
## Using 'hello_world.txt.dvc' as a stage file
## Reproducing 'hello_world.txt.dvc':
## 	paste hello.txt world.txt > hello_world.txt

dvc status
## (No changes)

dvc repro hello_world.txt.dvc
## Reproducing 'hello.txt.dvc':
## 	yes "Hello" | head -10 > hello.txt
## Reproducing 'world.txt.dvc':
## 	yes "World" | head -10 > world.txt
```

Is this expected behavior?


Discussion:
Hi @prihoda !

Yes, it is an expected behaviour. hello.dvc and world.dvc have no dependencies but have a command to execute, thus dvc always considers them as stages that are changed. This behavior was introduced recently to allow creating so-called `callback` stages, that can check if anything has changed that is outside of dvc's scope. Note how in your example dvc only reproduced `hello.dvc` and `world.dvc` but not `hello_world.dvc`, this is because dvc detected that dependencies of `hello_world.dvc` didn't change after reproducing `hello.dvc` and `world.dvc` and thus there was no reason to reproduce `hello_world.dvc`. To help with that we should really return `dvc lock` feature back(https://github.com/iterative/dvc/issues/523 ) so that you could lock such stages from reproducing, while still retaining commands in your dvc files, so that you know where your files came from. For now, there is a workaround that you could use: just add a dummy file and specify it as a dependency in your dvc file. I.e.:
```
$ echo dummy > dummy.txt
$ git add dummy.txt # No need to add it with dvc, but you can do that if you want
$ dvc run -d dummy.txt -o hello.txt 'yes "Hello" | head -10 > hello.txt'
$ dvc run -d dummy.txt -o world.txt 'yes "World" | head -10 > world.txt'
```
This way as long as `dummy` doesn't change, dvc will consider `hello.txt.dvc` and `world.txt.dvc` as not-changed as well.
Oh I see. That is a useful feature, just not something I would personally expect. It would be nice to display these callback stages in `dvc status` so that when I want to reproduce the `hello_world.dvc` file, I know that more stages will be reproduced - as we discussed in #667.
Hm, you are right, `dvc status` should reflect that. I'll prepare a fix soon. Thank you!

Resolving patch:
diff --git a/dvc/stage.py b/dvc/stage.py
--- a/dvc/stage.py
+++ b/dvc/stage.py
@@ -91,10 +91,14 @@ def changed_md5(self):
         self.project.logger.debug(msg.format(self.relpath, self.md5, md5))
         return True
 
+    @property
+    def is_callback(self):
+        return not self.is_data_source and len(self.deps) == 0
+
     def changed(self):
         ret = False
 
-        if not self.is_data_source and len(self.deps) == 0:
+        if self.is_callback:
             ret = True
 
         for entry in itertools.chain(self.outs, self.deps):
@@ -268,7 +272,7 @@ def status(self):
         ret.update(self._status(self.deps, 'deps'))
         ret.update(self._status(self.outs, 'outs'))
 
-        if ret or self.changed_md5():
+        if ret or self.changed_md5() or self.is_callback:
             return {self.relpath: ret}
 
         return {}


Test patch: