Repository: iterative/dvc

Issue:
Virtualenv on macOS not working (fish)
**Please provide information about your setup**
* DVC version: 0.20.1
* Platform: macOS
* method of installation: pip

It looks like the commands run by dvc are ignoring currently used virtualenv 
![screenshot 2018-11-08 at 13 48 32](https://user-images.githubusercontent.com/6491400/48199798-82c1c200-e35d-11e8-8f5c-a0522805f596.png)

Steps to reproduce the issue:
```
$ mkdir dvc_test && cd dvc_test
$ git init
$ virtualenv -p python3.6 .env
$ source .env/bin/activate
$ pip install dvc
$ dvc init
$ which python
$ dvc run which python
```
The output of two last commands is different, which means, that python... comannd run by DVC is not taking into acount activated virtualenv. I've tested with virtualenv as well as with python venv.

I've done the same steps on linux - it works fine, so it's seems that the issue apears only on macOS


Discussion:
Hi @k-lyda !

I will try to reproduce this later today. Thank you for the feedback!
Hi @k-lyda , was not able to reproduce it on my MacOS machine. Would you mind to share `echo $SHELL`, is it the same shell that you are using? And `echo $PATH` as well. Can you also check your shell config (`cat ~/.bash_profile` in my case) - does it override PATH in some way that is related to anaconda?

We'll try anaconda a little bit later today.

Hi @shcheklein, indeed it looks like an issue is with adding anaconda inside shell profile script. I will do some more testing to check exactly what is causing the issue
Thanks, @k-lyda. I'll reopen it because I think it's important design question - the way we execute commands with DVC. It seems that ideally behavior should be the same in both cases, does not matter if you run it via DVC or directly from the command line. May be we should use some shell options to prevent it from re-reading profiles/config. What do you think, @efiop ?
@shcheklein You are right about it being an env vars overwriting issue by bashrc/zshrc/etc when launching a new instance for command execution inside dvc. I will look into it shortly.
@shcheklein, @efiop, as you mentioned, in my case the issue occurs only when I have in my fish config (I use fish shell), line `export PATH=<anaconda path>:$PATH`. When I comment out this line, DVC works properly.

Best Regards,
Konrad 
@k-lyda Oh, no wonder I can't reproduce, I should've asked right away about  the shell you were using, but I didn't think it could be anything special :slightly_smiling_face:  . Unlike other shells like bash/zsh, which don't parse configs when you invoke them like `bash script.sh` or `bash -c 'mycommand'`,
fish actually is a special one and reads your configs every time, which is where your issue comes from. This happens because when you run `dvc run mycommand`, dvc looks at your SHELL env variable and uses it to run your command with your current env, which gets overwritten by fish. And there is currently no option to prevent it from reading those configs :slightly_frowning_face: . We could introduce a special hack for it, that would save environment to some temporary file and then make fish source it before running your command. Effectively something like:
```
$ printenv &> /tmp/tmpenv.sh
$ fish -c 'source /tmp/tmpenv.sh && yourcommand'
$ rm /tmp/tmpenv.sh
```

To be more exact, we would need to modify run() method https://github.com/iterative/dvc/blob/master/dvc/stage.py#L419. Would you like to contribute a patch? :slightly_smiling_face: 

Thanks,
Ruslan
Ok, tried to create a patch for this, but ran into problems when sourcing saved env in fish, since it is really angry about overwriting some read-only vars and stuff like that, overcoming which makes the solution even more hackish, at which point I think it is no longer worth it to mess with, since more elegant workarounds are available(such as removing lines from fishrc that overwrite critical env vars; defining SHELL to something like /bin/bash;). Closing this for now. Please feel free to reopen.

Resolving patch:
diff --git a/dvc/stage.py b/dvc/stage.py
--- a/dvc/stage.py
+++ b/dvc/stage.py
@@ -416,6 +416,33 @@ def _check_missing_deps(self):
         if len(missing) > 0:
             raise MissingDep(missing)
 
+    def _check_if_fish(self, executable):  # pragma: no cover
+        if executable is None \
+           or os.path.basename(os.path.realpath(executable)) != 'fish':
+            return
+
+        msg = "DVC detected that you are using fish as your default " \
+              "shell. Be aware that it might cause problems by overwriting " \
+              "your current environment variables with values defined " \
+              "in '.fishrc', which might affect your command. See " \
+              "https://github.com/iterative/dvc/issues/1307. "
+        self.project.logger.warn(msg)
+
+    def _run(self):
+        self._check_missing_deps()
+        executable = os.getenv('SHELL') if os.name != 'nt' else None
+        self._check_if_fish(executable)
+
+        p = subprocess.Popen(self.cmd,
+                             cwd=self.cwd,
+                             shell=True,
+                             env=fix_env(os.environ),
+                             executable=executable)
+        p.communicate()
+
+        if p.returncode != 0:
+            raise StageCmdFailedError(self)
+
     def run(self, dry=False):
         if self.locked:
             msg = u'Verifying outputs in locked stage \'{}\''
@@ -439,16 +466,7 @@ def run(self, dry=False):
             self.project.logger.info(msg)
 
             if not dry:
-                self._check_missing_deps()
-                executable = os.getenv('SHELL') if os.name != 'nt' else None
-                p = subprocess.Popen(self.cmd,
-                                     cwd=self.cwd,
-                                     shell=True,
-                                     env=fix_env(os.environ),
-                                     executable=executable)
-                p.communicate()
-                if p.returncode != 0:
-                    raise StageCmdFailedError(self)
+                self._run()
 
         if not dry:
             self.save()


Test patch: