Repository: conan-io/conan

Issue:
Conan virtualenv on Windows should generate shell script as well as .bat and .ps1
Hi,

It's a fairly common usecase in Windows for people to develop software using bash shells such as git-bash, MSYS2, cygwin, etc. Right now, conan only generates environment scripts for CMD and powershell, although these terminals often are not the preferred choice. This means that users have to open a windows command prompt just to execute the programs they've built. It would be far more ideal to just generate the bash script which already gets generated in unix. I don't see this change really breaking anyone's workflow either, it would only make it easier to write software on windows using conan.

Many thanks.


Discussion:
Yes, I wouldn't oppose to this generator generating .sh files in addition to .bat and .ps1 files. No, I think it wouldn't be breaking to have a couple of files more.

What do you think @lasote?
I still working on the virtualenv generator and I am happy to add those files if you guys want to. However, I would really appreciate some feedback on #2556 in order to fix the tests first.
I just started to look at this feature. In order to implement and test it I think that we need to specify the expected behavior in more detail.

Some questions:
* Should paths be in Windows (`some\\path`) or in Linux (`some/path`) style?
* Do we want to have Windows (`;`) or Linux (`:`) path separators?
* Do we want to have [space separated variables](https://github.com/conan-io/conan/blob/d1b30b346233cb550d3f3152986bc63b62e3cc9b/conans/client/generators/virtualenv.py#L8) which are currently only used on non Windows systems?
*  Do we want to have Windows (`'\r\n'`) or Posix (`\n`) line endings?

Hi Mario, 

In so far as the format of the commands to be run, it should be compatible with both MSYS2 and cygwin. That is:
- It should use the foo/bar folder style.
- The : separator should be used in env vars.
- Bash tends to have difficulties with \r\n, so I would recommend keeping with the POSIX \n.
- For the space separated variables, I think it should be sufficient to only define the variables which are already being declared in the current .bat files, although I don't know if anything more than that is needed.

Thanks.
Thank you for your feedback! Based on my rewrite of the virtualenv generator (see #2621) I gave it a try on the weekend. However, I am not sure if we can generate useful sh files in the regular windows environment (i.e., from cmd or powershell).

I currently see two problems:
1. Absolute paths (including drive letter) have to be re-rooted depending on the used bash implementation. On MSYS, drive letters are directly converted into [top level directories](https://stackoverflow.com/questions/11376899/browsing-folders-in-msys) (e.g., `C:\folder\file` is mapped to `/c/folder/file`). Cygwin, on the other hand, [seems to map them into the `cygdrive`](http://magicmonster.com/kb/app/cygwin/change_drive.html) directory (e.g., `C:\folder\file` is mapped to `/cygdrive/c/folder/file`). Without additional information which environment we target, performing the correct substitution is not possible.
2. The deactivate scripts will most certainly be useless given that they capture the environment at the `conan install` time and not at the `source activate.sh` time. Given that `cmd` and MSYS/cygwin `bash` have different default environment (including PATHs), sourcing a `deactivate.sh` that has been created by `conan install` from `cmd`, would most likely break the respective `bash`.

If we really wan't to have this feature in the virtualenv generator, the best I came up with to workaround (1) is to introduce a config variable with a regex that replaces drive letters. Alternatively, a dedicated MSYS/cygwin generator could be added. For (2), on the other hand, I am out of ideas.

I am really not sure if generating `sh` files from a `cmd`/`ps1` environment is worth the complexity. If you wan't to use `bash` anyway, would it be ok to call `conan install` from MSYS/cygwin bash? At least in my MSYS environment the virtualenv generator already generates `sh` files that work perfectly.

Best,
Mario
Thanks for the work Mario.

I think most people will be wanting to use this from a bash terminal explicitly to avoid touching the windows command prompts, so only generating the script from bash I think would be acceptable.

For the absolute paths, I think you should be able to search the environment variables like $PATH and be able to determine which substitution to use from that. For example, my $PATH variable in cygwin uses /cygdrive/c/..., whereas my MSYS2 terminal uses /c/... . Perhaps searching for `/cygdrive/c/WINDOWS` might be sufficient.

I can't speak for others, but personally deactivate.sh is a secondary concern for me. I only really need to be able to activate the environment so I can run my program, deactivating it can be achieved easily for me by just loading a new terminal. I've also found deactivate.sh to be unreliable for me using windows batch and linux, so I often tend not to use it, so it's not a major change to my workflow, although again I can't speak for others.

You are welcome. Just to be sure that we are on the same page. It is ok for you that we only generate the `sh` files when we use `conan install` from `bash`? If yes, then I think it should work now with #2621 and it would be great if you could give it a try.

The problems I mentioned earlier only exist if we want to generate the `sh` files although `conan install` is executed from `cmd` or power shell. Regarding `deactivate.sh`, I fixed a few bugs during the rewrite but agree that it is still not perfect.

[shameless self-plug] Since you mentioned that you simply load a new terminal instead of using deactivate you may also be interested in `conan exec` which I work on (see #2461 and #2516). With it you can basically run `conan exec --dir . bash` and have a bash with the correct environment. At the end you simply call `exit` and return to your previous bash.
I finally got around to giving this a try. I installed the version on your git repo directly into my virtualenv, although when I ran a conan install, it still only generated the .bat/.ps1 files and no shell scripts. I tried this on both MSYS2 and git-bash, same results. Below are the commands I ran:

	$ pip install git+git://github.com/niosHD/conan.git@feature/virtualenv
	Collecting git+git://github.com/niosHD/conan.git@feature/virtualenv
	...
	Installing collected packages: conan
	Successfully installed conan-1.2.0.dev0
	$ conan --version
	Conan version 1.2.0-dev
	$ conan install .. -s compiler.version=14 -s build_type=Debug
	...
	PROJECT: Generator cmake created conanbuildinfo.cmake
	PROJECT: Generator virtualenv created activate.bat
	PROJECT: Generator virtualenv created deactivate.bat
	PROJECT: Generator virtualenv created activate.ps1
	PROJECT: Generator virtualenv created deactivate.ps1
	PROJECT: Generator virtualrunenv created activate_run.bat
	PROJECT: Generator virtualrunenv created deactivate_run.bat
	PROJECT: Generator virtualrunenv created activate_run.ps1
	PROJECT: Generator virtualrunenv created deactivate_run.ps1
	PROJECT: Generator txt created conanbuildinfo.txt
	PROJECT: Generated conaninfo.txt



@niosHD I can reproduce by executing the following commands in a clean virtual env:

	pip install git+git://github.com/niosHD/conan.git@feature/virtualenv
	conan new test/1.0.0@user/channel
	conan install . -g "virtualenv"

This is using the MSYS2 terminal provided by git-bash.

Thanks!
Any progress on this?

Thanks!
We will try to solve it for the next release. It should be very easy.

Resolving patch:
diff --git a/conans/client/generators/virtualenv.py b/conans/client/generators/virtualenv.py
--- a/conans/client/generators/virtualenv.py
+++ b/conans/client/generators/virtualenv.py
@@ -127,9 +127,8 @@ def content(self):
             result["activate.ps1"] = os.linesep.join(activate)
             result["deactivate.ps1"] = os.linesep.join(deactivate)
 
-        if os_info.is_posix:
-            activate, deactivate = self._sh_lines()
-            result["activate.sh"] = os.linesep.join(activate)
-            result["deactivate.sh"] = os.linesep.join(deactivate)
+        activate, deactivate = self._sh_lines()
+        result["activate.sh"] = os.linesep.join(activate)
+        result["deactivate.sh"] = os.linesep.join(deactivate)
 
         return result


Test patch:
diff --git a/conans/test/generators/generators_test.py b/conans/test/generators/generators_test.py
--- a/conans/test/generators/generators_test.py
+++ b/conans/test/generators/generators_test.py
@@ -40,7 +40,8 @@ def test_base(self):
 
         virtualenvFiles = ["activate.sh", "deactivate.sh"]
         if platform.system() == "Windows":
-            virtualenvFiles = ["activate.bat", "deactivate.bat", "activate.ps1", "deactivate.ps1"]
+            virtualenvFiles.extend(["activate.bat", "deactivate.bat", "activate.ps1",
+                                    "deactivate.ps1"])
 
         self.assertEqual(sorted(['conanfile.txt', 'conaninfo.txt', 'conanbuildinfo.cmake',
                                  'conanbuildinfo.gcc', 'conanbuildinfo.qbs', 'conanbuildinfo.pri',
diff --git a/conans/test/generators/virtualenv_test.py b/conans/test/generators/virtualenv_test.py
--- a/conans/test/generators/virtualenv_test.py
+++ b/conans/test/generators/virtualenv_test.py
@@ -76,53 +76,73 @@ def package_info(self):
             self.assertIn('$env:SPECIAL_VAR = "dummyValue"', activate)
 
             deactivate = load(os.path.join(client.current_folder, "deactivate.bat"))
-            self.assertIn('SET PROMPT=%s' % env.setdefault('PROMPT',''), deactivate)
-            self.assertIn('SET BASE_LIST=%s' % env.setdefault('BASE_LIST',''), deactivate)
-            self.assertIn('SET BASE_VAR=%s' % env.setdefault('BASE_VAR',''), deactivate)
-            self.assertIn('SET CPPFLAGS=%s' % env.setdefault('CPPFLAGS',''), deactivate)
-            self.assertIn('SET LD_LIBRARY_PATH=%s' % env.setdefault('LD_LIBRARY_PATH',''), deactivate)
-            self.assertIn('SET PATH=%s' % env.setdefault('PATH',''), deactivate)
-            self.assertIn('SET SPECIAL_VAR=%s' % env.setdefault('SPECIAL_VAR',''), deactivate)
+            self.assertIn('SET PROMPT=%s' % env.setdefault('PROMPT', ''), deactivate)
+            self.assertIn('SET BASE_LIST=%s' % env.setdefault('BASE_LIST', ''), deactivate)
+            self.assertIn('SET BASE_VAR=%s' % env.setdefault('BASE_VAR', ''), deactivate)
+            self.assertIn('SET CPPFLAGS=%s' % env.setdefault('CPPFLAGS', ''), deactivate)
+            self.assertIn('SET LD_LIBRARY_PATH=%s' % env.setdefault('LD_LIBRARY_PATH', ''), deactivate)
+            self.assertIn('SET PATH=%s' % env.setdefault('PATH', ''), deactivate)
+            self.assertIn('SET SPECIAL_VAR=%s' % env.setdefault('SPECIAL_VAR', ''), deactivate)
 
             deactivate = load(os.path.join(client.current_folder, "deactivate.ps1"))
-            self.assertIn('$env:BASE_LIST = "%s"' % env.setdefault('BASE_LIST',''), deactivate)
-            self.assertIn('$env:BASE_VAR = "%s"' % env.setdefault('BASE_VAR',''), deactivate)
-            self.assertIn('$env:CPPFLAGS = "%s"' % env.setdefault('CPPFLAGS',''), deactivate)
-            self.assertIn('$env:LD_LIBRARY_PATH = "%s"' % env.setdefault('LD_LIBRARY_PATH',''), deactivate)
-            self.assertIn('$env:PATH = "%s"' % env.setdefault('PATH',''), deactivate)
-            self.assertIn('$env:SPECIAL_VAR = "%s"' % env.setdefault('SPECIAL_VAR',''), deactivate)
+            self.assertIn('$env:BASE_LIST = "%s"' % env.setdefault('BASE_LIST', ''), deactivate)
+            self.assertIn('$env:BASE_VAR = "%s"' % env.setdefault('BASE_VAR', ''), deactivate)
+            self.assertIn('$env:CPPFLAGS = "%s"' % env.setdefault('CPPFLAGS', ''), deactivate)
+            self.assertIn('$env:LD_LIBRARY_PATH = "%s"' % env.setdefault('LD_LIBRARY_PATH', ''), deactivate)
+            self.assertIn('$env:PATH = "%s"' % env.setdefault('PATH', ''), deactivate)
+            self.assertIn('$env:SPECIAL_VAR = "%s"' % env.setdefault('SPECIAL_VAR', ''), deactivate)
 
-        if os_info.is_posix:
-            activate = load(os.path.join(client.current_folder, "activate.sh"))
-            self.assertIn('OLD_PS1="$PS1"\nexport OLD_PS1', activate)
-            self.assertIn('PS1="(conanenv) $PS1"\nexport PS1', activate)
-            self.assertIn('BASE_LIST="dummyValue1":"dummyValue2":"baseValue1":"baseValue2":$BASE_LIST\nexport BASE_LIST', activate)
-            self.assertIn('BASE_VAR="baseValue"\nexport BASE_VAR', activate)
-            self.assertIn('CPPFLAGS="-flag1 -flag2 -baseFlag1 -baseFlag2 $CPPFLAGS"\nexport CPPFLAGS', activate)
-            self.assertIn('LD_LIBRARY_PATH="dummydir/lib":"basedir/lib":$LD_LIBRARY_PATH\nexport LD_LIBRARY_PATH', activate)
-            self.assertIn('PATH="dummydir/bin":"basedir/bin":"samebin":$PATH\nexport PATH', activate)
-            self.assertIn('SPECIAL_VAR="dummyValue"\nexport SPECIAL_VAR', activate)
+        activate = load(os.path.join(client.current_folder, "activate.sh"))
+        self.assertIn('OLD_PS1="$PS1"', activate)
+        self.assertIn('export OLD_PS1', activate)
+        self.assertIn('PS1="(conanenv) $PS1"', activate)
+        self.assertIn('export PS1', activate)
+        self.assertIn('BASE_LIST="dummyValue1":"dummyValue2":"baseValue1":"baseValue2":$BASE_LIST', activate)
+        self.assertIn('export BASE_LIST', activate)
+        self.assertIn('BASE_VAR="baseValue"', activate)
+        self.assertIn('export BASE_VAR', activate)
+        self.assertIn('CPPFLAGS="-flag1 -flag2 -baseFlag1 -baseFlag2 $CPPFLAGS"', activate)
+        self.assertIn('export CPPFLAGS', activate)
+        self.assertIn('SPECIAL_VAR="dummyValue"', activate)
+        self.assertIn('export SPECIAL_VAR', activate)
+        if os_info.is_windows:
+            self.assertIn('LD_LIBRARY_PATH="dummydir\\lib":"basedir\\lib":$LD_LIBRARY_PATH', activate)
+            self.assertIn('PATH="dummydir\\bin":"basedir\\bin":"samebin":$PATH', activate)
+        else:
+            self.assertIn('LD_LIBRARY_PATH="dummydir/lib":"basedir/lib":$LD_LIBRARY_PATH', activate)
+            self.assertIn('PATH="dummydir/bin":"basedir/bin":"samebin":$PATH', activate)
+        self.assertIn('export LD_LIBRARY_PATH', activate)
+        self.assertIn('export PATH', activate)
 
-            deactivate = load(os.path.join(client.current_folder, "deactivate.sh"))
-            if posix_empty_vars:
-                self.assertIn('unset OLD_PS1', deactivate)
-                if not get_env("PS1"):
-                    self.assertIn('unset PS1', deactivate)
-                self.assertIn('unset BASE_LIST', deactivate)
-                self.assertIn('unset BASE_VAR', deactivate)
-                self.assertIn('unset CPPFLAGS', deactivate)
-                self.assertIn('unset LD_LIBRARY_PATH', deactivate)
-                self.assertIn('PATH="%s"\nexport PATH' % env.setdefault('PATH',''), deactivate)
-                self.assertIn('unset SPECIAL_VAR', deactivate)
-            else:
-                self.assertIn('OLD_PS1="%s"\nexport OLD_PS1' % env.setdefault('OLD_PS1', ''), deactivate)
-                self.assertIn('PS1="%s"\nexport PS1' % env.setdefault('PS1', ''), deactivate)
-                self.assertIn('BASE_LIST="%s"\nexport BASE_LIST' % env.setdefault('BASE_LIST', ''), deactivate)
-                self.assertIn('BASE_VAR="%s"\nexport BASE_VAR' % env.setdefault('BASE_VAR', ''), deactivate)
-                self.assertIn('CPPFLAGS="%s"\nexport CPPFLAGS' % env.setdefault('CPPFLAGS', ''), deactivate)
-                self.assertIn('LD_LIBRARY_PATH="%s"\nexport LD_LIBRARY_PATH' % env.setdefault('LD_LIBRARY_PATH', ''), deactivate)
-                self.assertIn('PATH="%s"\nexport PATH' % env.setdefault('PATH', ''), deactivate)
-                self.assertIn('SPECIAL_VAR="%s"\nexport SPECIAL_VAR' % env.setdefault('SPECIAL_VAR', ''), deactivate)
+        deactivate = load(os.path.join(client.current_folder, "deactivate.sh"))
+        if posix_empty_vars:
+            self.assertIn('unset OLD_PS1', deactivate)
+            if not get_env("PS1"):
+                self.assertIn('unset PS1', deactivate)
+            self.assertIn('unset BASE_LIST', deactivate)
+            self.assertIn('unset BASE_VAR', deactivate)
+            self.assertIn('unset CPPFLAGS', deactivate)
+            self.assertIn('unset LD_LIBRARY_PATH', deactivate)
+            self.assertIn('PATH="%s"' % env.setdefault('PATH', ''), deactivate)
+            self.assertIn('export PATH', deactivate)
+            self.assertIn('unset SPECIAL_VAR', deactivate)
+        else:
+            self.assertIn('OLD_PS1="%s"' % env.setdefault('OLD_PS1', ''), deactivate)
+            self.assertIn('export OLD_PS1', deactivate)
+            self.assertIn('PS1="%s"' % env.setdefault('PS1', ''), deactivate)
+            self.assertIn('export PS1', deactivate)
+            self.assertIn('BASE_LIST="%s"' % env.setdefault('BASE_LIST', ''), deactivate)
+            self.assertIn('export BASE_LIST', deactivate)
+            self.assertIn('BASE_VAR="%s"' % env.setdefault('BASE_VAR', ''), deactivate)
+            self.assertIn('export BASE_VAR', deactivate)
+            self.assertIn('CPPFLAGS="%s"' % env.setdefault('CPPFLAGS', ''), deactivate)
+            self.assertIn('export CPPFLAGS', deactivate)
+            self.assertIn('LD_LIBRARY_PATH="%s"' % env.setdefault('LD_LIBRARY_PATH', ''), deactivate)
+            self.assertIn('export LD_LIBRARY_PATH', deactivate)
+            self.assertIn('PATH="%s"' % env.setdefault('PATH', ''), deactivate)
+            self.assertIn('export PATH', deactivate)
+            self.assertIn('SPECIAL_VAR="%s"' % env.setdefault('SPECIAL_VAR', ''), deactivate)
+            self.assertIn('export SPECIAL_VAR', deactivate)
 
     def environment_test(self):
         os.environ["PROMPT"] = "old_PROMPT"