Issue 1:
[Note: My primary motivation for creating this issue is to:

Create a link between the potentially impacted projects (because the symptoms make it difficult to ascertain where the problem lies) i.e. Ansible + virtualenv (+ Homebrew); and,

Document a workaround/solution where it might be found.]

ISSUE TYPE

Documentation Report

COMPONENT NAME

Installation of Ansible into a virtual environment (optionally, via Homebrew)

ANSIBLE VERSION

ansible 2.3.2.0
config file =
configured module search path = Default w/o overrides
python version = 2.7.2 (default, Oct 11 2012, 20:14:37) [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)]

CONFIGURATION

N/A

OS / ENVIRONMENT

Ansible was installed via Homebrew which uses virtualenv. Output of brew --version:

Homebrew 1.3.2-25-g914378c
Homebrew/homebrew-core (git revision 9a5c8; last commit 2017-09-09)

Output of brew info ansible:

ansible: stable 2.3.2.0, HEAD
Automate deployment, configuration, and upgrading
https://www.ansible.com/
/usr/local/Cellar/ansible/2.3.2.0_1 (17,942 files, 187.8MB) *
Built from source on 2017-09-10 at 18:00:29
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/ansible.rb
==> Dependencies
Build: pkg-config ✔
Required: libyaml ✔, openssl@1.1 ✔

Disclosure: This is being run on an (unsupported by Homebrew and pretty much everyone else) Mac OS X 10.8 machine but AFAICT that is not connected to the issue.

The issue occurs before any control machine is contacted.

SUMMARY

Attempting to use the ec2.py inventory script (via) results in an error when used with an Ansible instance installed into a virtualenv (via Homebrew, in my specific case) that is not activated.

The error occurs because the ec2.py script cannot find the ansible Python module it expects because it is not being executed by the same Python interpreter binary/configuration as the ansible tool.

STEPS TO REPRODUCE

As far as I recall (modulo a compilation issue I've not yet reported to Homebrew):

brew install ansible
# Download https://raw.github.com/ansible/ansible/devel/contrib/inventory/ec2.py
ansible -i ec2.py --list-hosts all

EXPECTED RESULTS

Expected either:

A list of hosts; or,

an error message boto.exception.NoAuthHandlerFound: No handler was ready to authenticate which occurs when the script is executed correctly but no authorisation credentials are provided.

ACTUAL RESULTS

ERROR! Attempted to execute "ec2.py" as inventory script: Inventory script (ec2.py) had an execution error: Traceback (most recent call last):
File "/<path>/ec2.py", line 135, in <module>
from ansible.module_utils import ec2 as ec2_utils
ImportError: No module named ansible.module_utils

Explanation

There is much more relevant detail/context given in the associated issues in the other projects (pypa/virtualenv#1080, pypa/virtualenv#1081, Homebrew/brew#3167) which for the sake of brevity I won't repeat here but the summary is:

virtualenv intentionally distinguishes between "using" and "activating" a virtualenv.

Executing the Python interpreter binary within the virtualenv directly (without "activating" the virtualenv) only "uses" the virtualenv, it doesn't automatically "activate" it.

There is a potential difference in behaviour between "using" a virtualenv and "activating and then using" a virtualenv if a Python script with an env-based shebang line is executed as a child process of a script executed by the Python interpreter located within the virtualenv.

Specifically:

Manually "activating" a virtualenv changes the value of PATH so that the Python interpreter binary located within the virtualenv is used and this change is propagated to child processes and affects the action of /usr/bin/env.

Merely "using" an unactivated virtualenv does not modify PATH so in this situation any child process relying on env uses the default Python interpreter binary (and associated libraries).

Homebrew installs Ansible in a virtualenv and the ansible command is configured to use the Python interpreter located within the virtualenv directly:

$ head -1 `which ansible`
#!/usr/local/Cellar/ansible/2.3.2.0_1/libexec/bin/python

The ec2.py inventory script uses a env-based shebang line and now assumes it is using the same Python interpreter binary/configuration as ansible tool because it attempts to import from the ansible.module_utils module.

But because the Ansible virtualenv is only being "used" and not "activated and then used" the ec2.py script is not executed by the same Python interpreter binary/configuration as the ansible tool.

A potential solution/workaround (and alternative to an activation wrapper script in other situations)

The solution/workaround I proposed in pypa/virtualenv#1081:

The existing modification of sys.path by virtualenv is achieved by way of a custom site.py file. Researching this approach led to the discovery that further customization is made possible by the use of a sitecustomize module located in the site-packages folder of the virtualenv's Python installation.

Thus, the required modification of the PATH and other environment variables can be achieved via the creation of /<virtualenv path>/lib/python2.7/site-packages/sitecustomize.py (based on activate_this.py):

import os
import sys

old_os_path = os.environ.get('PATH', '')
os.environ['PATH'] = os.path.dirname(os.path.abspath(sys.executable)) + os.pathsep + old_os_path

(Clearly this is just a quick proof-of-concept, to be robust the code would need to avoid duplication of paths and also modify the other environment variables.)

This approach can also function as an alternative to activation wrapper scripts that are used in other situations when the "automatic activation before use" of a virtualenv is desired when the Python interpreter located within the virtualenv is executed directly.

Current status

From the conversation in pypa/virtualenv#1080 it now appears that the virtualenv project is unlikely to incorporate the proposed solution due to (as I understand it): (a) the issue being seen as specific to Ansible and/or Homebrew's packaging; and, (b) my proposed solution seeming too "magic" and non-obvious to people who may encounter it.

I have now proposed the solution to Homebrew for use in the tool itself or as part of the formula for Ansible but as the issue affects non-Homebrew Ansible virtualenv installations I wanted to at least document the approach here.

It may be appropriate to add a note about the issue/solution to the docs or, at least, add a warning to http://docs.ansible.com/ansible/latest/intro_installation.html#latest-releases-via-pip where it says "Readers that use virtualenv can also install Ansible under virtualenv...".

Related issues

Likely: #17147, #11689, #5335, #10452

Possibly #18414, #10378, #29987, #18227, #16724, #15019, #4083

Thanks for your time.

Issue 2:
ISSUE TYPE

Bug Report

COMPONENT NAME

ansible-galaxy

ANSIBLE VERSION

ansible 2.3.0.0
config file = /Users/callen/.ansible.cfg
configured module search path = Default w/o overrides
python version = 2.7.13 (default, Dec 17 2016, 23:03:43) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]

CONFIGURATION

Nothing special.

OS / ENVIRONMENT

macOS Sierra 10.12.4

SUMMARY

When using ansible-galaxy to check out a requirements.yml role, if a template inside the role contains the name of the role, the template file gets mangled during checkout. It should leave files named as they are in the repo.

STEPS TO REPRODUCE

Create a requirements.yml file with a single, simple role reference:

---
- src: https://github.com/commandtab/nginx.git
scm: git
version: 1.0.0

Install the role:

ansible-galaxy install -r requirements.yml --force

EXPECTED RESULTS

I expect the role to be cloned from the git repo to disk, leaving all filenames intact.

ACTUAL RESULTS

If you compare the contents of the role git repo's templates/ directory to the checked-out role, the templates/ directory on disk has had two of the files renamed:

nginx.conf.j2 became .conf.j2

nginx.repo.j2 became .repo.j2

Note that mime.types.j2 was not renamed.