Repository: ansible/ansible

Issue:
ansible-galaxy install does not work
<!---
Please do not report issues/requests related to Ansible modules here !!

Report them to the appropriate modules-core or modules-extras project:
  - https://github.com/ansible/ansible-modules-core/issues
  - https://github.com/ansible/ansible-modules-extras/issues

Also verify first that your issue/request is not already reported in GitHub
-->
##### ISSUE TYPE
- Bug Report
##### COMPONENT NAME

ansible-galaxy
##### ANSIBLE VERSION

```
master (2.2.0)
```
##### CONFIGURATION

<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->
##### OS / ENVIRONMENT

<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
-->

mac, Linux
##### SUMMARY

<!--- Explain the problem briefly -->

ansible-galaxy install -r requirements.yml fails.
##### STEPS TO REPRODUCE

<!---
For bugs, show exactly how to reproduce the problem.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->

```
ansible-galaxy install -r requirements.yml
```

<!--- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS

<!--- What did you expect to happen when running the steps above? -->

It will install the roles in requirements.yml file.
##### ACTUAL RESULTS

<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->

```
(ansible) ~/P/c/ansible-role-django-app ❯❯❯ ~/miniconda2/envs/ansible/bin/ansible-galaxy install -r requirements.yml             master
Usage: ansible-galaxy install [options] [-r FILE | role_name(s)[,version] | scm+role_repo_url[,version] | tar_file(s)]

Options:
  -f, --force           Force overwriting an existing role
  -h, --help            show this help message and exit
  -c, --ignore-certs    Ignore SSL certificate validation errors.
  -i, --ignore-errors   Ignore errors and continue with the next specified
                        role.
  -n, --no-deps         Don't download roles listed as dependencies
  -r ROLE_FILE, --role-file=ROLE_FILE
                        A file containing a list of roles to be imported
  -p ROLES_PATH, --roles-path=ROLES_PATH
                        The path to the directory containing your roles. The
                        default is the roles_path configured in your
                        ansible.cfg file (/etc/ansible/roles if not
                        configured)
  -s API_SERVER, --server=API_SERVER
                        The API server destination
  -v, --verbose         verbose mode (-vvv for more, -vvvv to enable
                        connection debugging)
  --version             show program's version number and exit
ERROR! - please specify a user/role name, or a roles file, but not both
```

ansible-galaxy  init test-role creates 'init' directory, ignores role name
##### ISSUE TYPE

<!--- Pick one below and delete the rest: -->
- Bug Report
##### COMPONENT NAME

ansible-galaxy
##### ANSIBLE VERSION

<!--- Paste verbatim output from “ansible --version” between quotes below -->

```
$ ansible --version
ansible 2.2.0 (devel c654a4617a) last updated 2016/09/14 10:05:44 (GMT -500)
  lib/ansible/modules/core: (detached HEAD ae6992bf8c) last updated 2016/09/13 10:06:39 (GMT -500)
  lib/ansible/modules/extras: (detached HEAD afd0b23836) last updated 2016/09/13 10:06:39 (GMT -500)
  config file =
  configured module search path = Default w/o overrides
```
##### CONFIGURATION

Default configuration
##### OS / ENVIRONMENT

OS X 10.11.6
##### SUMMARY

ansible-galaxy init rolename creates a directory called init, ignores rolename
##### STEPS TO REPRODUCE

ansible-galaxy  init test-role
- init was created successfully

<!--- Paste example playbooks or commands between quotes below -->

```
$ ansible-galaxy -vvvv init test-role
No config file found; using defaults
Opened /Users/tanner/.ansible_galaxy
Initial connection to galaxy_server: https://galaxy.ansible.com
Base API: https://galaxy.ansible.com/api/v1
https://galaxy.ansible.com/api/v1/platforms/?page_size
- init was created successfully
```
##### EXPECTED RESULTS

A role created in a directory called test-role
##### ACTUAL RESULTS

A role created in a directory called init



Discussion:
I can see the same issue, and it looks like this has been introduced in the last 24 hours. Prior to this everything worked fine.

One of my nightly builds (which also runs `ansible-galaxy` from `ansible@devel`) failed around two hours ago, showing the same error message:
https://travis-ci.org/goodplay/goodplay/jobs/159504690#L520

Not sure about the root cause yet, but it might be related to commit bf2996194.

My last working test used this commit: f4af154beff2b281e464b616f504293d67187da5

Have a duplicate issue open against ansible-galaxy (https://github.com/ansible/galaxy-issues/issues/191). git bisect identified [this commit](https://github.com/ansible/ansible/commit/bf2996194716c7d58b9b543603c2d917e3eac060) as introducing the issue.

Quick debug shows role_name getting set to init here:

```
ansible/lib/ansible/cli/galaxy.py(178)execute_init()
-> role_name = self.args.pop(0).strip() if self.args else None
(Epdb) self.args
['init', 'test-role']
```

Which I think is because init.py:set_action() is not working?

Don't really understand the logic of init.py:set_action()

this is probably an effect of the recent changes to make it py3 compatible

galaxy.py(123)parse() changes self.args.

Before the call self.args is ["ansible-galaxy", "test-role"] after the called to parse it's ['init', 'test-role']

```
0: self.args = [u'/Users...e-galaxy', u'test-role']

> /Users/tanner/projects/ansible.git/ansible/lib/ansible/cli/galaxy.py(123)parse()
-> self.options, self.args =self.parser.parse_args()
(Epdb)

0: self.args = ['init', 'test-role']
```

Super hackery that fixes my problem.

```
diff --git a/lib/ansible/cli/galaxy.py b/lib/ansible/cli/galaxy.py
index 457e15d..db8cbed 100644
--- a/lib/ansible/cli/galaxy.py
+++ b/lib/ansible/cli/galaxy.py
@@ -175,7 +175,7 @@ class GalaxyCLI(CLI):
         force      = self.get_opt('force', False)
         offline    = self.get_opt('offline', False)

-        role_name = self.args.pop(0).strip() if self.args else None
+        role_name = self.args.pop(1).strip() if self.args else None
         if not role_name:
             raise AnsibleOptionsError("- no role name specified for init")
         role_path = os.path.join(init_path, role_name)
```


Resolving patch:
diff --git a/lib/ansible/cli/__init__.py b/lib/ansible/cli/__init__.py
--- a/lib/ansible/cli/__init__.py
+++ b/lib/ansible/cli/__init__.py
@@ -1,4 +1,5 @@
 # (c) 2012-2014, Michael DeHaan <michael.dehaan@gmail.com>
+# (c) 2016, Toshio Kuratomi <tkuratomi@ansible.com>
 #
 # This file is part of Ansible
 #
@@ -52,6 +53,42 @@ def format_help(self, formatter=None, epilog=None):
         return optparse.OptionParser.format_help(self, formatter=None)
 
 
+# Note: Inherit from SortedOptParser so that we get our format_help method
+class InvalidOptsParser(SortedOptParser):
+    '''Ignore invalid options.
+
+    Meant for the special case where we need to take care of help and version
+    but may not know the full range of options yet.  (See it in use in set_action)
+    '''
+    def __init__(self, parser):
+        # Since this is special purposed to just handle help and version, we
+        # take a pre-existing option parser here and set our options from
+        # that.  This allows us to give accurate help based on the given
+        # option parser.
+        SortedOptParser.__init__(self, usage=parser.usage,
+                                 option_list=parser.option_list,
+                                 option_class=parser.option_class,
+                                 conflict_handler=parser.conflict_handler,
+                                 description=parser.description,
+                                 formatter=parser.formatter,
+                                 add_help_option=False,
+                                 prog=parser.prog,
+                                 epilog=parser.epilog)
+        self.version=parser.version
+
+    def _process_long_opt(self, rargs, values):
+        try:
+            optparse.OptionParser._process_long_opt(self, rargs, values)
+        except optparse.BadOptionError:
+            pass
+
+    def _process_short_opts(self, rargs, values):
+        try:
+            optparse.OptionParser._process_short_opts(self, rargs, values)
+        except optparse.BadOptionError:
+            pass
+
+
 class CLI(object):
     ''' code behind bin/ansible* programs '''
 
@@ -82,7 +119,7 @@ def set_action(self):
         """
         Get the action the user wants to execute from the sys argv list.
         """
-        for i in range(0,len(self.args)):
+        for i in range(0, len(self.args)):
             arg = self.args[i]
             if arg in self.VALID_ACTIONS:
                 self.action = arg
@@ -90,8 +127,13 @@ def set_action(self):
                 break
 
         if not self.action:
-            # if no need for action if version/help
-            tmp_options, tmp_args = self.parser.parse_args()
+            # if we're asked for help or version, we don't need an action.
+            # have to use a special purpose Option Parser to figure that out as
+            # the standard OptionParser throws an error for unknown options and
+            # without knowing action, we only know of a subset of the options
+            # that could be legal for this command
+            tmp_parser = InvalidOptsParser(self.parser)
+            tmp_options, tmp_args = tmp_parser.parse_args(self.args)
             if not(hasattr(tmp_options, 'help') and tmp_options.help) or (hasattr(tmp_options, 'version') and tmp_options.version):
                 raise AnsibleOptionsError("Missing required action")
 
diff --git a/lib/ansible/cli/galaxy.py b/lib/ansible/cli/galaxy.py
--- a/lib/ansible/cli/galaxy.py
+++ b/lib/ansible/cli/galaxy.py
@@ -120,7 +120,7 @@ def parse(self):
         if self.action in ("init","install"):
             self.parser.add_option('-f', '--force', dest='force', action='store_true', default=False, help='Force overwriting an existing role')
 
-        self.options, self.args =self.parser.parse_args()
+        self.options, self.args = self.parser.parse_args(self.args[1:])
         display.verbosity = self.options.verbosity
         self.galaxy = Galaxy(self.options)
         return True


Test patch:
diff --git a/test/units/cli/test_galaxy.py b/test/units/cli/test_galaxy.py
--- a/test/units/cli/test_galaxy.py
+++ b/test/units/cli/test_galaxy.py
@@ -46,9 +46,8 @@ def setUpClass(cls):
             shutil.rmtree("./delete_me")
 
         # creating framework for a role
-        gc = GalaxyCLI(args=["init"])
-        with patch('sys.argv', ["-c", "--offline", "delete_me"]):
-            gc.parse()
+        gc = GalaxyCLI(args=["init", "-c", "--offline", "delete_me"])
+        gc.parse()
         gc.run()
         cls.role_dir = "./delete_me"
         cls.role_name = "delete_me"
@@ -117,9 +116,8 @@ def test_display_galaxy_info(self):
 
     def test_execute_remove(self):
         # installing role
-        gc = GalaxyCLI(args=["install"])
-        with patch('sys.argv', ["--offline", "-p", self.role_path, "-r", self.role_req]):
-            galaxy_parser = gc.parse()
+        gc = GalaxyCLI(args=["install", "--offline", "-p", self.role_path, "-r", self.role_req])
+        galaxy_parser = gc.parse()
         gc.run()
 
         # checking that installation worked
@@ -127,9 +125,8 @@ def test_execute_remove(self):
         self.assertTrue(os.path.exists(role_file))
 
         # removing role
-        gc = GalaxyCLI(args=["remove"])
-        with patch('sys.argv', ["-c", "-p", self.role_path, self.role_name]):
-            galaxy_parser = gc.parse()
+        gc = GalaxyCLI(args=["remove", "-c", "-p", self.role_path, self.role_name])
+        galaxy_parser = gc.parse()
         super(GalaxyCLI, gc).run()
         gc.api = ansible.galaxy.api.GalaxyAPI(gc.galaxy)
         completed_task = gc.execute_remove()
@@ -140,19 +137,18 @@ def test_execute_remove(self):
 
     def test_exit_without_ignore(self):
         ''' tests that GalaxyCLI exits with the error specified unless the --ignore-errors flag is used '''
-        gc = GalaxyCLI(args=["install"])
+        gc = GalaxyCLI(args=["install", "-c", "fake_role_name"])
 
         # testing without --ignore-errors flag
-        with patch('sys.argv', ["-c", "fake_role_name"]):
-            galaxy_parser = gc.parse()
+        galaxy_parser = gc.parse()
         with patch.object(ansible.utils.display.Display, "display", return_value=None) as mocked_display:
             # testing that error expected is raised
             self.assertRaises(AnsibleError, gc.run)
             self.assertTrue(mocked_display.called_once_with("- downloading role 'fake_role_name', owned by "))
 
         # testing with --ignore-errors flag
-        with patch('sys.argv', ["-c", "fake_role_name", "--ignore-errors"]):
-            galalxy_parser = gc.parse()
+        gc = GalaxyCLI(args=["install", "-c", "fake_role_name", "--ignore-errors"])
+        galalxy_parser = gc.parse()
         with patch.object(ansible.utils.display.Display, "display", return_value=None) as mocked_display:
             # testing that error expected is not raised with --ignore-errors flag in use
             gc.run()
@@ -160,53 +156,50 @@ def test_exit_without_ignore(self):
 
     def run_parse_common(self, galaxycli_obj, action):
         with patch.object(ansible.cli.SortedOptParser, "set_usage") as mocked_usage:
-            with patch('sys.argv', ["-c"]):
-                galaxy_parser = galaxycli_obj.parse()
-
-                # checking that the common results of parse() for all possible actions have been created/called
-                self.assertTrue(galaxy_parser)
-                self.assertTrue(isinstance(galaxycli_obj.parser, ansible.cli.SortedOptParser))
-                self.assertTrue(isinstance(galaxycli_obj.galaxy, ansible.galaxy.Galaxy))
-                if action in ['import', 'delete']:
-                    formatted_call = 'usage: %prog ' + action + ' [options] github_user github_repo'
-                elif action == 'info':
-                    formatted_call = 'usage: %prog ' + action + ' [options] role_name[,version]'
-                elif action == 'init':
-                    formatted_call = 'usage: %prog ' + action + ' [options] role_name'
-                elif action == 'install':
-                    formatted_call = 'usage: %prog ' + action + ' [options] [-r FILE | role_name(s)[,version] | scm+role_repo_url[,version] | tar_file(s)]'
-                elif action == 'list':
-                    formatted_call = 'usage: %prog ' + action + ' [role_name]'
-                elif action == 'login':
-                    formatted_call = 'usage: %prog ' + action + ' [options]'
-                elif action == 'remove':
-                    formatted_call = 'usage: %prog ' + action + ' role1 role2 ...'
-                elif action == 'search':
-                    formatted_call = 'usage: %prog ' + action + ' [searchterm1 searchterm2] [--galaxy-tags galaxy_tag1,galaxy_tag2] [--platforms platform1,platform2] [--author username]'
-                elif action == 'setup':
-                    formatted_call = 'usage: %prog ' + action + ' [options] source github_user github_repo secret'
-                calls = [call('usage: %prog [delete|import|info|init|install|list|login|remove|search|setup] [--help] [options] ...'), call(formatted_call)]
-                mocked_usage.assert_has_calls(calls)
+            galaxy_parser = galaxycli_obj.parse()
+
+            # checking that the common results of parse() for all possible actions have been created/called
+            self.assertTrue(galaxy_parser)
+            self.assertTrue(isinstance(galaxycli_obj.parser, ansible.cli.SortedOptParser))
+            self.assertTrue(isinstance(galaxycli_obj.galaxy, ansible.galaxy.Galaxy))
+            if action in ['import', 'delete']:
+                formatted_call = 'usage: %prog ' + action + ' [options] github_user github_repo'
+            elif action == 'info':
+                formatted_call = 'usage: %prog ' + action + ' [options] role_name[,version]'
+            elif action == 'init':
+                formatted_call = 'usage: %prog ' + action + ' [options] role_name'
+            elif action == 'install':
+                formatted_call = 'usage: %prog ' + action + ' [options] [-r FILE | role_name(s)[,version] | scm+role_repo_url[,version] | tar_file(s)]'
+            elif action == 'list':
+                formatted_call = 'usage: %prog ' + action + ' [role_name]'
+            elif action == 'login':
+                formatted_call = 'usage: %prog ' + action + ' [options]'
+            elif action == 'remove':
+                formatted_call = 'usage: %prog ' + action + ' role1 role2 ...'
+            elif action == 'search':
+                formatted_call = 'usage: %prog ' + action + ' [searchterm1 searchterm2] [--galaxy-tags galaxy_tag1,galaxy_tag2] [--platforms platform1,platform2] [--author username]'
+            elif action == 'setup':
+                formatted_call = 'usage: %prog ' + action + ' [options] source github_user github_repo secret'
+            calls = [call('usage: %prog [delete|import|info|init|install|list|login|remove|search|setup] [--help] [options] ...'), call(formatted_call)]
+            mocked_usage.assert_has_calls(calls)
 
     def test_parse(self):
         ''' systematically testing that the expected options parser is created '''
         # testing no action given
-        gc = GalaxyCLI(args=[])
-        with patch('sys.argv', ["-c"]):
-            self.assertRaises(AnsibleOptionsError, gc.parse)
+        gc = GalaxyCLI(args=["-c"])
+        self.assertRaises(AnsibleOptionsError, gc.parse)
 
         # testing action that doesn't exist
-        gc = GalaxyCLI(args=["NOT_ACTION"])
-        with patch('sys.argv', ["-c"]):
-            self.assertRaises(AnsibleOptionsError, gc.parse)
+        gc = GalaxyCLI(args=["NOT_ACTION", "-c"])
+        self.assertRaises(AnsibleOptionsError, gc.parse)
 
         # testing action 'delete'
-        gc = GalaxyCLI(args=["delete"])
+        gc = GalaxyCLI(args=["delete", "-c"])
         self.run_parse_common(gc, "delete")
         self.assertTrue(gc.options.verbosity==0)
 
         # testing action 'import'
-        gc = GalaxyCLI(args=["import"])
+        gc = GalaxyCLI(args=["import", "-c"])
         self.run_parse_common(gc, "import")
         self.assertTrue(gc.options.wait==True)
         self.assertTrue(gc.options.reference==None)
@@ -214,18 +207,18 @@ def test_parse(self):
         self.assertTrue(gc.options.verbosity==0)
 
         # testing action 'info'
-        gc = GalaxyCLI(args=["info"])
+        gc = GalaxyCLI(args=["info", "-c"])
         self.run_parse_common(gc, "info")
         self.assertTrue(gc.options.offline==False)
 
         # testing action 'init'
-        gc = GalaxyCLI(args=["init"])
+        gc = GalaxyCLI(args=["init", "-c"])
         self.run_parse_common(gc, "init")
         self.assertTrue(gc.options.offline==False)
         self.assertTrue(gc.options.force==False)
 
         # testing action 'install'
-        gc = GalaxyCLI(args=["install"])
+        gc = GalaxyCLI(args=["install", "-c"])
         self.run_parse_common(gc, "install")
         self.assertTrue(gc.options.ignore_errors==False)
         self.assertTrue(gc.options.no_deps==False)
@@ -233,30 +226,30 @@ def test_parse(self):
         self.assertTrue(gc.options.force==False)
 
         # testing action 'list'
-        gc = GalaxyCLI(args=["list"])
+        gc = GalaxyCLI(args=["list", "-c"])
         self.run_parse_common(gc, "list")
         self.assertTrue(gc.options.verbosity==0)
 
         # testing action 'login'
-        gc = GalaxyCLI(args=["login"])
+        gc = GalaxyCLI(args=["login", "-c"])
         self.run_parse_common(gc, "login")
         self.assertTrue(gc.options.verbosity==0)
         self.assertTrue(gc.options.token==None)
 
         # testing action 'remove'
-        gc = GalaxyCLI(args=["remove"])
+        gc = GalaxyCLI(args=["remove", "-c"])
         self.run_parse_common(gc, "remove")
         self.assertTrue(gc.options.verbosity==0)
 
         # testing action 'search'
-        gc = GalaxyCLI(args=["search"])
+        gc = GalaxyCLI(args=["search", "-c"])
         self.run_parse_common(gc, "search")
         self.assertTrue(gc.options.platforms==None)
         self.assertTrue(gc.options.tags==None)
         self.assertTrue(gc.options.author==None)
 
         # testing action 'setup'
-        gc = GalaxyCLI(args=["setup"])
+        gc = GalaxyCLI(args=["setup", "-c"])
         self.run_parse_common(gc, "setup")
         self.assertTrue(gc.options.verbosity==0)
         self.assertTrue(gc.options.remove_id==None)