Repository: ansible/ansible

Issue:
tower_* modules not updated for Tower 3.2/ tower_cli 3.2

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
tower_credential
tower_*

##### ANSIBLE VERSION
```
ansible 2.4.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/notting/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.13 (default, Sep  5 2017, 08:53:59) [GCC 7.1.1 20170622 (Red Hat 7.1.1-3)]
```

##### CONFIGURATION
```
ANSIBLE_SSH_PIPELINING(/etc/ansible/ansible.cfg) = True
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False
PERSISTENT_CONNECT_TIMEOUT(/etc/ansible/ansible.cfg) = 30
```

##### OS / ENVIRONMENT

Fedora 26

##### SUMMARY

With tower_cli 3.2.0 installed, assorted Tower modules fail:

```
[notting@nostromo: ~]$ ansible -m tower_credential localhost -a "kind=ssh username=wow password=wow name=frobozznicate organization=TGP"
[WARNING]: Could not match supplied host pattern, ignoring: all

[WARNING]: provided hosts list is empty, only localhost is available

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: tower_cli.exceptions.BadRequest: Missing required fields: credential-type
localhost | FAILED! => {
   "changed": false,
   "failed": true,
   "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_imDa8H/ansible_module_tower_credential.py\", line 292, in <module>\n    main()\n  File \"/tmp/ansible_imDa8H/ansible_module_tower_credential.py\", line 277, in main\n    result = credential.modify(**params)\n  File \"/usr/lib/python2.7/site-packages/tower_cli/models/base.py\", line 659, in modify\n    return self.write(pk, create_on_missing=create_on_missing, force_on_exists=True, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/tower_cli/models/base.py\", line 357, in write\n    raise exc.BadRequest('Missing required fields: %s' % ', '.join(missing_fields).replace('_', '-'))\ntower_cli.exceptions.BadRequest: Missing required fields: credential-type\n",
   "module_stdout": "",
   "msg": "MODULE FAILURE",
   "rc": 0
}
```

##### STEPS TO REPRODUCE

see above

##### EXPECTED RESULTS

works


##### ACTUAL RESULTS

attempts to use /api/v1 parameters from 3.1 and earlier with tower-cli 3.2's /api/v2 usage. Things go kablooie.



Discussion:
cc @ghjm @jlaska @matburt @wwitzel3
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
is there any workaround for this? I'm using tower-cli 3.2.0, Tower  3.2.1 , Ansible 2.4.0
@hanengarcia et al

No work around for this, other than to go back to `tower-cli <3.2`.

I've started going through the modules, and there is quite a bit of other work that would be required to update this to api/v2.

For example, in the `tower_credentials.py` module alone, you need to create a couple helper modules that look up the required fields, and the fields needed in the list of `{ "inputs": { "fields": [...] } }` for the `credential-type`. You'll then need to dynamically build up the list of k,v pairs and generate the JSON appropriately.

I started looking at this, and I'm pretty sure I could come up with something, but unfortunately this isn't really a task I should be tackling at work probably...

The other question then lies with if the modules should be backwards compatible with API v1 and v2, or if we just clean break from v1 to v2 in a new Ansible version.

If we need to be backwards compatible, then the modules are going to get a lot more complicated.

I started building (and successfully made work) a new `tower_inventory_source` module, but it requires the use of tower-cli 3.2.0 (because 3.1.8 doesn't have the `inventory_source` resource), so I'm not really sure where to go from here.
Actually, there might be a work around :)

First, downgrade to using the v1 API by installing `ansible-tower-cli` to version <3.2.

Then you can pretty much use the existing modules. For the functionality / modules that don't exist yet, I'm going to be consuming the API directly via the `uri` module for now (primarily so that I can create the inventory cloud sources).
@leifmadsen  I downgraded to ansible-tower-cli 3.1.0 but it still seems to be using the v2 API.  Do I need to back further than that?
@leifmadsen @mpetrive-rh Same problem. I downgraded to ansible-tower-cli 3.1.7.
An idea please ?
`ansible-tower-cli` version 3.1.7 uses API v1 as far as I can tell. I had things working and not working when I was changing environments around. Make sure you're able to communicate with `tower-cli` directly, because all the modules are doing is wrapping that library. If it works with `tower-cli` it should work fine with the modules (assuming you're not using `tower-cli >= 3.2`
@leifmadsen No I confirm it's not ok. Others errors appear : 
`'tower_verify_ssl' is not a valid field for Machine`
`'state' is not a valid field for Machine`

My versions:
- `Tower 3.2.1` 
- `Ansible 2.4.1.0`
- `tower-cli 3.1.7`
Needs more information. I'm not really sure where you're getting that output from.

From the `~/.tower_cli.cfg` file that I have, my setting is `verify_ssl: False`. In the modules I'm using, I have `tower_verify_ssl: False`, like:

```
  tasks:
    - name: Drop demo items
      block:
        - name: Drop demo project
          tower_project:
            name: "Demo Project"
            state: absent
            tower_verify_ssl: False

        - name: Drop demo credential
          tower_credential:
            name: "Demo Credential"
            organization: null
            kind: ssh
            state: absent
            tower_verify_ssl: False

        - name: Drop Demo inventory
          tower_inventory:
            name: "Demo Inventory"
            organization: null
            state: absent
            tower_verify_ssl: False

        - name: Drop Demo job template
          tower_job_template:
            name: "Demo Job Template"
            job_type: run
            playbook: null
            project: null
            state: absent
            tower_verify_ssl: False
```

All of this works against a relatively recent version of AWX, with `tower-cli` version 3.1.8, and Ansible 2.4.1.0.
@leifmadsen any chance you can try it against a Tower 3.2.1 instance?
Sorry, I don't really have access to a Tower install, just AWX.
@leifmadsen I have the same configuration as you except I use tower and not awx. And it's not ok.
I'm also seeing this bug - tower 3.2.2, tower-cli 3.2.0. Let me know if you'd like me to test anything. 

Resolving patch:
diff --git a/lib/ansible/modules/web_infrastructure/ansible_tower/tower_credential.py b/lib/ansible/modules/web_infrastructure/ansible_tower/tower_credential.py
--- a/lib/ansible/modules/web_infrastructure/ansible_tower/tower_credential.py
+++ b/lib/ansible/modules/web_infrastructure/ansible_tower/tower_credential.py
@@ -54,7 +54,7 @@
       description:
         - Type of credential being added.
       required: True
-      choices: ["ssh", "net", "scm", "aws", "rax", "vmware", "satellite6", "cloudforms", "gce", "azure", "azure_rm", "openstack"]
+      choices: ["ssh", "vault", "net", "scm", "aws", "vmware", "satellite6", "cloudforms", "gce", "azure_rm", "openstack", "rhv", "insights", "tower"]
     host:
       description:
         - Host for this credential.
@@ -165,6 +165,43 @@
     pass
 
 
+KIND_CHOICES = {
+    'ssh': 'Machine',
+    'vault': 'Ansible Vault',
+    'net': 'Network',
+    'scm': 'Source Control',
+    'aws': 'Amazon Web Services',
+    'vmware': 'VMware vCenter',
+    'satellite6': 'Red Hat Satellite 6',
+    'cloudforms': 'Red Hat CloudForms',
+    'gce': 'Google Compute Engine',
+    'azure_rm': 'Microsoft Azure Resource Manager',
+    'openstack': 'OpenStack',
+    'rhv': 'Red Hat Virtualization',
+    'insights': 'Insights',
+    'tower': 'Ansible Tower',
+}
+
+
+def credential_type_for_v1_kind(params, module):
+    credential_type_res = tower_cli.get_resource('credential_type')
+    kind = params.pop('kind')
+    arguments = {'managed_by_tower': True}
+    if kind == 'ssh':
+        if params.get('vault_password'):
+            arguments['kind'] = 'vault'
+        else:
+            arguments['kind'] = 'ssh'
+    elif kind in ('net', 'scm', 'insights', 'vault'):
+        arguments['kind'] = kind
+    elif kind in KIND_CHOICES:
+        arguments.update(dict(
+            kind='cloud',
+            name=KIND_CHOICES[kind]
+        ))
+    return credential_type_res.get(**arguments)
+
+
 def main():
 
     argument_spec = tower_argument_spec()
@@ -173,8 +210,7 @@ def main():
         user=dict(),
         team=dict(),
         kind=dict(required=True,
-                  choices=["ssh", "net", "scm", "aws", "rax", "vmware", "satellite6",
-                           "cloudforms", "gce", "azure", "azure_rm", "openstack"]),
+                  choices=KIND_CHOICES.keys()),
         host=dict(),
         username=dict(),
         password=dict(no_log=True),
@@ -213,22 +249,48 @@ def main():
         tower_check_mode(module)
         credential = tower_cli.get_resource('credential')
         try:
-            params = module.params.copy()
+            params = {}
             params['create_on_missing'] = True
+            params['name'] = name
 
             if organization:
                 org_res = tower_cli.get_resource('organization')
                 org = org_res.get(name=organization)
                 params['organization'] = org['id']
 
-            if params['ssh_key_data']:
-                filename = params['ssh_key_data']
+            credential_type = credential_type_for_v1_kind(module.params, module)
+            params['credential_type'] = credential_type['id']
+
+            if module.params.get('description'):
+                params['description'] = module.params.get('description')
+
+            if module.params.get('user'):
+                user_res = tower_cli.get_resource('user')
+                user = user_res.get(username=module.params.get('user'))
+                params['user'] = user['id']
+
+            if module.params.get('team'):
+                team_res = tower_cli.get_resource('team')
+                team = team_res.get(name=module.params.get('team'))
+                params['team'] = team['id']
+
+            params['inputs'] = {}
+            if module.params.get('ssh_key_data'):
+                filename = module.params.get('ssh_key_data')
                 if not os.path.exists(filename):
                     module.fail_json(msg='file not found: %s' % filename)
                 if os.path.isdir(filename):
                     module.fail_json(msg='attempted to read contents of directory: %s' % filename)
                 with open(filename, 'rb') as f:
-                    params['ssh_key_data'] = f.read()
+                    params['inputs']['ssh_key_data'] = f.read()
+
+            for key in ('authorize', 'authorize_password', 'client', 'secret',
+                        'tenant', 'subscription', 'domain', 'become_method',
+                        'become_username', 'become_password', 'vault_password',
+                        'project', 'host', 'username', 'password',
+                        'ssh_key_unlock'):
+                if module.params.get(key):
+                    params['inputs'][key] = module.params.get(key)
 
             if state == 'present':
                 result = credential.modify(**params)


Test patch: