Repository: ansible/ansible

Issue:
ANSIBLE_AZURE_AUTH_SOURCE no long seems to be picking up cli authentication
##### SUMMARY
With the release of 2.6 it seems that the azure auth source configuration is no longer picking up the cli credentials when the client_id and secret are specified. Previously, the context ignored the client_id and secret when the auth source was set to cli.
##### ISSUE TYPE
 - Bug Report
##### COMPONENT NAME
all azure modules

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
2.6.0
```
##### CONFIGURATION
ansible-config 2.6.0
  config file = /home/smgv/.ansible.cfg
  configured module search path = [u'/home/smgv/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/smgv/.ansible-virtualenv/lib/python2.7/site-packages/ansible
  executable location = /home/smgv/.ansible-virtualenv/bin/ansible-config
  python version = 2.7.5 (default, Aug  4 2017, 00:39:18) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]
Using /home/smgv/.ansible.cfg as config file
DEFAULT_CALLBACK_WHITELIST(/home/smgv/.ansible.cfg) = ['profile_task', 'profile_roles', 'timer\x13\x11']
DEFAULT_VERBOSITY(env: ANSIBLE_VERBOSITY) = 3
HOST_KEY_CHECKING(/home/smgv/.ansible.cfg) = False
RETRY_FILES_ENABLED(env: ANSIBLE_RETRY_FILES_ENABLED) = False

$echo $ANSIBLE_AZURE_AUTH_SOURCE
cli

##### OS / ENVIRONMENT
CentOS 7
##### STEPS TO REPRODUCE
$ export $ANSIBLE_AZURE_AUTH_SOURCE=cli
$ az login

Then run the  play book:

```yaml
---
- hosts: localhost
  connection: local

  tasks:

  - name: Create Azure Resource Group
    azure_rm_resourcegroup:
      name: testing-smgv
      location: southcentralus
      client_id: "<shouldn't matter>"
      subscription_id: "<sub_id>"
      tenant: "<tenant_id>"
      secret: "<shouldn't matter>"
```
##### EXPECTED RESULTS
down grading to Ansible 2.5.0
```bash
TASK [Create Azure Resource Group] *************************************************************************************************************************************************************
task path: /home/########################/TomCat/test.yml:8
Wednesday 11 July 2018  14:26:00 +0000 (0:00:12.963)       0:00:13.042 ********
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/azure_rm_common.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/basic.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/six/__init__.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/_text.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/parsing/convert_bool.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/parsing/__init__.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/pycompat24.py
Using module file /usr/lib/python2.7/site-packages/ansible/modules/cloud/azure/azure_rm_resourcegroup.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: ########################
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/########################/.ansible/tmp/ansible-tmp-1531319161.09-110646833200342 `" && echo ansible-tmp-1531319161.09-110646833200342="` echo /home/########################/.ansible/tmp/ansible-tmp-1531319161.09-110646833200342 `" ) && sleep 0'
<127.0.0.1> PUT /home/########################/.ansible/tmp/ansible-local-10001Hic6vi/tmpbyt_o_ TO /home/########################/.ansible/tmp/ansible-tmp-1531319161.09-110646833200342/azure_rm_resourcegroup.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/########################/.ansible/tmp/ansible-tmp-1531319161.09-110646833200342/ /home/########################/.ansible/tmp/ansible-tmp-1531319161.09-110646833200342/azure_rm_resourcegroup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python2 /home/########################/.ansible/tmp/ansible-tmp-1531319161.09-110646833200342/azure_rm_resourcegroup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/########################/.ansible/tmp/ansible-tmp-1531319161.09-110646833200342/ > /dev/null 2>&1 && sleep 0'
changed: [localhost] => {
    "changed": true,
    "contains_resources": false,
    "invocation": {
        "module_args": {
            "ad_user": null,
            "api_profile": "latest",
            "append_tags": true,
            "auth_source": null,
            "cert_validation_mode": null,
            "client_id": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "cloud_environment": null,
            "force": false,
            "location": "southcentralus",
            "name": "testing",
            "password": null,
            "profile": null,
            "secret": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "state": "present",
            "subscription_id": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "tags": null,
            "tenant": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER"
        }
    },
    "state": {
        "id": "/subscriptions/********/resourceGroups/testing",
        "location": "southcentralus",
        "name": "testing",
        "provisioning_state": "Succeeded",
        "tags": null
    }
}
META: ran handlers
META: ran handlers

PLAY RECAP *************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0

```
##### ACTUAL RESULTS
```bash
TASK [Create Azure Resource Group] *************************************************************************************************************************************************************
task path:########################
Wednesday 11 July 2018  14:23:07 +0000 (0:00:13.029)       0:00:13.112 ********
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: ########################
<127.0.0.1> EXEC /bin/sh -c 'echo ~######################## && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/########################/.ansible/tmp/ansible-tmp-1531318987.68-179767745454717 `" && echo ansible-tmp-1531318987.68-179767745454717="` echo /home/########################/.ansible/tmp/ansible-tmp-1531318987.68-179767745454717 `" ) && sleep 0'
Using module file /home/########################/.ansible-virtualenv/lib/python2.7/site-packages/ansible/modules/cloud/azure/azure_rm_resourcegroup.py
<127.0.0.1> PUT /home/########################/.ansible/tmp/ansible-local-9630F2fBp3/tmp1bkotM TO /home/########################/.ansible/tmp/ansible-tmp-1531318987.68-179767745454717/azure_rm_resourcegroup.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/########################/.ansible/tmp/ansible-tmp-1531318987.68-179767745454717/ /home/########################/.ansible/tmp/ansible-tmp-1531318987.68-179767745454717/azure_rm_resourcegroup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/home/########################/.ansible-virtualenv/bin/python2 /home/########################/.ansible/tmp/ansible-tmp-1531318987.68-179767745454717/azure_rm_resourcegroup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/########################/.ansible/tmp/ansible-tmp-1531318987.68-179767745454717/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_bG6pqU/ansible_module_azure_rm_resourcegroup.py", line 255, in <module>
    main()
  File "/tmp/ansible_bG6pqU/ansible_module_azure_rm_resourcegroup.py", line 252, in main
    AzureRMResourceGroup()
  File "/tmp/ansible_bG6pqU/ansible_module_azure_rm_resourcegroup.py", line 136, in __init__
    supports_tags=True)
  File "/tmp/ansible_bG6pqU/ansible_modlib.zip/ansible/module_utils/azure_rm_common.py", line 348, in __init__
  File "/home/########################/.ansible-virtualenv/lib/python2.7/site-packages/msrestazure/azure_active_directory.py", line 453, in __init__
    self.set_token()
  File "/home/########################/.ansible-virtualenv/lib/python2.7/site-packages/msrestazure/azure_active_directory.py", line 480, in set_token
    raise_with_traceback(AuthenticationError, "", err)
  File "/home/########################/.ansible-virtualenv/lib/python2.7/site-packages/msrest/exceptions.py", line 48, in raise_with_traceback
    raise error
msrest.exceptions.AuthenticationError: , UnauthorizedClientError: (unauthorized_client) AADSTS70001: Application with identifier 'something' was not found in the directory ########################
Trace ID: 71555177-6213-4471-adc2-0174c2df1000
Correlation ID: e7470409-f586-40d0-8c8d-1089d22e5319
Timestamp: 2018-07-11 14:23:38Z

fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_bG6pqU/ansible_module_azure_rm_resourcegroup.py\", line 255, in <module>\n    main()\n  File \"/tmp/ansible_bG6pqU/ansible_module_azure_rm_resourcegroup.py\", line 252, in main\n    AzureRMResourceGroup()\n  File \"/tmp/ansible_bG6pqU/ansible_module_azure_rm_resourcegroup.py\", line 136, in __init__\n    supports_tags=True)\n  File \"/tmp/ansible_bG6pqU/ansible_modlib.zip/ansible/module_utils/azure_rm_common.py\", line 348, in __init__\n  File \"/home/########################/.ansible-virtualenv/lib/python2.7/site-packages/msrestazure/azure_active_directory.py\", line 453, in __init__\n    self.set_token()\n  File \"/home/########################/.ansible-virtualenv/lib/python2.7/site-packages/msrestazure/azure_active_directory.py\", line 480, in set_token\n    raise_with_traceback(AuthenticationError, \"\", err)\n  File \"/home/########################/.ansible-virtualenv/lib/python2.7/site-packages/msrest/exceptions.py\", line 48, in raise_with_traceback\n    raise error\nmsrest.exceptions.AuthenticationError: , UnauthorizedClientError: (unauthorized_client) AADSTS70001: Application with identifier 'something' was not found in the directory chevron.onmicrosoft.com\r\nTrace ID: 71555177-6213-4471-adc2-0174c2df1000\r\nCorrelation ID: e7470409-f586-40d0-8c8d-1089d22e5319\r\nTimestamp: 2018-07-11 14:23:38Z\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 1
}
```



Discussion:
Files identified in the description:
* [lib/ansible/modules/cloud/azure](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/azure)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
Hrm, yep, looks like a side-effect of changing the auth_source argspec default to `'auto'` from `None` in  b20a88c39d5f144077564407fe438013551ef36e - means we're never actually consulting the environment variable anymore. That change had to be made to keep our docs<->argspec compliance test happy, but in this case we need to figure out how to keep the "logical" default of `auto` while keeping the compliance test happy. I think the answer is to remove both the argspec and doc defaults, and just mention in the docs that if it's not specified, we'll consult `ANSIBLE_AZURE_AUTH_SOURCE` and use `auto` if it's not found. 

@zikalino - do you agree with that?
@nitzmahone yes, i definitely agree. i can create pr and backport pr for 2.6 for this.
but can we remove default value from the doc entirely?
Yes, that'd be what we'd have to do to keep the compliance test happy (so just document the logical default and the env fallback in the description of the `auth_source` arg)

Resolving patch:
diff --git a/lib/ansible/module_utils/azure_rm_common.py b/lib/ansible/module_utils/azure_rm_common.py
--- a/lib/ansible/module_utils/azure_rm_common.py
+++ b/lib/ansible/module_utils/azure_rm_common.py
@@ -23,8 +23,7 @@
 AZURE_COMMON_ARGS = dict(
     auth_source=dict(
         type='str',
-        choices=['auto', 'cli', 'env', 'credential_file', 'msi'],
-        default='auto'
+        choices=['auto', 'cli', 'env', 'credential_file', 'msi']
     ),
     profile=dict(type='str'),
     subscription_id=dict(type='str', no_log=True),
diff --git a/lib/ansible/utils/module_docs_fragments/azure.py b/lib/ansible/utils/module_docs_fragments/azure.py
--- a/lib/ansible/utils/module_docs_fragments/azure.py
+++ b/lib/ansible/utils/module_docs_fragments/azure.py
@@ -70,6 +70,7 @@ class ModuleDocFragment(object):
     auth_source:
         description:
             - Controls the source of the credentials to use for authentication.
+            - If not specified, ANSIBLE_AZURE_AUTH_SOURCE environment variable will be used and default to C(auto) if variable is not defined.
             - C(auto) will follow the default precedence of module parameters -> environment variables -> default profile in credential file
               C(~/.azure/credentials).
             - When set to C(cli), the credentials will be sources from the default Azure CLI profile.
@@ -84,7 +85,6 @@ class ModuleDocFragment(object):
         - credential_file
         - env
         - msi
-        default: auto
         version_added: 2.5
     api_profile:
         description:


Test patch: