Repository: ansible/ansible

Issue:
azure_rm_functionapp module doesn't work
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
azure_rm_functionapp

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.5.0
  config file = None
  configured module search path = [u'/Users/zhijzhao/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.14 (default, Sep 25 2017, 09:53:22) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.37)]
```

##### CONFIGURATION
<!---
If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"

Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).

-->
No change at all.

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->
macOS 10.13.1

##### SUMMARY
<!--- Explain the problem briefly -->
Exceptions as below when creating azure function.

TASK [create basic function app] ************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: 'AzureRMFunctionApp' object has no attribute 'base_url'
fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"/var/folders/0r/_5p333ds2q30tf1pcq6p_wc40000gn/T/ansible_Aun21J/ansible_module_azure_rm_functionapp.py\", line 297, in <module>\n    main()\n  File \"/var/folders/0r/_5p333ds2q30tf1pcq6p_wc40000gn/T/ansible_Aun21J/ansible_module_azure_rm_functionapp.py\", line 294, in main\n    AzureRMFunctionApp()\n  File \"/var/folders/0r/_5p333ds2q30tf1pcq6p_wc40000gn/T/ansible_Aun21J/ansible_module_azure_rm_functionapp.py\", line 156, in __init__\n    required_if=required_if\n  File \"/var/folders/0r/_5p333ds2q30tf1pcq6p_wc40000gn/T/ansible_Aun21J/ansible_modlib.zip/ansible/module_utils/azure_rm_common.py\", line 303, in __init__\n  File \"/var/folders/0r/_5p333ds2q30tf1pcq6p_wc40000gn/T/ansible_Aun21J/ansible_module_azure_rm_functionapp.py\", line 174, in exec_module\n    function_app = self.web_client.web_apps.get(\n  File \"/var/folders/0r/_5p333ds2q30tf1pcq6p_wc40000gn/T/ansible_Aun21J/ansible_modlib.zip/ansible/module_utils/azure_rm_common.py\", line 783, in web_client\nAttributeError: 'AzureRMFunctionApp' object has no attribute 'base_url'\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 0}

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
- hosts: localhost
  tasks:
  - name: create storage account for function apps
    azure_rm_storageaccount:
      resource_group: zhijzhao
      name: azfunccistor4
      account_type: Standard_LRS

  - name: create basic function app
    azure_rm_functionapp:
      resource_group: zhijzhao
      name: azfuncci
      storage_account: azfunccistor4

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```

```



Discussion:
@zikalino @yaweiw 
cc @tstringer
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/module_utils/azure_rm_common.py b/lib/ansible/module_utils/azure_rm_common.py
--- a/lib/ansible/module_utils/azure_rm_common.py
+++ b/lib/ansible/module_utils/azure_rm_common.py
@@ -780,12 +780,14 @@ def dns_client(self):
     def web_client(self):
         self.log('Getting web client')
         if not self._web_client:
-            self._web_client = self.get_mgmt_svc_client(WebSiteManagementClient, base_url=self.base_url)
+            self._web_client = self.get_mgmt_svc_client(WebSiteManagementClient,
+                                                        base_url=self._cloud_environment.endpoints.resource_manager)
         return self._web_client
 
     @property
     def containerservice_client(self):
         self.log('Getting container service client')
         if not self._containerservice_client:
-            self._containerservice_client = self.get_mgmt_svc_client(ContainerServiceClient)
+            self._containerservice_client = self.get_mgmt_svc_client(ContainerServiceClient,
+                                                                     base_url=self._cloud_environment.endpoints.resource_manager)
         return self._containerservice_client
diff --git a/lib/ansible/modules/cloud/azure/azure_rm_acs.py b/lib/ansible/modules/cloud/azure/azure_rm_acs.py
--- a/lib/ansible/modules/cloud/azure/azure_rm_acs.py
+++ b/lib/ansible/modules/cloud/azure/azure_rm_acs.py
@@ -5,14 +5,13 @@
 # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
 from __future__ import absolute_import, division, print_function
-__metaclass__ = type
 
+__metaclass__ = type
 
 ANSIBLE_METADATA = {'metadata_version': '1.1',
                     'status': ['preview'],
                     'supported_by': 'community'}
 
-
 DOCUMENTATION = '''
 ---
 module: azure_rm_acs
@@ -30,7 +29,6 @@
         description:
             - Name of the Container Services instance.
         required: true
-        default: null
     state:
         description:
             - Assert the state of the ACS. Use 'present' to create or update an ACS and 'absent' to delete it.
@@ -52,7 +50,6 @@
         description:
             - Master profile suboptions.
         required: true
-        default: null
         suboptions:
             count:
                 description:
@@ -62,6 +59,11 @@
                   - 1
                   - 3
                   - 5
+            vm_size:
+                description:
+                    - The VM Size of each of the Agent Pool VM's (e.g. Standard_F1 / Standard_D2v2).
+                required: true
+                version_added: 2.5
             dns_prefix:
                 description:
                   - The DNS Prefix to use for the Container Service master nodes.
@@ -70,13 +72,11 @@
         description:
             - The linux profile suboptions.
         required: true
-        default: null
         suboptions:
             admin_username:
                 description:
                   - The Admin Username for the Cluster.
                 required: true
-                default: azureuser
             ssh_key:
                 description:
                     - The Public SSH Key used to access the cluster.
@@ -85,7 +85,6 @@
         description:
             - The agent pool profile suboptions.
         required: true
-        default: null
         suboptions:
             name:
                 description:
@@ -95,7 +94,6 @@
                 description:
                     - Number of agents (VMs) to host docker containers. Allowed values must be in the range of 1 to 100 (inclusive).
                 required: true
-                default: 1
             dns_prefix:
                 description:
                     - The DNS Prefix given to Agents in this Agent Pool.
@@ -104,7 +102,6 @@
                 description:
                     - The VM Size of each of the Agent Pool VM's (e.g. Standard_F1 / Standard_D2v2).
                 required: true
-                default: Standard_D2v2
     service_principal:
         description:
             - The service principal suboptions.
@@ -123,7 +120,6 @@
         description:
             - Should VM Diagnostics be enabled for the Container Service VM's.
         required: true
-        default: false
 
 extends_documentation_fragment:
     - azure
@@ -144,6 +140,7 @@
         master_profile:
             - count: 3
               dns_prefix: acsk8smasterdns
+              vm_size: Standard_D2_v2
         linux_profile:
             - admin_username: azureuser
               ssh_key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAA...
@@ -168,6 +165,7 @@
         master_profile:
             - count: 3
               dns_prefix: acsdcosmasterdns
+              vm_size: Standard_D2_v2
         linux_profile:
             - admin_username: azureuser
               ssh_key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAA...
@@ -189,6 +187,7 @@
         master_profile:
             - count: 3
               dns_prefix: acsswarmmasterdns
+              vm_size: Standard_D2_v2
         linux_profile:
             - admin_username: azureuser
               ssh_key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAA...
@@ -214,13 +213,11 @@
         orchestration_platform: Swarm
         master_profile:
             - count: 1
+              vm_size: Standard_A0
               dns_prefix: acstestingmasterdns5
         linux_profile:
             - admin_username: azureuser
               ssh_key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAA...
-        service_principal:
-            - client_id: 7fb4173c-3ca3-4d5b-87f8-1daac941207a
-              client_secret: MPNSuM1auUuITefiLGBrpZZnLMDKBLw2
         agent_pool_profiles:
             - name: default
               count: 4
@@ -320,12 +317,17 @@ def create_ssh_configuration_instance(sshconf):
 def create_master_profile_instance(masterprofile):
     '''
     Helper method to serialize a dict to a ContainerServiceMasterProfile
+    Note: first_consecutive_static_ip is specifically set to None, for Azure server doesn't accept
+    request body with this property. This should be an inconsistency bug before Azure client SDK
+    and Azure server.
     :param: masterprofile: dict with the parameters to setup the ContainerServiceMasterProfile
     :return: ContainerServiceMasterProfile
     '''
     return ContainerServiceMasterProfile(
         count=masterprofile[0]['count'],
-        dns_prefix=masterprofile[0]['dns_prefix']
+        dns_prefix=masterprofile[0]['dns_prefix'],
+        vm_size=masterprofile[0]['vm_size'],
+        first_consecutive_static_ip=None
     )
 
 
@@ -357,7 +359,11 @@ def create_acs_dict(acs):
     :param: acs: ContainerService or AzureOperationPoller with the Azure callback object
     :return: dict with the state on Azure
     '''
-    results = dict(
+    service_principal_profile_dict = None
+    if acs.orchestrator_profile.orchestrator_type == 'Kubernetes':
+        service_principal_profile_dict = create_service_principal_profile_dict(acs.service_principal_profile)
+
+    return dict(
         id=acs.id,
         name=acs.name,
         location=acs.location,
@@ -365,13 +371,12 @@ def create_acs_dict(acs):
         orchestrator_profile=create_orchestrator_profile_dict(acs.orchestrator_profile),
         master_profile=create_master_profile_dict(acs.master_profile),
         linux_profile=create_linux_profile_dict(acs.linux_profile),
-        service_principal_profile=acs.service_principal_profile,
+        service_principal_profile=service_principal_profile_dict,
         diagnostics_profile=create_diagnotstics_profile_dict(acs.diagnostics_profile),
         provisioning_state=acs.provisioning_state,
         agent_pool_profiles=create_agent_pool_profiles_dict(acs.agent_pool_profiles),
         type=acs.type
     )
-    return results
 
 
 def create_linux_profile_dict(linuxprofile):
@@ -380,11 +385,10 @@ def create_linux_profile_dict(linuxprofile):
     :param: linuxprofile: ContainerServiceLinuxProfile with the Azure callback object
     :return: dict with the state on Azure
     '''
-    results = dict(
+    return dict(
         ssh_key=linuxprofile.ssh.public_keys[0].key_data,
         admin_username=linuxprofile.admin_username
     )
-    return results
 
 
 def create_master_profile_dict(masterprofile):
@@ -393,12 +397,24 @@ def create_master_profile_dict(masterprofile):
     :param: masterprofile: ContainerServiceMasterProfile with the Azure callback object
     :return: dict with the state on Azure
     '''
-    results = dict(
+    return dict(
         count=masterprofile.count,
         fqdn=masterprofile.fqdn,
+        vm_size=masterprofile.vm_size,
         dns_prefix=masterprofile.dns_prefix
     )
-    return results
+
+
+def create_service_principal_profile_dict(serviceprincipalprofile):
+    '''
+    Helper method to deserialize a ContainerServiceServicePrincipalProfile to a dict
+    Note: For security reason, the service principal secret is skipped on purpose.
+    :param: serviceprincipalprofile: ContainerServiceServicePrincipalProfile with the Azure callback object
+    :return: dict with the state on Azure
+    '''
+    return dict(
+        client_id=serviceprincipalprofile.client_id
+    )
 
 
 def create_diagnotstics_profile_dict(diagnosticsprofile):
@@ -407,10 +423,9 @@ def create_diagnotstics_profile_dict(diagnosticsprofile):
     :param: diagnosticsprofile: ContainerServiceVMDiagnostics with the Azure callback object
     :return: dict with the state on Azure
     '''
-    results = dict(
+    return dict(
         vm_diagnostics=diagnosticsprofile.vm_diagnostics.enabled
     )
-    return results
 
 
 def create_orchestrator_profile_dict(orchestratorprofile):
@@ -419,10 +434,9 @@ def create_orchestrator_profile_dict(orchestratorprofile):
     :param: orchestratorprofile: ContainerServiceOrchestratorProfile with the Azure callback object
     :return: dict with the state on Azure
     '''
-    results = dict(
+    return dict(
         orchestrator_type=str(orchestratorprofile.orchestrator_type)
     )
-    return results
 
 
 def create_agent_pool_profiles_dict(agentpoolprofiles):
@@ -431,17 +445,13 @@ def create_agent_pool_profiles_dict(agentpoolprofiles):
     :param: agentpoolprofiles: ContainerServiceAgentPoolProfile with the Azure callback object
     :return: dict with the state on Azure
     '''
-    results = []
-    for profile in agentpoolprofiles:
-        result = dict(
-            count=profile.count,
-            vm_size=profile.vm_size,
-            name=profile.name,
-            dns_prefix=profile.dns_prefix,
-            fqdn=profile.fqdn
-        )
-        results.append(result)
-    return results
+    return [dict(
+        count=profile.count,
+        vm_size=profile.vm_size,
+        name=profile.name,
+        dns_prefix=profile.dns_prefix,
+        fqdn=profile.fqdn
+    ) for profile in agentpoolprofiles]
 
 
 class AzureRMContainerService(AzureRMModuleBase):
@@ -563,16 +573,33 @@ def exec_module(self, **kwargs):
                     if update_tags:
                         to_be_updated = True
 
+                    def is_property_changed(profile, property, ignore_case=False):
+                        base = response[profile].get(property)
+                        new = getattr(self, profile)[0].get(property)
+                        if ignore_case:
+                            return base.lower() != new.lower()
+                        else:
+                            return base != new
+
                     # Cannot Update the master count for now // Uncomment this block in the future to support it
-                    if response['master_profile'].get('count') != self.master_profile[0].get('count'):
+                    if is_property_changed('master_profile', 'count'):
                         # self.log(("Master Profile Count Diff, Was {0} / Now {1}"
                         #           .format(response['master_profile'].count,
                         #           self.master_profile[0].get('count'))))
                         # to_be_updated = True
                         self.module.warn("master_profile.count cannot be updated")
 
+                    # Cannot Update the master vm_size for now. Could be a client SDK bug
+                    # Uncomment this block in the future to support it
+                    if is_property_changed('master_profile', 'vm_size', True):
+                        # self.log(("Master Profile VM Size Diff, Was {0} / Now {1}"
+                        #           .format(response['master_profile'].get('vm_size'),
+                        #                   self.master_profile[0].get('vm_size'))))
+                        # to_be_updated = True
+                        self.module.warn("master_profile.vm_size cannot be updated")
+
                     # Cannot Update the SSH Key for now // Uncomment this block in the future to support it
-                    if response['linux_profile'].get('ssh_key') != self.linux_profile[0].get('ssh_key'):
+                    if is_property_changed('linux_profile', 'ssh_key'):
                         # self.log(("Linux Profile Diff SSH, Was {0} / Now {1}"
                         #          .format(response['linux_profile'].ssh.public_keys[0].key_data,
                         #          self.linux_profile[0].get('ssh_key'))))
@@ -582,7 +609,7 @@ def exec_module(self, **kwargs):
                     # self.log("linux_profile response : {0}".format(response['linux_profile'].get('admin_username')))
                     # self.log("linux_profile self : {0}".format(self.linux_profile[0].get('admin_username')))
                     # Cannot Update the Username for now // Uncomment this block in the future to support it
-                    if response['linux_profile'].get('admin_username') != self.linux_profile[0].get('admin_username'):
+                    if is_property_changed('linux_profile', 'admin_username'):
                         # self.log(("Linux Profile Diff User, Was {0} / Now {1}"
                         #          .format(response['linux_profile'].admin_username,
                         #          self.linux_profile[0].get('admin_username'))))
@@ -599,10 +626,11 @@ def exec_module(self, **kwargs):
                         for profile_self in self.agent_pool_profiles:
                             if profile_result['name'] == profile_self['name']:
                                 matched = True
-                                if profile_result['count'] != profile_self['count'] or profile_result['vm_size'] != profile_self['vm_size']:
+                                if profile_result['count'] != profile_self['count'] or profile_result['vm_size'] != \
+                                        profile_self['vm_size']:
                                     self.log(("Agent Profile Diff - Count was {0} / Now {1} - Vm_size was {2} / Now {3}"
-                                             .format(profile_result['count'], profile_self['count'],
-                                              profile_result['vm_size'], profile_self['vm_size'])))
+                                              .format(profile_result['count'], profile_self['count'],
+                                                      profile_result['vm_size'], profile_self['vm_size'])))
                                     to_be_updated = True
                         if not matched:
                             self.log("Agent Pool not found")
@@ -666,7 +694,8 @@ def create_update_acs(self):
         # self.log("vm_diagnostics : {0}".format(parameters.diagnostics_profile.vm_diagnostics))
 
         try:
-            poller = self.containerservice_client.container_services.create_or_update(self.resource_group, self.name, parameters)
+            poller = self.containerservice_client.container_services.create_or_update(self.resource_group, self.name,
+                                                                                      parameters)
             response = self.get_poller_result(poller)
         except CloudError as exc:
             self.log('Error attempting to create the ACS instance.')
@@ -717,5 +746,6 @@ def main():
     """Main execution"""
     AzureRMContainerService()
 
+
 if __name__ == '__main__':
     main()


Test patch:
diff --git a/test/runner/requirements/integration.cloud.azure.txt b/test/runner/requirements/integration.cloud.azure.txt
--- a/test/runner/requirements/integration.cloud.azure.txt
+++ b/test/runner/requirements/integration.cloud.azure.txt
@@ -13,4 +13,4 @@ azure-mgmt-keyvault>=0.40.0,<0.41
 azure-mgmt-batch>=4.1.0,<5
 azure-mgmt-sql>=0.7.1,<0.8
 azure-mgmt-web>=0.32.0,<0.33
-azure-mgmt-containerservice>=1.0.0
\ No newline at end of file
+azure-mgmt-containerservice>=2.0.0,<3.0.0
\ No newline at end of file