Repository: ansible/ansible

Issue:
azure_rm_virtualmachine: creating data_disks fails when storage container is specified
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
cloud/azure/azure_rm_virtualmachine
<!--- Name of the module/plugin/task/feature -->


##### ANSIBLE VERSION
```
ansible 2.4.0 (devel 7cc3c0577e) last updated 2017/08/31 09:28:15 (GMT +900)
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/gamlor/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/gamlor/ansible/lib/ansible
  executable location = /home/gamlor/ansible/bin/ansible
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]
```
##### CONFIGURATION
Default configuration


##### OS / ENVIRONMENT
Ubuntu 16.04 / NA

##### SUMMARY
Creating an Azure virtual machine with a new data disk fails with 'local variable 'data_disk_storage_account' referenced before assignment'

##### STEPS TO REPRODUCE
1. Have a playbook which first creates an Azure storage account
2. Then create a Azure virtual machine, which a data disk, which uses that storage account   
3. Run the playbook



<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
- hosts: localhost
  connection: local
  tasks:
  - name: Create test roup
    azure_rm_resourcegroup:
      name: Testing
      location: koreacentral
  - name: Create newtork
    azure_rm_virtualnetwork:
      name: testnetwork
      resource_group: Testing
      address_prefixes_cidr:
          - "10.0.0.0/16"
      dns_servers:
          - "8.8.8.8"
          - "8.8.4.4"
  - name: Create subnet
    azure_rm_subnet:
      name: testsubnet
      virtual_network_name: testnetwork
      resource_group: Testing
      address_prefix_cidr: "10.0.2.0/24"
## Required storage. The crash occurs when using a existing storage
  - name: Storage account for virtual machine images
    azure_rm_storageaccount:
      resource_group: Testing
      name: gamlorteststore
      type: Standard_RAGRS
  - name: Create storage container for the images
    azure_rm_storageblob:
      resource_group: Testing
      storage_account_name: gamlorteststore
      container: testcontainer

## Crashing virtual machine creation
  - name: Create virtual machine
    azure_rm_virtualmachine:
      resource_group: Testing
      name: Testing-Machine
      admin_username: testing
      admin_password: Test!Passw0rd
      vm_size: Basic_A0
      storage_account_name: gamlorteststore
      storage_container_name: testcontainer
      os_type: Windows
      data_disks:
        - lun: 0
          storage_account_name: gamlorteststore
          storage_container_name: testcontainer
          storage_blob_name: 'data-disk.vhd'
          disk_size_gb: 80
      image:
        publisher: MicrosoftWindowsServer
        offer: WindowsServer
        sku: 2012-R2-Datacenter
        version: latest
```

##### EXPECTED RESULTS
- Virtual Machine is created
- Datadisk is created
- Data disk is attached to virtual machine

##### ACTUAL RESULTS
Ansible crashes with error 

```
The full traceback is:                                                                                                       │ .../modules/remote_management/oneview/test_oneview_san_manager.py | 239 ++++++++++++++++++++++++++
Traceback (most recent call last):                                                                                           │ 67 files changed, 1975 insertions(+), 217 deletions(-)
  File "/tmp/ansible_HDBf4r/ansible_module_azure_rm_virtualmachine.py", line 1524, in <module>                               │ create mode 100644 lib/ansible/modules/network/aci/aci_config_rollback.py
    main()                                                                                                                   │ create mode 100644 lib/ansible/modules/network/aci/aci_config_snapshot.py
  File "/tmp/ansible_HDBf4r/ansible_module_azure_rm_virtualmachine.py", line 1521, in main                                   │ create mode 100644 lib/ansible/modules/remote_management/oneview/oneview_ethernet_network_facts.py
    AzureRMVirtualMachine()                                                                                                  │ create mode 100644 lib/ansible/modules/remote_management/oneview/oneview_fcoe_network_facts.py
  File "/tmp/ansible_HDBf4r/ansible_module_azure_rm_virtualmachine.py", line 652, in __init__                                │ create mode 100644 lib/ansible/modules/remote_management/oneview/oneview_network_set_facts.py
    supports_check_mode=True)                                                                                                │ create mode 100644 lib/ansible/modules/remote_management/oneview/oneview_san_manager.py
  File "/tmp/ansible_HDBf4r/ansible_modlib.zip/ansible/module_utils/azure_rm_common.py", line 255, in __init__               │ rename test/integration/targets/{ec2_facts => ec2_metadata_facts}/aliases (100%)
  File "/tmp/ansible_HDBf4r/ansible_module_azure_rm_virtualmachine.py", line 908, in exec_module                             │ rename test/integration/targets/{ec2_facts => ec2_metadata_facts}/defaults/main.yml (100%)
    data_disk_storage_account.name = data_disk['storage_account_name']                                                       │ rename test/integration/targets/{ec2_facts => ec2_metadata_facts}/meta/main.yml (100%)
UnboundLocalError: local variable 'data_disk_storage_account' referenced before assignment                                   │ rename test/integration/targets/{ec2_facts => ec2_metadata_facts}/tasks/main.yml (100%)
                                                                                                                             │ rename test/integration/targets/{ec2_facts => ec2_metadata_facts}/vars/main.yml (100%)
fatal: [localhost]: FAILED! => {                                                                                             │ rename test/integration/targets/{ec2_elb_lb => elb_classic_lb}/aliases (100%)
    "changed": false,                                                                                                        │ rename test/integration/targets/{ec2_elb_lb => elb_classic_lb}/defaults/main.yml (100%)
    "failed": true,                                                                                                          │ rename test/integration/targets/{ec2_elb_lb => elb_classic_lb}/meta/main.yml (100%)
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_HDBf4r/ansible_module_azure_rm_virtualmachine│ rename test/integration/targets/{ec2_elb_lb => elb_classic_lb}/tasks/main.yml (100%)
.py\", line 1524, in <module>\n    main()\n  File \"/tmp/ansible_HDBf4r/ansible_module_azure_rm_virtualmachine.py\", line 152│ rename test/integration/targets/{ec2_elb_lb => elb_classic_lb}/vars/main.yml (100%)
1, in main\n    AzureRMVirtualMachine()\n  File \"/tmp/ansible_HDBf4r/ansible_module_azure_rm_virtualmachine.py\", line 652, │ create mode 100644 test/integration/targets/github_issue/aliases
in __init__\n    supports_check_mode=True)\n  File \"/tmp/ansible_HDBf4r/ansible_modlib.zip/ansible/module_utils/azure_rm_com│ create mode 100644 test/integration/targets/github_issue/tasks/main.yml
mon.py\", line 255, in __init__\n  File \"/tmp/ansible_HDBf4r/ansible_module_azure_rm_virtualmachine.py\", line 908, in exec_│ create mode 100644 test/integration/targets/github_issue/vars/main.yml
module\n    data_disk_storage_account.name = data_disk['storage_account_name']\nUnboundLocalError: local variable 'data_disk_│ rename test/integration/targets/{aws_lambda_policy => lambda_policy}/aliases (100%)
storage_account' referenced before assignment\n",                                                                            │ rename test/integration/targets/{aws_lambda_policy => lambda_policy}/defaults/main.yml (100%)
    "module_stdout": "",                                                                                                     │ rename test/integration/targets/{aws_lambda_policy => lambda_policy}/files/mini_http_lambda.py (100%)
    "msg": "MODULE FAILURE",                                                                                                 │ rename test/integration/targets/{aws_lambda_policy => lambda_policy}/tasks/main.yml (100%)
    "rc": 0                                                                                                                  │ rename test/integration/targets/{aws_lambda_policy => lambda_policy}/templates/endpoint-test-swagger-ap
}
```



Discussion:
cc @haroldwongms @nitzmahone @tstringer
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Fix PR: https://github.com/ansible/ansible/pull/28860

Resolving patch:
diff --git a/lib/ansible/modules/cloud/azure/azure_rm_virtualmachine.py b/lib/ansible/modules/cloud/azure/azure_rm_virtualmachine.py
--- a/lib/ansible/modules/cloud/azure/azure_rm_virtualmachine.py
+++ b/lib/ansible/modules/cloud/azure/azure_rm_virtualmachine.py
@@ -71,8 +71,7 @@
     vm_size:
         description:
             - A valid Azure VM size value. For example, 'Standard_D4'. The list of choices varies depending on the
-              subscription and location. Check your subscription for available choices.
-        required: true
+              subscription and location. Check your subscription for available choices. Required when creating a VM.
     admin_username:
         description:
             - Admin username used to access the host after it is created. Required when creating a VM.
@@ -582,7 +581,7 @@ def __init__(self):
             state=dict(choices=['present', 'absent'], default='present', type='str'),
             location=dict(type='str'),
             short_hostname=dict(type='str'),
-            vm_size=dict(type='str', required=True),
+            vm_size=dict(type='str'),
             admin_username=dict(type='str'),
             admin_password=dict(type='str', no_log=True),
             ssh_password_enabled=dict(type='bool', default=True),
@@ -761,7 +760,7 @@ def exec_module(self, **kwargs):
                     changed = True
                     vm_dict['properties']['osProfile']['computerName'] = self.short_hostname
 
-                if self.started and vm_dict['powerstate'] != 'running':
+                if self.started and vm_dict['powerstate'] not in ['starting', 'running'] and self.allocated:
                     self.log("CHANGED: virtual machine {0} not running and requested state 'running'".format(self.name))
                     changed = True
                     powerstate_change = 'poweron'
@@ -770,7 +769,7 @@ def exec_module(self, **kwargs):
                              .format(self.name, vm_dict['powerstate']))
                     changed = True
                     powerstate_change = 'restarted'
-                elif self.state == 'present' and not self.allocated and vm_dict['powerstate'] != 'deallocated':
+                elif self.state == 'present' and not self.allocated and vm_dict['powerstate'] not in ['deallocated', 'deallocating']:
                     self.log("CHANGED: virtual machine {0} {1} and requested state 'deallocated'"
                              .format(self.name, vm_dict['powerstate']))
                     changed = True
@@ -802,6 +801,7 @@ def exec_module(self, **kwargs):
 
         if changed:
             if self.state == 'present':
+                default_storage_account = None
                 if not vm:
                     # Create the VM
                     self.log("Create virtual machine {0}".format(self.name))
@@ -835,6 +835,7 @@ def exec_module(self, **kwargs):
                             self._cloud_environment.suffixes.storage_endpoint,
                             self.storage_container_name,
                             self.storage_blob_name)
+                        default_storage_account = storage_account  # store for use by data disks if necessary
 
                     if not self.short_hostname:
                         self.short_hostname = self.name
@@ -904,18 +905,22 @@ def exec_module(self, **kwargs):
                                     count += 1
 
                                 if data_disk.get('storage_account_name'):
-                                    self.get_storage_account(data_disk['storage_account_name'])
-                                    data_disk_storage_account.name = data_disk['storage_account_name']
+                                    data_disk_storage_account = self.get_storage_account(data_disk['storage_account_name'])
                                 else:
-                                    data_disk_storage_account = self.create_default_storage_account()
-                                    self.log("data disk storage account:")
-                                    self.log(self.serialize_obj(data_disk_storage_account, 'StorageAccount'), pretty_print=True)
+                                    if(not default_storage_account):
+                                        data_disk_storage_account = self.create_default_storage_account()
+                                        self.log("data disk storage account:")
+                                        self.log(self.serialize_obj(data_disk_storage_account, 'StorageAccount'), pretty_print=True)
+                                        default_storage_account = data_disk_storage_account  # store for use by future data disks if necessary
+                                    else:
+                                        data_disk_storage_account = default_storage_account
 
                                 if not data_disk.get('storage_container_name'):
                                     data_disk['storage_container_name'] = 'vhds'
 
-                                data_disk_requested_vhd_uri = 'https://{0}.blob.core.windows.net/{1}/{2}'.format(
+                                data_disk_requested_vhd_uri = 'https://{0}.blob.{1}/{2}/{3}'.format(
                                     data_disk_storage_account.name,
+                                    self._cloud_environment.suffixes.storage_endpoint,
                                     data_disk['storage_container_name'],
                                     data_disk['storage_blob_name']
                                 )
@@ -1182,20 +1187,31 @@ def deallocate_vm(self):
 
     def delete_vm(self, vm):
         vhd_uris = []
+        managed_disk_ids = []
         nic_names = []
         pip_names = []
 
         if self.remove_on_absent.intersection(set(['all','virtual_storage'])):
             # store the attached vhd info so we can nuke it after the VM is gone
-            self.log('Storing VHD URI for deletion')
-            vhd_uris.append(vm.storage_profile.os_disk.vhd.uri)
+            if(vm.storage_profile.os_disk.managed_disk):
+                self.log('Storing managed disk ID for deletion')
+                managed_disk_ids.append(vm.storage_profile.os_disk.managed_disk.id)
+            elif(vm.storage_profile.os_disk.vhd):
+                self.log('Storing VHD URI for deletion')
+                vhd_uris.append(vm.storage_profile.os_disk.vhd.uri)
 
             data_disks = vm.storage_profile.data_disks
             for data_disk in data_disks:
-                vhd_uris.append(data_disk.vhd.uri)
+                if(data_disk.vhd):
+                    vhd_uris.append(data_disk.vhd.uri)
+                elif(data_disk.managed_disk):
+                    managed_disk_ids.append(data_disk.managed_disk.id)
 
+            # FUTURE enable diff mode, move these there...
             self.log("VHD URIs to delete: {0}".format(', '.join(vhd_uris)))
             self.results['deleted_vhd_uris'] = vhd_uris
+            self.log("Managed disk IDs to delete: {0}".format(', '.join(managed_disk_ids)))
+            self.results['deleted_managed_disk_ids'] = managed_disk_ids
 
         if self.remove_on_absent.intersection(set(['all','network_interfaces'])):
             # store the attached nic info so we can nuke them after the VM is gone
@@ -1228,8 +1244,10 @@ def delete_vm(self, vm):
         # TODO: parallelize nic, vhd, and public ip deletions with begin_deleting
         # TODO: best-effort to keep deleting other linked resources if we encounter an error
         if self.remove_on_absent.intersection(set(['all','virtual_storage'])):
-            self.log('Deleting virtual storage')
+            self.log('Deleting VHDs')
             self.delete_vm_storage(vhd_uris)
+            self.log('Deleting managed disks')
+            self.delete_managed_disks(managed_disk_ids)
 
         if self.remove_on_absent.intersection(set(['all','network_interfaces'])):
             self.log('Deleting network interfaces')
@@ -1270,7 +1288,16 @@ def delete_pip(self, name):
         # Delete returns nada. If we get here, assume that all is well.
         return True
 
+    def delete_managed_disks(self, managed_disk_ids):
+        for mdi in managed_disk_ids:
+            try:
+                poller = self.rm_client.resources.delete_by_id(mdi, '2017-03-30')
+                self.get_poller_result(poller)
+            except Exception as exc:
+                self.fail("Error deleting managed disk {0} - {1}".format(mdi, str(exc)))
+
     def delete_vm_storage(self, vhd_uris):
+        # FUTURE: figure out a cloud_env indepdendent way to delete these
         for uri in vhd_uris:
             self.log("Extracting info from blob uri '{0}'".format(uri))
             try:
@@ -1354,7 +1381,7 @@ def create_default_storage_account(self):
         valid_name = False
 
         # Attempt to find a valid storage account name
-        storage_account_name_base = self.name[:20].lower()
+        storage_account_name_base = re.sub('[^a-zA-Z0-9]', '', self.name[:20].lower())
         for i in range(0, 5):
             rand = random.randrange(1000, 9999)
             storage_account_name = storage_account_name_base + str(rand)
@@ -1392,8 +1419,11 @@ def check_storage_account_name(self, name):
         self.log("Checking storage account name availability for {0}".format(name))
         try:
             response = self.storage_client.storage_accounts.check_name_availability(name)
+            if response.reason == 'AccountNameInvalid':
+                raise Exception("Invalid default storage account name: {0}".format(name))
         except Exception as exc:
             self.fail("Error checking storage account name availability for {0} - {1}".format(name, str(exc)))
+
         return response.name_available
 
     def create_default_nic(self):


Test patch: