Repository: ansible/ansible

Issue:
azure_rm_virtualmachine with network interface and security group in another resource group
<!---
Verify first that your issue/request is not already reported on GitHub.
THIS FORM WILL BE READ BY A MACHINE, COMPLETE ALL SECTIONS AS DESCRIBED.
Also test if the latest release, and devel branch are affected too.
ALWAYS add information AFTER (OUTSIDE) these html comments.
Otherwise it may end up being automatically closed by our bot. -->

##### SUMMARY
<!--- Explain the problem briefly -->
One of customer asked for below scenario:
create a azure VM with network interface and security group in another resource group.

##### ISSUE TYPE
 - Feature Idea

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->
azure_rm_virtualmachine

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.5.2
```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->

##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->

##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->

<!--- Paste example playbooks or commands between quotes below -->
```yaml

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```

```

azure_rm_virtualmachine dosn't use virtual_network_resource_group value when validating resource groups on VM create
<!---
Verify first that your issue/request is not already reported on GitHub.
THIS FORM WILL BE READ BY A MACHINE, COMPLETE ALL SECTIONS AS DESCRIBED.
Also test if the latest release, and devel branch are affected too.
ALWAYS add information AFTER (OUTSIDE) these html comments.
Otherwise it may end up being automatically closed by our bot. -->

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->

azure_rm_virtualmachine

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.5.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/rprasad34/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, May  3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]

```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->
DEFAULT_FORKS(/etc/ansible/ansible.cfg) = 100
DEFAULT_POLL_INTERVAL(/etc/ansible/ansible.cfg) = 15
DEFAULT_ROLES_PATH(/etc/ansible/ansible.cfg) = [u'/etc/ansible/roles', u'/usr/share/ansible/roles']
DEFAULT_STDOUT_CALLBACK(/etc/ansible/ansible.cfg) = debug
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False

##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->
Linux RHEL 7.5
##### SUMMARY
<!--- Explain the problem briefly -->
When creating a VM machine using an existing network interface in a different resource group and supplying   "virtual_network_resource_group" and  "virtual_network_name" to azure_rm_virtualmachine. The machine fails to create with the following error:
````
Error fetching network interface mynetworkinterface123 - Azure Error: ResourceNotFound
Message: The Resource 'Microsoft.Network/networkInterfaces/mynetworkinterface123' under resource group 'MyComputeResourceGroup' was not found.
````
The  "def get_network_interface" function in azure_rm_virtualmachine.py  assumes it's under the same resource group.

##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
  tasks:
  - name: Create a network interface using existing security group and disable public IP
    azure_rm_networkinterface:
      name: mynetworkinterface123
      resource_group: myNetWorkResourceGroup
      virtual_network_name: MyVNet
      subnet_name:  MySubnet
      security_group_name: MyNSG
      public_ip: false      

  - name: Create VM
    azure_rm_virtualmachine:
      resource_group: MyComputeResourceGroup
      location: somewhere
      virtual_network_resource_group: myNetWorkResourceGroup
      virtual_network_name: MyVNet
      network_interface_names: mynetworkinterface123
      subnet_name:  MySubnet
      name: MyHostName123

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->

success

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
Error fetching network interface mynetworkinterface123 - Azure Error: ResourceNotFound
Message: The Resource 'Microsoft.Network/networkInterfaces/mynetworkinterface123' under resource group 'MyComputeResourceGroup' was not found.
```
The following patch fixes the issue:
````
--- /usr/lib/python2.7/site-packages/ansible/modules/cloud/azure/azure_rm_virtualmachine.py     2018-04-27 03:48:29.000000000 +1000
+++ azure_rm_virtualmachine.py  2018-05-10 11:21:45.469600851 +1000
@@ -1377,8 +1377,12 @@
         return True

     def get_network_interface(self, name):
+        virtual_network_resource_group = self.resource_group
+        if self.virtual_network_resource_group:
+            virtual_network_resource_group = self.virtual_network_resource_group
         try:
-            nic = self.network_client.network_interfaces.get(self.resource_group, name)
+            nic = self.network_client.network_interfaces.get(virtual_network_resource_group, name)
             return nic
         except Exception as exc:
             self.fail("Error fetching network interface {0} - {1}".format(name, str(exc)))

````


Discussion:
Files identified in the description:
* [lib/ansible/modules/cloud/azure/azure_rm_virtualmachine.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/azure/azure_rm_virtualmachine.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @haroldwongms @nitzmahone @trstringer @xscript @yuwzho @zikalino
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Files identified in the description:
* [lib/ansible/modules/cloud/azure/azure_rm_virtualmachine.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/azure/azure_rm_virtualmachine.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @haroldwongms @nitzmahone @trstringer @xscript @yuwzho @zikalino
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
1st 3 lines of my patch can be condensed into one line with if else.
I patched it with 3 lines for readability. Please change to match your coding standards.

Cheers.
This topic should be "support VM refer NSG in other resource group" instead of simply use the virtual network's resource group for NSG. They can in 3 different rgs.
Hi @yuwzho I have tested the report YML file， meet same problem as report describe , when create VM by other resource group network interface, it will prompt this error, Could you help check and confirm it is question or not , thanks!

Resolving patch:
diff --git a/lib/ansible/modules/cloud/azure/azure_rm_virtualmachine.py b/lib/ansible/modules/cloud/azure/azure_rm_virtualmachine.py
--- a/lib/ansible/modules/cloud/azure/azure_rm_virtualmachine.py
+++ b/lib/ansible/modules/cloud/azure/azure_rm_virtualmachine.py
@@ -226,9 +226,11 @@
               providing a list of ports.
     network_interface_names:
         description:
-            - List of existing network interface names to add to the VM. If a network interface name is not provided
-              when the VM is created, a default network interface will be created. In order for the module to create
-              a network interface, at least one Virtual Network with one Subnet must exist.
+            - List of existing network interface names to add to the VM.
+            - Item can be a str of name or resource id of the network interface.
+            - Item can also be a dict contains C(resource_group) and C(name) of the network interface.
+            - If a network interface name is not provided when the VM is created, a default network interface will be created.
+            - In order for the module to create a new network interface, at least one Virtual Network with one Subnet must exist.
         aliases:
             - network_interfaces
     virtual_network_resource_group:
@@ -239,15 +241,17 @@
     virtual_network_name:
         description:
             - When creating a virtual machine, if a network interface name is not provided, one will be created.
-              The new network interface will be assigned to the first virtual network found in the resource group.
-              Use this parameter to provide a specific virtual network instead.
+            - The network interface will be assigned to the first virtual network found in the resource group.
+            - Use this parameter to provide a specific virtual network instead.
+            - If the virtual network in in another resource group, specific resource group by C(virtual_network_resource_group).
         aliases:
             - virtual_network
     subnet_name:
         description:
             - When creating a virtual machine, if a network interface name is not provided, one will be created.
-              The new network interface will be assigned to the first subnet found in the virtual network.
-              Use this parameter to provide a specific subnet instead.
+            - The new network interface will be assigned to the first subnet found in the virtual network.
+            - Use this parameter to provide a specific subnet instead.
+            - If the subnet is in another resource group, specific resource group by C(virtual_network_resource_group).
         aliases:
             - subnet
     remove_on_absent:
@@ -614,7 +618,7 @@
     pass
 
 from ansible.module_utils.basic import to_native, to_bytes
-from ansible.module_utils.azure_rm_common import AzureRMModuleBase, azure_id_to_dict, normalize_location_name
+from ansible.module_utils.azure_rm_common import AzureRMModuleBase, azure_id_to_dict, normalize_location_name, format_resource_id
 
 
 AZURE_OBJECT_CLASS = 'VirtualMachine'
@@ -660,7 +664,7 @@ def __init__(self):
             public_ip_allocation_method=dict(type='str', choices=['Dynamic', 'Static', 'Disabled'], default='Static',
                                              aliases=['public_ip_allocation']),
             open_ports=dict(type='list'),
-            network_interface_names=dict(type='list', aliases=['network_interfaces']),
+            network_interface_names=dict(type='list', aliases=['network_interfaces'], elements='raw'),
             remove_on_absent=dict(type='list', default=['all']),
             virtual_network_resource_group=dict(type='str'),
             virtual_network_name=dict(type='str', aliases=['virtual_network']),
@@ -753,9 +757,9 @@ def exec_module(self, **kwargs):
                 ))
 
             if self.network_interface_names:
-                for name in self.network_interface_names:
-                    nic = self.get_network_interface(name)
-                    network_interfaces.append(nic.id)
+                for nic_name in self.network_interface_names:
+                    nic = self.parse_network_interface(nic_name)
+                    network_interfaces.append(nic)
 
             if self.ssh_public_keys:
                 msg = "Parameter error: expecting ssh_public_keys to be a list of type dict where " \
@@ -1241,7 +1245,7 @@ def serialize_vm(self, vm):
         # Expand network interfaces to include config properties
         for interface in vm.network_profile.network_interfaces:
             int_dict = azure_id_to_dict(interface.id)
-            nic = self.get_network_interface(int_dict['networkInterfaces'])
+            nic = self.get_network_interface(int_dict['resourceGroups'], int_dict['networkInterfaces'])
             for interface_dict in result['properties']['networkProfile']['networkInterfaces']:
                 if interface_dict['id'] == interface.id:
                     nic_dict = self.serialize_obj(nic, 'NetworkInterface')
@@ -1254,7 +1258,7 @@ def serialize_vm(self, vm):
                 if config['properties'].get('publicIPAddress'):
                     pipid_dict = azure_id_to_dict(config['properties']['publicIPAddress']['id'])
                     try:
-                        pip = self.network_client.public_ip_addresses.get(self.resource_group,
+                        pip = self.network_client.public_ip_addresses.get(pipid_dict['resourceGroups'],
                                                                           pipid_dict['publicIPAddresses'])
                     except Exception as exc:
                         self.fail("Error fetching public ip {0} - {1}".format(pipid_dict['publicIPAddresses'],
@@ -1341,18 +1345,18 @@ def delete_vm(self, vm):
             self.log('Storing NIC names for deletion.')
             for interface in vm.network_profile.network_interfaces:
                 id_dict = azure_id_to_dict(interface.id)
-                nic_names.append(id_dict['networkInterfaces'])
-            self.log('NIC names to delete {0}'.format(', '.join(nic_names)))
+                nic_names.append(dict(name=id_dict['networkInterfaces'], resource_group=id_dict['resourceGroups']))
+            self.log('NIC names to delete {0}'.format(str(nic_names)))
             self.results['deleted_network_interfaces'] = nic_names
             if self.remove_on_absent.intersection(set(['all', 'public_ips'])):
                 # also store each nic's attached public IPs and delete after the NIC is gone
-                for name in nic_names:
-                    nic = self.get_network_interface(name)
+                for nic_dict in nic_names:
+                    nic = self.get_network_interface(nic_dict['resource_group'], nic_dict['name'])
                     for ipc in nic.ip_configurations:
                         if ipc.public_ip_address:
                             pip_dict = azure_id_to_dict(ipc.public_ip_address.id)
-                            pip_names.append(pip_dict['publicIPAddresses'])
-                self.log('Public IPs to  delete are {0}'.format(', '.join(pip_names)))
+                            pip_names.append(dict(name=pip_dict['publicIPAddresses'], resource_group=pip_dict['resourceGroups']))
+                self.log('Public IPs to  delete are {0}'.format(str(pip_names)))
                 self.results['deleted_public_ips'] = pip_names
 
         self.log("Deleting virtual machine {0}".format(self.name))
@@ -1374,37 +1378,37 @@ def delete_vm(self, vm):
 
         if self.remove_on_absent.intersection(set(['all', 'network_interfaces'])):
             self.log('Deleting network interfaces')
-            for name in nic_names:
-                self.delete_nic(name)
+            for nic_dict in nic_names:
+                self.delete_nic(nic_dict['resource_group'], nic_dict['name'])
 
         if self.remove_on_absent.intersection(set(['all', 'public_ips'])):
             self.log('Deleting public IPs')
-            for name in pip_names:
-                self.delete_pip(name)
+            for pip_dict in pip_names:
+                self.delete_pip(pip_dict['resource_group'], pip_dict['name'])
         return True
 
-    def get_network_interface(self, name):
+    def get_network_interface(self, resource_group, name):
         try:
-            nic = self.network_client.network_interfaces.get(self.resource_group, name)
+            nic = self.network_client.network_interfaces.get(resource_group, name)
             return nic
         except Exception as exc:
             self.fail("Error fetching network interface {0} - {1}".format(name, str(exc)))
 
-    def delete_nic(self, name):
+    def delete_nic(self, resource_group, name):
         self.log("Deleting network interface {0}".format(name))
         self.results['actions'].append("Deleted network interface {0}".format(name))
         try:
-            poller = self.network_client.network_interfaces.delete(self.resource_group, name)
+            poller = self.network_client.network_interfaces.delete(resource_group, name)
         except Exception as exc:
             self.fail("Error deleting network interface {0} - {1}".format(name, str(exc)))
         self.get_poller_result(poller)
         # Delete doesn't return anything. If we get this far, assume success
         return True
 
-    def delete_pip(self, name):
+    def delete_pip(self, resource_group, name):
         self.results['actions'].append("Deleted public IP {0}".format(name))
         try:
-            poller = self.network_client.public_ip_addresses.delete(self.resource_group, name)
+            poller = self.network_client.public_ip_addresses.delete(resource_group, name)
             self.get_poller_result(poller)
         except Exception as exc:
             self.fail("Error deleting {0} - {1}".format(name, str(exc)))
@@ -1692,6 +1696,16 @@ def create_default_nic(self):
             self.fail("Error creating network interface {0} - {1}".format(network_interface_name, str(exc)))
         return new_nic
 
+    def parse_network_interface(self, nic):
+        nic = self.parse_resource_to_dict(nic)
+        if 'name' not in nic:
+            self.fail("Invalid network interface {0}".format(str(nic)))
+        return format_resource_id(val=nic['name'],
+                                  subscription_id=nic['subscription_id'],
+                                  resource_group=nic['resource_group'],
+                                  namespace='Microsoft.Network',
+                                  types='networkInterfaces')
+
 
 def main():
     AzureRMVirtualMachine()


Test patch:
diff --git a/test/integration/targets/azure_rm_virtualmachine/tasks/virtualmachine.yml b/test/integration/targets/azure_rm_virtualmachine/tasks/virtualmachine.yml
--- a/test/integration/targets/azure_rm_virtualmachine/tasks/virtualmachine.yml
+++ b/test/integration/targets/azure_rm_virtualmachine/tasks/virtualmachine.yml
@@ -256,23 +256,38 @@
       state: absent
       vm_size: Standard_A0
 
+- set_fact:
+      niclist:
+         - name: testvm011
+           resource_group: "{{ resource_group_secondary }}"
+         - name: testvm012
+           resource_group: "{{ resource_group_secondary  }}"
+
+- name: Create virtual network
+  azure_rm_virtualnetwork:
+      resource_group: "{{ resource_group_secondary }}"
+      name: testvm001
+      address_prefixes: "10.10.0.0/16"
+  register: vn
+
+- name: Add subnet
+  azure_rm_subnet:
+      resource_group: "{{ resource_group_secondary }}"
+      name: testvm001
+      address_prefix: "10.10.0.0/24"
+      virtual_network: testvm001
+
 - name: Create NICs for dual nic VM
   azure_rm_networkinterface:
-      resource_group: "{{ resource_group }}"
-      name: "{{ item }}"
-      virtual_network: testvm001
+      resource_group: "{{ item.resource_group }}"
+      name: "{{ item.name }}"
+      virtual_network: "{{ vn.state.id }}"
       subnet: testvm001
       security_group: testvm001
-  loop:
-    - testvm011
-    - testvm012
+  loop: "{{ niclist }}"
 
 - name: Create virtual machine with two NICs
   register: output
-  vars:
-    niclist:
-      - testvm011
-      - testvm012
   azure_rm_virtualmachine:
       resource_group: "{{ resource_group }}"
       name: testvm003
@@ -297,10 +312,6 @@
         - azure_vm.properties.availabilitySet.id
 
 - name: Should be idempotent with a dual NICs
-  vars:
-    niclist:
-      - testvm011
-      - testvm012
   azure_rm_virtualmachine:
       resource_group: "{{ resource_group }}"
       name: testvm003