Repository: ansible/ansible
Pull request: Added ability to specify plan information for images that require it.
##### SUMMARY
Added ability to specify billing plan information for images that require it. Images like CIS hardened images on Azure marketplace require plan field to be populated.
Fixes #65268 

##### ISSUE TYPE
- Feature Pull Request

##### COMPONENT NAME
azure_rm_virtualmachinescaleset

##### ADDITIONAL INFORMATION
Tests were done locally that:
1. generated the VM scale set with CIS image
2. generated VM scale set with standard Ubuntu image to prove nothing broke
3. as expected, failed to generate a scale set when specifying plan for image that does not require it
4. as expected, failed to generate a scale set when specifying image without required plan

The module azure_rm_virtualmachinescaleset handles plan information. Based on how plan and accept_terms was used I copied:
1. over documentation
2. over parameter definition
3. over code blocks for validating plan parameter
4. over code for accepting terms
and
added an example in the examples section.





Problem statement:
azure_rm_virtualmachinescaleset fails when using image that requires plan information
The module azure_rm_virtualmachine has fields for plan information. Module azure_rm_virtualmachinescaleset needs to have them and forward them when creating a VM with an image that requires plan info to be specified. The CIS hardened images from the Azure Marketplace is a prime example.

Hints from issue discussion:
Files identified in the description:
None

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
@seanw122: Greetings! Thanks for taking the time to open this issue. In order for the community to handle your issue effectively, we need a bit more information.

Here are the items we could not find in your description:
- issue type
- ansible version
- component name

Please set the description of this issue with this template:
https://raw.githubusercontent.com/ansible/ansible/devel/.github/ISSUE_TEMPLATE.md

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: issue_missing_data --->
ISSUE TYPE
Bug Report

COMPONENT NAME
azure_rm_virtualmachinescaleset 

ANSIBLE VERSION
ansible 2.9.0
Files identified in the description:
* [`lib/ansible/modules/cloud/azure/azure_rm_virtualmachinescaleset.py`](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/azure/azure_rm_virtualmachinescaleset.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @haroldwongms @nitzmahone @sozercan @yungezz @yuwzho @zikalino
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
@seanw122 Thank you for opening this question! In order to analyze and solve the problem you met, according to https://github.com/ansible/ansible/issues/new/choose, please provide the format of the report? Thank you very much!
> ISSUE TYPE
> Bug Report
> COMPONENT NAME
> azure_rm_virtualmachinescaleset
> ANSIBLE VERSION
> ansible 2.9.0

@seanw122 Also need you'r playbook, reproduce steps, expected results and actual results. Thank you very much!

Playbook to create a VM Scale Set using a shared image in the Azure Image Gallery listed below. The image is a CIS hardened image from the Azure Marketplace. It requires a plan to be specified.

- name: Create Azure VM Scale Set
  hosts: localhost
  connection: local
  vars:
    resource_group: dscrsgcisvm
    admin_username: notadmin
    admin_password: 1GoodPwd@Here
    vnet_name: dscvnet
    subnet_name: subnet1
    vmss_name: dscvmss
    custom_image_name: cis-ubuntu-1804-img
    shared_gallery_name: dscimggallery
    shared_image_version: 1.0.9
    shared_image_name: cisubuntu1804-img
    subscription_id: a5aaaaa2-7a89-4a0a-a66a-aaaaa7aaaaaa
  tasks:
  - name: Create an availability set for managed disk vm
    azure_rm_availabilityset:
      name: dscavailset
      resource_group: "{{ resource_group }}"
      platform_update_domain_count: 5
      platform_fault_domain_count: 2
      sku: Aligned
  - name: Create virtual network
    azure_rm_virtualnetwork:
      resource_group: "{{ resource_group }}"
      name: "{{ vnet_name }}"
      address_prefixes: "10.0.0.0/24"
  - name: Add subnet
    azure_rm_subnet:
      resource_group: "{{ resource_group }}"
      name: "{{ subnet_name }}"
      address_prefix: "10.0.0.0/24"
      virtual_network: "{{ vnet_name }}"
  - name: Create Network Security Group that allows SSH and 80
    azure_rm_securitygroup:
      resource_group: "{{ resource_group }}"
      name: dscnsg
      rules:
        - name: SSH
          protocol: Tcp
          destination_port_range: 22
          access: Allow
          priority: 1001
          direction: Inbound
        - name: Web
          protocol: Tcp
          destination_port_range: 80
          access: Allow
          priority: 500
          direction: Inbound
  - name: Create virtual network interface card
    azure_rm_networkinterface:
      resource_group: "{{ resource_group }}"
      name: dscvmnic
      virtual_network: "{{ vnet_name }}"
      subnet: "{{ subnet_name }}"
      security_group: dscnsg
  - name: Create VM Scale Set
    azure_rm_virtualmachinescaleset:
      resource_group: "{{ resource_group }}"
      name: "{{ vmss_name }}-vm"
      vm_size: Standard_B1s
      admin_username: "{{ admin_username }}"
      admin_password: "{{ admin_password }}"
      ssh_password_enabled: true
      capacity: 2
      virtual_network_name: "{{ vnet_name }}"
      subnet_name: "{{ subnet_name }}"
      upgrade_policy: Manual
      tier: Standard
      managed_disk_type: Standard_LRS
      os_type: Linux
      os_disk_caching: ReadWrite
      image:
        id: "/subscriptions/{{ subscription_id }}/resourceGroups/{{ resource_group }}/providers/Microsoft.Compute/galleries/{{ shared_gallery_name }}/images/{{ shared_image_name }}/versions/{{ shared_image_version }}"
    register: vmss_output
  - debug:
      var: vmss_output



Error Message:
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Error creating or updating virtual machine dscvmss-vm - Azure Error: VMMarketplaceInvalidInput\nMessage: Creating a virtual machine from Marketplace image requires Plan information in the request. VM: '/subscriptions/a5aaaaa2-7a89-4a0a-a66a-aaaaa7aaaaaa/resourceGroups/dscrsgcisvm/providers/Microsoft.Compute/virtualMachineScaleSets/dscvmss-vm/virtualMachines/dscvmss-vm_0'."}


I have tried:

image:
      offer: cis-ubuntu-linux-1804-l1
      publisher: center-for-internet-security-inc
      sku: cis-ubuntu1804-l1
      version: 1.0.9

but received same error. If I create a single VM using azure_rm_virtualmachine I can specify plan information and it works fine.

plan:
  name: cis-ubuntu1804-l1
  product: cis-ubuntu-linux-1804-l1
  publisher: center-for-internet-security-inc


What I think needs to happen is for the azure_rm_virtualmachinescaleset to have new field for plan and have that passed to each VM on creation. Being a programmer I know this is easier said than done. Thanks for any help on this!

I forked the code and made some updates. Tomorrow I'll attempt to run and test this locally. I've been using Azure Shell until now.
[azure_rm_virtualmachinescaleset.py.txt](https://github.com/ansible/ansible/files/3894294/azure_rm_virtualmachinescaleset.py.txt)


Trying to build the module locally. I was not expecting it to reach out to Azure. Should it?    If so, do I simply execute _az login_ ?


$ python3 -m azure_rm_virtualmachinescaleset /tmp/args.json

{"msg": "Error retrieving resource group dscrsgcisvm - Get Token request returned http error: 400 and server response: {\"error\":\"invalid_grant\",\"error_description\":\"AADSTS700082: The refresh token has expired due to inactivity.\u00a0The token was issued on 2019-08-27T21:39:44.1947898Z and was inactive for 90.00:00:00.\\r\\nTrace ID: 8a1f5765-fe13-4d45-9edc-34fe54f67c00\\r\\nCorrelation ID: d7029ed5-e19a-4777-a8c6-a7db01b41799\\r\\nTimestamp: 2019-11-26 23:45:40Z\",\"error_codes\":[700082],\"timestamp\":\"2019-11-26 23:45:40Z\",\"trace_id\":\"8a1f5765-fe13-4d45-9edc-34fe54f67c00\",\"correlation_id\":\"d7029ed5-e19a-4777-a8c6-a7db01b41799\",\"error_uri\":\"https://login.microsoftonline.com/error?code=700082\"}", "failed": true, ....


--- Update---
I did an _az login_ and retried. I wasn't expecting this to attempt to build anything in the Azure environment. I assumed (my problem) that the command would generate a module to test later using ansible-playbook. Instead, it did try to build a Virtual Machine Scale Set in my Azure resource group. It did fail due to another module needing to also be updated to handle plan information.

--- Update---
I had code passing the plan that was to wrong object. I moved to parent object. I works now!  Submitted PR after I ran tests locally to prove it works and it didn't break creation of Scale Set using images that do no require plan information.

Commit message: Removed extra blank line. Added promotion_code to plan param list.

Patch to review:
diff --git a/changelogs/fragments/65335-add-plan-to-azure-vmscaleset-module.yaml b/changelogs/fragments/65335-add-plan-to-azure-vmscaleset-module.yaml
new file mode 100644
index 00000000000000..1b48dff1a35495
--- /dev/null
+++ b/changelogs/fragments/65335-add-plan-to-azure-vmscaleset-module.yaml
@@ -0,0 +1,2 @@
+minor_changes:
+  - azure_rm_virtualmachinescaleset - added ability to use plan information for images that require billing plan details
diff --git a/lib/ansible/modules/cloud/azure/azure_rm_virtualmachinescaleset.py b/lib/ansible/modules/cloud/azure/azure_rm_virtualmachinescaleset.py
index da27187dfc834d..feb58aa60ecef2 100644
--- a/lib/ansible/modules/cloud/azure/azure_rm_virtualmachinescaleset.py
+++ b/lib/ansible/modules/cloud/azure/azure_rm_virtualmachinescaleset.py
@@ -207,6 +207,35 @@
         type: bool
         default: True
         version_added: "2.9"
+    plan:
+        description:
+            - Third-party billing plan for the VM.
+        version_added: "2.10"
+        type: dict
+        suboptions:
+            name:
+                description:
+                    - Billing plan name.
+                required: true
+            product:
+                description:
+                    - Product name.
+                required: true
+            publisher:
+                description:
+                    - Publisher offering the plan.
+                required: true
+            promotion_code:
+                description:
+                    - Optional promotion code.
+    accept_terms:
+        description:
+            - Accept terms for Marketplace images that require it.
+            - Only Azure service admin/account admin users can purchase images from the Marketplace.
+            - Only valid when a I(plan) is specified.
+        type: bool
+        default: false
+        version_added: "2.10"
     zones:
         description:
             - A list of Availability Zones for your virtual machine scale set.
@@ -257,6 +286,37 @@
         caching: ReadWrite
         managed_disk_type: Standard_LRS
 
+- name: Create VMSS with an image that requires plan information
+  azure_rm_virtualmachinescaleset:
+    resource_group: myResourceGroup
+    name: testvmss
+    vm_size: Standard_DS1_v2
+    capacity: 2
+    virtual_network_name: testvnet
+    upgrade_policy: Manual
+    subnet_name: testsubnet
+    admin_username: adminUser
+    ssh_password_enabled: false
+    ssh_public_keys:
+      - path: /home/adminUser/.ssh/authorized_keys
+        key_data: < insert yor ssh public key here... >
+    managed_disk_type: Standard_LRS
+    image:
+      offer: cis-ubuntu-linux-1804-l1
+      publisher: center-for-internet-security-inc
+      sku: Stable
+      version: latest
+    plan:
+      name: cis-ubuntu-linux-1804-l1
+      product: cis-ubuntu-linux-1804-l1
+      publisher: center-for-internet-security-inc
+    accept_terms: True
+    data_disks:
+      - lun: 0
+        disk_size_gb: 64
+        caching: ReadWrite
+        managed_disk_type: Standard_LRS
+
 - name: Create a VMSS with a custom image
   azure_rm_virtualmachinescaleset:
     resource_group: myResourceGroup
@@ -453,7 +513,11 @@ def __init__(self):
             overprovision=dict(type='bool', default=True),
             single_placement_group=dict(type='bool', default=True),
             zones=dict(type='list'),
-            custom_data=dict(type='str')
+            custom_data=dict(type='str'),
+            plan=dict(type='dict', options=dict(publisher=dict(type='str', required=True),
+                                 product=dict(type='str', required=True), name=dict(type='str', required=True), 
+                                 promotion_code=dict(type='str'))),
+            accept_terms=dict(type='bool', default=False),
         )
 
         self.resource_group = None
@@ -487,6 +551,8 @@ def __init__(self):
         self.single_placement_group = None
         self.zones = None
         self.custom_data = None
+        self.plan = None
+        self.accept_terms = None
 
         required_if = [
             ('state', 'present', [
@@ -758,6 +824,12 @@ def exec_module(self, **kwargs):
                         if nsg:
                             self.security_group = self.network_models.NetworkSecurityGroup(id=nsg.get('id'))
 
+                    plan = None
+                    if self.plan:
+                        plan = self.compute_models.Plan(name=self.plan.get('name'), product=self.plan.get('product'),
+                                                        publisher=self.plan.get('publisher'),
+                                                        promotion_code=self.plan.get('promotion_code'))
+
                     os_profile = None
                     if self.admin_username or self.custom_data or self.ssh_public_keys:
                         os_profile = self.compute_models.VirtualMachineScaleSetOSProfile(
@@ -779,6 +851,7 @@ def exec_module(self, **kwargs):
                             capacity=self.capacity,
                             tier=self.tier,
                         ),
+                        plan=plan,
                         virtual_machine_profile=self.compute_models.VirtualMachineScaleSetVMProfile(
                             os_profile=os_profile,
                             storage_profile=self.compute_models.VirtualMachineScaleSetStorageProfile(
@@ -852,6 +925,24 @@ def exec_module(self, **kwargs):
 
                         vmss_resource.virtual_machine_profile.storage_profile.data_disks = data_disks
 
+                    # Before creating VM accept terms of plan if `accept_terms` is True
+                    if self.accept_terms is True:
+                        if not self.plan or not all([self.plan.get('name'), self.plan.get('product'), self.plan.get('publisher')]):
+                            self.fail("parameter error: plan must be specified and include name, product, and publisher")
+                        try:
+                            plan_name = self.plan.get('name')
+                            plan_product = self.plan.get('product')
+                            plan_publisher = self.plan.get('publisher')
+                            term = self.marketplace_client.marketplace_agreements.get(
+                                publisher_id=plan_publisher, offer_id=plan_product, plan_id=plan_name)
+                            term.accepted = True
+                            self.marketplace_client.marketplace_agreements.create(
+                                publisher_id=plan_publisher, offer_id=plan_product, plan_id=plan_name, parameters=term)
+                        except Exception as exc:
+                            self.fail(("Error accepting terms for virtual machine {0} with plan {1}. " +
+                                       "Only service admin/account admin users can purchase images " +
+                                       "from the marketplace. - {2}").format(self.name, self.plan, str(exc)))
+
                     self.log("Create virtual machine with parameters:")
                     self.create_or_update_vmss(vmss_resource)
 


Code review comments:
diff_hunk: @@ -852,6 +927,24 @@ def exec_module(self, **kwargs):
 
                         vmss_resource.virtual_machine_profile.storage_profile.data_disks = data_disks
 
+                    # Before creating VM accept terms of plan if `accept_terms` is True
+                    if self.accept_terms is True:
+                        if not self.plan or not all([self.plan.get('name'), self.plan.get('product'), self.plan.get('publisher')]):
+                            self.fail("parameter error: plan must be specified and include name, product, and publisher")
path: lib/ansible/modules/cloud/azure/azure_rm_virtualmachinescaleset.py
text: @user1:
Your logic from the code above is: if `accept_terms` is `True`, `plan` must be specified. (That can also be modelled by `required_if`.) If you want the logic the other way around, you have to change your code (and you can't use `required_if` AFAIK).