Repository: ansible/ansible
Pull request: Meraki scenario guide - describe how to merge new data with old data
##### SUMMARY
Ansible and Ansible's Meraki modules don't allow for merging and splitting data. This PR describes how to do this.

Fixes #48944

##### ISSUE TYPE
- Docs Pull Request

##### COMPONENT NAME
meraki scenario guide

Problem statement:
Meraki scenario guide should show how to merge data
##### SUMMARY
Meraki Ansible modules don't do a lot of backend work around merging existing data with new data. This puts the burden on the playbook to gather existing data and reconcile the data structure proposed by the playbook against the existing production configuration. The scenario guide should show how to merge these components.

One example is with firewall rules where it makes sense to merge existing firewalls with a new proposed one, in an order that works.

##### ISSUE TYPE
- Documentation Report

##### COMPONENT NAME
meraki

##### ANSIBLE VERSION
```
ansible 2.8.0.dev0 (meraki/network_disable_params 3547cf9991) last updated 2018/11/20 20:36:55 (GMT -500)
  config file = None
  configured module search path = ['/Users/kbreit/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/kbreit/Documents/Programming/ansible/lib/ansible
  executable location = /Users/kbreit/Documents/Programming/ansible/bin/ansible
  python version = 3.5.4 (default, Feb 25 2018, 14:56:02) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)]

```

Hints from issue discussion:
Hi @kbreit, thank you for submitting this issue!

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: thank_submitter --->
Files identified in the description:
* [lib/ansible/module_utils/network/meraki/meraki.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/network/meraki/meraki.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
@kbreit: in order for the community to handle your issue effectively, we need a bit more information.

Here are the items we could not find in your description:
- ansible version

Please set the description of this issue with this template:
https://raw.githubusercontent.com/ansible/ansible/devel/.github/ISSUE_TEMPLATE.md

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: issue_missing_data --->
cc @dagwieers
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Commit message: Described how to merge new data with old data

Patch to review:
diff --git a/docs/docsite/rst/scenario_guides/guide_meraki.rst b/docs/docsite/rst/scenario_guides/guide_meraki.rst
index e37c68c3b22235..50cc45c3453858 100644
--- a/docs/docsite/rst/scenario_guides/guide_meraki.rst
+++ b/docs/docsite/rst/scenario_guides/guide_meraki.rst
@@ -37,58 +37,58 @@ Meraki modules provide a user-friendly interface to manage your Meraki environme
 
 .. code-block:: yaml
 
-	- name: Query SNMP settings
-	  meraki_snmp:
-	  	api_key: abc123
-	  	org_name: AcmeCorp
-	  	state: query
-	  delegate_to: localhost
+    - name: Query SNMP settings
+      meraki_snmp:
+        api_key: abc123
+        org_name: AcmeCorp
+        state: query
+      delegate_to: localhost
 
 Information about a particular object can be queried. For example, the `meraki_admin <meraki_admin_module>` module supports
 
 .. code-block:: yaml
 
-	- name: Gather information about Jane Doe
-	  meraki_admin:
-	  	api_key: abc123
-	  	org_name: AcmeCorp
-	  	state: query
-	  	email: janedoe@email.com
-	  delegate_to: localhost
+    - name: Gather information about Jane Doe
+      meraki_admin:
+        api_key: abc123
+        org_name: AcmeCorp
+        state: query
+        email: janedoe@email.com
+      delegate_to: localhost
 
 Common Parameters
 =================
 
 All Ansible Meraki modules support the following parameters which affect communication with the Meraki Dashboard API. Most of these should only be used by Meraki developers and not the general public.
 
-	host
-		Hostname or IP of Meraki Dashboard.
+    host
+        Hostname or IP of Meraki Dashboard.
 
-	use_https
-		Specifies whether communication should be over HTTPS. (Defaults to ``yes``)
+    use_https
+        Specifies whether communication should be over HTTPS. (Defaults to ``yes``)
 
-	use_proxy
-		Whether to use a proxy for any communication.
+    use_proxy
+        Whether to use a proxy for any communication.
 
-	validate_certs
-		Determine whether certificates should be validated or trusted. (Defaults to ``yes``)
+    validate_certs
+        Determine whether certificates should be validated or trusted. (Defaults to ``yes``)
 
 These are the common parameters which are used for most every module.
 
-	org_name
-		Name of organization to perform actions in.
+    org_name
+        Name of organization to perform actions in.
 
-	org_id
-		ID of organization to perform actions in.
+    org_id
+        ID of organization to perform actions in.
 
-	net_name
-		Name of network to perform actions in.
+    net_name
+        Name of network to perform actions in.
 
-	net_id
-		ID of network to perform actions in.
+    net_id
+        ID of network to perform actions in.
 
-	state
-		General specification of what action to take. ``query`` does lookups. ``present`` creates or edits. ``absent`` deletes.
+    state
+        General specification of what action to take. ``query`` does lookups. ``present`` creates or edits. ``absent`` deletes.
 
 .. hint:: Use the ``org_id`` and ``net_id`` parameters when possible. ``org_name`` and ``net_name`` require additional behind-the-scenes API calls to learn the ID values. ``org_id`` and ``net_id`` will perform faster. 
 
@@ -108,22 +108,83 @@ Meraki and its related Ansible modules return most information in the form of a
 
 .. code-block:: json
 
-	[
-		{
-			"orgAccess": "full", 
-			"name": "John Doe",
-			"tags": [],
-			"networks": [],
-			"email": "john@doe.com",
-			"id": "12345677890"
-		}
-	]
+    [
+        {
+            "orgAccess": "full", 
+            "name": "John Doe",
+            "tags": [],
+            "networks": [],
+            "email": "john@doe.com",
+            "id": "12345677890"
+        }
+    ]
 
 Handling Returned Data
 ======================
 
 Since Meraki's response data uses lists instead of properly keyed dictionaries for responses, certain strategies should be used when querying data for particular information. For many situations, use the ``selectattr()`` Jinja2 function.
 
+Merging Existing and New Data
+=============================
+
+Ansible's Meraki modules do not allow for manipulating data. For example, there is likely a need to insert a rule in the middle of a firewall ruleset. Ansible and the Meraki modules lack a way to directly merge to manipulate data. However, a playlist can use a few tasks to split the list where a rule needs to be inserted and then merge them together again with the new rule added. The steps involved are as follows:
+
+1. Create blank "front" and "back" lists.
+    ::
+
+        vars:
+          - front_rules: []
+          - back_rules: []
+2. Get existing firewall rules from Meraki and create a new variable.
+    ::
+
+        - name: Get firewall rules
+          meraki_mx_l3_firewall:
+            auth_key: abc123
+            org_name: YourOrg
+            net_name: YourNet
+            state: query
+          delegate_to: localhost
+          register: rules
+        - set_fact:
+            original_ruleset: '{{rules.data}}'
+3. Write the new rule.
+    ::
+
+        - set_fact:
+            new_rule:
+              - 
+                - comment: Block traffic to server
+                  src_cidr: 192.0.1.0/24
+                  src_port: any
+                  dst_cidr: 192.0.1.2/32
+                  dst_port: any
+                  protocol: any
+                  policy: deny
+4. Split the rules into two lists. This assumes the existing ruleset is 2 rules long.
+    ::
+
+        - set_fact:
+            front_rules: '{{front_rules + [ original_ruleset[:1] ]}}'
+        - set_fact:
+            back_rules: '{{back_rules + [ original_ruleset[1:] ]}}'
+5. Merge rules with the new rule in the middle.
+    ::
+
+        - set_fact:
+            new_ruleset: '{{front_rules + new_rule + back_rules}}'
+6. Upload new ruleset to Meraki.
+    ::
+
+        - name: Set two firewall rules
+          meraki_mx_l3_firewall:
+            auth_key: abc123
+            org_name: YourOrg
+            net_name: YourNet
+            state: present
+            rules: '{{ new_ruleset }}'
+          delegate_to: localhost
+
 Error Handling
 ==============
 


Code review comments:
diff_hunk: @@ -108,22 +108,83 @@ Meraki and its related Ansible modules return most information in the form of a
 
 .. code-block:: json
 
-	[
-		{
-			"orgAccess": "full", 
-			"name": "John Doe",
-			"tags": [],
-			"networks": [],
-			"email": "john@doe.com",
-			"id": "12345677890"
-		}
-	]
+    [
+        {
+            "orgAccess": "full", 
+            "name": "John Doe",
+            "tags": [],
+            "networks": [],
+            "email": "john@doe.com",
+            "id": "12345677890"
+        }
+    ]
 
 Handling Returned Data
 ======================
 
 Since Meraki's response data uses lists instead of properly keyed dictionaries for responses, certain strategies should be used when querying data for particular information. For many situations, use the ``selectattr()`` Jinja2 function.
 
+Merging Existing and New Data
+=============================
+
+Ansible's Meraki modules do not allow for manipulating data. For example, there is likely a need to insert a rule in the middle of a firewall ruleset. Ansible and the Meraki modules lack a way to directly merge to manipulate data. However, a playlist can use a few tasks to split the list where a rule needs to be inserted and then merge them together again with the new rule added. The steps involved are as follows:
path: docs/docsite/rst/scenario_guides/guide_meraki.rst
text: @user1:
```suggestion
Ansible's Meraki modules do not allow for manipulating data. For example, you may need to insert a rule in the middle of a firewall ruleset. Ansible and the Meraki modules lack a way to directly merge to manipulate data. However, a playlist can use a few tasks to split the list where you need to insert a rule and then merge them together again with the new rule added. The steps involved are as follows:
```