Repository: ansible/ansible

Issue:
aci_ap fails with parameters containing quotes
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and devel branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
 - Bug Report

##### COMPONENT NAME
aci_ap but I guess similar for other aci_* modules

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.4.2.0
  config file = /home/dhamann/ddsdnlab.at/ansible_aci_buildout/ansible.cfg
  configured module search path = [u'/home/dhamann/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.12 (default, Nov 20 2017, 18:23:56) [GCC 5.4.0 20160609]

```

##### CONFIGURATION
<!---
If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->
```
DEFAULT_STDOUT_CALLBACK(/home/dhamann/ddsdnlab.at/ansible_aci_buildout/ansible.cfg) = skippy
RETRY_FILES_ENABLED(/home/dhamann/ddsdnlab.at/ansible_aci_buildout/ansible.cfg) = False
```

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->
N/A

##### SUMMARY
<!--- Explain the problem briefly -->
aci_ap module will fail for parameters contianing quotes (double or single)

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
    - name: Add Application Network Profile
      tags:
        - network
        - application
        - anp
        - test
      aci_ap:
        host: "{{ apic_hostname }}"
        username: "{{ apic_username }}"
        password: "{{ apic_password }}"
        validate_certs: false
        ap: legacy
        tenant: SDNLAB
        description: This ANP contains "legacy" networks
        state: present
      delegate_to: localhost
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
Now that I think about it I'm not sure if this is even feasible to fix in Ansible, but since I already started with this bug report, please let me know what you think. It seems to me that APIC does not accept quotes (even properly escaped) in json payload. Should the aci modules therefore remove quotes from strings it passes on to APIC?

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
fatal: [apic.ddsdnlab.at -> localhost]: FAILED! => {
    "changed": false, 
    "config": {
        "fvAp": {
            "attributes": {
                "descr": "This ANP contains \"legacy\" networks"
            }
        }
    }, 
    "error_code": "801", 
    "error_text": "property descr of uni/tn-SDNLAB/ap-legacy failed validation for value 'This ANP contains \"legacy\" networks'", 
    "existing": [
        {
            "fvAp": {
                "attributes": {
                    "descr": "This ANP contains legacy networks", 
                    "dn": "uni/tn-SDNLAB/ap-legacy", 
                    "name": "legacy", 
                    "nameAlias": "", 
                    "ownerKey": "", 
                    "ownerTag": "", 
                    "prio": "unspecified"
                }
            }
        }
    ], 
    "filter_string": "?rsp-prop-include=config-only", 
    "imdata": [
        {
            "error": {
                "attributes": {
                    "code": "801", 
                    "text": "property descr of uni/tn-SDNLAB/ap-legacy failed validation for value 'This ANP contains \"legacy\" networks'"
                }
            }
        }
    ], 
    "invocation": {
        "module_args": {
            "ap": "legacy", 
            "description": "This ANP contains \"legacy\" networks", 
            "host": "apic.ddsdnlab.at", 
            "hostname": "apic.ddsdnlab.at", 
            "method": null, 
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", 
            "protocol": "https", 
            "state": "present", 
            "tenant": "SDNLAB", 
            "timeout": 30, 
            "use_proxy": true, 
            "use_ssl": true, 
            "username": "apic:Local\\\\ansible", 
            "validate_certs": false
        }
    }, 
    "method": "POST", 
    "msg": "Request failed: 801 property descr of uni/tn-SDNLAB/ap-legacy failed validation for value 'This ANP contains \"legacy\" networks'", 
    "proposed": {
        "fvAp": {
            "attributes": {
                "descr": "This ANP contains \"legacy\" networks", 
                "name": "legacy"
            }
        }
    }, 
    "response": "HTTP Error 400: Bad Request", 
    "status": 400, 
    "totalCount": "1", 
    "url": "https://apic.ddsdnlab.at/api/mo/uni/tn-[SDNLAB]/ap-[legacy].json"
}

```



Discussion:
Files identified in the description:
* [lib/ansible/modules/network/aci/aci_aep.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/aci/aci_aep.py)
* [lib/ansible/modules/network/aci/aci_ap.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/aci/aci_ap.py)
* [lib/ansible/modules/network/aci/aci_bd.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/aci/aci_bd.py)
* [lib/ansible/modules/network/aci/aci_epg.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/aci/aci_epg.py)
* [lib/ansible/modules/network/aci/aci_vrf.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/aci/aci_vrf.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @brunocalogero @dagwieers @jmcgill298 @schunduri
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
@marinus81 Thanks for reporting this.

The error for me was pretty clear that it is the APIC's own validation that does not like quotes in the description. So to verify that this is the case, I looked in the MIM and this confirms my suspicion:
https://pubhub-prod.s3.amazonaws.com/media/apic-mim-ref/docs/MO-fvAp.html#descr

I would be expecting the ACI Web GUI to have a similar issue, and indeed, the entry-form immediately indicates that the input is considered invalid, so the GUI blocks this off effecitvely.

I wonder what we could do to make this more obvious to users ?

- Would it help if we added this to the docs ? And where would we add it ?
  - As a note to the module docs, or as part of the description of the "description" parameter ?
    - I am concerned that adding too much information would make it less likely people would be reading up to these details. Especially as we would then end up doing this for other parameters as well.
  - In the general ACI documentation ?
    - Will people find it there ? Maybe if we added the exact error in a section with common error messages, and then reference the docs from there
    - I prefer this over stuffing this in every module documentation
- Can we improve the error to the user ?
  - Should we validate ourselves ?
    - I am concerned with doing something like this. I like the fact that the error from the APIC is already clear and correct. If we start to validate input, we would have to implement all the validation checks that are already part of the APIC. We risk not being 100% compatible, or not taking into account changes in the APIC.
  - Should we rewrite user input ?
    - I am hesitant to start doing this for the same reason as above. We would be adding validation checks, or at least have rules implemented that are already in the APIC. I am not so keen to duplicating this kind of validation as it can lead to issues with the intermediate layer.

If you have any ideas on how to handle this better, let me know.
I am in agreement with that last section @dagwieers we should not unnecessarily manipulate user input, nor should we overwrite error messages that are already clearly indicating the error.

Perhaps we could put a general note about special characters in the general ACI docs. Maybe the BU would be able to provide us the details of what is allowed?
@jmcgill298 So in this case for `descr` it is: https://pubhub-prod.s3.amazonaws.com/media/apic-mim-ref/docs/MO-fvAp.html#descr

I would make a section about common error messages, and then give examples of strings with specific conventions like for `descr`. The nice thing is that we have a unique error number as well, so people will find it with ease if they search for the error.
who said regex couldn't be used for documentation 😆 

Resolving patch:
diff --git a/lib/ansible/module_utils/network/aci/aci.py b/lib/ansible/module_utils/network/aci/aci.py
--- a/lib/ansible/module_utils/network/aci/aci.py
+++ b/lib/ansible/module_utils/network/aci/aci.py
@@ -380,7 +380,7 @@ def request(self, path, payload=None):
             try:
                 # APIC error
                 self.response_json(info['body'])
-                self.fail_json(msg='Request failed: %(code)s %(text)s' % self.error)
+                self.fail_json(msg='APIC Error %(code)s: %(text)s' % self.error)
             except KeyError:
                 # Connection error
                 self.fail_json(msg='Connection failed for %(url)s. %(msg)s' % info)
@@ -416,7 +416,7 @@ def query(self, path):
             try:
                 # APIC error
                 self.response_json(query['body'])
-                self.fail_json(msg='Query failed: %(code)s %(text)s' % self.error)
+                self.fail_json(msg='APIC Error %(code)s: %(text)s' % self.error)
             except KeyError:
                 # Connection error
                 self.fail_json(msg='Connection failed for %(url)s. %(msg)s' % query)
@@ -687,7 +687,7 @@ def delete_config(self):
                 try:
                     # APIC error
                     self.response_json(info['body'])
-                    self.fail_json(msg='Request failed: %(code)s %(text)s' % self.error)
+                    self.fail_json(msg='APIC Error %(code)s: %(text)s' % self.error)
                 except KeyError:
                     # Connection error
                     self.fail_json(msg='Connection failed for %(url)s. %(msg)s' % info)
@@ -820,7 +820,7 @@ def get_existing(self):
             try:
                 # APIC error
                 self.response_json(info['body'])
-                self.fail_json(msg='Request failed: %(code)s %(text)s' % self.error)
+                self.fail_json(msg='APIC Error %(code)s: %(text)s' % self.error)
             except KeyError:
                 # Connection error
                 self.fail_json(msg='Connection failed for %(url)s. %(msg)s' % info)
@@ -920,7 +920,7 @@ def post_config(self):
                 try:
                     # APIC error
                     self.response_json(info['body'])
-                    self.fail_json(msg='Request failed: %(code)s %(text)s' % self.error)
+                    self.fail_json(msg='APIC Error %(code)s: %(text)s' % self.error)
                 except KeyError:
                     # Connection error
                     self.fail_json(msg='Connection failed for %(url)s. %(msg)s' % info)


Test patch: