Repository: ansible/ansible

Issue:
cs_instance "user_data" error only when using python3 working in python2
##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
 - Bug Report


##### COMPONENT NAME
<!---
Name of the module, plugin, task or feature
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path
-->
cs_instance

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.4.3.0
  config file = /Users/nbari/projects/ansible/ansible.cfg
  configured module search path = ['/Users/nbari/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/nbari/Library/Python/3.6/lib/python/site-packages/ansible
  executable location = /Users/nbari/Library/Python/3.6/bin/ansible
  python version = 3.6.4 (default, Mar  1 2018, 18:36:50) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)]

```

##### CONFIGURATION
<!---
If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->
macOS High Sierra, brew instal python

##### SUMMARY
<!--- Explain the problem briefly -->

Error when running the playbook only using python3

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->
Run playbook using python 3 ` ansible-playbook play.yml`

<!--- Paste example playbooks or commands between quotes below -->
```yaml
 - name: Creat VM
      cs_instance:
        name: cloudinit-test
        template: Linux CentOS 7.4 64-bit 10G Disk (2018-01-08-d617dd)
        service_offering: Micro
        ssh_key: devops
        api_region: exoscale
        state: started
        tags:
          - key: provisioner
            value: ansible
        user_data: |
            #cloud-config
            package_upgrade: true
            packages:
              - tmux
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
no error

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: a bytes-like object is required, not 'str'
fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"/var/folders/75/8jzhscr56j13rv6tj6805ch40000gn/T/ansible_j03a0m9f/ansible_module_cs_instance.py\", line 1088, in <module>\n    main()\n  File \"/var/folders/75/8jzhscr56j13rv6tj6805ch40000gn/T/ansible_j03a0m9f/ansible_module_cs_instance.py\", line 1068, in main\n    acs_instance.present_instance()\n  File \"/var/folders/75/8jzhscr56j13rv6tj6805ch40000gn/T/ansible_j03a0m9f/ansible_module_cs_instance.py\", line 668, in present_instance\n    instance = self.update_instance(instance=instance, start_vm=start_vm)\n  File \"/var/folders/75/8jzhscr56j13rv6tj6805ch40000gn/T/ansible_j03a0m9f/ansible_module_cs_instance.py\", line 765, in update_instance\n    args_instance_update['userdata'] = self.get_user_data()\n  File \"/var/folders/75/8jzhscr56j13rv6tj6805ch40000gn/T/ansible_j03a0m9f/ansible_module_cs_instance.py\", line 682, in get_user_data\n    user_data = base64.b64encode(str(user_data))\n  File \"/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/base64.py\", line 58, in b64encode\n    encoded = binascii.b2a_base64(s, newline=False)\nTypeError: a bytes-like object is required, not 'str'\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1}
```




Discussion:
Files identified in the description:
None

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
!component cs_instance
bot_status
Files identified in the description:
* [lib/ansible/modules/cloud/cloudstack/cs_instance.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/cloudstack/cs_instance.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @dpassante @resmo
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
### Components
[lib/ansible/modules/cloud/cloudstack/cs_instance.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/cloudstack/cs_instance.py)
support: community
maintainers: dpassante resmo

### Metadata
**waiting_on**: maintainer
[needs_info](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md#label-needs_info): False

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: bot_status --->
While testing with python 2 and changing the `state` to `destroyed` is giving this error:

```
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'id'
fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"/var/folders/75/8jzhscr56j13rv6tj6805ch40000gn/T/ansible_b9HigX/ansible_module_cs_instance.py\", line 1088, in <module>\n    main()\n  File \"/var/folders/75/8jzhscr56j13rv6tj6805ch40000gn/T/ansible_b9HigX/ansible_module_cs_instance.py\", line 1078, in main\n    result = acs_instance.get_result(instance)\n  File \"/var/folders/75/8jzhscr56j13rv6tj6805ch40000gn/T/ansible_b9HigX/ansible_module_cs_instance.py\", line 973, in get_result\n    self.result['user_data'] = self._get_instance_user_data(instance)\n  File \"/var/folders/75/8jzhscr56j13rv6tj6805ch40000gn/T/ansible_b9HigX/ansible_module_cs_instance.py\", line 544, in _get_instance_user_data\n    res = self.cs.getVirtualMachineUserData(virtualmachineid=instance['id'])\nKeyError: 'id'\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1}
```
@nbari 1 bug report per issue, please file a new issue with exact workflow what you did so we can reproduce and handle cover it in integration tests.
status:reproducing

Resolving patch:
diff --git a/lib/ansible/modules/cloud/cloudstack/cs_instance.py b/lib/ansible/modules/cloud/cloudstack/cs_instance.py
--- a/lib/ansible/modules/cloud/cloudstack/cs_instance.py
+++ b/lib/ansible/modules/cloud/cloudstack/cs_instance.py
@@ -348,8 +348,13 @@
 
 import base64
 from ansible.module_utils.basic import AnsibleModule
-from ansible.module_utils.cloudstack import (AnsibleCloudStack, CS_HYPERVISORS, cs_argument_spec,
-                                             cs_required_together)
+from ansible.module_utils._text import to_bytes, to_text
+from ansible.module_utils.cloudstack import (
+    AnsibleCloudStack,
+    CS_HYPERVISORS,
+    cs_argument_spec,
+    cs_required_together
+)
 
 
 class AnsibleCloudStackInstance(AnsibleCloudStack):
@@ -592,7 +597,7 @@ def present_instance(self, start_vm=True):
     def get_user_data(self):
         user_data = self.module.params.get('user_data')
         if user_data is not None:
-            user_data = base64.b64encode(str(user_data))
+            user_data = to_text(base64.b64encode(to_bytes(user_data)))
         return user_data
 
     def get_details(self):


Test patch:
diff --git a/test/integration/targets/cs_instance/tasks/present.yml b/test/integration/targets/cs_instance/tasks/present.yml
--- a/test/integration/targets/cs_instance/tasks/present.yml
+++ b/test/integration/targets/cs_instance/tasks/present.yml
@@ -15,6 +15,11 @@
     affinity_group: "{{ cs_resource_prefix }}-ag"
     security_group: "{{ cs_resource_prefix }}-sg"
     ssh_key: "{{ cs_resource_prefix }}-sshkey"
+    user_data: |
+        #cloud-config
+        package_upgrade: true
+        packages:
+          - tmux
     tags: []
   register: instance
   check_mode: true
@@ -32,6 +37,11 @@
     affinity_group: "{{ cs_resource_prefix }}-ag"
     security_group: "{{ cs_resource_prefix }}-sg"
     ssh_key: "{{ cs_resource_prefix }}-sshkey"
+    user_data: |
+        #cloud-config
+        package_upgrade: true
+        packages:
+          - tmux
     tags: []
   register: instance
 - name: verify create instance
@@ -54,6 +64,11 @@
     affinity_group: "{{ cs_resource_prefix }}-ag"
     security_group: "{{ cs_resource_prefix }}-sg"
     ssh_key: "{{ cs_resource_prefix }}-sshkey"
+    user_data: |
+        #cloud-config
+        package_upgrade: true
+        packages:
+          - tmux
     tags: []
   register: instance
 - name: verify create instance idempotence
@@ -84,7 +99,6 @@
     - instance.service_offering == "{{ test_cs_instance_offering_1 }}"
     - instance.state == "Running"
 
-
 - name: test running instance not updated
   cs_instance:
     name: "{{ cs_resource_prefix }}-vm-{{ instance_number }}"