Repository: ansible/ansible

Issue:
maven_artifact Python3 Support
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
<!--- Please do not include extra details here, e.g. "vyos_command" not "the network module vyos_command"
mavent_repository

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.4.2.0
  config file = None
  configured module search path = [u'/Users/joellow/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/Cellar/ansible/2.4.2.0_1/libexec/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.14 (default, Sep 25 2017, 09:53:17) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]
```

##### CONFIGURATION
<!---
If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"

Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).

-->
Generated by Packer. Nothing specific, except specifying `ansible_python_interpreter=/usr/bin/python3` as required.

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->
macOS El Capitan, managing Ubuntu 16.04 with default Python3 runtime (3.5.2)

##### SUMMARY
<!--- Explain the problem briefly -->
Cannot use `maven_artifact` with python3 on managed host.

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
  maven_artifact:
    group_id: "{{ item.group_id }}"
    artifact_id: "{{ item.artifact_id }}"
    version: "{{ item.version }}"
    repository_url: <url>
    extension: hpi
    dest: "/path/to/dir/{{ item.group_id }}.{{ item.artifact_id }}-{{ item.version }}.hpi"
    group: root
    owner: root
    mode: 0644
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
Should succeed.

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
Error running the command on the remote. I got one error:

<!--- Paste verbatim command output between quotes below -->
```
failed: [default] (item={u'group_id': u'<group_id>', u'artifact_id': u'<artifact>', u'version': u'<version>'}) => {"changed": false, "item": {"artifact_id": "<artifact>", "group_id": "<group_id>", "version": "<version>"}, "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_58_aoltx/ansible_module_maven_artifact.py\", line 499, in <module>\n    main()\n  File \"/tmp/ansible_58_aoltx/ansible_module_maven_artifact.py\", line 481, in main\n    if downloader.download(artifact, b_dest):\n  File \"/tmp/ansible_58_aoltx/ansible_module_maven_artifact.py\", line 350, in download\n    self._write_chunks(response, f, report_hook=self.chunk_report)\n  File \"/tmp/ansible_58_aoltx/ansible_module_maven_artifact.py\", line 368, in _write_chunks\n    total_size = response.info().getheader('Content-Length').strip()\nAttributeError: 'HTTPMessage' object has no attribute 'getheader'\nShared connection to 127.0.0.1 closed.\r\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 0}
```

Replacing `getheader` with `get` on line 368 in the Ansible source yields:

```
failed: [default] (item={u'group_id': u'<group_id>', u'artifact_id': u'<artifact>', u'version': u'<version>'}) => {"changed": false, "item": {"artifact_id": "<artifact>", "group_id": "<group_id>", "version": "<version>"}, "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_gs91uuvm/ansible_module_maven_artifact.py\", line 499, in <module>\n    main()\n  File \"/tmp/ansible_gs91uuvm/ansible_module_maven_artifact.py\", line 481, in main\n    if downloader.download(artifact, b_dest):\n  File \"/tmp/ansible_gs91uuvm/ansible_module_maven_artifact.py\", line 350, in download\n    self._write_chunks(response, f, report_hook=self.chunk_report)\n  File \"/tmp/ansible_gs91uuvm/ansible_module_maven_artifact.py\", line 379, in _write_chunks\n    file.write(chunk)\nTypeError: write() argument must be str, not bytes\nShared connection to 127.0.0.1 closed.\r\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 0}
```

Replacing `open(..., 'w')` with `open(..., 'wb')` on line 348 seems to fix it.


Discussion:
(just pasting a formatted version of the errors mentioned above for slightly easier reading)

```
failed: [default] (item={u'group_id': u'<group_id>', u'artifact_id': u'<artifact>', u'version': u'<version>'}) => {"changed": false, "item": {"artifact_id": "<artifact>", "group_id": "<group_id>", "version": "<version>"}, "module_stderr": 
"Traceback (most recent call last):
  File "/tmp/ansible_58_aoltx/ansible_module_maven_artifact.py", line 499, in <module>
    main()
  File "/tmp/ansible_58_aoltx/ansible_module_maven_artifact.py", line 481, in main
    if downloader.download(artifact, b_dest):
  File "/tmp/ansible_58_aoltx/ansible_module_maven_artifact.py", line 350, in download
    self._write_chunks(response, f, report_hook=self.chunk_report)
  File "/tmp/ansible_58_aoltx/ansible_module_maven_artifact.py", line 368, in _write_chunks
    total_size = response.info().getheader('Content-Length').strip()
AttributeError: 'HTTPMessage' object has no attribute 'getheader'
Shared connection to 127.0.0.1 closed.
", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 0}
```

Replacing `getheader` with `get` on line 368 in the Ansible source yields:



```

failed: [default] (item={u'group_id': u'<group_id>', u'artifact_id': u'<artifact>', u'version': u'<version>'}) => {"changed": false, "item": {"artifact_id": "<artifact>", "group_id": "<group_id>", "version": "<version>"}, "module_stderr": 
"Traceback (most recent call last):
  File "/tmp/ansible_gs91uuvm/ansible_module_maven_artifact.py", line 499, in <module>
    main()
  File "/tmp/ansible_gs91uuvm/ansible_module_maven_artifact.py", line 481, in main
    if downloader.download(artifact, b_dest):
  File "/tmp/ansible_gs91uuvm/ansible_module_maven_artifact.py", line 350, in download
    self._write_chunks(response, f, report_hook=self.chunk_report)
  File "/tmp/ansible_gs91uuvm/ansible_module_maven_artifact.py", line 379, in _write_chunks
    file.write(chunk)
TypeError: write() argument must be str, not bytes
Shared connection to 127.0.0.1 closed.
", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 0}
```
pinging @abadger 
Looks like get() was added as an alias of getheader in python 2.2.  So +1
to making that change.

Looks like we will always write bytes in that function so +1 to that change
as well.

On Dec 11, 2017 10:54 AM, "Adrian Likins" <notifications@github.com> wrote:

> pinging @abadger <https://github.com/abadger>
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/ansible/ansible/issues/33761#issuecomment-350821078>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAMxWr72w876xc9BO202H5j67JYbXP4Vks5s_XpVgaJpZM4Q9EUd>
> .
>

Sorry about that @alikins -- forgot to add the newlines back after pasting it in.
@lowjoel Not a problem it wasn't your fault. The stdout output from ansible in that case has the embedded "\n"'s which looks weird everywhere and I find hard to read so I fix it sometimes ;->
Seeing this issue as well. 

Resolving patch:
diff --git a/lib/ansible/modules/packaging/language/maven_artifact.py b/lib/ansible/modules/packaging/language/maven_artifact.py
--- a/lib/ansible/modules/packaging/language/maven_artifact.py
+++ b/lib/ansible/modules/packaging/language/maven_artifact.py
@@ -349,8 +349,7 @@ def download(self, artifact, filename=None):
         if not self.verify_md5(filename, url + ".md5"):
             response = self._request(url, "Failed to download artifact " + str(artifact), lambda r: r)
             if response:
-                f = open(filename, 'w')
-                # f.write(response.read())
+                f = open(filename, 'wb')
                 self._write_chunks(response, f, report_hook=self.chunk_report)
                 f.close()
             else:
@@ -366,19 +365,19 @@ def chunk_report(self, bytes_so_far, chunk_size, total_size):
         if bytes_so_far >= total_size:
             sys.stdout.write('\n')
 
-    def _write_chunks(self, response, file, chunk_size=8192, report_hook=None):
-        total_size = response.info().getheader('Content-Length').strip()
+    def _write_chunks(self, response, filehandle, chunk_size=8192, report_hook=None):
+        total_size = response.info().get('Content-Length').strip()
         total_size = int(total_size)
         bytes_so_far = 0
 
-        while 1:
+        while True:
             chunk = response.read(chunk_size)
             bytes_so_far += len(chunk)
 
             if not chunk:
                 break
 
-            file.write(chunk)
+            filehandle.write(chunk)
             if report_hook:
                 report_hook(bytes_so_far, chunk_size, total_size)
 


Test patch: