Repository: ansible/ansible

Issue:
UnicodeDecodeError: 'utf8' codec when starting fail2ban on Amazon Linux
##### Issue Type:

Bug Report
##### Ansible Version:

1.5.3 but has been there since at least 1.3... just never got around to filing a report
##### Environment:

Playbook is running on Mac OSX - target machine is Amazon Linux
##### Summary:

When I run the following playbook snipped against Amazon Linux, I get one of those - 
UnicodeDecodeError: 'utf8' codec tracebacks.
##### Steps To Reproduce:

``` yaml
- name: Install security packages (yum version)
  yum: pkg={{ item }} state=latest enablerepo=epel
  with_items:
    - fail2ban
  when: ansible_distribution == 'Amazon'

- name: Enable fail2ban service
  service: name=fail2ban enabled=yes
```
##### Expected Results:

Expect it to start the service (which it does), and display "changed"
##### Actual Results:

Traceback:

```
TASK: [base | Enable fail2ban service] **************************************** 
changed: [54.187.65.41]

TASK: [base | Starting fail2ban service] ************************************** 
Traceback (most recent call last):
  File "/Users/pas/ansible/bin/ansible-playbook", line 289, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/Users/pas/ansible/bin/ansible-playbook", line 229, in main
    pb.run()
  File "/Users/pas/ansible/lib/ansible/playbook/__init__.py", line 281, in run
    if not self._run_play(play):
  File "/Users/pas/ansible/lib/ansible/playbook/__init__.py", line 654, in _run_play
    if not self._run_task(play, task, False):
  File "/Users/pas/ansible/lib/ansible/playbook/__init__.py", line 406, in _run_task
    results = self._run_task_internal(task)
  File "/Users/pas/ansible/lib/ansible/playbook/__init__.py", line 365, in _run_task_internal
    results = runner.run()
  File "/Users/pas/ansible/lib/ansible/runner/__init__.py", line 1179, in run
    results = [ self._executor(h, None) for h in hosts ]
  File "/Users/pas/ansible/lib/ansible/runner/__init__.py", line 535, in _executor
    self.callbacks.on_unreachable(host, msg)
  File "/Users/pas/ansible/lib/ansible/callbacks.py", line 458, in on_unreachable
    msg = "fatal: [%s] => %s" % (host, results)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xbf in position 130: invalid start byte
```

Rerunning the playbook is the current work around, as fail2ban does indeed start, so the next run gets through this task.

Please let me know if you need more help with this.



Discussion:
Happy now ansibot? :)

@pas256 region and ami-ID ?

It really doesn't matter... but in my specific case: us-west-2
and amzn-ami-pv-2014.03.0.x86_64-ebs (ami-b8f69f88)

It happened on the previous generation of Amazon Linux as well.

@pas256 I just tried this on a us-east-1 Amazon 64-bit instance and it worked fine. Could you verify if you're seeing the same issue there?

Yeah, still not working. Here is the process:

```
$ ansible --version
ansible 1.6 (devel 09e0d1c51f) last updated 2014/04/16 21:45:01 (GMT -700)
$ git clone git@github.com:Answers4AWS/netflixoss-ansible.git
$ cd netflixoss-ansible
$ ansible-playbook -l tag_Name_test playbooks/base-amazon-linux.yml

PLAY [Base] ******************************************************************* 

GATHERING FACTS *************************************************************** 
The authenticity of host 'ec2-54-188-179-247.us-west-2.compute.amazonaws.com (54.188.179.247)' can't be established.
RSA key fingerprint is 6b:35:f1:20:ac:5f:0e:14:c6:1a:be:19:3a:fd:d8:3a.
Are you sure you want to continue connecting (yes/no)? yes
ok: [ec2-54-188-179-247.us-west-2.compute.amazonaws.com]
ec2-54-188-179-247.us-west-2.compute.amazonaws.com: importing /Users/pas/tmp/netflixoss-ansible/playbooks/vars/Amazon.yml

TASK: [base | Ensure APT sources list includes multiverse] ******************** 

...
...

TASK: [base | Copy rc.local template] ***************************************** 
changed: [ec2-54-188-179-247.us-west-2.compute.amazonaws.com]

TASK: [base | Install security packages (apt version)] ************************ 
skipping: [ec2-54-188-179-247.us-west-2.compute.amazonaws.com]

TASK: [base | Install security packages (yum version)] ************************ 
changed: [ec2-54-188-179-247.us-west-2.compute.amazonaws.com] => (item=fail2ban)

TASK: [base | Enable fail2ban service] **************************************** 
changed: [ec2-54-188-179-247.us-west-2.compute.amazonaws.com]

TASK: [base | Starting fail2ban service] ************************************** 
Traceback (most recent call last):
  File "/Users/pas/ansible/bin/ansible-playbook", line 314, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/Users/pas/ansible/bin/ansible-playbook", line 254, in main
    pb.run()
  File "/Users/pas/ansible/lib/ansible/playbook/__init__.py", line 291, in run
    if not self._run_play(play):
  File "/Users/pas/ansible/lib/ansible/playbook/__init__.py", line 644, in _run_play
    if not self._run_task(play, task, False):
  File "/Users/pas/ansible/lib/ansible/playbook/__init__.py", line 421, in _run_task
    results = self._run_task_internal(task)
  File "/Users/pas/ansible/lib/ansible/playbook/__init__.py", line 376, in _run_task_internal
    results = runner.run()
  File "/Users/pas/ansible/lib/ansible/runner/__init__.py", line 1223, in run
    results = [ self._executor(h, None) for h in hosts ]
  File "/Users/pas/ansible/lib/ansible/runner/__init__.py", line 533, in _executor
    self.callbacks.on_unreachable(host, msg)
  File "/Users/pas/ansible/lib/ansible/callbacks.py", line 464, in on_unreachable
    msg = "fatal: [%s] => %s" % (host, results)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xbf in position 132: invalid start byte
```

Please let me know if you need more info.

So that's interesting, you're hitting a callback that indicates the host is unreachable at that point, but obviously you were connecting to it. Is fail2ban killing the session? Regardless, it looks like the results may contain some unicode causing the traceback in the callback itself.

@pas256 could you try the following patch?

<pre>
diff --git a/lib/ansible/callbacks.py b/lib/ansible/callbacks.py
index 1abfe68..6d9b5a4 100644
--- a/lib/ansible/callbacks.py
+++ b/lib/ansible/callbacks.py
@@ -459,9 +459,9 @@ class PlaybookRunnerCallbacks(DefaultRunnerCallbacks):
         if type(results) == dict:
             item = results.get('item', None)
         if item:
-            msg = "fatal: [%s] => (item=%s) => %s" % (host, item, results)
+            msg = "fatal: [%s] => (item=%s) => %s" % (host, item, utils.jsonify(results))
         else:
-            msg = "fatal: [%s] => %s" % (host, results)
+            msg = "fatal: [%s] => %s" % (host, utils.jsonify(results))
         display(msg, color='red', runner=self.runner)
         super(PlaybookRunnerCallbacks, self).on_unreachable(host, results)
 
</pre>


Resolving patch:
diff --git a/lib/ansible/utils/__init__.py b/lib/ansible/utils/__init__.py
--- a/lib/ansible/utils/__init__.py
+++ b/lib/ansible/utils/__init__.py
@@ -884,10 +884,11 @@ def filter_leading_non_json_lines(buf):
     filter only leading lines since multiline JSON is valid.
     '''
 
+    kv_regex = re.compile(r'.*\w+=\w+.*')
     filtered_lines = StringIO.StringIO()
     stop_filtering = False
     for line in buf.splitlines():
-        if stop_filtering or "=" in line or line.startswith('{') or line.startswith('['):
+        if stop_filtering or kv_regex.match(line) or line.startswith('{') or line.startswith('['):
             stop_filtering = True
             filtered_lines.write(line + '\n')
     return filtered_lines.getvalue()


Test patch: