improve unicode handling

As a follow up to #578 and #595 I have another patch to improve unicode handling.

Added another unit test which check processing when output is encoded.

@josegaert @nicolasWDC @saljam @wjwwood @yukota Since you have all been part of the previous discussion I would like to ask you to try the changes of this branch with your specific scenario.

Since these unicode issues are highly specific to your environment, available locale and output of your tools the more people will validate that it works the more confident we can be that it does not break things (as it has happened before). Please comment on this ticket with either feedback ("it works for you" as well as "it doesn't (please with as much details as possible)").