basho/riak_cs — pull request #794 by shino (2014-02-05 14:00:31+00:00)
Title: Bugfix/non ascii in xml

This PR addresses two issues #787 and #628.

The plan to fix them and possible other XML-with-multibyte(non-ASCII)-character
issues are as follows:
- `riak_cs_xml:format_value(Val)` treats conversion between lists/binaries and
Unicode strings
(xmerl accepts only Unicode strings but not binary or latin-1-ish strings).
- In the function clause `when is_list(Val)`, use list_to_binary and call
`format_value` again with converted binary.
- In the function clause `when is_binary(Val)` use `unicode:characters_to_list`
to produce Unicode strings.
- Remove all other `unicode:characters_to_list` around XML output creation.

Some misc notes:
- When we take into account Unicode, lists created by `binary_to_list` should not
be considered as "strings". This conversion is NOT Unicode-aware, so lists and
binaries have byte-wise correspondence.
- On the other hand, `unicode:characters_to_list` is Unicode-aware, so converted
lists are not necessarily `[0..255]`. Each element of a converted list is
correspond to Unicode codepoint (not so precise but it's almost a character).
- The talk [1] is good reference for Latin-1 and Unicode in Erlang.
Codepoints, encodings, string literals, binaries and more.

```
> A = <<16#E3, 16#81, 16#82>>. % A binary of Japanese Hiragana "A" in UTF-8
<<227,129,130>>
> binary_to_list(A).
[227,129,130] % Just byte-wise
> unicode:characters_to_list(A).
[12354] % A list of single element (single character)
```

[1] http://www.erlang-factory.com/conference/ErlangUserConference2013/speakers/PatrikNyblom