localshred/protobuf — pull request #120 by localshred (2013-09-17 04:42:54+00:00)
Title: StringField get wrong encoding

I've got a StringField whose value contains unicode characters.

It seems that when the message is serialized to binary bytes and then deserialized to message, the unicode characters in the StringField is replaced with empty char "".

I found that in Protobuf::Field::StringFiled.encode(value) function, value's encoding is "ASCII-8BIT", but it should be UTF-8 actually. After calling value.encode!, because char in UTF-8 is not valid in ASCII-8BIT, these chars are replaced with char "".

I didn't find out when and where the value encoding is changed from UTF-8 to ASCII-8BIT.

I plan to comment out the encode! line to get rid of this temporarily.

Hope this will be resolved.

thanks!