rails/rails — Issue #3727: Emoji range encoded in JSON mangled
Author: pothibo
Created: 2011-11-22 23:38:45+00:00
State: open

I'm receiving emoji from an iOS device. Everything is fine from there (using the console I can verify that the right emoji is stored in postgresql)

Now, when I want to retrieve it via json on my iOS device, the emoji are not the proper one. I did some testing in irb and here's what I came up with:

irb(main):047:0> test = {"emoji" => "\u{1F618}"}
=> {"emoji"=>"\u{1F618}"}
irb(main):048:0> test = MultiJson::Engines::JsonGem.encode(test)
=> "{\"emoji\":\"\\uf618\"}"
irb(main):049:0> test
=> "{\"emoji\":\"\\uf618\"}"
irb(main):050:0> test = MultiJson::Engines::JsonGem.decode(test)
=> {"emoji"=>""}

Since I'm new to Unicode encoding/decoding, my assumptions could be wrong.

From what I can understand from [ActiveSupport::JSON::Encoding.escape](http://rubydoc.info/gems/activesupport/3.1.2/ActiveSupport/JSON/Encoding.escape), the encode method change the annotation to clean UTF-7. Doing so change the character value once decoded. I do understand the issues with UTF-7 and XSS, but is it possible to have something that returns valid UTF-8 code in the emoji's range while keeping the security?