jruby/jruby — pull request #1345 by DavidEGrayson (2013-12-17 03:05:09+00:00)
Title: UnmarshalStream: Look for encoding information in all instance variables

Unfortunately, I think the solution to issue #939 did not go far enough. When unmarshaling a string, MRI actually checks every instance variable for encoding information and uses the last encoding it found.

We can see this from `r_ivar` in MRI 2.0.0p353 and 1.9.3p484:
https://github.com/ruby/ruby/blob/v2_0_0_353/marshal.c#L1369-1380
https://github.com/ruby/ruby/blob/v1_9_3_484/marshal.c#L1255-1266

I also checked the latest version of MRI's master branch.

Here is some example code where we unmarshal a string and the encoding is specified as UTF-16BE in the second instance variable:

```ruby
puts Marshal.load("\x04\x08I\"\x06a\x07:\x07@aT:\x0Dencoding\"\x0DUTF-16BE").encoding
```

Here is a shell session showing how MRI respects the encoding but JRuby does not, giving the string an encoding of ASCII-8BIT:

```
$ jruby -v && jruby pr2_repro.rb
jruby 1.7.9 (1.9.3p392) 2013-12-06 87b108a on Java HotSpot(TM) 64-Bit Server VM
1.7.0_45-b18 [Windows 8-amd64]
ASCII-8BIT
$ source use_mri2.sh
$ ruby -v && ruby pr2_repro.rb
ruby 2.0.0p0 (2013-02-24) [x64-mingw32]
UTF-16BE
```

The code in this pull request fixes this issue. All I did was remove the "i==0" condition in `defaultVariablesUnmarshal` and adjusted the indentation accordingly.

The reason I am touching this code is because I plan to refactor the encoding-determination logic out of it into another method so I can use it for symbols for issue #1329, or maybe I will just call this method directly.