encoding.c: mask dummy flags

* encoding.c (must_encindex, rb_enc_from_index, rb_obj_encoding): mask
  encoding index and ignore dummy flags.  [ruby-core:59354] [Bug #9314]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44462 b2dd03c8-39d4-4d8f-98ff-823fe69b080e