Reapply 243 onto master

#243 got applied to the 2-13-maintenance branch but somehow I forgot to apply it to master :(.  Which means there was a regression in 2.14.0.rc1 that this fixes.

This already got code reviewed by @alindeman originally, so I'm just doing a PR to ensure the travis build is green with this against all supported rubies.