Backport #11049 and #11051 into 4-0-stable

I've just run into this on my app, this backports @neerajdotname's fix into Rails 4.0.x.

Related: #11049, #11051, #10509, #10566.