I've had a chat with @alext to fix this issue properly would take a while - a quicker solution would be to pin the tests to say Rails 3.