The rakefile did the same, but instead of using environments, it first executed tests with imagick loaded, then imagick was unloaded in favor of gmagick.

I don't think we can do better ATM