@gylaz I believe we have to check it in based on how Bundler and Appraisal work. /cc @jferris