I was running the `ci/travis.sh` locally with different ruby version and it would break if the rails app wasn't removed when switching the ruby version.