Permissions should be removed when the repository is deleted

Quick fix for travis-ci/travis-ci#1937.