Revert "cache site-packages between travis runs"

The feature only applies to private repositories

This reverts commit 4b6fd214cf69e83914310eee3c2551adcb31ba96.