Revert "cache site-packages between travis builds"

The feature only applies to private repositories.

This reverts commit fc6cedab25cce344d178b22b330fdae86e885168.