Separate downloads from build directory

Should lower the bandwidth use and build time for subsequent builds.