Reduce the size of the image

Current image is about **700 MB**, these changes reduce it to **460 MB**.
This can have unintended consequences (see [note in ENV](https://docs.docker.com/reference/builder/#env)) and doesn't really matter for the build server or building locally since the "errors" don't cause problems.