Split out non pure python dependencies that are not absolutely required