[RFC] Remove USE_SYSTEM and code only run under USE_SYSTEM

Needs a couple of extra eyes to make sure it doesn't break anything.

See #297