Clean up internal library dependencies so we can avoid building unused code