Add a few minor tweaks and reorganizations to improve caching and overall build