Pull back more "build tests with world" pieces so buildworld is healthier