Remove unneeded iOS libraries
Fix builds for iOS x86_64, arm64, improve code caching, speed up build