Cleaning up the build matrix a bit as well