Exclude some numpy/python combinations from build matrix