Switch to only using PyMem_Malloc