prevent assertion in calloc when num * size overflows (return NULL instead)