fix prime size in resultant_modular
speed up fmpz_mat_rank by using rref for large sizes
redefine fmpz_invmod to consider any integer invertible mod 1 (for gmp 6.0 compatibility)