check malloc return for code in rabin2.c