Fix bad return in modInverse(), add endless loop bug to test