curve: fix endomorphism criteria