tests: fix false-positive under win64

BN_ULONG isn't necessarily an unsigned long, as is the case on win64.
Merge pull request #5015 from theuni/win64test

tests: fix false-positive under win64