run test fixtures on invalid signatures

...and get rid of unused tests on invalid public key recovery. Those tests were
originally designed for BitcoinJS with an interface that is too different to
test conveniently on fullnode.