Fix two OAEP bug - we didn't prohibit tiny keys properly due to an
integer underflow, and would crash instead of failing if the input was
too large to have been produced by the associated key.