VM: better version of bignum_to_fixnum_strict that doesn't allocate