rb_check_convert_type compatibility with MRI

This PR falls in the categories "bikeshed" (in a way) and "do we really care?".
This PR changes Rbx' implementation of `rb_check_convert_type` (and its callers
to keep things going) so that its behaviour is the same as MRI's. Before this
commit the main difference was that MRI would raise when a coercion method
returned an unexpected type whereas Rbx would return `nil` instead.

A quick Github search
(<https://github.com/search?q=rb_check_convert_type&ref=cmdform&type=Code>)
shows that people do use the function and probably also rely on the behaviour
of raising errors.

The current changes are a bit of a mess as I wanted to get things going. In the
long term we probably should refactor things a bit but I'd like to have some
feedback first. In particular I'd like to know if it's even worth bothering,
though I'd say it would be.