improve custom-protocol-handler.js

As documented in the Undetectables -
https://github.com/Modernizr/Modernizr/wiki/Undetectables -
navigator.registerProtocolHandler was stubbed in webkit for a while, and
doesn't actually do anything there. This test forces a failure and ensures it's
the right type of failure.