Recover from panics

I can't figure out how to prevent the panics, so we can at least just
throw an error instead.
Less chatty test