make sure to handle 5xx errors

If an unxpected error occurs we will not get a JSON back
and serialization will fail. This makes sure we handle those edge cases.