Throw on 4xx and 5xx

As we don't give the user back a response object we should at least blow up if sending data to backdrop failed.