Return a bytes object if we were given a bytes object
Raise a ValueError instead of an Exception