Make --quiet --no-ignore-invalid still exit for invalid options

--quiet should reduce verbosity, it should not change the
underlying behavior.