Turn off colors with -v. Plus a test for --color, --nocolor and -v. Fixes #478.

Minor nitpick: Instead of referencing a GitHub issue which may one day disappear, maybe explain why it makes sense to disable colors when doing an inverted filter. This isn't a blocker, just feedback if you submit a PR in the future.