Fix issues in --invert-match logic.

There were a few issues with the new --invert-match logic added in 525e4f0, which caused it to produce incorrect output. This pull request should fix these issues.