Handle nil tagsets

If the tags don't exist in the record being compared we shouldn't throw an
error, we should just assume it doesn't match.

Side benefit: fixes CI.