Cleanup InvertVisitor

Use the methods provided by the parent
class instead of a conditional ladder
Just reuse the same instance for recursive visits