Fix FindVisitor to not start the visiting in the constructor

We used to call visitAny in the constructor, which lead to bad
API where we had to set things on this before calling super.