More refactoring

Builds on top of #2916 

Removes Map[PatternExpression, QueryGraph] and QueryPlanInput