Only change the XML where it is needed

Refs #10300