modify factor to leave unchanged expressions alone

replaces and fixes #2074