Make 'removeBuildTreeRefs' tolerate non-existing paths.

Fixes #1360.