Fix #1132 -- don’t CWD to dir-with-conf.py if not needed

This is #1132.