Only ever enable swig's directors with python (Closes #843)