Properly support when PythonPath is supplied in the config