pass exit_on_error through to obtain_path function
create single log file to all options tests, remove it in suite.py