Transform notices/warnings to exceptions in CLI runner