Fix #2 - move external config directory creation and add attribute to skip the cleanup