Refactoring to support forever and better handling of failure files