adds ability to gracefully crash out of infinite loops when processing supermassive files