added fault tolerance and retries to it all