added fix for correctly maintaining metadata about the job incase of failure