added retry loop for creating intermediate files