Clobber/delete destination ftp files as needed

That's sort of why I suggested punting, because we wouldn't really be doing
the job.  The point was that if you actually delete the final file before
being 100% sure that the new file is there and in place with the correct
name, then you are out-of-luck if the last rename fails.  E.g. I would
certainly hope in the linux kernel that if you 'mv file1 file2' where file2
already exists, and then abort that process midway, that file1 and file2
are still there.


On Mon, Aug 4, 2014 at 3:30 PM, Jean Connelly <notifications@github.com>
wrote:

> In kadi/occweb.py:
>
> >
> >      for local_file in local_files:
> >          file_dir, file_base = os.path.split(os.path.abspath(local_file))
> >          ftp_file = str(uuid.uuid4())  # random unique identifier
> >          with Ska.File.chdir(file_dir):
> >              ftp.put(file_base, ftp_file)
> > +            # If final destination of file already exists, do
> > +            # a rename and delete of that file.
> > +            if file_base in dir_files:
> > +                ftp_del_file = "del_{}".format(str(uuid.uuid4()))
> > +                ftp.rename("{}/{}".format(ftp_dirname, file_base),
> > +                           ftp_del_file)
> > +                ftp.delete(ftp_del_file)
>
> I'm not entirely clear on why it is more atomic to do the final rename
> before the delete... but maybe I'm just assuming that if you can't delete
> the temp file you probably ran into other problems here anyway. Which other
> exceptions would you like to handle so you might be able to get more ftp
> commands through and how would that help?
>
> To support being truly robust and atomic I think you'd really need to have
> rename() check file status in Ska.ftp, but I figured that wasn't my
> purview. And regarding errors I think all of the ftp commands can fail with
> unhandled IOErrors (though I didn't try to make permissions errors) and
> there's still the unhandled recursive object delete problem if the SFTP
> isn't correctly initialized.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/sot/kadi/pull/50/files#r15775615>.
>