mne-tools/mne-python — pull request #564 by rgoj (2013-04-14 14:28:18+00:00)
Title: WIP Resume dataset downloads issue #557

Here's my first go at issue #557 (resuming dataset downloads). I copied the code
from NISL into utils.py (first commit) and integrated the ProgressBar (second
commit). The code is WIP, so the final version will be better integrated into
existing MNE code, but I'm submitting the pull request now because of a problem.

Unfortunately it seems that the code from NISL works only for HTTP
downloads, not for FTP. Specifically, if I understand things correctly, adding
the Range header to the HTTP request:

urlOpener.addheader("Range", "bytes=%s-" % (local_file_size))

and making that request to an FTP server, results in receiving the whole file
again instead of just the bytes from local_file_size onwards. The NISL code was
adapted from [here](http://code.activestate.com/recipes/83208-resuming-download-of-a-file/) and [this stack overflow question](http://stackoverflow.com/questions/3581296/how-to-resume-download-in-python-using-urlretrieve-function) also mentions the problem of using that resume code for FTP downloads.

I'm considering three solutions (leaning towards the last one):

* Use HTTP to access the file if that's at all possible?
* Use a download library that supports resume for FTP ([urlgrabber](http://urlgrabber.baseurl.org/), [fileDownloader](https://pypi.python.org/pypi/fileDownloader.py), [PycURL](http://pycurl.sourceforge.net/)), but this would be an additional, not very important requirement
* Include the source of e.g. fileDownloader.py (roughly 200 lines of MIT-licensed code) and use it
* Take (i.e. learn) the FTP resume commands from e.g. fileDownloader and reimplement them in MNE

The last option seems doable, I'm trying it now to see if it works. Meanwhile,
do you have any comments, suggestions, is another option better?