opened
eaquigley
username_0
---


Author Name: **Kevin Condon** (@username_1)
Original Redmine Issue: 4086, https://redmine.hmdc.harvard.edu/issues/4086
Original Date: 2014-06-06
Original Assignee: Gustavo Durand

---


This was requested by a dv admin, Janina:

Allow a user to download all files from a dataverse at once, see RT#179817

do you know if there is a way to download all of the files in our Dataverse at once?
File Download: Allow user to download all files from a dataverse at once.
issue
created
kcondon
username_1
Under the Download button, the UI currently says, All files from this dataset, but it is grayed out. 
A user was asking whether this was broken, see RT #196683, also Liz added a ticket to allow selecting all files at once for download: #1988
comment
created
raprasad
username_2
Based on # of files + sizes, can there be a quick estimate of whether:
- This can be done fairly fast (in seconds) for that user
- Goes into a queue
   - User sees message saying to expect an email + dv notification for when the download is ready.
comment
created
sbarbosadataverse
username_3
@scolapasta 
This was requested by Harvard GSD just today and had to be passed to Kevin.
comment
created
kcondon
username_1
Will bring up at GIRT to see whether it is realistic to pursue this.
comment
created
donsizemore
username_4
Feature request (which might better belong in a separate issue?):

Thu-Mai often needs to download all files from a dataset in their original format. I'm looking into scripting this through the Native and Data Access APIs, but a GUI option for archivists would be extremely helpful.
comment
created
pdurbin
username_5
@username_4 yeah, a separate issue might be nice. What you and Thu-Main want is actually a smaller "user story" since it's limited to a single dataset. This issue is about a whole dataverse, which might contain sub-dataverses (which might contain sub-dataverses).
comment
created
pdurbin
username_5
@username_4 this issue came up today. Weren't you saying in IRC that you cooked up a script? Want us to take a look? 😄
comment
created
pdurbin
username_5
"all mine does is download all files in a given dataset in original format writing out the original filename, but it could be smartly rewritten and extended" -- @username_4 at http://irclog.iq.harvard.edu/dataverse/2017-04-07#i_51359
comment
created
pdurbin
username_5
When we tried to estimate #4529 about downloading all file based on a persistent ID (DOI or Handle) we decided against implementing that feature due to concerns over performance problems: https://github.com/IQSS/dataverse/issues/4529#issuecomment-385772705

This issue represents even more load on the server so by the same logic we wouldn't implement this either.
comment
closed
djbrooke
username_6
issue
created
djbrooke
username_6
I'm going to close this for now. For the performance concerns, we could possibly revisit after implementing Lambda functions (#6093) that would take zipping datasets off the application server, or if we decide to pre-zip and store content in support of #6085. We'd need to take non-S3 installations into consideration.
comment
created
mankoff
username_7
Hello. I'm interested in this feature (and commented recently on a related issue). I have a question after reading this thread: 

Why is zipping required? 

Based on my (limited, ancient) webserver admin experience, if the dataset is exposed as a folder, the individual files could be downloaded with wget. Compression can happen on-the-fly (and only by certain filetype?) by the webserver (e.g. Apache), or no compression at all, and the Dataverse does not need to bulk-zip everything before the download begins.
comment
created
pdurbin
username_5
@username_7 I appreciate your out of the box thinking! Thanks for commenting on #4529 and #6505 as well! Let's move the conversation to one of those issues since they're still open. Alternatively, you're welcome to open a dedicated issue about this idea.
comment