weecology/retriever — pull request #173 by ethanwhite (2014-03-25 15:36:25+00:00)
Title: WIP: Add ability to directly download files with no processing

The Retriever is designed to handle tabular data. These are the kinds of data that typically need the most work done on them, but there are also lots of other useful formats that would be good to include. For example, environmental data that is stored in different kinds of raster/shape files or phylogenetic trees. These kinds of files are typically pretty standardized meaning that the main thing we need to be able to do is just download them directly and place them in our location of choice.

This is a work in progress that attempts to allow this to work with the current system. I'm not sure if this is a good way to think about solving this problem or not so I'm posting for comments before moving forward.

It is currently lacking the ability to pass in an argument for where to place the file (it is currently copied from the raw_data directory into DATA_DIR). This results because I'm handling moving the file in `templates.DownloadOnlyTemplate` rather than in `engines/download_only.py`. In addition, the download only scripts will currently run with any chosen engine selected, because they don't really use the engine. I added the engine itself since `retriever install csv some_spatial_file_script` would be confusing and so being explicit using `retriever install download some_spatial_file_script` seemed better.