Package: datarig
Features|Installation|Dependencies|Documentation|Attribution|Contributions|Issues|AcknowledgementsFeaturesProviding large testing and demo data alongside your package releases is
challenging for two reasons. First, code repositories have strict limits on file
sizes. Second, you don't want your users to wait forever to download your cool
package because you've included large data files.  If you're a python developer
and have hit these issues thenDataRigis for you.  DataRig allows you to
move data from web-based repositories into your user's local directories
post-installation. This "just-in-time" data fetching is perfect for users to
test or run your package's demos.InstallationDataRig can be installed into your projects environment using pip:Activate the virtual or conda environment of your package$source<YOUR_ENV>/bin/activate# python virtual environment$condaactivate<YOUR_ENV>Install DataRig to your active environment(<YOUR_ENV>)$pipinstalldatarigDependenciesDataRig is super lightweight requiring justPython≥3.9and the request library available here:packagepypicondarequestshttps://pypi.org/project/requests/✓DocumentationUsing DataRig to access a repository is simple. Just build aRecordinstance and all the data will be at your fingertips. Here's how to do it for
a sample Zenodo repository:$ipython>>>fromdatarigimportZenodo>>># set the url to the api endpoint url for the record id 7868945>>>url='http://zenodo.org/api/records/7868945'>>>record=Zenodo(url)This record contains all of the repositories information stored as attributes.
To see everything at once, just print the record.>>>print(record)You will see a datasets attribute with a list of Dataset objects. These Datasets
contain the name, url link, size and file type of the datasets that can be
downloaded from the repository record. Let's print each of them.>>>fordsetinrecord.datasets:...print(dset)Notice that a Dataset instance describes the data but does not contain the
actual data. To get the data to your machine, you call call the records
'download' method. Let's get help for this method before calling it.>>>help(record.download)To call this method we need a directory to place the downloaded data, the name
of the dataset to download, the amount of memory to use during downloading
(chunksize) and a boolean of whether the download should be streamed to disk.
Streaming is usually the right choice since the files you will download are
likely large. Let's download the "sample_arr.npy" file from this record into
your current working dir.>>>frompathlibimportPath>>>record.download(directory=None,name='sample_arr.npy')That's it! You've just downloaded a dataset from a Zenodo record :sunglasses:AttributionIf you find DataRig useful, please cite the Zenodo archive of this repository.If you really like DataRig, you can also star therepository⭐!ContributionsContributions are what makes open-source fun and we would love for you to
contribute. Please check out ourcontribution guideto get started.IssuesDataRig provides custom issue templates for filing bugs, requesting
feature enhancements, suggesting documentation changes, or just asking
questions.Ready to discuss?File an issuehere.AcknowledgementsThis work is generously supported through the Ting Tsung and Wei Fong Chao
Foundation and the National Institute of Neurological Disorders and Stroke
(Grant 2R01 NS100738-05A1).