Package: daman
damandescriptiondamanaims at providing a relatively simple solution to the challenge of sharing data between various multiple individuals/machines in the context of a python-development workflow. This solution relies on cloud storage and allows to control the amount of local disc space used for data management.setup & configureInstallThis package is available throughpipand can directly be installed as follows by runningpip install daman.cloud providerAWS: S3 BucketShould your AWS account not yet configured on your current machine, retrieve the following information:access_key_idsecret_access_keyusingdm_awsyou can then simply running the following command:dm_aws--access_key_id<access_key_id>--secret_access_key<secret_access_key>manuallyAlternatively you can create a file at~/.aws/credentialsand type in the following :[default]aws_access_key_id=<access_key_id>aws_secret_access_key=<secret_access_key>configure damanTo finalise the setup phase, it is required to run to provide the following information:storage_name: Name of the bucket to be usedservice: Type of cloud storage used (Currently onlyawsis available).local_dir[Optional]: local directory to store data in.default is~/.daman/data/allocated_space[Optional]: Disc space to allocate to local directory. By default no limit is set.dm_configure--storage_name<storage_name>
--service<service>how touploadCurrently it is only possible to push python object todamandata manager.python:pushdm.push(obj,key=key,meta=None,local=True,force=False,persist=False)Inputobj:object- Any pickle serialisable object.key:str- name under which to store the object. will be used to retrieve the object.meta[OPTIONAL]:object- Any pickle serialisable meta information to store with the object.local[OPTIONAL]:bool- If set toTruealso adds the uploaded data to your local registery.force[OPTIONAL]:bool- Ifkeyis already in use,forcemust be set toTruein order to force the overwriting of the already stored object.persist[OPTIONAL]:bool- If set toTrueensures the file will not be deleted unless manually requested.Output -Nonedownloadpython:pullobj,meta=dm.push(key=key,force=False,persist=False)Inputkey:str- key under which the data is stored.force[OPTIONAL]:bool- when set toTruedownloads the dataset from cloud service ignoring local version.persist[OPTIONAL]:bool- If set toTrueensures the file will not be deleted unless manually requested.memory_only[OPTIONAL]:bool- is set toTrueonly loads the data into memory and does not keep a local version unless already available.Outputobj:objectstored object.meta:objectstore meta data.terminal:dm_pulldm_pull--help

usage:dm_pull[-h]--key{}[--force][--persist]Setsupdamanpackage.

optionalarguments:-h,--helpshowthishelpmessageandexit--key{}keyofthefiletodelete--forceWhenprovidedforcesthedownloadevenwhenthefileisalreadyavailable.--persistWhenprovidedensuresthatthedownloadedfileisalwayskeptondisconmanuallydeleted.deletepython:deleteobj,meta=dm.delete(key=key,local=True,remote=False)Inputkey:str- key under which the data to delete is stored.local[OPTIONAL]:bool- if set toTruewill delete local file.remote[OPTIONAL]:bool- if set toTruewill delete remote version of the requested key.Output -Noneterminal:dm_deletedm_delete--help

Setsupdamanpackage.
optionalarguments:-h,--helpshowthishelpmessageandexit--key{}keyofthefiletodelete--remoteWhenprovided,deletestherequestedfileonthecloudserviceaswell.

usage:dm_delete[-h]--key{}[--remote]