Package: darwin-pyproj-test
V7 Darwin Python SDKOfficial library to manage datasets along withV7 Darwin annotation platform.Darwin-py can both be used from thecommand lineand as apython library.Main functions are (but not limited to):Client authenticationListing local and remote datasetsCreate/remove datasetsUpload/download data to/from remote datasetsDirect integration with PyTorch dataloadersSupport tested for python 3.8.Installationpip install darwin-pyYou can now typedarwinin your terminal and access the command line interface.If you wish to use the PyTorch bindings, then you can use themlflag to install all the additional requirementspip install darwin-py[ml]To run test, first install thetestextra packagepip install darwin-py[test]Usage as a Command Line Interface (CLI)Once installed,darwinis accessible as a command line tool.
A useful way to navigate the CLI usage is through the help command-h/--helpwhich will
provide additional information for each command available.Client AuthenticationTo perform remote operations on Darwin you first need to authenticate.
This requires ateam-specific API-key.
If you do not already have a Darwin account, you cancontact usand we can set one up for you.To start the authentication process:$ darwin authenticate
API key:
Make example-team the default team? [y/N] y
Datasets directory [~/.darwin/datasets]:
Authentication succeeded.You will be then prompted to enter your API-key, whether you want to set the corresponding team as
default and finally the desired location on the local file system for the datasets of that team.
This process will create a configuration file at~/.darwin/config.yaml.
This file will be updated with future authentications for different teams.Listing local and remote datasetsLists a summary of local existing datasets$ darwin dataset local
NAME            IMAGES     SYNC_DATE         SIZE
mydataset       112025     yesterday     159.2 GBLists a summary of remote datasets accessible by the current user.$ darwin dataset remote
NAME                       IMAGES     PROGRESS
example-team/mydataset     112025        73.0%Create/remove a datasetTo create an empty dataset remotely:$ darwin dataset create test
Dataset 'test' (example-team/test) has been created.
Access at https://darwin.v7labs.com/datasets/579The dataset will be created in the team you're authenticated for.To delete the project on the server:$ darwin dataset remove test
About to delete example-team/test on darwin.
Do you want to continue? [y/N] yUpload/download data to/from a remote datasetUploads data to an existing remote project.
It takes the dataset name and a single image (or directory) with images/videos to upload as
parameters.The-e/--excludeargument allows to indicate file extension/s to be ignored from the data_dir.
e.g.:-e .jpgFor videos, the frame rate extraction rate can be specified by adding--fps <frame_rate>Supported extensions:Video files: [.mp4,.bpm,.movformats].Image files [.jpg,.jpeg,.pngformats].$ darwin dataset push test /path/to/folder/with/images
100%|████████████████████████| 2/2 [00:01<00:00,  1.27it/s]Before a dataset can be downloaded, a release needs to be generated:$ darwin dataset export test 0.1
Dataset test successfully exported to example-team/test:0.1This version is immutable, if new images / annotations have been added you will have to create a new release to included them.To list all available releases$ darwin dataset releases test
NAME                           IMAGES     CLASSES                   EXPORT_DATE
example-team/test:0.1               4           0     2019-12-07 11:37:35+00:00And to finally download a release.$ darwin dataset pull test:0.1
Dataset example-team/test:0.1 downloaded at /directory/choosen/at/authentication/time.Usage as a Python libraryThe framework is designed to be usable as a standalone python library.
Usage can be inferred from looking at the operations performed indarwin/cli_functions.py.
A minimal example to download a dataset is provided below and a more extensive one can be found indarwin_demo.py.fromdarwin.clientimportClientclient=Client.local()# use the configuration in ~/.darwin/config.yamldataset=client.get_remote_dataset("example-team/test")dataset.pull()# downloads annotations and images for the latest exported versionFollowthis guidefor how to integrate darwin datasets directly in PyTorch.