Package: aind-codeocean-utils
aind-codeocean-utilsLibrary to contain useful utility methods to interface with Code Ocean.InstallationTo use the package, you can install it frompypi:pipinstallaind-codeocean-utilsTo install the package from source, in the root directory, runpipinstall-e.To develop the code, runpipinstall-e.[dev]UsageThe package includes helper functions to interact with Code Ocean:CodeOceanJobThis class enables one to run a job that:Registers a new asset to Code Ocean from s3Runs a capsule/pipeline on the newly registered asset (or an existing assey)Captures the run results into a new assetSteps 1 and 3 are optional, while step 2 (running the computation) is mandatory.Here is a full example that registers a new ecephys asset, runs the spike sorting
capsule with some parameters, and registers the results:importosfromaind_codeocean_api.codeoceanimportCodeOceanClientfromaind_codeocean_utils.codeocean_jobimport(CodeOceanJob,CodeOceanJobConfig)# Set up the CodeOceanClient from aind_codeocean_apiCO_TOKEN=os.environ["CO_TOKEN"]CO_DOMAIN=os.environ["CO_DOMAIN"]co_client=CodeOceanClient(domain=CO_DOMAIN,token=CO_TOKEN)# Define Job Parametersjob_config_dict=dict(register_config=dict(asset_name="test_dataset_for_codeocean_job",mount="ecephys_701305_2023-12-26_12-22-25",bucket="aind-ephys-data",prefix="ecephys_701305_2023-12-26_12-22-25",tags=["codeocean_job_test","ecephys","701305","raw"],custom_metadata={"modality":"extracellular electrophysiology","data level":"raw data",},viewable_to_everyone=True),run_capsule_config=dict(data_assets=None,# when None, the newly registered asset will be usedcapsule_id="a31e6c81-49a5-4f1c-b89c-2d47ae3e02b4",run_parameters=["--debug","--no-remove-out-channels"]),capture_result_config=dict(process_name="sorted",tags=["np-ultra"]# additional tags to the ones inherited from input))# instantiate config modeljob_config=CodeOceanJobConfig(**job_config_dict)# instantiate code ocean jobco_job=CodeOceanJob(co_client=co_client,job_config=job_config)# run and wait for resultsjob_response=co_job.run_job()This job will:Register thetest_dataset_for_codeocean_jobasset from the specified s3 bucket and prefixRun the capsulea31e6c81-49a5-4f1c-b89c-2d47ae3e02b4with the specified parametersRegister the result astest_dataset_for_codeocean_job_sorter_{date-time}To run a computation on existing data assets, do not provide theregister_configand
provide thedata_assetfield in therun_capsule_config.To skip capturing the result, do not provide thecapture_result_configoption.ContributingLinters and testingThere are several libraries used to run linters, check documentation, and run tests.Please test your changes using thecoveragelibrary, which will run the tests and log a coverage report:coveragerun-munittestdiscover&&coveragereportUseinterrogateto check that modules, methods, etc. have been documented thoroughly:interrogate.Useflake8to check that code is up to standards (no unused imports, etc.):flake8.Useblackto automatically format the code into PEP standards:black.Useisortto automatically sort import statements:isort.Pull requestsFor internal members, please create a branch. For external members, please fork the repository and open a pull request from the fork. We'll primarily useAngularstyle for commit messages. Roughly, they should follow the pattern:<type>(<scope>): <short summary>where scope (optional) describes the packages affected by the code changes and type (mandatory) is one of:build: Changes that affect build tools or external dependencies (example scopes: pyproject.toml, setup.py)ci: Changes to our CI configuration files and scripts (examples: .github/workflows/ci.yml)docs: Documentation only changesfeat: A new featurefix: A bugfixperf: A code change that improves performancerefactor: A code change that neither fixes a bug nor adds a featuretest: Adding missing tests or correcting existing testsSemantic ReleaseThe table below, fromsemantic release, shows which commit message gets you which release type whensemantic-releaseruns (using the default configuration):Commit messageRelease typefix(pencil): stop graphite breaking when too much pressure appliedPatchFix Release, Default releasefeat(pencil): add 'graphiteWidth' optionMinorFeature Releaseperf(pencil): remove graphiteWidth optionBREAKING CHANGE: The graphiteWidth option has been removed.The default graphite width of 10mm is always used for performance reasons.MajorBreaking Release(Note that theBREAKING CHANGE:token must be in the footer of the commit)DocumentationTo generate the rst files source files for documentation, runsphinx-apidoc-odoc_template/source/srcThen to create the documentation HTML files, runsphinx-build-bhtmldoc_template/source/doc_template/build/htmlMore info on sphinx installation can be foundhere.