Package: libica
libicaPython SDK to programmatically call Illumina Connected Analytics (ICA) Genomic (multi-omics) data platform and BioInformatics web services. This SDK supports both ICA v1 and v2 APIs:ICAv1 API:https://illumina.gitbook.io/ica-v1/reference/r-apiICAv2 API:https://help.ica.illumina.com/reference/r-apiInstall throughpiplike so:pip install libicaOverviewTested for Python 3.7, 3.8, 3.9, 3.10, 3.11SeeChangeLogandMilestonesTest CoverageWikiSDK Guide:PyDocUser Guide:https://umccr-illumina.github.io/libica/Getting started with SDK for ICA v2Seepilot.py# List all data in a projectimportosfromcontextlibimportclosingfromlibica.openapi.v2importConfiguration,ApiClient,ApiExceptionfromlibica.openapi.v2.api.project_data_apiimportProjectDataApifromlibica.openapi.v2.model.project_dataimportProjectDatafromlibica.openapi.v2.model.project_data_paged_listimportProjectDataPagedListif__name__=='__main__':project_id=os.environ['ICAV2_PROJECT_ID']file_path=[""]# empty will list everything under projecticav2_access_token=os.environ['ICAV2_ACCESS_TOKEN']ica_url="https://ica.illumina.com/ica/rest"configuration=Configuration(host=ica_url,access_token=icav2_access_token,)# configuration.debug = True  # uncomment to debug API call loggingapi_client=ApiClient(configuration=configuration,header_name="Content-Type",header_value="application/vnd.illumina.v3+json",)withclosing(api_client)asctx_api_client:project_data_api:ProjectDataApi=ProjectDataApi(api_client=ctx_api_client)try:page_token=""whileTrue:project_data_paged_list:ProjectDataPagedList=project_data_api.get_project_data_list(project_id=project_id,file_path=file_path,page_size=str(1000),page_token=page_token,)foriteminproject_data_paged_list.items:project_data:ProjectData=itemprint((project_data.data.id,# fil.<ID> (or) fol.<ID>project_data.data.details.path,project_data.data.details.data_type,))page_token=project_data_paged_list.next_page_tokenifnotproject_data_paged_list.next_page_token:breakexceptApiExceptionase:print(e)You may considerUsing App Package(see below).Getting started with SDK for ICA v1Export ICA base URL and JWT Bearer token:export ICA_BASE_URL=<baseUrl>
export ICA_ACCESS_TOKEN=<tok>Generate Bearer JWT token usingICA CLIlike so:ica tokens create --helpSomewhere in your Python code:importosfromlibica.openapiimportlibwesfromlibica.openapi.libwesimportWorkflowList,WorkflowCompactica_base_url=os.getenv("ICA_BASE_URL")ica_access_token=os.getenv("ICA_ACCESS_TOKEN")configuration=libwes.Configuration(host=ica_base_url,api_key={'Authorization':ica_access_token},api_key_prefix={'Authorization':"Bearer"},)withlibwes.ApiClient(configuration)asapi_client:wfl_api:libwes.WorkflowsApi=libwes.WorkflowsApi(api_client)try:page_token=NonewhileTrue:wfl_list:WorkflowList=wfl_api.list_workflows(page_size=100,page_token=page_token)# print(wfl_list)foriteminwfl_list.items:wfl:WorkflowCompact=itemprint(wfl.id)print(wfl.name)page_token=wfl_list.next_page_tokenifnotwfl_list.next_page_token:breakexceptlibwes.ApiExceptionase:print(e)UsingApp PackageNOTE:libica.apppackage contains reusable modules that are based on use cases around UMCCRData Portal backend,Workflows automation and orchestrationimplementations. Hence, it may be a specific domain logic implementation. However, it may still be reusable for your use cases. Starter examples are as follows.App for ICA v2Seepilotapp.pyExample:ProjectDataOpsapp to list project files, download a file, etc...fromcontextlibimportclosingfromlibica.appimportAppHelperfromlibica.app.dataopsimportProjectDataOpsfromlibica.openapi.v2importApiClientfromlibica.openapi.v2.model.project_dataimportProjectData# --- Use AppHelper to build SDK API clientapp_helper=AppHelper(debug=False)project_id=app_helper.get_icav2_cli_session_project_id()cli_session_api_client:ApiClient=app_helper\.build_icav2_configuration_with_cli_session()\.get_icav2_api_client()# --- Construct ProjectDataOps from dataops moduleproject_dataops=ProjectDataOps(project_id=project_id,api_client=cli_session_api_client)# --- List all files under given project's folder# If you do not cd into the folder, it will list all files under the projectproject_dataops.cd("/test_folder/")foriteminproject_dataops.list_files():project_data:ProjectData=itemprint((project_data.data.id,# fil.<ID> (or) fol.<ID>project_data.data.details.path,project_data.data.details.data_type,project_data.data.details.name,project_data.data.details.status,project_data.data.details.file_size_in_bytes,project_data.data.details.time_created,))# --- Download csv file from given project and file pathfile_path="/test_folder/SampleSheet.csv"print(f"Downloading{file_path}from project_id{project_id}")project_dataops.cd(file_path=file_path)ntf=project_dataops.download_file()withclosing(ntf)ascf:withopen(cf.name,'r')asf:forlineinf.readlines():print(line)For more, see PyDoc:https://umccr-illumina.github.io/libica/libica/app/dataops.htmlApp for ICA v1Example: Configuration Object Builderfromlibica.appimportconfigurationfromlibica.openapiimportlibgdsgds_config=configuration(lib=libgds,# pass in library of interest e.g. libwes, libtes, etcsecret_name=["FROM_AWS_SECRET_MANAGER_THAT_STORE_ICA_ACCESS_TOKEN"],base_url="https://use1.platform.illumina.com",# overwrite if not https://aps2.platform.illumina.comdebug=False,# True if you like to debug API calls, False by default anyway, just for demo)withlibgds.ApiClient(gds_config)asapi_client:...Example: Listing Files from GDSfromtypingimportListfromlibica.appimportgdsfromlibica.openapiimportlibgdsvol,path=gds.parse_path("gds://development/some/folder/path/")files:List[libgds.FileResponse]=gds.get_file_list(volume_name=vol,path=path)forfileinfiles:print(f"{file.name},{file.volume_name},{file.path},{file.presigned_url}")DevelopmentSetup virtual environment and activate itInstall dev dependenciesmake installTo bring upmockAPIμ-servicesmake upTo run tests suitesmake unit
make autounitTo run full suite, smoke testmake testAutoGen WorkflowSDK is autogenerated from OpenAPI (Swagger) definitionsThere are few tools required for this autogen workflow to work.openapi-generator-cli-- used to generate code and docswagger-cli-- validate definitionsThese tools are Node.js app, hence, required build toolsnode,npm,npxandyarnas follows.node -v
v18.16.1

npm i -g yarn
yarn -v
3.5.1

yarn install

npx openapi-generator-cli help
npx swagger-cli --helpAPI mock serverprismis set up through docker compose as follows.make up
make ps
make test_ica_mock
make test_icav2_mock
docker compose logsAll the autogen config and arrangement refer tosyncapi.shandsyncapi2.shwhich is called throughMakefiletargets.Generator VersionSee generatorversion compatibilitySelect generator version as follows:npx openapi-generator-cli version-manager listNoticeMIT License and DISCLAIMERThe Advanced Genomics Collaboration (TAGC)