Package: hi-ml-cpath
Histopathology Models and WorkflowsModels on public dataThis repository contains a set of models built on and for public datasets (PANDA, TCGA). Detailed instructions to
on-board the datasets and run the models are providedon
readthedocs.Getting startedSetting up PythonFor working on the histopathology folder, please create a separate Conda environment.cdhi-ml-cpath
makeenvYou can then activate the environment viaconda activate HimlHisto. Set VSCode to use this Conda environment, by choosing "Python: Select Interpreter"
from the command palette.If the dependencies need to be updated, please modifyhi-ml-cpath/primary_deps.yml, and then run the scripthi-ml-cpath/create_and_lock_environment.sh. This will create a full "locked" environment specification with pinned
versions of all depdencies.Setting up AzureMLIn addition, please download an AzureML workspace configuration file for the workspace that you wish to use:In the browser, navigate to the workspace in questionClick on the drop-down menu on upper right of the page, to the left of your account picture.Select "Download config file".Save that file into the the repository root.Once that config file is in place, all Python runs that you start inside thehi-ml-cpathfolder will automatically use this config file.Running histopathology modelsTo test your setup, please execute in thehi-ml-cpathfolder:condaactivateHimlHisto
python../hi-ml/src/health_ml/runner.py--modelhealth_cpath.TcgaCrckImageNetMIL--cluster=training-nd24This should start an AzureML job in the AzureML workspace that you configured above viaconfig.json. You may need to adjust the name of
the compute cluster (training-nd24in the above example).Conda environmentIf you start your jobs in thehi-ml-cpathfolder, they will automatically pick up the Conda environment file that is present in that folder.
If you start your jobs in a different folder, you need to add the--conda_envoption to point to the file<repo_root>/hi-ml-cpath/environment.yml.Running histopathology testsIn thehi-ml-cpathfolder, runmakecall_pytestInside of VSCode, all tests in the repository should be picked up automatically. You can exclude the tests for thehi-mlandhi-ml-azurepackages by
modifyingpython.testing.pytestArgsin the VSCode.vscode/settings.jsonfile.Tests that require a GPUThe test pipeline for the histopathology folder contains a run ofpyteston a machine with 2 GPUs. Only tests that are
marked with thepytestmarkgpuare executed on that GPU machine. Note that all tests that bear thegpumark willalsobe executed when running on a CPU machine. You need to manually add askipifflag for tests that are meant to
exclusively run on GPU machines. This also helps to ensure that the test suite can pass when executed outside of the
build agents.Tests that run only on a CPU machine: Provide nopytestmarksdeftest_my_code()->None:passTests that run on both on a CPU and on a GPU machine: Add@pytest.mark.gpu@pytest.mark.gpudeftest_my_code()->None:passTests that run only on a GPU machine:fromhealth_ml.utils.common_utilsimportis_gpu_availableno_gpu=notis_gpu_available()@pytest.mark.skipif(no_gpu,reason="Test requires GPU")@pytest.mark.gpudeftest_my_code()->None:pass