Package: tfcaidm
TensorFlow CAIDMDeep learning pipeline for medical imagingSite|Slides|LicenseIntroductionTFCAIDMis a unified framework for building and training medical imaging deep learning models built on top ofTensorFlowandJarvisMD. The library supports interfacing custom datasets withjarvis, model development withtensorflow, and built-in reproducibility, traceability, and performance logging for all experiments. User's can train or extend pre-existing models that have been implemented inMODEL_ZOO.mdor define their own.Available FeaturesReusable state-of-the-art deep learning model blocksSupport for training multiple models in parallelHigh-level interface for customizing datasets, models, loss functions, training routines, etc.Reproducibility, performance logging, model checkpointing, and hyperparameter trackingUpcoming FeaturesAutoML / efficient hyperparameter searchDistributed data and model trainingVision transformer modelsBetter documentationMore InformationYAML configuration filesHyperparameter tuningSupported modelsCustomizabilityViewing resultsBenchmarks (coming soon)Disclaimer: The library is primarily built for users with access to the caidm clusters, though general users are also supported.InstallationThe current library is supported on python 3.7 and tensorflow 2.5+, and the installation instructions provided below assume that your system is already equipped with cuda and nvcc.Local InstallationInstall using thecondavirtual environment.Whereuseris your account username.user$condacreate--nametfcaidmpython=3.7
user$condaactivatetfcaidm
user(tfcaidm)$pipinstalltensorflow
user(tfcaidm)$pipinstalljarvis-md
user(tfcaidm)$pipinstalltfcaidmExampleTraining a set of models require two separate python scripts: a training submission script and a training routine script.Training Submissionfromjarvis.utils.generalimportgpusfromtfcaidmimportJobs# --- Define pathsYML_CONFIG="pipeline.yml"TRAIN_ROUTINE_PATH="main.py"# --- Submit a training jobJobs(path=YML_CONFIG).setup(producer=__file__,consumer=TRAIN_ROUTINE_PATH,).train_cluster()Automated Training Routinefromjarvis.trainimportparamsfromjarvis.utils.generalimportgpusfromtfcaidmimportTrainer# --- Autoselect GPU (use only on caidm cluster)gpus.autoselect()# --- Get hyperparameters (args passed by environment variables)hyperparams=params.load()# --- Train model (dataset and model created within trainer)trainer=Trainer(hyperparams)results=trainer.cross_validation(save=True)trainer.save_results(results)Custom Training Routinefromjarvis.trainimportparamsfromjarvis.utils.generalimportgpus,overloadfromtfcaidmimportJClientfromtfcaidmimportModelfromtfcaidmimportTrainer# --- Autoselect GPU (use only on caidm cluster)gpus.autoselect()# --- Get hyperparameters (args passed by environment variables)hyperparams=params.load()# --- Setup custom dataset generator (more details in notebooks)@overload(JClient)defcreate_generator(self,gen_data):forxs,ysingen_data:# --- User defined codexs=DataAugment(xs)yieldxs,ys# --- Setup custom model (more details in notebooks)@overload(Model)defcreate(self):# --- User defined codemodel=ViT(...)model.compile(...)returnmodel# --- Train model (dataset and model created within trainer)trainer=Trainer(hyperparams)results=trainer.cross_validation(save=True)trainer.save_results(results)# See notebooks for a breakdown on customizabilityFor an example project, seeexamples/projects. For a more detailed walkthrough of the library, seenotebooks.Sister Repositoriespeterchang77/caidmpeterchang77/dl_tutor