Initial version

README.md

@@ -0,0 +1,71 @@
+---
+tags:
+- monai
+- medical
+library_name: monai
+license: unknown
+---
+
+# MedNIST GAN Hand Model
+
+This model is a generator for creating images like the Hand category in the MedNIST dataset. It was trained as a GAN and accepts random values as inputs to produce an image output. The `train.json` file describes the training process along with the definition of the discriminator network used, and is based on the [MONAI GAN tutorials](https://github.com/Project-MONAI/tutorials/blob/main/modules/mednist_GAN_workflow_dict.ipynb).
+
+This is a demonstration network meant to just show the training process for this sort of network with MONAI, its outputs are not particularly good and are of the same tiny size as the images in MedNIST. The training process was very short so a network with a longer training time would produce better results.
+
+### Downloading the Dataset
+
+Download the dataset from [here](https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/MedNIST.tar.gz) and extract the contents to a convenient location.
+
+The MedNIST dataset was gathered from several sets from [TCIA](https://wiki.cancerimagingarchive.net/display/Public/Data+Usage+Policies+and+Restrictions),
+[the RSNA Bone Age Challenge](http://rsnachallenges.cloudapp.net/competitions/4),
+and [the NIH Chest X-ray dataset](https://cloud.google.com/healthcare/docs/resources/public-datasets/nih-chest).
+
+The dataset is kindly made available by [Dr. Bradley J. Erickson M.D., Ph.D.](https://www.mayo.edu/research/labs/radiology-informatics/overview) (Department of Radiology, Mayo Clinic)
+under the Creative Commons [CC BY-SA 4.0 license](https://creativecommons.org/licenses/by-sa/4.0/).
+
+
+If you use the MedNIST dataset, please acknowledge the source.
+
+### Training
+
+Assuming the current directory is the bundle directory, and the dataset was extracted to the directory `./MedNIST`, the following command will train the network for 50 epochs:
+
+```
+PYTHONPATH=./scripts python -m monai.bundle run training --meta_file configs/metadata.json --config_file configs/train.json --logging_file configs/logging.conf --bundle_root .
+```
+
+Note that the training code relies on extra scripts in the `scripts` directory which are made accessible by changing the `PYTHONPATH` variable in this invocation. If your `PYTHONPATH` is already used for other things you will have to add `./scripts` to the variable rather than replace it.
+
+Not also the output from the training will be placed in the `models` directory but will not overwrite the `model.pt` file that may be there already. You will have to manually rename the most recent checkpoint file to `model.pt` to use the inference script mentioned below after checking the results are correct. This saved checkpoint contains a dictionary with the generator weights stored as `model` and omits the discriminator.
+
+Another feature in the training file is the addition of sigmoid activation to the network by modifying it's structure at runtime. This is done with a line in the `training` section calling `add_module` on a layer of the network. This works best for training although the definition of the model now doesn't strictly match what it is in the `generator` section.
+
+The generator and discriminator networks were both trained with the `Adam` optimizer with a learning rate of 0.0002 and `betas` values `[0.5, 0.999]`. These have been emperically found to be good values for the optimizer and this GAN problem.
+
+### Inference
+
+The included `inference.json` generates a set number of png samples from the network and saves these to the directory `./outputs`. The output directory can be changed by setting the `output_dir` value, and the number of samples changed by setting the `num_samples` value. The following command line assumes it is invoked in the bundle directory:
+
+```
+python -m monai.bundle run inferring --meta_file configs/metadata.json --config_file configs/inference.json --logging_file configs/logging.conf --bundle_root .
+```
+
+Note this script uses postprocessing to apply the sigmoid activation the model's outputs and to save the results to image files.
+
+
+### Export
+
+The generator can be exported to a Torchscript bundle with the following:
+
+```
+python -m monai.bundle ckpt_export network_def --filepath mednist_gan.ts --ckpt_file models/model.pt --meta_file configs/metadata.json --config_file configs/inference.json
+```
+
+The model can be loaded without MONAI code after this operation. For example, an image can be generated from a set of random values with:
+
+```python
+import torch
+net = torch.jit.load("mednist_gan.ts")
+latent = torch.rand(1,64)
+img = net(latent)  # (1,1,64,64)
+```

configs/inference.json

@@ -0,0 +1,88 @@
+{
+    "imports": [
+        "$import glob"
+    ],
+    "device": "$torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')",
+    "ckpt_path": "$@bundle_root + '/models/model.pt'",
+    "output_dir": "./outputs",
+    "latent_size": 64,
+    "num_samples": 10,
+    "network_def": {
+        "_target_": "Generator",
+        "latent_shape": "@latent_size",
+        "start_shape": [
+            64,
+            8,
+            8
+        ],
+        "channels": [
+            32,
+            16,
+            8,
+            1
+        ],
+        "strides": [
+            2,
+            2,
+            2,
+            1
+        ]
+    },
+    "network": "$@network_def.to(@device)",
+    "dataset": {
+        "_target_": "Dataset",
+        "data": "$[torch.rand(@latent_size) for i in range(@num_samples)]"
+    },
+    "dataloader": {
+        "_target_": "DataLoader",
+        "dataset": "@dataset",
+        "batch_size": 1,
+        "shuffle": false,
+        "num_workers": 0
+    },
+    "inferer": {
+        "_target_": "SimpleInferer"
+    },
+    "postprocessing": {
+        "_target_": "Compose",
+        "transforms": [
+            {
+                "_target_": "Activationsd",
+                "keys": "pred",
+                "sigmoid": true
+            },
+            {
+                "_target_": "SaveImaged",
+                "keys": "pred",
+                "output_dir": "@output_dir",
+                "output_ext": "png",
+                "separate_folder": false,
+                "scale": 255,
+                "output_dtype": "$np.uint8",
+                "meta_key_postfix": null
+            }
+        ]
+    },
+    "handlers": [
+        {
+            "_target_": "CheckpointLoader",
+            "load_path": "@ckpt_path",
+            "load_dict": {
+                "model": "@network"
+            }
+        }
+    ],
+    "evaluator": {
+        "_target_": "SupervisedEvaluator",
+        "device": "@device",
+        "val_data_loader": "@dataloader",
+        "network": "@network",
+        "inferer": "@inferer",
+        "postprocessing": "@postprocessing",
+        "prepare_batch": "$lambda batchdata, *_,**__: (batchdata.to(@device),None,(),{})",
+        "val_handlers": "@handlers"
+    },
+    "inferring": [
+        "$@evaluator.run()"
+    ]
+}

configs/logging.conf

@@ -0,0 +1,21 @@
+[loggers]
+keys=root
+
+[handlers]
+keys=consoleHandler
+
+[formatters]
+keys=fullFormatter
+
+[logger_root]
+level=INFO
+handlers=consoleHandler
+
+[handler_consoleHandler]
+class=StreamHandler
+level=INFO
+formatter=fullFormatter
+args=(sys.stdout,)
+
+[formatter_fullFormatter]
+format=%(asctime)s - %(name)s - %(levelname)s - %(message)s

configs/metadata.json

@@ -0,0 +1,60 @@
+{
+    "schema": "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_generator_20220718.json",
+    "version": "0.1.0",
+    "changelog": {
+        "0.1.0": "Initial version"
+    },
+    "monai_version": "0.9.0",
+    "pytorch_version": "1.10.0",
+    "numpy_version": "1.21.0",
+    "optional_packages_version": {
+        "pytorch-ignite": "0.4.8",
+        "pillow": "8.4.0"
+    },
+    "task": "Generate random hand images from the MedNIST dataset",
+    "description": "This example of a GAN generator produces hand xray images like those in the MedNIST dataset",
+    "authors": "MONAI Team",
+    "copyright": "Copyright (c) MONAI Consortium",
+    "intended_use": "This is an example of a GAN with generator discriminator networks using MONAI, suitable for demonstration purposes only.",
+    "data_source": "MedNIST dataset kindly made available by Dr. Bradley J. Erickson M.D., Ph.D. (Department of Radiology, Mayo Clinic)",
+    "data_type": "jpeg",
+    "network_data_format": {
+        "inputs": {
+            "latent": {
+                "type": "tuples",
+                "format": "latent",
+                "num_channels": 0,
+                "spatial_shape": [
+                    64
+                ],
+                "dtype": "float32",
+                "value_range": [
+                    0,
+                    1
+                ],
+                "is_patch_data": false,
+                "channel_def": {}
+            }
+        },
+        "outputs": {
+            "pred": {
+                "type": "image",
+                "format": "magnitude",
+                "num_channels": 1,
+                "spatial_shape": [
+                    64,
+                    64
+                ],
+                "dtype": "float32",
+                "value_range": [
+                    0,
+                    1
+                ],
+                "is_patch_data": false,
+                "channel_def": {
+                    "0": "image"
+                }
+            }
+        }
+    }
+}

configs/train.json

@@ -0,0 +1,164 @@
+{
+    "imports": [
+        "$from functools import partial",
+        "$import glob",
+        "$from losses import discriminator_loss",
+        "$from losses import generator_loss"
+    ],
+    "device": "$torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')",
+    "ckpt_path": "$@bundle_root + '/models/model.pt'",
+    "dataset_dir": "./MedNIST/Hand",
+    "datalist": "$list(sorted(glob.glob(@dataset_dir + '/*.jpeg')))",
+    "latent_size": 64,
+    "discriminator": {
+        "_target_": "Discriminator",
+        "in_shape": [
+            1,
+            64,
+            64
+        ],
+        "channels": [
+            8,
+            16,
+            32,
+            64,
+            1
+        ],
+        "strides": [
+            2,
+            2,
+            2,
+            2,
+            1
+        ],
+        "num_res_units": 1,
+        "kernel_size": 5
+    },
+    "generator": {
+        "_target_": "Generator",
+        "latent_shape": "@latent_size",
+        "start_shape": [
+            64,
+            8,
+            8
+        ],
+        "channels": [
+            32,
+            16,
+            8,
+            1
+        ],
+        "strides": [
+            2,
+            2,
+            2,
+            1
+        ]
+    },
+    "dnetwork": "$@discriminator.apply(monai.networks.normal_init).to(@device)",
+    "gnetwork": "$@generator.apply(monai.networks.normal_init).to(@device)",
+    "preprocessing": {
+        "_target_": "Compose",
+        "transforms": [
+            {
+                "_target_": "LoadImaged",
+                "keys": "reals"
+            },
+            {
+                "_target_": "AddChanneld",
+                "keys": "reals"
+            },
+            {
+                "_target_": "ScaleIntensityd",
+                "keys": "reals"
+            },
+            {
+                "_target_": "RandRotated",
+                "keys": "reals",
+                "range_x": "$np.pi/12",
+                "prob": 0.5,
+                "keep_size": true
+            },
+            {
+                "_target_": "RandFlipd",
+                "keys": "reals",
+                "spatial_axis": 0,
+                "prob": 0.5
+            },
+            {
+                "_target_": "RandZoomd",
+                "keys": "reals",
+                "min_zoom": 0.9,
+                "max_zoom": 1.1,
+                "prob": 0.5
+            },
+            {
+                "_target_": "EnsureTyped",
+                "keys": "reals"
+            }
+        ]
+    },
+    "real_dataset": {
+        "_target_": "CacheDataset",
+        "data": "$[{'reals': i} for i in @datalist]",
+        "transform": "@preprocessing"
+    },
+    "real_dataloader": {
+        "_target_": "DataLoader",
+        "dataset": "@real_dataset",
+        "batch_size": 600,
+        "shuffle": true,
+        "num_workers": 12
+    },
+    "doptimizer": {
+        "_target_": "torch.optim.Adam",
+        "params": "$@dnetwork.parameters()",
+        "lr": 0.0002,
+        "betas": [
+            0.5,
+            0.999
+        ]
+    },
+    "goptimizer": {
+        "_target_": "torch.optim.Adam",
+        "params": "$@gnetwork.parameters()",
+        "lr": 0.0002,
+        "betas": [
+            0.5,
+            0.999
+        ]
+    },
+    "handlers": [
+        {
+            "_target_": "CheckpointSaver",
+            "save_dir": "$@bundle_root + '/models'",
+            "save_dict": {
+                "model": "@gnetwork"
+            },
+            "save_interval": 0,
+            "save_final": true,
+            "epoch_level": true
+        }
+    ],
+    "trainer": {
+        "_target_": "GanTrainer",
+        "device": "@device",
+        "max_epochs": 50,
+        "train_data_loader": "@real_dataloader",
+        "g_network": "@gnetwork",
+        "g_optimizer": "@goptimizer",
+        "g_loss_function": "$partial(generator_loss, disc_net=@dnetwork)",
+        "d_network": "@dnetwork",
+        "d_optimizer": "@doptimizer",
+        "d_loss_function": "$partial(discriminator_loss, disc_net=@dnetwork)",
+        "d_train_steps": 5,
+        "g_update_latents": true,
+        "latent_shape": "@latent_size",
+        "key_train_metric": "$None",
+        "train_handlers": "@handlers"
+    },
+    "training": [
+        "$@gnetwork.conv.add_module('activation', torch.nn.Sigmoid())",
+        "$@trainer.run()"
+    ]
+}

docs/README.md

@@ -0,0 +1,64 @@
+
+# MedNIST GAN Hand Model
+
+This model is a generator for creating images like the Hand category in the MedNIST dataset. It was trained as a GAN and accepts random values as inputs to produce an image output. The `train.json` file describes the training process along with the definition of the discriminator network used, and is based on the [MONAI GAN tutorials](https://github.com/Project-MONAI/tutorials/blob/main/modules/mednist_GAN_workflow_dict.ipynb).
+
+This is a demonstration network meant to just show the training process for this sort of network with MONAI, its outputs are not particularly good and are of the same tiny size as the images in MedNIST. The training process was very short so a network with a longer training time would produce better results.
+
+### Downloading the Dataset
+
+Download the dataset from [here](https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/MedNIST.tar.gz) and extract the contents to a convenient location.
+
+The MedNIST dataset was gathered from several sets from [TCIA](https://wiki.cancerimagingarchive.net/display/Public/Data+Usage+Policies+and+Restrictions),
+[the RSNA Bone Age Challenge](http://rsnachallenges.cloudapp.net/competitions/4),
+and [the NIH Chest X-ray dataset](https://cloud.google.com/healthcare/docs/resources/public-datasets/nih-chest).
+
+The dataset is kindly made available by [Dr. Bradley J. Erickson M.D., Ph.D.](https://www.mayo.edu/research/labs/radiology-informatics/overview) (Department of Radiology, Mayo Clinic)
+under the Creative Commons [CC BY-SA 4.0 license](https://creativecommons.org/licenses/by-sa/4.0/).
+
+
+If you use the MedNIST dataset, please acknowledge the source.
+
+### Training
+
+Assuming the current directory is the bundle directory, and the dataset was extracted to the directory `./MedNIST`, the following command will train the network for 50 epochs:
+
+```
+PYTHONPATH=./scripts python -m monai.bundle run training --meta_file configs/metadata.json --config_file configs/train.json --logging_file configs/logging.conf --bundle_root .
+```
+
+Note that the training code relies on extra scripts in the `scripts` directory which are made accessible by changing the `PYTHONPATH` variable in this invocation. If your `PYTHONPATH` is already used for other things you will have to add `./scripts` to the variable rather than replace it.
+
+Not also the output from the training will be placed in the `models` directory but will not overwrite the `model.pt` file that may be there already. You will have to manually rename the most recent checkpoint file to `model.pt` to use the inference script mentioned below after checking the results are correct. This saved checkpoint contains a dictionary with the generator weights stored as `model` and omits the discriminator.
+
+Another feature in the training file is the addition of sigmoid activation to the network by modifying it's structure at runtime. This is done with a line in the `training` section calling `add_module` on a layer of the network. This works best for training although the definition of the model now doesn't strictly match what it is in the `generator` section.
+
+The generator and discriminator networks were both trained with the `Adam` optimizer with a learning rate of 0.0002 and `betas` values `[0.5, 0.999]`. These have been emperically found to be good values for the optimizer and this GAN problem.
+
+### Inference
+
+The included `inference.json` generates a set number of png samples from the network and saves these to the directory `./outputs`. The output directory can be changed by setting the `output_dir` value, and the number of samples changed by setting the `num_samples` value. The following command line assumes it is invoked in the bundle directory:
+
+```
+python -m monai.bundle run inferring --meta_file configs/metadata.json --config_file configs/inference.json --logging_file configs/logging.conf --bundle_root .
+```
+
+Note this script uses postprocessing to apply the sigmoid activation the model's outputs and to save the results to image files.
+
+
+### Export
+
+The generator can be exported to a Torchscript bundle with the following:
+
+```
+python -m monai.bundle ckpt_export network_def --filepath mednist_gan.ts --ckpt_file models/model.pt --meta_file configs/metadata.json --config_file configs/inference.json
+```
+
+The model can be loaded without MONAI code after this operation. For example, an image can be generated from a set of random values with:
+
+```python
+import torch
+net = torch.jit.load("mednist_gan.ts")
+latent = torch.rand(1,64)
+img = net(latent)  # (1,1,64,64)
+```

docs/license.txt

@@ -0,0 +1,24 @@
+Copyright 2022 MONAI Consortium
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+    http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+Third Party Licenses
+-----------------------------------------------------------------------
+
+/*********************************************************************/
+i. MedNIST Dataset
+The dataset is kindly made available by Dr. Bradley J. Erickson M.D., Ph.D. (https://www.mayo.edu/research/labs/radiology-informatics/overview), Department of Radiology, Mayo Clinic under the Creative Commons [CC BY-SA 4.0 license](https://creativecommons.org/licenses/by-sa/4.0/).
+
+The MedNIST dataset was gathered from several sets from:
+ * TCIA (https://wiki.cancerimagingarchive.net/display/Public/Data+Usage+Policies+and+Restrictions)
+ * the RSNA Bone Age Challenge (http://rsnachallenges.cloudapp.net/competitions/4),
+ * the NIH Chest X-ray dataset (https://cloud.google.com/healthcare/docs/resources/public-datasets/nih-chest).
+
+If you use the MedNIST dataset, please acknowledge the source. For the license and usage conditions of the source datasets, please see their respective sites.

models/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ee4b4a812063936e95a5e39e628884b0160ab5be4897d99ede3dc7746c4308a
+size 1272514

scripts/losses.py

@@ -0,0 +1,22 @@
+import torch
+
+disc_loss_criterion = torch.nn.BCELoss()
+gen_loss_criterion = torch.nn.BCELoss()
+real_label = 1
+fake_label = 0
+
+
+def discriminator_loss(gen_images, real_images, disc_net):
+    real = real_images.new_full((real_images.shape[0], 1), real_label)
+    gen = gen_images.new_full((gen_images.shape[0], 1), fake_label)
+
+    realloss = disc_loss_criterion(disc_net(real_images), real)
+    genloss = disc_loss_criterion(disc_net(gen_images.detach()), gen)
+
+    return (genloss + realloss) / 2
+
+
+def generator_loss(gen_images, disc_net):
+    output = disc_net(gen_images)
+    cats = output.new_full(output.shape, real_label)
+    return gen_loss_criterion(output, cats)