Repository: mlcommons/GaNDLF

Issue:
Ensure correct augmentation order 
**Describe the bug**
For MR images, when multiple augmentations/preprocessing is selected, the correct order needs to be maintained.

**To Reproduce**
This is not really an error as much as incorrect computation.

**Expected behavior**
MR images need some augmentations to be handled BEFORE any normalization happens. The order should be:

- Affine / Resample 
- Blur 
- Gamma (to change the subject intrinsic contrast, before applying another artefact)
- Bias
- Ghosting / Spike / Motion
- Noise
- Normalize / intensity scaling

Rest of the augmentations can work on normalized images, so the current pipeline _should_ work.

**Screenshots**
N.A.

**GaNDLF Version**
0.0.10-dev

**Desktop (please complete the following information):**
N.A.

**Additional context**
Please see discussion in https://github.com/fepegar/torchio/discussions/600



Discussion:
This will touch into `gandlf_preprocess` as well. We would need to modify `ImagesFromDataFrame` to ensure the order is maintained (need to add another mode other than "train" and "test"). 

There will be 2 solutions:
1. Add augmentations for `gandlf_preprocess`, this would reduce the variability during training.
2. If any of these augmentations are detected, `gandlf_preprocess` stops running and asks uses to directly use `gandlf_run`.
Solution after discussion with @sbakas and @Geeks-Sid as of 2021/08/26:

1. First is augmentation
    - Leave order to the user (because this can be application-dependent)
    - Take order from configuration directly 
2. Second is preprocessing
    - Leave order to the user (because this can be application-dependent)
    - Take order from configuration directly 
    - Normalization should **always** happen at the end

Resolving patch:
diff --git a/GANDLF/data/ImagesFromDataFrame.py b/GANDLF/data/ImagesFromDataFrame.py
--- a/GANDLF/data/ImagesFromDataFrame.py
+++ b/GANDLF/data/ImagesFromDataFrame.py
@@ -174,7 +174,7 @@ def ImagesFromDataFrame(dataframe, parameters, train):
             subject_dict["value_" + str(valueCounter)] = np.array(
                 dataframe[values][patient]
             )
-            valueCounter = valueCounter + 1
+            valueCounter += 1
 
         # skip subject the condition was tripped
         if not skip_subject:
@@ -207,50 +207,9 @@ def ImagesFromDataFrame(dataframe, parameters, train):
             # Appending this subject to the list of subjects
             subjects_list.append(subject)
 
-    augmentation_list = []
+    transformations_list = []
 
-    # first, we want to do thresholding, followed by clipping, if it is present - required for inference as well
-    if not (preprocessing is None):
-        if "to_canonical" in preprocessing:
-            augmentation_list.append(ToCanonical())
-
-        if train:  # we want the crop to only happen during training
-            if "crop_external_zero_planes" in preprocessing:
-                augmentation_list.append(
-                    global_preprocessing_dict["crop_external_zero_planes"](patch_size)
-                )
-        for key in ["threshold", "clip"]:
-            if key in preprocessing:
-                augmentation_list.append(
-                    global_preprocessing_dict[key](
-                        min_thresh=preprocessing[key]["min"],
-                        max_thresh=preprocessing[key]["max"],
-                    )
-                )
-
-        # first, we want to do the resampling, if it is present - required for inference as well
-        if "resample" in preprocessing:
-            if "resolution" in preprocessing["resample"]:
-                # resample_split = str(aug).split(':')
-                resample_values = tuple(
-                    np.array(preprocessing["resample"]["resolution"]).astype(np.float)
-                )
-                if len(resample_values) == 2:
-                    resample_values = tuple(np.append(resample_values, 1))
-                augmentation_list.append(Resample(resample_values))
-
-        # next, we want to do the intensity normalize - required for inference as well
-        if "normalize" in preprocessing:
-            augmentation_list.append(global_preprocessing_dict["normalize"])
-        elif "normalize_nonZero" in preprocessing:
-            augmentation_list.append(global_preprocessing_dict["normalize_nonZero"])
-        elif "normalize_nonZero_masked" in preprocessing:
-            augmentation_list.append(
-                global_preprocessing_dict["normalize_nonZero_masked"]
-            )
-
-    # other augmentations should only happen for training - and also setting the probabilities
-    # for the augmentations
+    # augmentations are applied to the training set only
     if train and not (augmentations == None):
         for aug in augmentations:
             if aug != "default_probability":
@@ -269,7 +228,7 @@ def ImagesFromDataFrame(dataframe, parameters, train):
                     )
                 elif aug in ["rotate_90", "rotate_180"]:
                     for axis in augmentations[aug]["axis"]:
-                        augmentation_list.append(
+                        transformations_list.append(
                             global_augs_dict[aug](
                                 axis=axis, p=augmentations[aug]["probability"]
                             )
@@ -300,10 +259,68 @@ def ImagesFromDataFrame(dataframe, parameters, train):
                         p=augmentations[aug]["probability"]
                     )
                 if actual_function is not None:
-                    augmentation_list.append(actual_function)
+                    transformations_list.append(actual_function)
+
+    # first, we want to do thresholding, followed by clipping, if it is present - required for inference as well
+    current_idx = 0
+    normalize_to_apply = None
+    if not (preprocessing is None):
+        # go through preprocessing in the order they are specified
+        for preprocess in preprocessing:
+            preprocess_lower = preprocess.lower()
+            # special check for resample
+            if preprocess_lower == "resample":
+                if "resolution" in preprocessing[preprocess_lower]:
+                    # resample_split = str(aug).split(':')
+                    resample_values = tuple(
+                        np.array(preprocessing["resample"]["resolution"]).astype(
+                            np.float
+                        )
+                    )
+                    if len(resample_values) == 2:
+                        resample_values = tuple(np.append(resample_values, 1))
+                    transformations_list.append(Resample(resample_values))
+
+            # special check for to_canonical
+            if preprocess_lower == "to_canonical":
+                transformations_list.append(ToCanonical())
+
+            for key in ["threshold", "clip"]:
+                if preprocess_lower == key:
+                    transformations_list.append(
+                        global_preprocessing_dict[key](
+                            min_thresh=preprocessing[key]["min"],
+                            max_thresh=preprocessing[key]["max"],
+                        )
+                    )
+
+            # intensity normalization is always applied last
+            if preprocess_lower == "normalize":
+                normalize_to_apply = global_preprocessing_dict["normalize"]
+            elif preprocess_lower == "normalize_nonZero":
+                normalize_to_apply = global_preprocessing_dict["normalize_nonZero"]
+            elif preprocess_lower == "normalize_nonZero_masked":
+                normalize_to_apply = global_preprocessing_dict[
+                    "normalize_nonZero_masked"
+                ]
+
+            # preprocessing routines that we only want for training
+            if train:
+                if preprocess_lower == "crop_external_zero_planes":
+                    transformations_list.append(
+                        global_preprocessing_dict["crop_external_zero_planes"](
+                            patch_size
+                        )
+                    )
+            current_idx += 1
+
+    # normalization type is applied at the end
+    if normalize_to_apply is not None:
+        transformations_list.append(normalize_to_apply)
 
-    if augmentation_list:
-        transform = Compose(augmentation_list)
+    # compose the transformations
+    if transformations_list:
+        transform = Compose(transformations_list)
     else:
         transform = None
     subjects_dataset = torchio.SubjectsDataset(subjects_list, transform=transform)
diff --git a/GANDLF/parseConfig.py b/GANDLF/parseConfig.py
--- a/GANDLF/parseConfig.py
+++ b/GANDLF/parseConfig.py
@@ -336,9 +336,10 @@ def parseConfig(config_file_path, version_check=True):
                 )
             for key in params["data_augmentation"]:
                 if key != "default_probability":
+                    # when probability is not present for an augmentation, default to '1'
                     if (params["data_augmentation"][key] == None) or not (
                         "probability" in params["data_augmentation"][key]
-                    ):  # when probability is not present for an augmentation, default to '1'
+                    ):
                         if not isinstance(params["data_augmentation"][key], dict):
                             params["data_augmentation"][key] = {}
                         params["data_augmentation"][key][


Test patch: