Repository: ivadomed/ivadomed
Issue category: Performance Issue

Issue:
Time issue when running `--test` on large microscopy images
**Disclaimer**
This issue is based on preliminary observations and does not contain a lot of details/examples at the moment.

## Issue description

I tried running the `--test` command on large microscopy images on `rosenberg`, and it takes a very long time.

I'll reference the following screenshot:
![test_on_om](https://user-images.githubusercontent.com/54086142/138940644-3f9618f9-d75e-41fb-9b3a-f10e8dc7a782.png)

From the `test` function called in main [here](https://github.com/ivadomed/ivadomed/blob/5207e7a5f0c882ae66248322ffbc518c0d2d6ce7/ivadomed/main.py#L470), the following happen:

At 19h20m56s, the segmentation of the first image starts (8557x7115 pixels).
At 19h29m59s, the segmentation of the second image starts  (5895x5266 pixels).
At 19h32m25s, the segmentations are completed, those timings are about normal (maybe a bit long, as describe in #972)

At this point, the test function is at line 85  in `testing.py` [here](https://github.com/ivadomed/ivadomed/blob/5207e7a5f0c882ae66248322ffbc518c0d2d6ce7/ivadomed/testing.py#L85), so it seems that the `metric_mgr.get_results()` function takes 40 minutes to compute the evaluation metrics (until 20h12m09s), which seems excessive.

After that, the `evaluate` function called in main [here](https://github.com/ivadomed/ivadomed/blob/5207e7a5f0c882ae66248322ffbc518c0d2d6ce7/ivadomed/main.py#L480) runs on the first image (8557x7115 pixels).
I killed the process after 2.5 hours without progress.
In that case, I suspect that the bottleneck may be in [run_eval](https://github.com/ivadomed/ivadomed/blob/5207e7a5f0c882ae66248322ffbc518c0d2d6ce7/ivadomed/evaluation.py#L87). 

Further investigations are needed.
I would be happy to make my model and images available if someone is interested to look into this.
Thanks!



Hints:
Update: I ran the same test again to have a little bit more details.
Turns out, the `evaluate` function took ~5h30min to compute metrics on my 2 images 😬.

![eval_om_02](https://user-images.githubusercontent.com/54086142/139498300-1c13bd3f-72af-4ce2-b0d7-daa66aa4f9f5.png)

I also ran the command with a cProfile (below) so we may have an idea where to look first.

<details>
<summary> cProfile output </summary>

```
         5766383 function calls (5632045 primitive calls) in 22538.294 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   2852/1    0.044    0.000 22538.307 22538.307 {built-in method builtins.exec}
        1    0.000    0.000 22538.307 22538.307 profile_test.py:1(<module>)
        1    0.032    0.032 22536.006 22536.006 main.py:295(run_command)
        1    1.535    1.535 19570.459 19570.459 evaluation.py:20(evaluate)
        2    0.001    0.001 19544.642 9772.321 evaluation.py:414(run_eval)
        4    0.019    0.005 14119.131 3529.783 evaluation.py:396(get_lfdr)
        8 4776.952  597.119 10804.197 1350.525 evaluation.py:305(_get_ltp_lfn)
        4 4290.862 1072.716 8717.269 2179.317 evaluation.py:344(_get_lfp)
    65904 7953.343    0.121 7953.343    0.121 {method 'astype' of 'numpy.ndarray' objects}
        4    0.008    0.002 5402.362 1350.591 evaluation.py:379(get_ltpr)
        1    0.018    0.018 2953.772 2953.772 testing.py:26(test)
69279/68504    1.798    0.000 2733.607    0.040 {built-in method numpy.core._multiarray_umath.implement_array_function}
        1    0.000    0.000 2381.068 2381.068 metrics.py:40(__call__)
        2    0.114    0.057 2357.215 1178.608 metrics.py:144(hausdorff_score)
    28944    0.160    0.000 2357.101    0.081 distance.py:365(directed_hausdorff)
    28944 2351.782    0.081 2355.295    0.081 {scipy.spatial._hausdorff.directed_hausdorff}
    27641    0.146    0.000 2327.396    0.084 <__array_function__ internals>:2(count_nonzero)
    27641    0.081    0.000 2327.107    0.084 numeric.py:424(count_nonzero)
    27641 2327.026    0.084 2327.026    0.084 {built-in method numpy.core._multiarray_umath.count_nonzero}
        1    0.061    0.061  570.711  570.711 testing.py:91(run_inference)
      141    0.003    0.000  531.454    3.769 std.py:1157(__iter__)
      135    0.001    0.000  531.343    3.936 dataloader.py:344(__next__)
      135    0.431    0.003  531.341    3.936 dataloader.py:383(_next_data)
      134    0.002    0.000  530.769    3.961 fetch.py:42(fetch)
      134    0.003    0.000  530.618    3.960 fetch.py:44(<listcomp>)
      402    1.184    0.003  530.616    1.320 mri2d_segmentation_dataset.py:169(__getitem__)
    12293  362.383    0.029  362.383    0.029 {method 'reduce' of 'numpy.ufunc' objects}
     9989    0.091    0.000  335.476    0.034 fromnumeric.py:70(_wrapreduction)
     9407    0.059    0.000  330.356    0.035 <__array_function__ internals>:2(amax)
     9407    0.083    0.000  330.240    0.035 fromnumeric.py:2617(amax)
      406    0.527    0.001  287.663    0.709 postprocessing.py:24(wrapper)
      406  107.077    0.264  287.136    0.707 postprocessing.py:75(threshold_predictions)
     1206    0.228    0.000  204.129    0.169 transforms.py:147(__call__)
127600/127428  168.456    0.001  173.261    0.001 {built-in method numpy.array}
      402    0.004    0.000  107.236    0.267 transforms.py:31(wrapper)
      402   41.435    0.103  107.231    0.267 transforms.py:298(__call__)
      804    0.012    0.000   96.664    0.120 transforms.py:212(__call__)
      409    0.002    0.000   65.816    0.161 <__array_function__ internals>:2(copy)
      409    0.001    0.000   65.809    0.161 function_base.py:715(copy)
      404    0.004    0.000   57.760    0.143 {method 'std' of 'numpy.ndarray' objects}
      404    0.234    0.001   57.756    0.143 _methods.py:259(_std)
      404   41.825    0.104   57.521    0.142 _methods.py:194(_var)
       22   33.902    1.541   37.671    1.712 metrics.py:62(numeric_score)
        4    0.000    0.000   21.153    5.288 inference.py:211(pred_to_png)
        8    0.000    0.000   20.135    2.517 functions.py:270(imwrite)
        8    0.007    0.001   20.129    2.516 format.py:477(append_data)
        8    0.013    0.002   20.122    2.515 pillow.py:359(_append_data)
        8    0.001    0.000   16.736    2.092 pillow.py:202(_append_data)
    39030   15.270    0.000   15.270    0.000 {method 'encode' of 'ImagingEncoder' objects}
      134   14.787    0.110   14.787    0.110 {method 'cpu' of 'torch._C._TensorBase' objects}
        8    0.000    0.000   14.650    1.831 Image.py:2095(save)
        8    0.000    0.000   14.470    1.809 PngImagePlugin.py:1178(_save)
        8    0.000    0.000   14.470    1.809 ImageFile.py:478(_save)
        1    0.004    0.004    9.926    9.926 loader.py:12(load_dataset)
        1    0.001    0.001    9.909    9.909 mri2d_segmentation_dataset.py:90(load_filenames)
        6    0.000    0.000    9.660    1.610 metrics.py:215(specificity_score)
        6    0.000    0.000    9.653    1.609 metrics.py:194(recall_score)
        6    0.000    0.000    9.651    1.608 metrics.py:172(precision_score)
        4    0.000    0.000    9.160    2.290 segmentation_pair.py:38(__init__)
        8    0.000    0.000    9.156    1.144 segmentation_pair.py:272(read_file)
        8    0.720    0.090    9.156    1.144 segmentation_pair.py:296(convert_file_to_nifti)
        2    1.160    0.580    8.364    4.182 inference.py:121(pred_to_nib)
      402    0.003    0.000    8.293    0.021 {method 'mean' of 'numpy.ndarray' objects}
      402    0.022    0.000    8.290    0.021 _methods.py:161(_mean)
        4    0.000    0.000    7.991    1.998 loadsave.py:81(save)
        4    0.000    0.000    7.991    1.998 filebasedimages.py:318(to_filename)
        4    0.000    0.000    7.991    1.998 analyze.py:996(to_file_map)
        4    0.000    0.000    7.863    1.966 arraywriters.py:539(to_fileobj)
        4    0.002    0.000    7.863    1.966 volumeutils.py:538(array_to_file)
        4    0.627    0.157    7.860    1.965 volumeutils.py:763(_write_data)
        8    0.000    0.000    6.200    0.775 functions.py:237(imread)
        2    0.391    0.196    5.694    2.847 evaluation.py:145(__init__)
      174    0.001    0.000    5.258    0.030 <__array_function__ internals>:2(sum)
      174    0.001    0.000    5.257    0.030 fromnumeric.py:2111(sum)
1928/1896    0.003    0.000    5.087    0.003 _asarray.py:110(asanyarray)
       34    0.000    0.000    5.080    0.149 dataobj_images.py:212(get_fdata)
        8    0.000    0.000    4.726    0.591 measurements.py:44(label)
        8    4.725    0.591    4.726    0.591 {built-in method scipy.ndimage._ni_label._label}
       16    0.000    0.000    4.460    0.279 openers.py:164(write)
       20    0.000    0.000    4.460    0.223 gzip.py:263(write)
        2    0.000    0.000    4.356    2.178 metrics.py:253(accuracy_score)
        2    0.000    0.000    4.353    2.176 metrics.py:236(intersection_over_union)
      209    0.008    0.000    4.020    0.019 __init__.py:1(<module>)
       10    2.341    0.234    3.618    0.362 metrics.py:91(dice_score)
       72    3.577    0.050    3.577    0.050 {method 'tobytes' of 'numpy.ndarray' objects}
        2    0.009    0.004    3.478    1.739 postprocessing.py:271(apply)
        2    0.540    0.270    3.470    1.735 postprocessing.py:284(binarize_maxpooling)
        8    0.698    0.087    3.372    0.422 util.py:45(image_as_uint)
        8    0.000    0.000    3.216    0.402 format.py:333(get_data)
        8    0.000    0.000    3.216    0.402 pillow.py:300(_get_data)
        8    0.000    0.000    3.216    0.402 pillow.py:167(_get_data)
        8    0.043    0.005    3.215    0.402 pillow.py:698(pil_get_frame)
       10    0.000    0.000    3.065    0.306 arrayproxy.py:370(__array__)
       10    0.022    0.002    3.065    0.306 arrayproxy.py:347(_get_scaled)
    28945    0.078    0.000    3.061    0.000 contextlib.py:72(inner)
       16    2.991    0.187    2.991    0.187 {method 'compress' of 'zlib.Compress' objects}
        8    0.000    0.000    2.983    0.373 functions.py:148(get_reader)
       16    0.000    0.000    2.978    0.186 format.py:215(__init__)
        8    0.000    0.000    2.977    0.372 format.py:157(get_reader)
        8    0.000    0.000    2.977    0.372 pillow.py:297(_open)
        8    0.000    0.000    2.977    0.372 pillow.py:118(_open)
        8    0.000    0.000    2.976    0.372 pillow.py:666(pil_try_read)
       16    0.000    0.000    2.976    0.186 Image.py:1274(getdata)
       24    0.003    0.000    2.975    0.124 ImageFile.py:154(load)
     1412    2.937    0.002    2.937    0.002 {method 'decode' of 'ImagingDecoder' objects}
    28945    2.643    0.000    2.643    0.000 {function SeedSequence.generate_state at 0x7f6986d91160}
10050/134    0.051    0.000    2.415    0.018 module.py:540(__call__)
      134    0.002    0.000    2.410    0.018 models.py:509(forward)
      539    2.342    0.004    2.342    0.004 {method 'copy' of 'numpy.ndarray' objects}
  2674/13    0.014    0.000    2.312    0.178 <frozen importlib._bootstrap>:986(_find_and_load)
  2668/13    0.010    0.000    2.311    0.178 <frozen importlib._bootstrap>:956(_find_and_load_unlocked)
  2223/13    0.011    0.000    2.310    0.178 <frozen importlib._bootstrap>:650(_load_unlocked)
  1934/13    0.006    0.000    2.309    0.178 <frozen importlib._bootstrap_external>:777(exec_module)
  3094/13    0.002    0.000    2.309    0.178 <frozen importlib._bootstrap>:211(_call_with_frames_removed)
5733/2516    0.008    0.000    2.227    0.001 <frozen importlib._bootstrap>:1017(_handle_fromlist)
   952/26    0.003    0.000    2.224    0.086 {built-in method builtins.__import__}
        2    0.000    0.000    2.190    1.095 main.py:1(<module>)
     1206    0.034    0.000    2.150    0.002 models.py:255(forward)
      120    0.000    0.000    2.112    0.018 fromnumeric.py:52(_wrapfunc)
120752/120617    0.057    0.000    2.106    0.000 _asarray.py:23(asarray)
        8    0.001    0.000    2.084    0.261 pillow.py:795(ndarray_to_pil)
        2    0.000    0.000    2.013    1.006 testing.py:412(get_gt)
        4    0.476    0.119    2.000    0.500 metrics.py:120(mse)
        1    0.000    0.000    1.970    1.970 serialization.py:499(load)
        1    0.000    0.000    1.970    1.970 serialization.py:613(_legacy_load)
        1    0.002    0.002    1.941    1.941 {method 'load' of '_pickle.Unpickler' objects}
      139    0.001    0.000    1.935    0.014 serialization.py:713(persistent_load)
     2546    0.006    0.000    1.925    0.001 conv.py:348(forward)
     1200    1.921    0.002    1.921    0.002 {built-in method zlib.crc32}
     2546    0.009    0.000    1.917    0.001 conv.py:340(_conv_forward)
    29058    1.912    0.000    1.912    0.000 {built-in method __new__ of type object at 0xab0800}
     2546    1.906    0.001    1.906    0.001 {built-in method conv2d}
      128    0.001    0.000    1.903    0.015 serialization.py:810(restore_location)
      128    0.000    0.000    1.902    0.015 serialization.py:176(default_restore_location)
      128    0.001    0.000    1.902    0.015 serialization.py:152(_cuda_deserialize)
      128    0.001    0.000    1.899    0.015 __init__.py:428(_lazy_new)
        8    0.046    0.006    1.736    0.217 Image.py:679(__array_interface__)
       76    0.000    0.000    1.693    0.022 <__array_function__ internals>:2(argmax)
       76    0.000    0.000    1.693    0.022 fromnumeric.py:1119(argmax)
       76    1.693    0.022    1.693    0.022 {method 'argmax' of 'numpy.ndarray' objects}
        8    0.013    0.002    1.690    0.211 Image.py:710(tobytes)
    65183    0.538    0.000    1.658    0.000 __init__.py:266(__getattr__)
       10    0.000    0.000    1.527    0.153 arrayproxy.py:328(_get_unscaled)
       10    0.288    0.029    1.525    0.153 volumeutils.py:449(array_from_file)
       24    0.000    0.000    1.370    0.057 evaluation.py:270(get_vol)
        8    0.000    0.000    1.338    0.167 Image.py:2736(fromarray)
      134    0.008    0.000    1.290    0.010 models.py:434(forward)
       39    0.000    0.000    1.277    0.033 {method 'sum' of 'numpy.ndarray' objects}
       39    0.000    0.000    1.277    0.033 _methods.py:45(_sum)
        1    0.000    0.000    1.245    1.245 bids_dataframe.py:31(__init__)
        1    0.001    0.001    1.241    1.241 bids_dataframe.py:69(create_bids_dataframe)
       10    0.140    0.014    1.236    0.124 {method 'readinto' of '_io._BufferedIOBase' objects}
    29558    0.035    0.000    1.217    0.000 <__array_function__ internals>:2(concatenate)
        1    0.000    0.000    1.195    1.195 evaluation.py:1(<module>)
      536    0.008    0.000    1.176    0.002 models.py:286(forward)
20458/18466    0.078    0.000    1.170    0.000 {method 'read' of '_io.BufferedReader' objects}
    65393    1.121    0.000    1.121    0.000 {built-in method _warnings.warn}
        2    0.000    0.000    1.115    0.558 inference.py:1(<module>)
      134    0.008    0.000    1.115    0.008 models.py:340(forward)
       46    0.000    0.000    1.099    0.024 gzip.py:287(read)
     1012    0.290    0.000    1.098    0.001 _compression.py:66(readinto)
        2    0.000    0.000    1.052    0.526 metrics.py:274(multi_class_dice_score)
```
</details>

Ouf yeah! The number of false positive lesions and number of true positive and false negative lesions (`_get_ltp_lfn` and `get_lfp`) don't really makes sense here and are probably mostly responsible for this huge computing time! It takes every unconnected objects et verifies overlaps, but in the case of axon and myelin segmentation, there are far from only 1 or 2 objects segmented. A quick workaround would be to put these metrics as options so they can be used only for relevant tasks like MS lesion segmentation.

Patch:
diff --git a/docs/source/configuration_file.rst b/docs/source/configuration_file.rst
index 8e9169961..ab31be8a5 100644
--- a/docs/source/configuration_file.rst
+++ b/docs/source/configuration_file.rst
@@ -2278,8 +2278,28 @@ Postprocessing
 
 Evaluation Parameters
 ---------------------
-Dict. Parameters to get object detection metrics (true positive and false detection rates), and this, for defined
-object sizes.
+Dict. Parameters to get object detection metrics (lesions true positive rate, lesions false detection rate
+and Hausdorff score), and this, for defined object sizes.
+
+.. jsonschema::
+
+    {
+        "$schema": "http://json-schema.org/draft-04/schema#",
+        "title": "object_detection_metrics",
+        "$$description": [
+            "Indicate if object detection metrics (lesions true positive rate, lesions false detection rate\n",
+            "and Hausdorff score) are computed or not at evaluation time. Default: ``true``",
+        ],
+        "type": "boolean"
+    }
+
+.. code-block:: JSON
+
+    {
+        "evaluation_parameters": {
+            "object_detection_metrics": true
+        }
+    }
 
 
 .. jsonschema::
@@ -2295,7 +2315,8 @@ object sizes.
                     "These values will create several consecutive target size bins. For instance\n",
                     "with a list of two values, we will have three target size bins: minimal size\n",
                     "to first list element, first list element to second list element, and second\n",
-                    "list element to infinity. Default: ``[20, 100]``."
+                    "list element to infinity. Default: ``[20, 100]``.\n",
+                    "``object_detection_metrics`` must be ``true`` for the target_size to apply."
                 ]
             },
             "unit": {
@@ -2329,7 +2350,10 @@ object sizes.
         "options": {
             "thr": {
                 "type": "int",
-                "description": "Minimal object size overlapping to be considered a TP, FP, or FN. Default: ``3``."
+                "$$description": [
+                    "Minimal object size overlapping to be considered a TP, FP, or FN. Default: ``3``.\n",
+                    "``object_detection_metrics`` must be ``true`` for the overlap to apply."
+                ]
             },
             "unit": {
                 "type": "string",
diff --git a/docs/source/tutorials/two_class_microscopy_seg_2d_unet.rst b/docs/source/tutorials/two_class_microscopy_seg_2d_unet.rst
index d01bade35..555a1f1bf 100644
--- a/docs/source/tutorials/two_class_microscopy_seg_2d_unet.rst
+++ b/docs/source/tutorials/two_class_microscopy_seg_2d_unet.rst
@@ -114,9 +114,9 @@ microscopy segmentation training.
       "train_fraction": 0.6
       "test_fraction": 0.1
 
-- ``training_parameters:training_time:num_epochs``: The maximum number of epochs that will be run during training. Each epoch is composed
-  of a training part and a validation part. It should be a strictly positive integer. In our case, we will use
-  50 epochs.
+- ``training_parameters:training_time:num_epochs``: The maximum number of epochs that will be run during training.
+  Each epoch is composed of a training part and a validation part. It should be a strictly positive integer.
+  In our case, we will use 50 epochs.
 
   .. code-block:: xml
 
@@ -132,12 +132,21 @@ microscopy segmentation training.
      "length_2D": [256, 256]
      "stride_2D": [244, 244]
 
-- ``postprocessing:binarize_maxpooling``: Used to binarize predictions across all classes in multiclass models. For each pixel, the class, including the background class, with the highest output probability will be segmented.
+- ``postprocessing:binarize_maxpooling``: Used to binarize predictions across all classes in multiclass models.
+  For each pixel, the class, including the background class, with the highest output probability will be segmented.
 
   .. code-block:: xml
 
       "binarize_maxpooling": {}
 
+- ``evaluation_parameters:object_detection_metrics``: Used to indicate if object detection metrics
+  (lesions true positive rate, lesions false detection rate and Hausdorff score) are computed or
+  not at evaluation time. For the axons and myelin segmentation task, we set this parameter to ``false``.
+
+  .. code-block:: xml
+
+      "object_detection_metrics": false
+
 - ``transformation:Resample``: Used to resample images to a common resolution (in mm) before splitting into patches,
   according to each image real pixel size. In our case, we resample the images to a common resolution of 0.0001 mm
   (0.1 μm) in both dimensions.
diff --git a/ivadomed/config/config_default.json b/ivadomed/config/config_default.json
index 70aa7c830..70c971f05 100644
--- a/ivadomed/config/config_default.json
+++ b/ivadomed/config/config_default.json
@@ -95,6 +95,7 @@
     },
     "postprocessing": {},
     "evaluation_parameters": {
+        "object_detection_metrics": true
     },
     "transformation": {
     }
diff --git a/ivadomed/config/config_microscopy.json b/ivadomed/config/config_microscopy.json
index c80556b06..d17dab35d 100644
--- a/ivadomed/config/config_microscopy.json
+++ b/ivadomed/config/config_microscopy.json
@@ -91,6 +91,9 @@
     "postprocessing": {
         "binarize_maxpooling": {}
     },
+    "evaluation_parameters": {
+        "object_detection_metrics": false
+    },
     "transformation": {
         "Resample":
         {
diff --git a/ivadomed/evaluation.py b/ivadomed/evaluation.py
index 8e7c4826d..241ac5383 100644
--- a/ivadomed/evaluation.py
+++ b/ivadomed/evaluation.py
@@ -86,24 +86,25 @@ def evaluate(bids_df, path_output, target_suffix, eval_params):
                                    params=eval_params)
         results_pred, data_painted = eval.run_eval()
 
-        # SAVE PAINTED DATA, TP FP FN
-        fname_paint = str(fname_pred).split('.nii.gz')[0] + '_painted.nii.gz'
-        nib_painted = nib.Nifti1Image(
-            dataobj=data_painted,
-            affine=nib_pred.header.get_best_affine(),
-            header=nib_pred.header.copy()
-        )
-        nib.save(nib_painted, fname_paint)
-
-        # For Microscopy PNG/TIF files (TODO: implement OMETIFF behavior)
-        if "nii" not in extension:
-            painted_list = imed_inference.split_classes(nib_painted)
-            # Reformat target list to include class index and be compatible with multiple raters
-            target_list = ["_class-%d" % i for i in range(len(target_suffix))]
-            imed_inference.pred_to_png(painted_list,
-                                       target_list,
-                                       str(path_preds.joinpath(subj_acq)),
-                                       suffix="_pred_painted.png")
+        if eval_params['object_detection_metrics']:
+            # SAVE PAINTED DATA, TP FP FN
+            fname_paint = str(fname_pred).split('.nii.gz')[0] + '_painted.nii.gz'
+            nib_painted = nib.Nifti1Image(
+                dataobj=data_painted,
+                affine=nib_pred.header.get_best_affine(),
+                header=nib_pred.header.copy()
+            )
+            nib.save(nib_painted, fname_paint)
+
+            # For Microscopy PNG/TIF files (TODO: implement OMETIFF behavior)
+            if "nii" not in extension:
+                painted_list = imed_inference.split_classes(nib_painted)
+                # Reformat target list to include class index and be compatible with multiple raters
+                target_list = ["_class-%d" % i for i in range(len(target_suffix))]
+                imed_inference.pred_to_png(painted_list,
+                                           target_list,
+                                           str(path_preds.joinpath(subj_acq)),
+                                           suffix="_pred_painted.png")
 
         # SAVE RESULTS FOR THIS PRED
         results_pred['image_id'] = subj_acq
@@ -135,6 +136,8 @@ class Evaluation3DMetrics(object):
         bin_struct (ndarray): Binary structure.
         size_min (int): Minimum size of objects. Objects that are smaller than this limit can be removed if
             "removeSmall" is in params.
+        object_detection_metrics (bool): Indicate if object detection metrics (lesions true positive and false detection
+            rates) are computed or not.
         overlap_vox (int): A prediction and ground-truth are considered as overlapping if they overlap for at least this
             amount of voxels.
         overlap_ratio (float): A prediction and ground-truth are considered as overlapping if they overlap for at least
@@ -167,6 +170,8 @@ def __init__(self, data_pred, data_gt, dim_lst, params=None):
         self.postprocessing_dict = {}
         self.size_min = 0
 
+        self.object_detection_metrics = params["object_detection_metrics"]
+
         if "target_size" in params:
             self.size_rng_lst, self.size_suffix_lst = \
                 self._get_size_ranges(thr_lst=params["target_size"]["thr"],
@@ -389,8 +394,12 @@ def get_ltpr(self, label_size=None, class_idx=0):
             label_size (int): Size of label.
             class_idx (int): Label index. If monolabel 0, else ranges from 0 to number of output channels - 1.
 
-        Note: computed only if n_obj >= 1.
+        Note: computed only if n_obj >= 1 and "object_detection_metrics" evaluation parameter is True.
         """
+        if not self.object_detection_metrics:
+            n_obj = 0
+            return np.nan, n_obj
+
         ltp, lfn, n_obj = self._get_ltp_lfn(label_size, class_idx)
 
         denom = ltp + lfn
@@ -406,8 +415,12 @@ def get_lfdr(self, label_size=None, class_idx=0):
             label_size (int): Size of label.
             class_idx (int): Label index. If monolabel 0, else ranges from 0 to number of output channels - 1.
 
-        Note: computed only if n_obj >= 1.
+        Note: computed only if n_obj >= 1 and "object_detection_metrics" evaluation parameter is True.
         """
+
+        if not self.object_detection_metrics:
+            return np.nan
+
         ltp, _, n_obj = self._get_ltp_lfn(label_size, class_idx)
         lfp = self._get_lfp(label_size, class_idx)
 
diff --git a/ivadomed/main.py b/ivadomed/main.py
index 2ea68a458..375f65aa1 100644
--- a/ivadomed/main.py
+++ b/ivadomed/main.py
@@ -517,7 +517,8 @@ def run_command(context, n_gif=0, thr_increment=None, resume_training=False, no_
                                                                            'requires_undo': True}}, device=device,
                                            cuda_available=cuda_available)
 
-        metric_fns = imed_metrics.get_metric_fns(ds_test.task)
+        eval_params = context[ConfigKW.EVALUATION_PARAMETERS]
+        metric_fns = imed_metrics.get_metric_fns(ds_test.task, eval_params)
 
         if ModelParamsKW.FILM_LAYERS in model_params and any(model_params[ModelParamsKW.FILM_LAYERS]):
             ds_test, model_params = update_film_model_params(context, ds_test, model_params, path_output)
@@ -535,7 +536,7 @@ def run_command(context, n_gif=0, thr_increment=None, resume_training=False, no_
         # RUN EVALUATION
         df_results = imed_evaluation.evaluate(bids_df, path_output=path_output,
                                               target_suffix=loader_params[LoaderParamsKW.TARGET_SUFFIX],
-                                              eval_params=context[ConfigKW.EVALUATION_PARAMETERS])
+                                              eval_params=eval_params)
         return df_results, pred_metrics
 
 
diff --git a/ivadomed/metrics.py b/ivadomed/metrics.py
index a0ba1f466..f8a49ab73 100644
--- a/ivadomed/metrics.py
+++ b/ivadomed/metrics.py
@@ -6,7 +6,7 @@
 
 
 # METRICS
-def get_metric_fns(task):
+def get_metric_fns(task, eval_params=None):
     metric_fns = [dice_score,
                   multi_class_dice_score,
                   precision_score,
@@ -14,7 +14,11 @@ def get_metric_fns(task):
                   specificity_score,
                   intersection_over_union,
                   accuracy_score]
-    if task == "segmentation":
+    if eval_params:
+        object_detection_metrics = eval_params['object_detection_metrics']
+    else:
+        object_detection_metrics = True
+    if task == "segmentation" and object_detection_metrics:
         metric_fns = metric_fns + [hausdorff_score]
 
     return metric_fns


Test patch: