Repository: mlcommons/GaNDLF

Issue:
The `overall_stats` for classification and regression are not displaying headers in the logs
**Describe the bug**
The `overall_stats` are automatically (and always) calculated for [classification](https://github.com/mlcommons/GaNDLF/blob/master/GANDLF/metrics/classification.py)/[regression](https://github.com/mlcommons/GaNDLF/blob/master/GANDLF/metrics/regression.py) tasks. These do not have correct header information in the logs.

**To Reproduce**
Steps to reproduce the behavior:
1. Start any classification/regression task.
2. The `overall_stats` are visible in the console as expected:
```bash
patis@system_0>$ tail logs.stdout -n 35
     Epoch Final   train balanced_accuracy :  0.6859078927675051
********************
Starting validation :
********************
     Epoch Final   validation loss :  0.6173104731833139
     Epoch Final   validation accuracy_global :  0.6692481637001038
     Epoch Final   validation precision_global :  0.6692481637001038
     Epoch Final   validation recall_global :  0.6692481637001038
     Epoch Final   validation f1_global :  0.6692481637001038
     Epoch Final   validation accuracy_per_class :  [0.6538598537445068, 0.7173574566841125]
     Epoch Final   validation precision_per_class :  [0.8785290718078613, 0.39864030480384827]
     Epoch Final   validation recall_per_class :  [0.6538598537445068, 0.7173574566841125]
     Epoch Final   validation f1_per_class :  [0.7497246265411377, 0.5124877095222473]
     Epoch Final   validation accuracy_per_class_average :  0.6856086254119873
     Epoch Final   validation precision_per_class_average :  0.6385846734046936
     Epoch Final   validation recall_per_class_average :  0.6856086254119873
     Epoch Final   validation f1_per_class_average :  0.6311061382293701
     Epoch Final   validation accuracy_per_class_weighted :  0.6692482233047485
     Epoch Final   validation precision_per_class_weighted :  0.7622302770614624
     Epoch Final   validation recall_per_class_weighted :  0.6692482233047485
     Epoch Final   validation f1_per_class_weighted :  0.6922313570976257
     Epoch Final   validation auc :  0.0
     Epoch Final   validation classification_accuracy :  0.6692481851505786
     Epoch Final   validation f1 :  0.3346240925752893
     Epoch Final   validation accuracy :  0.3346240925752893
     Epoch Final   validation balanced_accuracy :  0.6692481851505786
Time taken for epoch :  55.49035133123398  mins
[SNIP!]
```
3. But looking at the logs (in this case for the validation set, as an example), these headers are absent:
```bash
patis@system_0>$ head output/logs_validation.csv -n 35
epoch_no,valid_loss,valid_classification_accuracy,valid_f1,valid_accuracy,valid_balanced_accuracy
0,0.6450672215569039,0.6378746628761292,0.6378746628761292,0.6378746628761292,0.6378746628761292,[0.6049093008041382, 0.740935742855072],[0.8795172572135925, 0.37494373321533203],[0.6049093008041382, 0.740935742855072],[0.7168131470680237, 0.4979197084903717],0.6729224920272827,0.6272305250167847,0.6729224920272827,0.6073664426803589,0.6378746628761292,0.7572362422943115,0.6378746628761292,0.6637653112411499,0.0,0.63787464960828,0.31893732480414,0.31893732480414,0.63787464960828
1,0.5819898751287665,0.7210522294044495,0.7210522294044495,0.7210522294044495,0.7210521697998047,[0.769619345664978, 0.5692148208618164],[0.8481482267379761, 0.44143521785736084],[0.769619345664978, 0.5692148208618164],[0.8069778084754944, 0.49724721908569336],0.6694170832633972,0.6447917222976685,0.6694170832633972,0.6521124839782715,0.7210522890090942,0.7495832443237305,0.7210522890090942,0.731916069984436,0.0,0.7210522532882915,0.36052612664414574,0.36052612664414574,0.7210522532882915
```

**Expected behavior**
The output should be:
```bash
patis@system_0>$ head output/logs_validation.csv -n 35
epoch_no,valid_loss,valid_accuracy_global,valid_precision_global,valid_recall_global, ......... (see from console log above)
0,0.6450672215569039,0.6378746628761292,0.6378746628761292,0.6378746628761292,0.6378746628761292,[0.6049093008041382, 0.740935742855072],[0.8795172572135925, 0.37494373321533203],[0.6049093008041382, 0.740935742855072],[0.7168131470680237, 0.4979197084903717],0.6729224920272827,0.6272305250167847,0.6729224920272827,0.6073664426803589,0.6378746628761292,0.7572362422943115,0.6378746628761292,0.6637653112411499,0.0,0.63787464960828,0.31893732480414,0.31893732480414,0.63787464960828
1,0.5819898751287665,0.7210522294044495,0.7210522294044495,0.7210522294044495,0.7210521697998047,[0.769619345664978, 0.5692148208618164],[0.8481482267379761, 0.44143521785736084],[0.769619345664978, 0.5692148208618164],[0.8069778084754944, 0.49724721908569336],0.6694170832633972,0.6447917222976685,0.6694170832633972,0.6521124839782715,0.7210522890090942,0.7495832443237305,0.7210522890090942,0.731916069984436,0.0,0.7210522532882915,0.36052612664414574,0.36052612664414574,0.7210522532882915
```

**Screenshots**
N.A.

**GaNDLF Version**
<!-- Put the output of the following command:
python -c 'import GANDLF as g;print(g.__version__)'
-->
0.0.16-dev

**Desktop (please complete the following information):**
N.A.

**Additional context**
N.A.


Discussion:

Resolving patch:
diff --git a/GANDLF/compute/loss_and_metric.py b/GANDLF/compute/loss_and_metric.py
--- a/GANDLF/compute/loss_and_metric.py
+++ b/GANDLF/compute/loss_and_metric.py
@@ -139,9 +139,4 @@ def get_loss_and_metrics(image, ground_truth, predicted, params):
                     metric_output[metric] = get_metric_output(
                         metric_function, predicted, ground_truth, params
                     )
-        else:
-            print(
-                "WARNING: Could not find the requested metric '" + metric,
-                file=sys.stderr,
-            )
     return loss, metric_output
diff --git a/GANDLF/compute/training_loop.py b/GANDLF/compute/training_loop.py
--- a/GANDLF/compute/training_loop.py
+++ b/GANDLF/compute/training_loop.py
@@ -287,6 +287,28 @@ def training_loop(
     # datetime object containing current date and time
     print("Initializing training at :", get_date_time(), flush=True)
 
+    calculate_overall_metrics = (params["problem_type"] == "classification") or (
+        params["problem_type"] == "regression"
+    )
+
+    # get the overall metrics that are calculated automatically for classification/regression problems
+    if params["problem_type"] == "regression":
+        overall_metrics = overall_stats(torch.Tensor([1]), torch.Tensor([1]), params)
+    elif params["problem_type"] == "classification":
+        # this is just used to generate the headers for the overall stats
+        org_num_classes = params["model"]["num_classes"]
+        params["model"]["num_classes"] = 3
+        overall_metrics = overall_stats(
+            torch.Tensor([0, 0, 2, 2, 1, 2]).to(dtype=torch.int32),
+            torch.Tensor([0, 0, 2, 2, 1, 2]).to(dtype=torch.int32),
+            params,
+        )
+        # original number of classes are restored
+        params["model"]["num_classes"] = org_num_classes
+    if calculate_overall_metrics:
+        for metric in overall_metrics:
+            params["metrics"][metric] = 0
+
     # Setup a few loggers for tracking
     train_logger = Logger(
         logger_csv_filename=os.path.join(output_dir, "logs_training.csv"),


Test patch: