Repository: Project-MONAI/MONAI

Issue:
If image reader chokes on a file, put the file name into exception message
**Is your feature request related to a problem? Please describe.**
I am running training on a big set (5000) of images, and somewhere in the middle I get `TypeError: stat: path should be string, bytes, os.PathLike or integer, not float`. I have no idea which image is the problem.

**Describe the solution you'd like**
I would like the path of the offending image to be mentioned somewhere in the stack trace. Possibly by special handling of exceptions by `read` method in `monai/data/image_reader.py`.

**Describe alternatives you've considered**
Create and run my own script which checks consistency of the data I work with.

**Additional context**
Code: https://github.com/dzenanz/miqa/blob/b8409864bd53de9de000aaf5085ad54729d4bc1e/nn_classifier.py

Here is a full stack trace:
```text
epoch 2 average loss: 0.4841
confusion matrix:
[[1516  565]
 [ 407 1701]]
Evaluating on validation set
............................................................
............................................................
............................................................
............................................................
............................................................
............................................................
............................................................
............................................................
............................................................
............................................................
............................................................
............................................................
............................................................
............................................................
............................................................
............................................................
............................................................
.......input data information of the runtime error transform:
Traceback (most recent call last):
  File "/home/dzenan/miqa/nn_classifier.py", line 402, in <module>
    process_folds(args.folds, args.vfold, args.evaluate, args.nfolds)
  File "/home/dzenan/miqa/nn_classifier.py", line 366, in process_folds
    sizes = train_and_save_model(df, count_train, save_path=model_path, num_epochs=50, val_interval=2,
  File "/home/dzenan/miqa/nn_classifier.py", line 321, in train_and_save_model
    auc_metric = evaluate_model(model, val_loader, device, writer, epoch, "val")
  File "/home/dzenan/miqa/nn_classifier.py", line 146, in evaluate_model
    for val_data in data_loader:
  File "/home/dzenan/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
    data = self._next_data()
  File "/home/dzenan/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1199, in _next_data
    return self._process_data(data)
  File "/home/dzenan/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data
    data.reraise()
  File "/home/dzenan/.local/lib/python3.8/site-packages/torch/_utils.py", line 429, in reraise
    raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/dzenan/.local/lib/python3.8/site-packages/monai/transforms/transform.py", line 48, in apply_transform
    return transform(data)
  File "/home/dzenan/.local/lib/python3.8/site-packages/monai/transforms/io/dictionary.py", line 105, in __call__
    data = self._loader(d[key], reader)
  File "/home/dzenan/.local/lib/python3.8/site-packages/monai/transforms/io/array.py", line 154, in __call__
    img = reader.read(filename)
  File "/home/dzenan/.local/lib/python3.8/site-packages/monai/data/image_reader.py", line 180, in read
    if os.path.isdir(name):
  File "/usr/lib/python3.8/genericpath.py", line 42, in isdir
    st = os.stat(s)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not float
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/home/dzenan/.local/lib/python3.8/site-packages/monai/transforms/transform.py", line 48, in apply_transform
    return transform(data)
  File "/home/dzenan/.local/lib/python3.8/site-packages/monai/transforms/compose.py", line 144, in __call__
    input_ = apply_transform(_transform, input_)
  File "/home/dzenan/.local/lib/python3.8/site-packages/monai/transforms/transform.py", line 71, in apply_transform
    raise RuntimeError(f"applying transform {transform}") from e
RuntimeError: applying transform <monai.transforms.io.dictionary.LoadImaged object at 0x7fea0f4f1910>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/home/dzenan/.local/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/dzenan/.local/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/dzenan/.local/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/dzenan/.local/lib/python3.8/site-packages/monai/data/dataset.py", line 92, in __getitem__
    return self._transform(index)
  File "/home/dzenan/.local/lib/python3.8/site-packages/monai/data/dataset.py", line 78, in _transform
    return apply_transform(self.transform, data_i) if self.transform is not None else data_i
  File "/home/dzenan/.local/lib/python3.8/site-packages/monai/transforms/transform.py", line 71, in apply_transform
    raise RuntimeError(f"applying transform {transform}") from e
RuntimeError: applying transform <monai.transforms.compose.Compose object at 0x7fea0f4c7b50>
INFO:DataStats:input data information of the runtime error transform:
img statistics:
Type: <class 'float'>
Value: nan
INFO:DataStats:img statistics:
Type: <class 'float'>
Value: nan
label statistics:
Type: <class 'numpy.int64'>
Value: 1
INFO:DataStats:label statistics:
Type: <class 'numpy.int64'>
Value: 1
input data information of the runtime error transform:
input data information of the runtime error transform:
INFO:DataStats:input data information of the runtime error transform:
img statistics:
Type: <class 'float'>
Value: nan
img statistics:
Type: <class 'float'>
Value: nan
INFO:DataStats:img statistics:
Type: <class 'float'>
Value: nan
label statistics:
Type: <class 'numpy.int64'>
Value: 1
label statistics:
Type: <class 'numpy.int64'>
Value: 1
INFO:DataStats:label statistics:
Type: <class 'numpy.int64'>
Value: 1
.input data information of the runtime error transform:
input data information of the runtime error transform:
input data information of the runtime error transform:
INFO:DataStats:input data information of the runtime error transform:
img statistics:
Type: <class 'float'>
Value: nan
img statistics:
Type: <class 'float'>
Value: nan
img statistics:
Type: <class 'float'>
Value: nan
INFO:DataStats:img statistics:
Type: <class 'float'>
Value: nan
label statistics:
Type: <class 'numpy.int64'>
Value: 1
label statistics:
Type: <class 'numpy.int64'>
Value: 1
label statistics:
Type: <class 'numpy.int64'>
Value: 1
INFO:DataStats:label statistics:
Type: <class 'numpy.int64'>
Value: 1
```



Discussion:
I wrote my own verification routine, only to find out that the problem was caused by the input CSV containing a few empty rows at the end (e.g. `,,,,,,,,`), so the file name passed to the reader was a pandas data frame cell created from an empty string, which happened to be a float value of `nan`. If it was a real file name, I guess it would have appeared somewhere in the stack trace.

Perhaps the issue can be closed?

Resolving patch:
diff --git a/monai/transforms/transform.py b/monai/transforms/transform.py
--- a/monai/transforms/transform.py
+++ b/monai/transforms/transform.py
@@ -51,7 +51,7 @@ def apply_transform(transform: Callable, data, map_items: bool = True):
         if not isinstance(transform, transforms.compose.Compose):
             # log the input data information of exact transform in the transform chain
             datastats = transforms.utility.array.DataStats(data_shape=False, value_range=False)
-            datastats._logger.info("input data information of the runtime error transform:")
+            datastats._logger.info(f"\n=== Transform input info -- {type(transform).__name__} ===")
             if isinstance(data, (list, tuple)):
                 data = data[0]
 


Test patch: