Repository: Project-MONAI/MONAI

Issue:
PersistentDataset errored out
Regular Dataset works fine 
train_dataset = Dataset(train_images, transform=train_transform)
But upon replacing this line with PersistentDataset as
train_dataset = PersistentDataset(train_images, transform=train_transform, cache_dir=persistent_cache)

Received error " Object of type 'ndarray' is not JSON serializable" at getting training sample, details below:

----------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/media/ziyuexu/Research/Code/Pathology/LAMP/Lamp_Pathology_On_The_Fly_PC/train.py", line 277, in <module>
    train(**input_dict)
  File "/media/ziyuexu/Research/Code/Pathology/LAMP/Lamp_Pathology_On_The_Fly_PC/train.py", line 134, in train
    first_sample = first(train_dataloader)
  File "/media/ziyuexu/Research/Tools/MONAI/MONAI/monai/utils/misc.py", line 41, in first
    for i in iterable:
  File "/home/ziyuexu/anaconda3/envs/camelyon/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 345, in __next__
    data = self._next_data()
  File "/home/ziyuexu/anaconda3/envs/camelyon/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data
    return self._process_data(data)
  File "/home/ziyuexu/anaconda3/envs/camelyon/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data
    data.reraise()
  File "/home/ziyuexu/anaconda3/envs/camelyon/lib/python3.6/site-packages/torch/_utils.py", line 395, in reraise
    raise self.exc_type(msg)
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/ziyuexu/anaconda3/envs/camelyon/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/ziyuexu/anaconda3/envs/camelyon/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/ziyuexu/anaconda3/envs/camelyon/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/media/ziyuexu/Research/Tools/MONAI/MONAI/monai/data/dataset.py", line 198, in __getitem__
    pre_random_item = self._pre_first_random_cachecheck(self.data[index])
  File "/media/ziyuexu/Research/Tools/MONAI/MONAI/monai/data/dataset.py", line 177, in _pre_first_random_cachecheck
    json.dumps(item_transformed, sort_keys=True).encode("utf-8")
  File "/home/ziyuexu/anaconda3/envs/camelyon/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/home/ziyuexu/anaconda3/envs/camelyon/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/home/ziyuexu/anaconda3/envs/camelyon/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/home/ziyuexu/anaconda3/envs/camelyon/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'ndarray' is not JSON serializable


Discussion:

Resolving patch:
diff --git a/monai/data/dataset.py b/monai/data/dataset.py
--- a/monai/data/dataset.py
+++ b/monai/data/dataset.py
@@ -94,13 +94,14 @@ class PersistentDataset(Dataset):
 
     def __init__(
         self,
-        data: Sequence,
+        data: Sequence[str],
         transform: Union[Sequence[Callable], Callable],
         cache_dir: Optional[Union[Path, str]] = None,
     ) -> None:
         """
         Args:
-            data: input data to load and transform to generate dataset for model.
+            data: input data file paths to load and transform to generate dataset for model.
+                `PersistentDataset` expects input data to be a list of file paths and hashes them as cache keys.
             transform: transforms to execute operations on input data.
             cache_dir: If specified, this is the location for persistent storage
                 of pre-computed transformed data tensors. The cache_dir is computed once, and


Test patch: