DEBUG:rasterio._env:PROJ_DATA found in environment.
DEBUG:rasterio._env:Started GDALEnv: self=<rasterio._env.GDALEnv object at 0x346718f70>.
DEBUG:rasterio.env:Entered env context: <rasterio.env.Env object at 0x346718dc0>
DEBUG:rasterio._base:Sharing flag: 0
Traceback (most recent call last):
  File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/train_multilabel3.py", line 820, in <module>
    main()
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/main.py", line 94, in decorated_main
    _run_hydra(
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
    _run_app(
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/_internal/utils.py", line 457, in _run_app
    run_and_report(
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/_internal/utils.py", line 223, in run_and_report
    raise ex
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/_internal/utils.py", line 220, in run_and_report
    return func()
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/_internal/utils.py", line 458, in <lambda>
    lambda: hydra.run(
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 132, in run
    _ = ret.return_value
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/core/utils.py", line 260, in return_value
    raise self._return_value
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/core/utils.py", line 186, in run_job
    ret.return_value = task_function(task_cfg)
  File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/train_multilabel3.py", line 816, in main
    pipeline.train()
  File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/train_multilabel3.py", line 533, in train
    train_loader, valid_loader, _, dataset_callbacks = self.build_datasets(config)
  File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/train_multilabel3.py", line 311, in build_datasets
    sample_batch = next(iter(train_loader))
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 630, in __next__
    data = self._next_data()
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1344, in _next_data
    return self._process_data(data)
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1370, in _process_data
    data.reraise()
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/_utils.py", line 706, in reraise
    raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 309, in _worker_loop
    data = fetcher.fetch(index)  # type: ignore[possibly-undefined]
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 55, in fetch
    return self.collate_fn(data)
  File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/multilabel_dataset.py", line 70, in centernet_collate
    batch_dict = default_collate(included_items)
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 317, in default_collate
    return collate(batch, collate_fn_map=default_collate_fn_map)
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 155, in collate
    clone.update({key: collate([d[key] for d in batch], collate_fn_map=collate_fn_map) for key in elem})
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 155, in <dictcomp>
    clone.update({key: collate([d[key] for d in batch], collate_fn_map=collate_fn_map) for key in elem})
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 142, in collate
    return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 212, in collate_tensor_fn
    storage = elem._typed_storage()._new_shared(numel, device=elem.device)
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/storage.py", line 968, in _new_shared
    untyped_storage = torch.UntypedStorage._new_shared(size * self._element_size(), device=device)
  File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/storage.py", line 300, in _new_shared
    return cls._new_using_filename_cpu(size)
RuntimeError: torch_shm_manager at "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/bin/torch_shm_manager": could not generate a random directory for manager socket

DEBUG:rasterio._base:Nodata success: 1, Nodata value: -32768.000000
DEBUG:rasterio._base:Dataset <open DatasetReader name='/Volumes/Crucial/data/full/train/6de3dd8d23684472t/VH_dB.tif' mode='r'> is started.
DEBUG:rasterio.env:Exiting env context: <rasterio.env.Env object at 0x346718dc0>
DEBUG:rasterio.env:Cleared existing <rasterio._env.GDALEnv object at 0x346718f70> options
DEBUG:rasterio._env:Stopped GDALEnv <rasterio._env.GDALEnv object at 0x346718f70>.
DEBUG:rasterio.env:Exiting outermost env
DEBUG:rasterio.env:Exited env context: <rasterio.env.Env object at 0x346718dc0>
DEBUG:rasterio.env:Entering env context: <rasterio.env.Env object at 0x346718fd0>
DEBUG:rasterio.env:Starting outermost env
DEBUG:rasterio.env:No GDAL environment exists
DEBUG:rasterio.env:New GDAL environment <rasterio._env.GDALEnv object at 0x346718f70> created
(sam3) samerkharboush@Samers-MBP SAR-MPS-OptunaNasFbnYoloTransformers %