I'm glad to hear that you've applied the suggestions and are seeking a review of your updated multilabel_circle_coder.py and multilabel_dataset.py. I'll thoroughly review your code, focusing on ensuring compatibility and correctness after integrating YOLO with your multi-label dataset.
Review of multilabel_circle_coder.py
1. Structure and Integration

You've successfully integrated yolo_targets into the MultilabelCircleNetEncodeResult namedtuple and are generating YOLO targets within the encode method. This integration aligns with the goal of combining CenterNet and YOLO-style target generation.
2. Issues in generate_yolo_targets Method

Incorrect YOLO Target Format
Your current implementation of generate_yolo_targets assigns absolute coordinates and lengths directly to the grid cells:

# Current implementation
yolo_targets[grid_y, grid_x, 0:4] = np.array([center[0], center[1], length, length])
Issue: This is not consistent with the YOLO format. In YOLO, targets within each grid cell are specified relative to the grid cell and are normalized.
Correct YOLO Target Format
Here's how YOLO typically expects target values within each grid cell:
tx, ty: Offsets of the object's center from the top-left corner of the grid cell (values between 0 and 1).
tw, th: Width and height of the object, normalized by the image dimensions (values between 0 and 1, or encoded differently if using anchors).
objectness (confidence): Generally 1 if an object is present in the cell.
class: Class index or one-hot encoded class vector.
Recommended Corrections
Updated generate_yolo_targets Method

def generate_yolo_targets(
    self,
    centers: np.ndarray,
    confidences: np.ndarray,
    is_vessel: np.ndarray,
    is_fishing: np.ndarray,
    lengths: np.ndarray
) -> np.ndarray:
    num_objects = len(centers)
    yolo_targets = np.zeros((self.grid_height, self.grid_width, 6), dtype=np.float32)
    
    for i in range(num_objects):
        center = centers[i]
        confidence_label = confidences[i]
        confidence = self.confidence_mapping.get(confidence_label, self.ignore_value)
        vessel_type = self.vessel_mapping.get(is_vessel[i], self.ignore_value)
        fishing_status = self.fishing_mapping.get(is_fishing[i], self.ignore_value)
        length = lengths[i]
        
        # Convert center coordinates to grid cell indices
        grid_x = int(center[0] // self.output_stride)
        grid_y = int(center[1] // self.output_stride)
        
        # Compute the offset of the center within the grid cell (values between 0 and 1)
        x_cell = (center[0] / self.output_stride) - grid_x
        y_cell = (center[1] / self.output_stride) - grid_y
        
        # Normalize width and height by the image dimensions
        w_normalized = length / self.image_width
        h_normalized = length / self.image_height
        
        # Ensure grid indices are within bounds
        grid_x = min(grid_x, self.grid_width - 1)
        grid_y = min(grid_y, self.grid_height - 1)
        
        # Assign values to the grid cell
        yolo_targets[grid_y, grid_x, 0] = x_cell
        yolo_targets[grid_y, grid_x, 1] = y_cell
        yolo_targets[grid_y, grid_x, 2] = w_normalized
        yolo_targets[grid_y, grid_x, 3] = h_normalized
        yolo_targets[grid_y, grid_x, 4] = confidence
        yolo_targets[grid_y, grid_x, 5] = vessel_type  # Use appropriate class index
Explanation of Changes:
Offsets within Cell: x_cell and y_cell represent the position of the object within the grid cell, normalized to [0, 1].
Normalized Width and Height: w_normalized and h_normalized are the object's dimensions relative to the entire image size.
Grid Indices: grid_x and grid_y are computed based on the object's center coordinates scaled by the output stride.
Assignment: The YOLO targets are updated with normalized values, adhering to the expected YOLO format.
3. Consistency in Data Types for is_vessel and is_fishing

There's a potential inconsistency in how is_vessel and is_fishing are handled:
In encode Method: You're converting is_vessel and is_fishing to NumPy arrays of type np.float32:

is_vessel = np.array(is_vessel)
is_fishing = np.array(is_fishing)
In generate_yolo_targets Method: You're using these arrays to retrieve mappings:

vessel_type = self.vessel_mapping.get(is_vessel[i], self.ignore_value)
fishing_status = self.fishing_mapping.get(is_fishing[i], self.ignore_value)
Issue: If is_vessel[i] and is_fishing[i] are float32 values (e.g., 0.0, 1.0), and your mappings use string keys (e.g., 'UNKNOWN', 'TYPE1'), the .get() method will not find the corresponding keys, leading to incorrect mappings.
Recommended Solution:
Ensure Consistent Data Types: Decide whether is_vessel and is_fishing should be strings or numerical labels. If they are categorical strings, make sure they are kept as strings throughout the code. If they are numerical labels, update your mappings to use numerical keys.
Update Mappings If Using Numerical Labels:

self.vessel_mapping = {
    0: 0,
    1: 1,
    2: 2,
    # Add more vessel types as necessary
}

self.fishing_mapping = {
    0: 0,
    1: 1,
    2: 2,
    # Add more fishing statuses as necessary
}
Alternatively, Keep is_vessel and is_fishing as Strings: If you prefer to use strings, ensure you pass them as strings throughout the code and that the mappings have string keys.
Adjustments in encode Method:
If using strings, remove the conversion to np.float32:

# Remove this conversion if 'is_vessel' and 'is_fishing' are strings
# is_vessel = np.array(is_vessel)
# is_fishing = np.array(is_fishing)
4. Conversion to Torch Tensors

In your encode method, after generating yolo_targets, you're converting it to a torch.Tensor:

yolo_targets = yolo_targets.transpose(2, 0, 1)
yolo_targets = torch.from_numpy(yolo_targets).float()
yolo_targets = yolo_targets.unsqueeze(0)
This is appropriate. However, in your MultilabelCircleNetEncodeResult, yolo_targets will already be a tensor.
Review of multilabel_dataset.py
1. Redundant generate_yolo_targets Method

In your MultilabelCircleNetTargetsMixin, you have the generate_yolo_targets method:

class MultilabelCircleNetTargetsMixin:
    # ...
    
    def generate_yolo_targets(self, centers, confidences, is_vessel, is_fishing, lengths):
        # Implementation
Issue: This creates redundancy and potential inconsistencies since you're already generating YOLO targets within the MultilabelCircleNetCoder.encode method.
Recommendation:
Remove generate_yolo_targets from MultilabelCircleNetTargetsMixin: This ensures that all YOLO target generation is centralized within the coder, eliminating redundancy.
2. Simplify build_centernet_sample Method

In your build_centernet_sample method, you're calling self.box_coder.generate_yolo_targets to generate YOLO targets. Since you're already generating these targets in the encode method of your coder, you can simplify your build_centernet_sample method.
Adjusted build_centernet_sample Method:

def build_centernet_sample(
    self,
    image,
    centers,
    confidences,
    lengths,
    is_vessel,
    is_fishing,
    index,
    image_id,
    is_near_shore,
    location,
    folder: str,
    crop_coords: Tuple[Tuple[int, int], Tuple[int, int]],
) -> Dict[str, Any]:
    # Mask NaN values
    image[~np.isfinite(image)] = 0
    box_coder = self.box_coder.create_box_coder_for_image_size(image.shape[:2])
    
    # Convert numerical fields to NumPy arrays with explicit dtypes
    centers_np = np.array(centers, dtype=np.float32)
    lengths_np = np.array(lengths, dtype=np.float32)
    # Ensure 'is_vessel' and 'is_fishing' are the correct data type (strings or numerical labels)
    # Adjust accordingly
    is_vessel_np = np.array(is_vessel)
    is_fishing_np = np.array(is_fishing)
    is_near_shore_np = np.array(is_near_shore, dtype=np.float32)
    
    # Validate consistency in lengths
    num_objects = len(centers_np)
    if not all([
        len(confidences) == num_objects,
        len(lengths_np) == num_objects,
        len(is_vessel_np) == num_objects,
        len(is_fishing_np) == num_objects,
    ]):
        raise ValueError("Mismatch in number of objects across targets")
    
    result = {
        INPUT_INDEX_KEY: index,
        INPUT_IMAGE_KEY: image_to_tensor(image),
        TARGET_CENTERS_KEY: torch.from_numpy(centers_np),
        TARGET_VESSEL_KEY: is_vessel_np,  # Keep as is if they are strings
        TARGET_FISHING_KEY: is_fishing_np,  # Keep as is if they are strings
        TARGET_LENGTHS_KEY: torch.from_numpy(lengths_np),
        INPUT_CONFIDENCE_KEY: confidences,  # Keep as list of strings
        INPUT_IS_NEAR_SHORE_KEY: torch.from_numpy(is_near_shore_np),
        INPUT_LOCATION_ID_KEY: location,
        INPUT_FOLDER_KEY: folder,
        INPUT_SCENE_ID_KEY: image_id,
        TARGET_SAMPLE_WEIGHT: {"train": 0.25, "validation": 1.0, "valid": 1.0, "test": 0.1}[folder],
    }
    if crop_coords is not None:
        result[INPUT_SCENE_CROP_KEY] = crop_coords
    
    # Encode targets using the box coder
    centernet_targets: MultilabelCircleNetEncodeResult = box_coder.encode(
        centers=centers_np,
        confidences=confidences,
        lengths=lengths_np,
        is_vessel=is_vessel_np,
        is_fishing=is_fishing_np,
    )
    
    # Extract and assign the targets
    result[CENTERNET_TARGET_OBJECTNESS_MAP] = torch.from_numpy(centernet_targets.heatmap).float()
    result[CENTERNET_TARGET_VESSEL_MAP] = torch.from_numpy(centernet_targets.is_vessel).float()
    result[CENTERNET_TARGET_FISHING_MAP] = torch.from_numpy(centernet_targets.is_fishing).float()
    result[CENTERNET_TARGET_SIZE] = torch.from_numpy(centernet_targets.lengths).float()
    result[CENTERNET_TARGET_OFFSET] = torch.from_numpy(centernet_targets.offset).float()
    result[CENTERNET_TARGET_YOLO] = centernet_targets.yolo_targets  # Already a torch.Tensor (from coder)
    
    return result
Key Changes:
Removed Redundant Generation: Eliminated the unnecessary call to generate_yolo_targets within build_centernet_sample.
Directly Used yolo_targets from Encoder: Extracted yolo_targets directly from centernet_targets.
Ensured Consistent Data Types: Adjusted is_vessel_np and is_fishing_np to match the expected data types in the coder.
3. Data Type Consistency for is_vessel and is_fishing

Ensure that is_vessel and is_fishing are consistently represented across your dataset and coder:
If Using Strings: Keep is_vessel and is_fishing as lists of strings and adjust your mappings accordingly.
If Using Numerical Labels: Convert is_vessel and is_fishing to integers and update your mappings to use numerical keys.
4. Adjusting the centernet_collate Function

In your centernet_collate function, you don't need to add CENTERNET_TARGET_YOLO to skip_keys because it's a tensor and will be collated properly using default_collate.
Additional Recommendations
1. Testing and Validation

Unit Tests: Write unit tests for generate_yolo_targets to confirm that the targets are generated correctly.
Data Loader Testing: Run your data loader with a small batch size and print out shapes and sample values to verify correctness.
Model Forward Pass: Perform a forward pass through your model to ensure that the outputs and target shapes match.
Visual Inspection: Visualize some images with overlaid targets to manually verify correctness.
2. Integration with Loss Function and Model

Adjust Loss Function: Ensure that your loss function correctly interprets the YOLO targets and computes the loss accordingly.
Model Output Layers: Modify your model's architecture to output predictions compatible with the YOLO target format.
3. Consistency in Mappings

Check Mappings: Verify that all mappings (confidence_mapping, vessel_mapping, fishing_mapping) are correctly aligned with the data types of the variables they map.
4. Data Normalization

Consistent Normalization: Ensure that widths and heights are consistently normalized throughout your code (e.g., when generating targets and when interpreting model outputs).
Summary
Corrected generate_yolo_targets: Adjusted your method to generate targets in the proper YOLO format.
Removed Redundancy: Eliminated the redundant generate_yolo_targets method in MultilabelCircleNetTargetsMixin.
Simplified build_centernet_sample: Streamlined your method to use targets from the coder directly.
Ensured Data Consistency: Highlighted the importance of consistent data types for is_vessel and is_fishing.
Recommended Thorough Testing: Emphasized testing at each stage to validate changes and ensure correctness.
By carefully applying these adjustments and thoroughly testing your code, you should achieve compatibility and correctness in integrating YOLO with your multi-label dataset. If you have further questions or encounter specific issues, feel free to reach out for additional assistance.

the is_vessel and is_fishing should be string as they are in the training data labels example which is aligned with the TIFF SAR images VV+VH used as inputs
##sample training data
##the output should be
the error
DEBUG:rasterio._base:Sharing flag: 0
Traceback (most recent call last):
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/train_multilabel3.py", line 966, in
main()
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/main.py", line 94, in decorated_main
_run_hydra(
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
_run_app(
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/_internal/utils.py", line 457, in _run_app
run_and_report(
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/_internal/utils.py", line 223, in run_and_report
raise ex
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/_internal/utils.py", line 220, in run_and_report
return func()
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/_internal/utils.py", line 458, in
lambda: hydra.run(
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 132, in run
_ = ret.return_value
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/core/utils.py", line 260, in return_value
raise self._return_value
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/hydra/core/utils.py", line 186, in run_job
ret.return_value = task_function(task_cfg)
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/train_multilabel3.py", line 962, in main
pipeline.train()
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/train_multilabel3.py", line 679, in train
train_loader, valid_loader, _, dataset_callbacks = self.build_datasets(config)
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/train_multilabel3.py", line 311, in build_datasets
sample_batch = next(iter(train_loader))
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 630, in next
data = self._next_data()
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1344, in _next_data
return self._process_data(data)
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1370, in _process_data
data.reraise()
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/_utils.py", line 706, in reraise
raise exception
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 155, in collate
clone.update({key: collate([d[key] for d in batch], collate_fn_map=collate_fn_map) for key in elem})
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 155, in
clone.update({key: collate([d[key] for d in batch], collate_fn_map=collate_fn_map) for key in elem})
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 192, in collate
raise TypeError(default_collate_err_msg_format.format(elem_type))
TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found <class 'NoneType'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 309, in _worker_loop
data = fetcher.fetch(index) # type: ignore[possibly-undefined]
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 55, in fetch
return self.collate_fn(data)
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/multilabel_dataset.py", line 70, in centernet_collate
batch_dict = default_collate(included_items)
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 317, in default_collate
return collate(batch, collate_fn_map=default_collate_fn_map)
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 162, in collate
return {key: collate([d[key] for d in batch], collate_fn_map=collate_fn_map) for key in elem}
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 162, in
return {key: collate([d[key] for d in batch], collate_fn_map=collate_fn_map) for key in elem}
File "/opt/anaconda3/envs/sam3/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 192, in collate
raise TypeError(default_collate_err_msg_format.format(elem_type))
TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found <class 'NoneType'>
DEBUG:rasterio._base:Nodata success: 1, Nodata value: -32768.000000
###some related parts and methods as encode_batch_ target, build_dataset from file train_multilabel3.py
class CenternetPipeline(Pipeline):
def init(self, cfg: DictConfig):
super().init(cfg)
self.box_coder: Optional[MultilabelCircleNetCoder] = None
self.model: Optional[nn.Module] = None
self.criterion: Optional[nn.Module] = None
self.scaler: Optional[GradScaler] = GradScaler() if torch.cuda.is_available() else None
self.device = TorchDevice.choose_torch_device()
self.experiment_dir = ""
self.yolo_decoder: Optional[YOLODecoder] = None
self.nas: Optional[OptunaNAS] = None

def build_datasets(self, config: DictConfig) -> Tuple[DataLoader, DataLoader, Optional[DataLoader], List[Callback]]:
    """
    Builds and returns DataLoaders for training and validation datasets.
    """
    dataset_module = MultilabelCircleNetDataModule(config=config)

    train_image_size = config.train.image_size if 'image_size' in config.train else [512, 512]
    valid_image_size = config.valid.image_size if 'image_size' in config.valid else [2048, 2048]

    train_df, valid_df, _, shore_root = dataset_module.train_val_split(
        splitter=config.dataset.splitter,
        fold=config.dataset.fold,
        num_folds=config.dataset.num_folds,
    )

    # Validate output_stride
    output_stride = config.model.get('output_stride', 32)
    if output_stride <= 0:
        raise ValueError(f"Output stride must be a positive integer, got {output_stride}")

    self.box_coder = MultilabelCircleNetCoder(
        image_size=(valid_image_size[0], valid_image_size[1]),
        output_stride=output_stride,
        max_objects=config.model.get('max_objects', 512),
        heatmap_encoding=config.model.get('heatmap_encoding', "umich"),
        labels_encoding=config.model.get('labels_encoding', "circle"),
        ignore_value=config.model.get('ignore_value', -1)
    )

    normalization = {
        channel: hydra.utils.instantiate(config.dataset.normalization.channels[channel])
        for channel in config.dataset.input_channels
    }

    # Add global normalization if available
    if 'mean' in config.dataset.normalization and 'std' in config.dataset.normalization:
        global_norm = A.Normalize(
            mean=config.dataset.normalization.mean,
            std=config.dataset.normalization.std
        )
    else:
        global_norm = A.NoOp()

    # Define data augmentation and normalization transforms
    transforms = A.Compose([
        A.HorizontalFlip(p=0.5),
        A.VerticalFlip(p=0.5),
        A.Rotate(limit=15, p=0.5),
        A.RandomBrightnessContrast(p=0.5),
        A.Normalize(mean=config.dataset.normalization.mean, std=config.dataset.normalization.std),
        ToTensorV2(), 
    ], bbox_params=A.BboxParams(format='coco', label_fields=['labels']))

    # Check and validate heatmap_size
    heatmap_size = valid_image_size[0] // output_stride
    if heatmap_size <= 0:
        raise ValueError(f"Heatmap size must be a positive integer, got {heatmap_size}")

    self.valid_df = valid_df.copy()  # Keep a copy of unmodified groundtruth

    # Logging dataset info
    self.master_print("Dataset", config.dataset.slug)
    self.master_print("\tignore_low_confidence_detections", config.dataset.ignore_low_confidence_detections)
    self.master_print("\tignore_low_confidence_labels    ", config.dataset.ignore_low_confidence_labels)
    self.master_print("\tfilter_low_confidence_objects   ", config.dataset.filter_low_confidence_objects)

    self.master_print("Train dataset")
    self.master_print("\tUnknown vessels", np.isnan(train_df.is_vessel.values.astype(float)).sum())
    self.master_print("\tUnknown fishing", np.isnan(train_df.is_fishing.values.astype(float)).sum())
    self.master_print("Valid dataset")
    self.master_print("\tUnknown vessels", np.isnan(valid_df.is_vessel.values.astype(float)).sum())
    self.master_print("\tUnknown fishing", np.isnan(valid_df.is_fishing.values.astype(float)).sum())

    # Check mutually exclusive configurations
    if config.dataset.filter_low_confidence_objects and config.dataset.ignore_low_confidence_labels:
        raise ValueError("filter_low_confidence_objects and ignore_low_confidence_labels are mutually exclusive")

    # Initialize datasets
    train_dataset, train_sampler = dataset_module.get_random_crop_training_dataset(
        train_df=train_df,
        train_image_size=train_image_size,
        input_channels=config.dataset.channels,
        box_coder=self.box_coder,
        normalization=normalization,
        num_samples=config.train.get('num_samples', None),
        individual_augmentations=config.train.get('individual_augmentations', {}),
        augmentations=config.train.get('augs', []),
        balance_crowd=config.train.get('balance_crowd', False),
        balance_location=config.train.get('balance_location', False),
        channels_last=config.dataset.get('channels_last', False),
        read_image_fn=read_multichannel_image,
    )

    valid_dataset = dataset_module.get_validation_dataset(
        valid_df=valid_df,
        valid_crop_size=valid_image_size,
        input_channels=config.dataset.channels,
        box_coder=self.box_coder,
        normalization=normalization,
        channels_last=config.dataset.get('channels_last', False),
        read_image_fn=read_multichannel_image,
    )

    # Setting sampler to None for distributed training or default sampling
    train_sampler = None

    # Prepare callbacks (e.g., for logging, early stopping)
    dataset_callbacks = []

    # Get the custom collate functions
    train_collate_fn = train_dataset.get_collate_fn()
    valid_collate_fn = valid_dataset.get_collate_fn()

    # Initialize DataLoaders
    train_loader = DataLoader(
        train_dataset,
        sampler=train_sampler,
        batch_size=config.train.batch_size,
        num_workers=config.train.num_workers,
        pin_memory=True,
        shuffle=(train_sampler is None),
        persistent_workers=config.train.get('persistent_workers', False),
        collate_fn=train_collate_fn
    )
    valid_loader = DataLoader(
        valid_dataset,
        batch_size=config.train.get('batch_size', config.train.batch_size),
        num_workers=config.train.get('num_workers', config.train.num_workers),
        pin_memory=True,
        shuffle=False,
        collate_fn=valid_collate_fn
    )

    # Add debug prints to check shapes and dtypes
    try:
        sample_batch = next(iter(train_loader))
        logger.debug(f"[DEBUG] Sample batch keys: {sample_batch.keys()}")
        for k, v in sample_batch.items():
            if isinstance(v, torch.Tensor):
                logger.debug(f"[DEBUG] Sample batch '{k}' dtype: {v.dtype}, shape: {v.shape}")
            elif isinstance(v, dict):
                for sub_k, sub_v in v.items():
                    if isinstance(sub_v, torch.Tensor):
                        logger.debug(f"[DEBUG] Sample batch '{k}.{sub_k}' dtype: {sub_v.dtype}, shape: {sub_v.shape}")
    except StopIteration:
        logger.error("[ERROR] Training loader is empty!")
        raise

    return train_loader, valid_loader, None, dataset_callbacks

def build_model(self, config: DictConfig, train_loader: DataLoader, valid_loader: DataLoader) -> nn.Module:
    """
    Builds the model using CenterNetUNet with integrated heads and NAS if configured.
    """
    logger.info("Building model with config:")
    logger.info(OmegaConf.to_yaml(config.model))

    # Initialize YOLODecoder
    self.yolo_decoder = YOLODecoder(
        anchors=config.model.anchors,
        num_classes=config.model.num_classes,
        input_image_size=tuple(config.model.image_size),
        stride=config.model.output_stride,
        conf_threshold=config.inference.get('thresholds', {}).get('objectness', 0.5),
        iou_threshold=config.inference.get('thresholds', {}).get('iou', 0.4)
    ).to(self.device)
    self.yolo_decoder = self.yolo_decoder.float()  # Ensure decoder is float32

    
    if config.model.nas.enabled:
        self.nas = OptunaNAS(
            search_space=config.model.nas.search_space,
            evaluate_architecture=self.evaluate_architecture,
            direction=config.model.nas.direction
        )
        best_trial = self.nas.optimize(
            n_trials=config.model.nas.num_trials,
            config=config,
            train_loader=train_loader,
            val_loader=valid_loader
        )
        logger.info(f"Best NAS trial: {best_trial}")
        # Update model config with best trial params
        config.model = OmegaConf.merge(config.model, best_trial.params)

    self.model = CenterNetUNetModel.from_config(config.model)
    self.model = self.model.to(self.device)
    self.model = self.model.float()  # Ensure model parameters are float32

    logger.info(f"Total parameters in the model: {count_parameters(self.model)}")

    return self.model

def build_criterions(self, config: DictConfig) -> Tuple[Dict[str, nn.Module], List[Callback]]:
    """
    Builds and returns loss functions and their corresponding callbacks.
    """
    criterions = {}
    criterions_callbacks = []

    # Initialize combined loss
    combined_loss = CircleNetLoss().to(self.device)
    combined_loss = combined_loss.float()  # Ensure float32
    criterions['combined_loss'] = combined_loss

    return criterions, criterions_callbacks

def evaluate_architecture(self, trial: optuna.Trial, config: Dict[str, Any], train_loader: DataLoader, val_loader: DataLoader) -> float:
    """
    Evaluates a given model architecture on the validation set.
    This function is used by OptunaNAS to determine the performance of different architectures.
    """
    model = self.nas.create_model(config, trial)
    model.to(self.device)
    model.eval()

    # Initialize YOLODecoder and attach it to the model
    model.yolo_decoder = YOLODecoder(
        anchors=config.model.anchors,
        num_classes=config.model.num_classes,
        input_image_size=tuple(config.model.image_size),
        stride=config.model.output_stride,
        conf_threshold=config.get('inference', {}).get('thresholds', {}).get('objectness', 0.5),
        iou_threshold=config.get('inference', {}).get('thresholds', {}).get('iou', 0.4)
    ).to(self.device)
    model.yolo_decoder = model.yolo_decoder.float()

    total_loss = 0.0
    total_samples = 0

    with torch.no_grad():
        for batch_idx, batch in enumerate(val_loader):
            try:
                inputs = batch['INPUT_IMAGE_KEY'].to(self.device)
                if inputs.dtype != torch.float32:
                    logger.debug(f"[DEBUG] Casting 'images' from {inputs.dtype} to float32.")
                    inputs = inputs.float()

                # Ensure correct input shape
                if inputs.dim() == 3:
                    logger.debug(f"[DEBUG] Expanding dimensions of 'images' from {inputs.shape} to include batch dimension.")
                    inputs = inputs.unsqueeze(0)
                
                logger.debug(f"[DEBUG] Input shape: {inputs.shape}, dtype: {inputs.dtype}")

                encoded_targets = self.encode_batch_targets(batch, self.device, model)

                outputs = model(inputs)

                yolo_out = outputs.get(CENTERNET_OUTPUT_YOLO, None)
                if yolo_out is not None:
                    decoded_yolo = model.yolo_decoder.decode(yolo_out)

                loss = self.nas.calculate_loss(outputs, encoded_targets)
                if isinstance(loss, torch.Tensor):
                    total_loss += loss.item() * inputs.size(0)
                else:
                    total_loss += loss * inputs.size(0)
                total_samples += inputs.size(0)

            except Exception as e:
                logger.error(f"Error in batch {batch_idx}: {str(e)}")
                logger.debug(f"Input shape: {inputs.shape}, dtype: {inputs.dtype}")
                raise

    average_loss = safe_divide(total_loss, total_samples, default=float('inf'))
    logger.debug(f"[DEBUG] Architecture Evaluation - Average Validation Loss: {average_loss}")

    return -average_loss   

def process_tensor(self, t, name):
    import numpy as np

    logger.debug(f"[DEBUG] Entering process_tensor - {name}: type={type(t)}, value={t}")

    if isinstance(t, torch.Tensor):
        # If `t` is a tensor, ensure it's float32 and on the correct device
        t = t.float().to(self.device)
    elif isinstance(t, (list, np.ndarray)):
        if any(isinstance(el, torch.Tensor) for el in t):
            t = torch.cat([el.float() for el in t]).to(self.device)
        else:
            t = torch.tensor(t, dtype=torch.float32).to(self.device)
    elif isinstance(t, (int, float, np.integer, np.floating)):
        t = torch.tensor([t], dtype=torch.float32).to(self.device)
    else:
        raise TypeError(f"{name} must be a tensor, list, numpy array, or scalar, got {type(t)}")

    if t.dim() == 0:
        t = t.unsqueeze(0).unsqueeze(0)
    elif t.dim() == 1:
        t = t.unsqueeze(0)

    logger.debug(f"[DEBUG] Exiting process_tensor - {name}: type={type(t)}, shape={t.shape}")
    return t

def encode_batch_targets(self, batch, device, model):
    print(f"Batch keys: {list(batch.keys())}")

    centers = batch['TARGET_CENTERS_KEY']
    lengths = batch['TARGET_LENGTHS_KEY']
    is_vessel = batch['TARGET_VESSEL_KEY']
    is_fishing = batch['TARGET_FISHING_KEY']
    confidences = batch['INPUT_CONFIDENCE_KEY']

    print(f"Type of centers: {type(centers)}, length: {len(centers)}")
    print(f"Type of lengths: {type(lengths)}, length: {len(lengths)}")
    print(f"Type of is_vessel: {type(is_vessel)}, length: {len(is_vessel)}")
    print(f"Type of is_fishing: {type(is_fishing)}, length: {len(is_fishing)}")
    print(f"Type of confidences: {type(confidences)}, length: {len(confidences)}")

    # Process as per your existing logic...
    # Convert to list if they are tensors
    batch_size = len(centers)
    if isinstance(centers, torch.Tensor):
        centers = [centers[i] for i in range(batch_size)]
    
    if len(confidences) != batch_size:
        raise ValueError(f"Confidences length {len(confidences)} does not match batch_size {batch_size}")

    targets = []
    targets = []

    PIX_TO_M = 10  # Adjust if necessary

    # If any of these are tensors, not lists, ensure they are batched correctly
    if isinstance(centers, torch.Tensor):
        centers = [centers[i] for i in range(batch_size)]
    if isinstance(lengths, torch.Tensor):
        lengths = [lengths[i] for i in range(batch_size)]
    if isinstance(is_vessel, torch.Tensor):
        is_vessel = [is_vessel[i] for i in range(batch_size)]
    if isinstance(is_fishing, torch.Tensor):
        is_fishing = [is_fishing[i] for i in range(batch_size)]
    if isinstance(confidences, torch.Tensor):
        confidences = [confidences[i] for i in range(batch_size)]

    print(f"Type of confidences: {type(confidences)}, Length: {len(confidences)}")
    print(f"Batch size: {batch_size}")

# If confidences is a tensor, check its shape
    if isinstance(confidences, torch.Tensor):
        print(f"Confidences tensor shape: {confidences.shape}")

    for b in range(batch_size):
        # Check if there are any objects in this sample
        if centers[b].numel() == 0:
            # Handle the case with no objects
            # You can create empty targets or skip
            targets.append({
                'boxes': torch.zeros((0, 4), device=device),
                'labels': torch.zeros((0,), dtype=torch.long, device=device),
                'confidences': torch.zeros((0,), device=device),
                'lengths': torch.zeros((0,), device=device),
            })
            continue

        # Convert lengths from meters to pixels
        lengths_in_pixels = lengths[b] / PIX_TO_M  # Tensor [Num_objects]

        # Compute half sizes
        half_size = lengths_in_pixels / 2  # Tensor [Num_objects]

        # Centers in (x, y) format
        x_centers = centers[b][:, 0]  # Tensor [Num_objects]
        y_centers = centers[b][:, 1]  # Tensor [Num_objects]

        # Compute bounding box coordinates
        x1 = x_centers - half_size
        y1 = y_centers - half_size
        x2 = x_centers + half_size
        y2 = y_centers + half_size

        # Stack coordinates into boxes Tensor [Num_objects, 4]
        boxes = torch.stack([x1, y1, x2, y2], dim=1)  # [Num_objects, 4]

        # Create labels
        # Assuming:
        # 0: Non-vessel
        # 1: Vessel
        labels = is_vessel[b].long()  # [Num_objects]

        # Exclude non-vessels if desired
        valid_indices = (is_vessel[b] == 1)
        if valid_indices.sum() == 0:
            # Handle the case with no valid objects
            targets.append({
                'boxes': torch.zeros((0, 4), device=device),
                'labels': torch.zeros((0,), dtype=torch.long, device=device),
                'confidences': torch.zeros((0,), device=device),
                'lengths': torch.zeros((0,), device=device),
            })
            continue

        boxes = boxes[valid_indices].to(device)
        labels = labels[valid_indices].to(device)
        confidences_b = confidences[b][valid_indices].to(device)
        lengths_b = lengths[b][valid_indices].to(device)

        # Prepare the target dictionary
        target_dict = {
            'boxes': boxes,  # [Num_valid_objects, 4]
            'labels': labels,  # [Num_valid_objects]
            'confidences': confidences_b,  # [Num_valid_objects]
            'lengths': lengths_b,  # [Num_valid_objects]
        }

        targets.append(target_dict)

    # Encode YOLO targets
    yolo_targets = model.yolo_decoder.encode(targets)

    # Return the prepared targets
    return {
        'yolo_targets': yolo_targets,
        
    }

The error you're encountering stems from a None value in your data that's being passed to the default_collate function within your centernet_collate function. The default_collate function cannot handle None values, and thus raises a TypeError when it encounters one.
Here's what's happening:
default_collate Function: This function expects each item in the batch to be a tensor, NumPy array, number, dict, or list. If it finds a NoneType, it raises the TypeError you're seeing.
centernet_collate Function: Your custom collate function separates items into included_items and excluded_items. The included_items are passed to default_collate. If any of these included_items contain a None value, the error will be triggered.
Given that is_vessel and is_fishing are strings in your dataset (e.g., 'True', 'False', 'Unknown'), you need to ensure that they are processed correctly throughout your code so that they don't introduce None values.
Steps to Debug and Fix the Issue

Identify None Values in build_centernet_sample Add logging in build_centernet_sample to check for None values:

def build_centernet_sample(...):
    # Existing code...

    # After constructing 'result', check for None values
    for key, value in result.items():
        if value is None:
            print(f"Warning: Key '{key}' has a None value in result for index {index}")
This will help you pinpoint if any keys in result are None.
Check the folder Variable In your build_centernet_sample method, you have:

TARGET_SAMPLE_WEIGHT: {"train": 0.25, "validation": 1.0, "valid": 1.0, "test": 0.1}[folder],
Potential Issue: If folder is not in the keys of the dictionary (e.g., it's None or contains an unexpected value), this will cause a KeyError, or you might inadvertently set TARGET_SAMPLE_WEIGHT to None. Solution:
Ensure that folder always has a valid value.
Modify the dictionary lookup to use .get() with a default value:

sample_weight_dict = {"train": 0.25, "validation": 1.0, "valid": 1.0, "test": 0.1}
result[TARGET_SAMPLE_WEIGHT] = sample_weight_dict.get(folder, 1.0)  # Default to 1.0 if folder is not found
Ensure All Values in included_items Are Valid In your centernet_collate function, make sure that none of the values in included_items are None:

for b in batch:
    included = {k: v for k, v in b.items() if k not in skip_keys and v is not None}
    included_items.append(included)
    for k in skip_keys:
        if k in b:
            excluded_items[k].append(b[k])
This will exclude any keys with None values from included_items, preventing default_collate from encountering them.
Verify Data Loading and Preprocessing Ensure that all data loading and preprocessing steps are correctly handling your data. Specifically:
Image Loading: Make sure that the images are correctly loaded and that image_to_tensor(image) isn't receiving a None image.
Data Augmentations: If you're using augmentations, verify that they don't introduce None values.
Data Types: Confirm that all data types are consistent and correct.
Check is_vessel and is_fishing Mappings Since is_vessel and is_fishing are strings, ensure that your mappings in MultilabelCircleNetCoder use string keys:

self.vessel_mapping = {
    'UNKNOWN': 0,
    'TYPE1': 1,
    'TYPE2': 2,
    # Add other vessel types as necessary
}

self.fishing_mapping = {
    'UNKNOWN': 0,
    'ACTIVE': 1,
    'INACTIVE': 2,
    # Add other fishing statuses as necessary
}
When you access these mappings:

vessel_type = self.vessel_mapping.get(is_vessel[i], self.ignore_value)
fishing_status = self.fishing_mapping.get(is_fishing[i], self.ignore_value)
This ensures that the mappings properly handle your string labels.
Update process_tensor Function in train_multilabel3.py In your encode_batch_targets method, ensure that you're correctly handling the possibility of empty tensors or batches without objects:

# If there are no valid objects in the batch
if len(valid_indices) == 0:
    targets.append({
        'boxes': torch.zeros((0, 4), device=device),
        'labels': torch.zeros((0,), dtype=torch.long, device=device),
        'confidences': torch.zeros((0,), device=device),
        'lengths': torch.zeros((0,), device=device),
    })
    continue
This avoids appending None values to your targets.
Print Batch Contents for Debugging Before the error occurs, print the contents of the batch to see if you can spot any None values:

for idx, batch in enumerate(train_loader):
    print(f"Batch {idx}:")
    for key, value in batch.items():
        print(f"  {key}: {type(value)}")
This can help you identify if any batches contain unexpected None values.
Use a Try-Except Block in Data Loading Temporarily wrap your data loading in a try-except block to catch any exceptions and print detailed information:

try:
    sample_batch = next(iter(train_loader))
except Exception as e:
    print(f"Exception during data loading: {e}")
    import traceback
    traceback.print_exc()
This provides more context about where the error is happening.
Conclusion

The key to resolving your issue is to ensure that no None values are being passed to default_collate through your included_items. By verifying all parts of your data loading and processing pipeline, particularly focusing on the construction of your batch data and the collate function, you should be able to identify and fix the problem.
If you continue to encounter issues after these steps, consider:
Debugging with Smaller Batches: Reduce your batch size to 1 to isolate problematic data samples.
Validation of All Input Data: Ensure that all input data files are present and correctly formatted.
Let me know if you need further assistance or if you have any questions about these steps!

I've followed the your suggestions all but in the train_multilabel3.py still not very aligned needs much correction as
##the train csv should look like
###the desired output should be like
in train csv
detect_lat,detect_lon,vessel_length_m,source,detect_scene_row,detect_scene_column,is_vessel,is_fishing,distance_from_shore_km,scene_id,confidence,top,left,bottom,right,detect_id
5.66292355123629,4.84242888997755,,ais,16722,22703,,,9999.99,e42a50089e03990ft,LOW,,,,,e42a50089e03990ft_005.66292355123628965430_004.84242888997755027702
5.83078557395605,4.79439446548728,,ais,14867,22165,False,,9999.99,e42a50089e03990ft,HIGH,,,,,e42a50089e03990ft_005.83078557395605034941_004.79439446548728032127
5.65028885294943,5.07650030801318,,ais,16853,25297,False,,4.1204849040804055,e42a50089e03990ft,HIGH,,,,,e42a50089e03990ft_005.65028885294942995188_005.07650030801317964091
5.86549528127106,4.93833517222189,,ais,14478,23758,False,,4.34404231490167,e42a50089e03990ft,HIGH,,,,,e42a50089e03990ft_005.86549528127105990194_004.93833517222188955742
in validate csv
detect_lat,detect_lon,vessel_length_m,source,detect_scene_row,detect_scene_column,is_vessel,is_fishing,distance_from_shore_km,scene_id,confidence,top,left,bottom,right,detect_id
44.28796374,13.04173288,,manual,9923,23071,True,,9999.99,264ed833a13b7f2av,LOW,9918.0,23068.0,9928.0,23074.0,264ed833a13b7f2av_044.28796374000000213300_013.04173287999999963915
44.61788761,12.35710138,,manual,6105,17727,True,,6.25220144977796,264ed833a13b7f2av,LOW,6099.0,17725.0,6111.0,17729.0,264ed833a13b7f2av_044.61788760999999681189_012.35710137999999957970
44.28847473,13.7108852,,manual,10023,28410,True,,9999.99,264ed833a13b7f2av,MEDIUM,10018.0,28408.0,10028.0,28412.0,264ed833a13b7f2av_044.28847472999999723697_013.71088519999999988386
vessel_length_m,detect_scene_row,detect_scene_column,scene_id,is_vessel,is_fishing
33.99261,11448,13456,1,True,True
28.166428,15632,12622,1,True,True
32.700775,19454,4840,1,True,True
37.80931,12862,2608,1,True,True
48.392456,4902,5522,1,True,False
###train_multilabel3.py some parts
class CenternetPipeline(Pipeline):
def init(self, cfg: DictConfig):
super().init(cfg)
self.box_coder: Optional[MultilabelCircleNetCoder] = None
self.model: Optional[nn.Module] = None
self.criterion: Optional[nn.Module] = None
self.scaler: Optional[GradScaler] = GradScaler() if torch.cuda.is_available() else None
self.device = TorchDevice.choose_torch_device()
self.experiment_dir = ""
self.yolo_decoder: Optional[YOLODecoder] = None
self.nas: Optional[OptunaNAS] = None

def build_datasets(self, config: DictConfig) -> Tuple[DataLoader, DataLoader, Optional[DataLoader], List[Callback]]:
    """
    Builds and returns DataLoaders for training and validation datasets.
    """
    dataset_module = MultilabelCircleNetDataModule(config=config)

    train_image_size = config.train.image_size if 'image_size' in config.train else [512, 512]
    valid_image_size = config.valid.image_size if 'image_size' in config.valid else [2048, 2048]

    train_df, valid_df, _, shore_root = dataset_module.train_val_split(
        splitter=config.dataset.splitter,
        fold=config.dataset.fold,
        num_folds=config.dataset.num_folds,
    )

    # Validate output_stride
    output_stride = config.model.get('output_stride', 32)
    if output_stride <= 0:
        raise ValueError(f"Output stride must be a positive integer, got {output_stride}")

    self.box_coder = MultilabelCircleNetCoder(
        image_size=(valid_image_size[0], valid_image_size[1]),
        output_stride=output_stride,
        max_objects=config.model.get('max_objects', 512),
        heatmap_encoding=config.model.get('heatmap_encoding', "umich"),
        labels_encoding=config.model.get('labels_encoding', "circle"),
        ignore_value=config.model.get('ignore_value', -1)
    )

    normalization = {
        channel: hydra.utils.instantiate(config.dataset.normalization.channels[channel])
        for channel in config.dataset.input_channels
    }

    # Add global normalization if available
    if 'mean' in config.dataset.normalization and 'std' in config.dataset.normalization:
        global_norm = A.Normalize(
            mean=config.dataset.normalization.mean,
            std=config.dataset.normalization.std
        )
    else:
        global_norm = A.NoOp()

    # Define data augmentation and normalization transforms
    transforms = A.Compose([
        A.HorizontalFlip(p=0.5),
        A.VerticalFlip(p=0.5),
        A.Rotate(limit=15, p=0.5),
        A.RandomBrightnessContrast(p=0.5),
        A.Normalize(mean=config.dataset.normalization.mean, std=config.dataset.normalization.std),
        ToTensorV2(), 
    ], bbox_params=A.BboxParams(format='coco', label_fields=['labels']))

    # Check and validate heatmap_size
    heatmap_size = valid_image_size[0] // output_stride
    if heatmap_size <= 0:
        raise ValueError(f"Heatmap size must be a positive integer, got {heatmap_size}")

    self.valid_df = valid_df.copy()  # Keep a copy of unmodified groundtruth

    # Logging dataset info
    self.master_print("Dataset", config.dataset.slug)
    self.master_print("\tignore_low_confidence_detections", config.dataset.ignore_low_confidence_detections)
    self.master_print("\tignore_low_confidence_labels    ", config.dataset.ignore_low_confidence_labels)
    self.master_print("\tfilter_low_confidence_objects   ", config.dataset.filter_low_confidence_objects)

    self.master_print("Train dataset")
    self.master_print("\tUnknown vessels", np.isnan(train_df.is_vessel.values.astype(float)).sum())
    self.master_print("\tUnknown fishing", np.isnan(train_df.is_fishing.values.astype(float)).sum())
    self.master_print("Valid dataset")
    self.master_print("\tUnknown vessels", np.isnan(valid_df.is_vessel.values.astype(float)).sum())
    self.master_print("\tUnknown fishing", np.isnan(valid_df.is_fishing.values.astype(float)).sum())

    # Check mutually exclusive configurations
    if config.dataset.filter_low_confidence_objects and config.dataset.ignore_low_confidence_labels:
        raise ValueError("filter_low_confidence_objects and ignore_low_confidence_labels are mutually exclusive")

    # Initialize datasets
    train_dataset, train_sampler = dataset_module.get_random_crop_training_dataset(
        train_df=train_df,
        train_image_size=train_image_size,
        input_channels=config.dataset.channels,
        box_coder=self.box_coder,
        normalization=normalization,
        num_samples=config.train.get('num_samples', None),
        individual_augmentations=config.train.get('individual_augmentations', {}),
        augmentations=config.train.get('augs', []),
        balance_crowd=config.train.get('balance_crowd', False),
        balance_location=config.train.get('balance_location', False),
        channels_last=config.dataset.get('channels_last', False),
        read_image_fn=read_multichannel_image,
    )

    valid_dataset = dataset_module.get_validation_dataset(
        valid_df=valid_df,
        valid_crop_size=valid_image_size,
        input_channels=config.dataset.channels,
        box_coder=self.box_coder,
        normalization=normalization,
        channels_last=config.dataset.get('channels_last', False),
        read_image_fn=read_multichannel_image,
    )

    # Setting sampler to None for distributed training or default sampling
    train_sampler = None

    # Prepare callbacks (e.g., for logging, early stopping)
    dataset_callbacks = []

    # Get the custom collate functions
    train_collate_fn = train_dataset.get_collate_fn()
    valid_collate_fn = valid_dataset.get_collate_fn()

    # Initialize DataLoaders
    train_loader = DataLoader(
        train_dataset,
        sampler=train_sampler,
        batch_size=config.train.batch_size,
        num_workers=config.train.num_workers,
        pin_memory=True,
        shuffle=(train_sampler is None),
        persistent_workers=config.train.get('persistent_workers', False),
        collate_fn=train_collate_fn
    )
    valid_loader = DataLoader(
        valid_dataset,
        batch_size=config.train.get('batch_size', config.train.batch_size),
        num_workers=config.train.get('num_workers', config.train.num_workers),
        pin_memory=True,
        shuffle=False,
        collate_fn=valid_collate_fn
    )

    # Add debug prints to check shapes and dtypes
    try:
        sample_batch = next(iter(train_loader))
        logger.debug(f"[DEBUG] Sample batch keys: {sample_batch.keys()}")
        for k, v in sample_batch.items():
            if isinstance(v, torch.Tensor):
                logger.debug(f"[DEBUG] Sample batch '{k}' dtype: {v.dtype}, shape: {v.shape}")
            elif isinstance(v, dict):
                for sub_k, sub_v in v.items():
                    if isinstance(sub_v, torch.Tensor):
                        logger.debug(f"[DEBUG] Sample batch '{k}.{sub_k}' dtype: {sub_v.dtype}, shape: {sub_v.shape}")
    except StopIteration:
        logger.error("[ERROR] Training loader is empty!")
        raise

    return train_loader, valid_loader, None, dataset_callbacks

def build_model(self, config: DictConfig, train_loader: DataLoader, valid_loader: DataLoader) -> nn.Module:
    """
    Builds the model using CenterNetUNet with integrated heads and NAS if configured.
    """
    logger.info("Building model with config:")
    logger.info(OmegaConf.to_yaml(config.model))

    # Initialize YOLODecoder
    self.yolo_decoder = YOLODecoder(
        anchors=config.model.anchors,
        num_classes=config.model.num_classes,
        input_image_size=tuple(config.model.image_size),
        stride=config.model.output_stride,
        conf_threshold=config.inference.get('thresholds', {}).get('objectness', 0.5),
        iou_threshold=config.inference.get('thresholds', {}).get('iou', 0.4)
    ).to(self.device)
    self.yolo_decoder = self.yolo_decoder.float()  # Ensure decoder is float32

    
    if config.model.nas.enabled:
        self.nas = OptunaNAS(
            search_space=config.model.nas.search_space,
            evaluate_architecture=self.evaluate_architecture,
            direction=config.model.nas.direction
        )
        best_trial = self.nas.optimize(
            n_trials=config.model.nas.num_trials,
            config=config,
            train_loader=train_loader,
            val_loader=valid_loader
        )
        logger.info(f"Best NAS trial: {best_trial}")
        # Update model config with best trial params
        config.model = OmegaConf.merge(config.model, best_trial.params)

    self.model = CenterNetUNetModel.from_config(config.model)
    self.model = self.model.to(self.device)
    self.model = self.model.float()  # Ensure model parameters are float32

    logger.info(f"Total parameters in the model: {count_parameters(self.model)}")

    return self.model

def build_criterions(self, config: DictConfig) -> Tuple[Dict[str, nn.Module], List[Callback]]:
    """
    Builds and returns loss functions and their corresponding callbacks.
    """
    criterions = {}
    criterions_callbacks = []

    # Initialize combined loss
    combined_loss = CircleNetLoss().to(self.device)
    combined_loss = combined_loss.float()  # Ensure float32
    criterions['combined_loss'] = combined_loss

    return criterions, criterions_callbacks

def evaluate_architecture(self, trial: optuna.Trial, config: Dict[str, Any], train_loader: DataLoader, val_loader: DataLoader) -> float:
    """
    Evaluates a given model architecture on the validation set.
    This function is used by OptunaNAS to determine the performance of different architectures.
    """
    model = self.nas.create_model(config, trial)
    model.to(self.device)
    model.eval()

    # Initialize YOLODecoder and attach it to the model
    model.yolo_decoder = YOLODecoder(
        anchors=config.model.anchors,
        num_classes=config.model.num_classes,
        input_image_size=tuple(config.model.image_size),
        stride=config.model.output_stride,
        conf_threshold=config.get('inference', {}).get('thresholds', {}).get('objectness', 0.5),
        iou_threshold=config.get('inference', {}).get('thresholds', {}).get('iou', 0.4)
    ).to(self.device)
    model.yolo_decoder = model.yolo_decoder.float()

    total_loss = 0.0
    total_samples = 0

    with torch.no_grad():
        for batch_idx, batch in enumerate(val_loader):
            try:
                inputs = batch['INPUT_IMAGE_KEY'].to(self.device)
                if inputs.dtype != torch.float32:
                    logger.debug(f"[DEBUG] Casting 'images' from {inputs.dtype} to float32.")
                    inputs = inputs.float()

                # Ensure correct input shape
                if inputs.dim() == 3:
                    logger.debug(f"[DEBUG] Expanding dimensions of 'images' from {inputs.shape} to include batch dimension.")
                    inputs = inputs.unsqueeze(0)
                
                logger.debug(f"[DEBUG] Input shape: {inputs.shape}, dtype: {inputs.dtype}")

                encoded_targets = self.encode_batch_targets(batch, self.device, model)

                outputs = model(inputs)

                yolo_out = outputs.get(CENTERNET_OUTPUT_YOLO, None)
                if yolo_out is not None:
                    decoded_yolo = model.yolo_decoder.decode(yolo_out)

                loss = self.nas.calculate_loss(outputs, encoded_targets)
                if isinstance(loss, torch.Tensor):
                    total_loss += loss.item() * inputs.size(0)
                else:
                    total_loss += loss * inputs.size(0)
                total_samples += inputs.size(0)

            except Exception as e:
                logger.error(f"Error in batch {batch_idx}: {str(e)}")
                logger.debug(f"Input shape: {inputs.shape}, dtype: {inputs.dtype}")
                raise

    average_loss = safe_divide(total_loss, total_samples, default=float('inf'))
    logger.debug(f"[DEBUG] Architecture Evaluation - Average Validation Loss: {average_loss}")

    return -average_loss   

def process_tensor(self, t, name):
    import numpy as np

    logger.debug(f"[DEBUG] Entering process_tensor - {name}: type={type(t)}, value={t}")

    if len(valid_indices) == 0:
        targets.append({
            'boxes': torch.zeros((0, 4), device=device),
            'labels': torch.zeros((0,), dtype=torch.long, device=device),
            'confidences': torch.zeros((0,), device=device),
            'lengths': torch.zeros((0,), device=device),
        })
        continue

    if isinstance(t, torch.Tensor):
        # If `t` is a tensor, ensure it's float32 and on the correct device
        t = t.float().to(self.device)
    elif isinstance(t, (list, np.ndarray)):
        if any(isinstance(el, torch.Tensor) for el in t):
            t = torch.cat([el.float() for el in t]).to(self.device)
        else:
            t = torch.tensor(t, dtype=torch.float32).to(self.device)
    elif isinstance(t, (int, float, np.integer, np.floating)):
        t = torch.tensor([t], dtype=torch.float32).to(self.device)
    else:
        raise TypeError(f"{name} must be a tensor, list, numpy array, or scalar, got {type(t)}")

    if t.dim() == 0:
        t = t.unsqueeze(0).unsqueeze(0)
    elif t.dim() == 1:
        t = t.unsqueeze(0)

    logger.debug(f"[DEBUG] Exiting process_tensor - {name}: type={type(t)}, shape={t.shape}")
    return t

def encode_batch_targets(self, batch, device, model):
    print(f"Batch keys: {list(batch.keys())}")

    centers = batch['TARGET_CENTERS_KEY']
    lengths = batch['TARGET_LENGTHS_KEY']
    is_vessel = batch['TARGET_VESSEL_KEY']
    is_fishing = batch['TARGET_FISHING_KEY']
    confidences = batch['INPUT_CONFIDENCE_KEY']

    print(f"Type of centers: {type(centers)}, length: {len(centers)}")
    print(f"Type of lengths: {type(lengths)}, length: {len(lengths)}")
    print(f"Type of is_vessel: {type(is_vessel)}, length: {len(is_vessel)}")
    print(f"Type of is_fishing: {type(is_fishing)}, length: {len(is_fishing)}")
    print(f"Type of confidences: {type(confidences)}, length: {len(confidences)}")

    # Process as per your existing logic...
    # Convert to list if they are tensors
    batch_size = len(centers)
    if isinstance(centers, torch.Tensor):
        centers = [centers[i] for i in range(batch_size)]
    
    if len(confidences) != batch_size:
        raise ValueError(f"Confidences length {len(confidences)} does not match batch_size {batch_size}")

    targets = []
    targets = []

    PIX_TO_M = 10  # Adjust if necessary

    # If any of these are tensors, not lists, ensure they are batched correctly
    if isinstance(centers, torch.Tensor):
        centers = [centers[i] for i in range(batch_size)]
    if isinstance(lengths, torch.Tensor):
        lengths = [lengths[i] for i in range(batch_size)]
    if isinstance(is_vessel, torch.Tensor):
        is_vessel = [is_vessel[i] for i in range(batch_size)]
    if isinstance(is_fishing, torch.Tensor):
        is_fishing = [is_fishing[i] for i in range(batch_size)]
    if isinstance(confidences, torch.Tensor):
        confidences = [confidences[i] for i in range(batch_size)]

    print(f"Type of confidences: {type(confidences)}, Length: {len(confidences)}")
    print(f"Batch size: {batch_size}")

# If confidences is a tensor, check its shape
    if isinstance(confidences, torch.Tensor):
        print(f"Confidences tensor shape: {confidences.shape}")

    for b in range(batch_size):
        # Check if there are any objects in this sample
        if centers[b].numel() == 0:
            # Handle the case with no objects
            # You can create empty targets or skip
            targets.append({
                'boxes': torch.zeros((0, 4), device=device),
                'labels': torch.zeros((0,), dtype=torch.long, device=device),
                'confidences': torch.zeros((0,), device=device),
                'lengths': torch.zeros((0,), device=device),
            })
            continue

        # Convert lengths from meters to pixels
        lengths_in_pixels = lengths[b] / PIX_TO_M  # Tensor [Num_objects]

        # Compute half sizes
        half_size = lengths_in_pixels / 2  # Tensor [Num_objects]

        # Centers in (x, y) format
        x_centers = centers[b][:, 0]  # Tensor [Num_objects]
        y_centers = centers[b][:, 1]  # Tensor [Num_objects]

        # Compute bounding box coordinates
        x1 = x_centers - half_size
        y1 = y_centers - half_size
        x2 = x_centers + half_size
        y2 = y_centers + half_size

        # Stack coordinates into boxes Tensor [Num_objects, 4]
        boxes = torch.stack([x1, y1, x2, y2], dim=1)  # [Num_objects, 4]

        # Create labels
        # Assuming:
        # 0: Non-vessel
        # 1: Vessel
        labels = is_vessel[b].long()  # [Num_objects]

        # Exclude non-vessels if desired
        valid_indices = (is_vessel[b] == 1)
        if valid_indices.sum() == 0:
            # Handle the case with no valid objects
            targets.append({
                'boxes': torch.zeros((0, 4), device=device),
                'labels': torch.zeros((0,), dtype=torch.long, device=device),
                'confidences': torch.zeros((0,), device=device),
                'lengths': torch.zeros((0,), device=device),
            })
            continue

        boxes = boxes[valid_indices].to(device)
        labels = labels[valid_indices].to(device)
        confidences_b = confidences[b][valid_indices].to(device)
        lengths_b = lengths[b][valid_indices].to(device)

        # Prepare the target dictionary
        target_dict = {
            'boxes': boxes,  # [Num_valid_objects, 4]
            'labels': labels,  # [Num_valid_objects]
            'confidences': confidences_b,  # [Num_valid_objects]
            'lengths': lengths_b,  # [Num_valid_objects]
        }

        targets.append(target_dict)

    # Encode YOLO targets
    yolo_targets = model.yolo_decoder.encode(targets)

    # Return the prepared targets
    return {
        'yolo_targets': yolo_targets,
        
    }

It seems like you're encountering issues with your current implementation, particularly with handling None values and properly mapping string labels of is_vessel and is_fishing to numerical values. Let's address each of these issues step by step and ensure your code is correctly aligned.
1. The NoneType Error in default_collate
Cause of the Error

The error:

TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found <class 'NoneType'>
indicates that the default_collate function received a NoneType value in one of the items it's trying to collate. Specifically, this happens in your centernet_collate function when collating included_items.
Solution

Update centernet_collate Function
In your centernet_collate function, you have a list of skip_keys which represents keys that should not be passed to default_collate. Any key not in skip_keys will be included in included_items and passed to default_collate.
Your current skip_keys are:

skip_keys = [
    TARGET_CENTERS_KEY,
    TARGET_FISHING_KEY,
    TARGET_VESSEL_KEY,
    TARGET_LABELS_KEY,
    TARGET_LENGTHS_KEY,
    INPUT_SCENE_ID_KEY,
    INPUT_SCENE_CROP_KEY,
    INPUT_CONFIDENCE_KEY,
    INPUT_LOCATION_ID_KEY,
    INPUT_IS_NEAR_SHORE_KEY,
]
Potential Missing Keys in skip_keys
INPUT_FOLDER_KEY
TARGET_SAMPLE_WEIGHT
If INPUT_FOLDER_KEY and TARGET_SAMPLE_WEIGHT are strings or None, passing them to default_collate will cause the error you're seeing.
Add Missing Keys to skip_keys
Update your skip_keys to include these keys:

skip_keys = [
    TARGET_CENTERS_KEY,
    TARGET_FISHING_KEY,
    TARGET_VESSEL_KEY,
    TARGET_LABELS_KEY,
    TARGET_LENGTHS_KEY,
    INPUT_SCENE_ID_KEY,
    INPUT_SCENE_CROP_KEY,
    INPUT_CONFIDENCE_KEY,
    INPUT_LOCATION_ID_KEY,
    INPUT_IS_NEAR_SHORE_KEY,
    INPUT_FOLDER_KEY,        # Added
    TARGET_SAMPLE_WEIGHT,    # Added
]
Explanation
By adding these keys to skip_keys, they will be handled separately, and you avoid passing None or unsupported types to default_collate.
2. Handling is_vessel and is_fishing
Issue

Your data contains is_vessel and is_fishing as strings (e.g., "True", "False", "Unknown"). In your code, you're treating them as tensors and attempting numerical operations without proper mapping.
Solution

Implement Mappings
In your MultilabelCircleNetCoder, ensure you have mappings for is_vessel and is_fishing that correspond to your string labels:

self.vessel_mapping = {
    'False': 0,
    'True': 1,
    'Unknown': -1,  # or another appropriate value
}

self.fishing_mapping = {
    'False': 0,
    'True': 1,
    'Unknown': -1,  # or another appropriate value
}
Update encode Method in MultilabelCircleNetCoder
When you process is_vessel and is_fishing, map them using your mappings:

vessel_type = self.vessel_mapping.get(is_vessel_sorted[i], self.ignore_value)
fishing_status = self.fishing_mapping.get(is_fishing_sorted[i], self.ignore_value)
Ensure Data Types in build_centernet_sample
In your build_centernet_sample method in multilabel_dataset.py, when converting is_vessel and is_fishing, keep them as lists of strings to pass into the coder:

# Keep 'is_vessel' and 'is_fishing' as lists of strings
is_vessel_list = is_vessel  # List of strings
is_fishing_list = is_fishing  # List of strings
You don't need to convert them to NumPy arrays if they're strings.
3. Correcting encode_batch_targets in train_multilabel3.py
Issue

In your encode_batch_targets method, you're treating is_vessel as a tensor, but it's actually a list of strings. Attempting to perform numerical operations directly on them will cause errors.
Solution

Map is_vessel and is_fishing to Numerical Values
Before performing any numerical operations, map the string labels to numerical values using your mappings.
Here's how you can adjust your encode_batch_targets method:

def encode_batch_targets(self, batch, device, model):
    centers = batch[TARGET_CENTERS_KEY]
    lengths = batch[TARGET_LENGTHS_KEY]
    is_vessel = batch[TARGET_VESSEL_KEY]
    is_fishing = batch[TARGET_FISHING_KEY]
    confidences = batch[INPUT_CONFIDENCE_KEY]

    batch_size = len(centers)
    targets = []

    for b in range(batch_size):
        if centers[b].numel() == 0:
            targets.append({
                'boxes': torch.zeros((0, 4), device=device),
                'labels': torch.zeros((0,), dtype=torch.long, device=device),
                'confidences': torch.zeros((0,), device=device),
                'lengths': torch.zeros((0,), device=device),
            })
            continue

        lengths_in_pixels = lengths[b] / 10  # Adjust PIX_TO_M if necessary

        # Centers in (x, y) format
        x_centers = centers[b][:, 0]  # Tensor [Num_objects]
        y_centers = centers[b][:, 1]  # Tensor [Num_objects]
        
        # Compute half sizes
        half_size = lengths_in_pixels / 2

        # Compute bounding box coordinates
        x1 = x_centers - half_size
        y1 = y_centers - half_size
        x2 = x_centers + half_size
        y2 = y_centers + half_size

        # Stack coordinates into boxes Tensor [Num_objects, 4]
        boxes = torch.stack([x1, y1, x2, y2], dim=1).to(device)

        # Map 'is_vessel' from strings to numerical labels
        is_vessel_labels = []
        for label in is_vessel[b]:
            numerical_label = self.box_coder.vessel_mapping.get(label, self.box_coder.ignore_value)
            is_vessel_labels.append(numerical_label)
        labels = torch.tensor(is_vessel_labels, dtype=torch.long, device=device)
        
        # Similarly for 'confidences' if needed

        valid_indices = labels != self.box_coder.ignore_value
        if valid_indices.sum() == 0:
            targets.append({
                'boxes': torch.zeros((0, 4), device=device),
                'labels': torch.zeros((0,), dtype=torch.long, device=device),
                'confidences': torch.zeros((0,), device=device),
                'lengths': torch.zeros((0,), device=device),
            })
            continue

        # Filter for valid indices
        boxes = boxes[valid_indices]
        labels = labels[valid_indices]
        lengths_b = lengths[b][valid_indices].to(device)
        # Process confidences if needed

        # Prepare the target dictionary
        target_dict = {
            'boxes': boxes,
            'labels': labels,
            'lengths': lengths_b,
            # Add 'confidences' if processed
        }

        targets.append(target_dict)

    # Further processing, encoding with YOLO decoder if needed
    yolo_targets = model.yolo_decoder.encode(targets)

    return {'yolo_targets': yolo_targets}
Explanation
Mapping Labels: Converts string labels to numerical values before using them in computations.
Filtering Valid Indices: Excludes objects with labels that map to ignore_value.
Consistent Data Types: Ensures all tensors are properly transferred to the correct device and have appropriate data types.
4. Verifying Data Types and Avoiding None Values
Ensure No None Types in Included Items

In your build_centernet_sample method, make sure that any value that might be None is either handled appropriately or added to skip_keys.
Specifically, verify that folder and any other variable used in your result dictionary are properly initialized and not None.
Update Sample Weight Calculation

Adjust your sample weight calculation to handle missing or unexpected folder names:

sample_weight_dict = {"train": 0.25, "validation": 1.0, "valid": 1.0, "test": 0.1}
result[TARGET_SAMPLE_WEIGHT] = sample_weight_dict.get(folder, 1.0)  # Default to 1.0 if 'folder' key is not found
Check All Included Items Before Collation

In your centernet_collate function, after assembling included_items, you can add a check to ensure no None values are present:

for included in included_items:
    for key, value in included.items():
        if value is None:
            raise ValueError(f"Included item '{key}' has None value")
5. Additional Recommendations
Print Data Types and Shapes for Debugging

Add debug prints in your data loading and processing functions to check the types and shapes of your data at each step.
Validate All Mappings

Ensure all your mappings cover all possible values in your datasets. For instance, if you have 'Unknown' labels, decide how you want to handle them (e.g., map to a special ignore_value or a separate class).
Test with Small Batches

Run your code with a small batch size (e.g., 1 or 2) to easily trace through the data processing steps and identify where issues might occur.
Summary of Steps to Resolve the Issues
Update skip_keys in centernet_collate to include INPUT_FOLDER_KEY and TARGET_SAMPLE_WEIGHT to prevent passing strings or None to default_collate.
Properly map is_vessel and is_fishing from strings to numerical labels in your encode_batch_targets method before performing numerical operations.
Ensure that build_centernet_sample and other data preprocessing steps handle strings and possible None values appropriately, avoiding None entries in included items.
Verify that all data passed to default_collate consists of acceptable types (tensors, arrays, numbers, dicts, lists) and that no None values are present.
Add debug statements throughout your code to track data types and shapes, and to catch any unexpected values early.
Run unit tests or small-scale tests to validate each component of your data pipeline.
By carefully implementing these changes and thoroughly testing your code, you should resolve the errors you're encountering and ensure that your data pipeline correctly handles string labels and avoids passing None values to functions that cannot process them.