this project indeed appears to be a complex integrated multi-model system for vessel detection and fishing activity classification using SAR (Synthetic Aperture Radar) TIFF images as input. The system incorporates several advanced deep learning architectures and techniques, including:
CenterNet UNet: A combination of CenterNet (for object detection) and U-Net (for image segmentation).
YOLO (You Only Look Once): A real-time object detection system.
NAS (Neural Architecture Search): For optimizing model architecture.
FBN (Feature Pyramid Network): For multi-scale feature extraction.
FAN (Feature Aggregation Network): For combining features from different levels.
Swin Transformer: A hierarchical vision transformer.
Optuna: A hyperparameter optimization framework.
The system is designed to process SAR images and output predictions in a CSV format, including:
Vessel length in meters
Detection coordinates (row and column in the scene)
Scene ID
Vessel classification (True/False)
Fishing activity classification (True/False)
This type of system is typically used for maritime surveillance, monitoring fishing activities, and enforcing maritime regulations.
The configuration files and code snippets you've provided show various aspects of the model architecture, training process, and inference pipeline. Some key components include:
Model configuration (encoder, decoder, head)
Data augmentation strategies
Loss functions and optimization settings
Inference settings (tile size, batch size, thresholds)
Ensemble methods and Test Time Augmentation (TTA)
If you have any specific questions about the system or need clarification on any part of the codebase, please feel free to ask.
the outputs looks as
vessel_length_m,detect_scene_row,detect_scene_column,scene_id,is_vessel,is_fishing
33.99261,11448,13456,1,True,True
28.166428,15632,12622,1,True,True
32.700775,19454,4840,1,True,True
37.80931,12862,2608,1,True,True
48.392456,4902,5522,1,True,False
understand this deeply and review the following two functions and just say "done" and remember everything as we will need all these details to fix errors
##custom_encoder.py
class UpsampleBlock(nn.Module):
def init(self, in_channels, out_channels, scale_factor=2, mode='bilinear', align_corners=False):
super(UpsampleBlock, self).init()
self.upsample = nn.Sequential(
nn.Upsample(scale_factor=scale_factor, mode=mode, align_corners=align_corners),
nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1, bias=False),
nn.GroupNorm(16, out_channels),
nn.ReLU(inplace=True)
)
print(f"UpsampleBlock initialized with in_channels={in_channels}, out_channels={out_channels}")
def forward(self, x):
print(f"UpsampleBlock input shape: {x.shape}")
output = self.upsample(x)
print(f"UpsampleBlock output shape: {output.shape}")
return output
class CustomTimmEncoder(nn.Module):
def init(
self,
model_name: str = "efficientnetv2_rw_s",
num_channels: int = 2,
features_only: bool = True,
pretrained: bool = False,
activation: str = "silu",
drop_rate: float = 0.0,
drop_path_rate: float = 0.0,
image_size: Tuple[int, int] = (224, 224),
):
super(CustomTimmEncoder, self).init()
self.model_name = model_name # Store model name for conditional handling
self.encoder = timm.create_model(
model_name,
features_only=features_only,
in_chans=num_channels,
pretrained=pretrained,
act_layer=activation,
drop_rate=drop_rate,
drop_path_rate=drop_path_rate,
)
Extract channels and strides
Extract actual feature channels and strides
In CustomTimmEncoder
self.feature_info = self.encoder.feature_info
self.channels = [f['num_chs'] for f in self.feature_info]
#self.channels = [f['num_chs'] for f in self.encoder.feature_info]
self.strides = [f['reduction'] for f in self.feature_info]
self.feature_info = self.feature_info
self.image_size = image_size
self.embed_dim = self.channels[-1]
self.depths = [2, 2, 6, 2]
self.num_heads = [3, 6, 12, 24]
self.window_size = 7
self.mlp_ratio = 4.
self.qkv_bias = True
self.qk_scale = None
self.drop_rate = drop_rate
self.attn_drop_rate = 0.0
self.drop_path_rate = drop_path_rate
self.norm_layer = nn.LayerNorm
self.use_dropout = True
self.use_custom_pos_embed = False
self.transformer_output_channels = self.channels[-1]
self.output_stride = 32
Conditional handling based on encoder type
if "swin" in model_name.lower() or "transformer" in model_name.lower():
# Assuming transformer-based encoder has 'patch_embed'
if hasattr(self.encoder, 'patch_embed'):
self.conv_stem = self.encoder.patch_embed.proj
else:
raise AttributeError(f"The encoder '{model_name}' does not have 'patch_embed' attribute.")
else:
# For CNN-based encoders like EfficientNet
if hasattr(self.encoder, 'conv_stem'):
self.conv_stem = self.encoder.conv_stem
elif hasattr(self.encoder, 'conv1'):
self.conv_stem = self.encoder.conv1 # Example for some other CNNs
else:
raise AttributeError(f"The encoder '{model_name}' does not have a recognized stem convolution layer.")
def forward(self, x):
encoder_features = self.encoder(x)
if not isinstance(encoder_features, list):
encoder_features = [encoder_features]
logger.debug(f"Encoder features shapes: {[f.shape for f in encoder_features]}")
for idx, encoder_feature in enumerate(encoder_features):
if encoder_feature.dim() == 3:
# Missing batch dimension, add it
encoder_feature = encoder_feature.unsqueeze(0)
encoder_features[idx] = encoder_feature
print(f"[WARNING] Feature {idx} missing batch dimension, adding it.")
elif encoder_feature.dim() < 3:
raise ValueError(f"Feature {idx} has unexpected number of dimensions: {encoder_feature.dim()}")
print(f"[DEBUG] Feature {idx} shape: {encoder_feature.shape}") # Existing debug
Additional debug to ensure all features have batch dimension
for idx, encoder_feature in enumerate(encoder_features):
if encoder_feature.dim() != 4:
print(f"[ERROR] Feature {idx} does not have 4 dimensions after processing: {encoder_feature.dim()}")
raise ValueError(f"Feature {idx} has incorrect number of dimensions: {encoder_feature.dim()}")
return encoder_features
def change_input_channels(self, input_channels: int, mode="auto", **kwargs):
if isinstance(self.conv_stem, Conv2dSame):
self.conv_stem = make_n_channel_input_conv2d_same(self.conv_stem, input_channels, mode, **kwargs)
else:
self.conv_stem = make_n_channel_input(self.conv_stem, input_channels, mode, **kwargs)
return self
class DebugUnetDecoder(nn.Module):
def init(
self,
encoder_channels,
decoder_channels,
unet_block,
upsample_block=UpsampleBlock,
center=False,
activation='relu'
):
super(DebugUnetDecoder, self).init()
self.center = center
self.encoder_channels = encoder_channels
self.decoder_channels = decoder_channels
self.num_stages = len(decoder_channels)
self.up_samples = nn.ModuleList()
self.unet_blocks = nn.ModuleList()
logger.debug(f"Initializing DebugUnetDecoder with encoder_channels={encoder_channels} and decoder_channels={decoder_channels}")
for i in range(self.num_stages):
if i == 0:
in_channels = encoder_channels[-1]
else:
in_channels = decoder_channels[i-1]
out_channels = decoder_channels[i]

logger.debug(f"Stage {i}: in_channels={in_channels}, out_channels={out_channels}")

# Check for non-zero channels
assert in_channels > 0, f"in_channels must be positive, got {in_channels}"
assert out_channels > 0, f"out_channels must be positive, got {out_channels}"

self.up_samples.append(
    upsample_block(
        in_channels=in_channels,
        out_channels=out_channels
    )
)

...

class DecoupledHeadGroupNormLateShuffle(nn.Module):
    def __init__(
        self,
        channels: int,
        classifier_dim: int,
        dropout_rate: float = 0.25,
        objectness_dim: int = 32,
        size_dim: int = 32,
        offset_dim: int = 32,
        num_blocks: int = 3,
        activation: str = 'relu',  # Assuming ACT_RELU maps to 'relu'
        num_groups: int = 16,
        yolo_dim: int = 128,
        use_se: bool = False,
        use_dropblock: bool = False,
        dropblock_prob: float = 0.1,
    ):
        super().__init__()
        
        # Objectness Branch
        self.objectness = build_regression_block(
            input_channels=channels,
            embedding_size=objectness_dim,
            dropout_rate=dropout_rate,
            num_blocks=num_blocks,
            activation=activation,
            extra_final_layers=[
                ("upsample", nn.Upsample(scale_factor=2, mode='bilinear', align_corners=False)),
                ("final", nn.Conv2d(objectness_dim, 1, kernel_size=(3, 3), padding=1)),
            ],
            num_groups=num_groups,
            use_se=use_se,
            use_dropblock=use_dropblock,
            dropblock_prob=dropblock_prob,
        )
        
        # Classifier Branch
        self.classifier = build_regression_block(
            input_channels=channels,
            embedding_size=classifier_dim,
            dropout_rate=dropout_rate,
            num_blocks=num_blocks,
            activation=activation,
            extra_final_layers=[],  # No extra layers
            num_groups=num_groups,
            use_se=use_se,
            use_dropblock=use_dropblock,
            dropblock_prob=dropblock_prob,
        )
        
        # Size Branch
        self.size = build_regression_block(
            input_channels=channels,
            embedding_size=size_dim,
            dropout_rate=dropout_rate,
            num_blocks=num_blocks,
            activation=activation,
            extra_final_layers=[],  # No extra layers
            num_groups=num_groups,
            use_se=use_se,
            use_dropblock=use_dropblock,
            dropblock_prob=dropblock_prob,
        )
        
        # Feature Recombination and Upsampling for Classifier
        self.classifier_recombination = FeatureRecombination(classifier_dim, size_dim)
        self.classifier_up = nn.Sequential(
            OrderedDict(
                [
                    ("conv1", nn.Conv2d(classifier_dim, classifier_dim, kernel_size=(3, 3), padding=1)),
                    ("upsample", nn.Upsample(scale_factor=2, mode='bilinear', align_corners=False)),
                    ("final", nn.Conv2d(classifier_dim, 2, kernel_size=(3, 3), padding=1)),
                ]
            )
        )
        
        # Feature Recombination and Upsampling for Size
        self.size_recombination = FeatureRecombination(size_dim, classifier_dim)
        self.size_up = nn.Sequential(
            OrderedDict(
                [
                    ("conv1", nn.Conv2d(size_dim, size_dim, kernel_size=(3, 3), padding=1)),
                    ("upsample", nn.Upsample(scale_factor=2, mode='bilinear', align_corners=False)),
                    ("final", nn.Conv2d(size_dim, 1, kernel_size=(3, 3), padding=1)),
                    ("activation", nn.Softplus()),
                ]
            )
        )
        
        # Offset Branch (Optional)
        if offset_dim:
            self.offset = build_regression_block(
                input_channels=channels,
                embedding_size=offset_dim,
                dropout_rate=dropout_rate,
                num_blocks=num_blocks,
                activation=activation,
                extra_final_layers=[
                    ("upsample", nn.Upsample(scale_factor=2, mode='bilinear', align_corners=False)),
                    ("final", nn.Conv2d(offset_dim, 2, kernel_size=(3, 3), padding=1)),
                ],
                num_groups=num_groups,
                use_se=use_se,
                use_dropblock=use_dropblock,
                dropblock_prob=dropblock_prob,
            )
        else:
            self.offset = None
        
        # YOLO Head corrected to match 6 anchors, resulting in 42 output channels
        self.yolo_head = build_regression_block(
            input_channels=channels,
            embedding_size=yolo_dim,
            dropout_rate=dropout_rate,
            num_blocks=num_blocks,
            activation=activation,
            extra_final_layers=[
                ("upsample", nn.Upsample(scale_factor=2, mode='bilinear', align_corners=False)),
                ("final", nn.Conv2d(yolo_dim, 42, kernel_size=(3, 3), padding=1)),  # 6 anchors * (5 + 2) = 42
            ],
            num_groups=num_groups,
            use_se=use_se,
            use_dropblock=use_dropblock,
            dropblock_prob=dropblock_prob,
        )
        
        # Determine the total output channels based on the presence of offset
        self.out_channels = 6 if self.offset is not None else 4
        
        # Initialize weights
        self._init_weights()
        
        # Initialize metrics
        self.accuracy = torchmetrics.Accuracy(task="binary")
        self.f1 = torchmetrics.F1Score(num_classes=2, average='macro', task='multiclass')
    
    def _init_weights(self):
        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
            elif isinstance(m, (nn.BatchNorm2d, nn.GroupNorm, SqueezeExcitation)):
                nn.init.constant_(m.weight, 1)
                nn.init.constant_(m.bias, 0)
    
        # Initialize biases for certain layers
        if hasattr(self.objectness, 'final'):
            self.objectness.final.bias.data.fill_(-5)
        if hasattr(self.classifier_up, 'final'):
            self.classifier_up.final.bias.data.fill_(-1)
        if hasattr(self.size_up, 'final'):
            self.size_up.final.bias.data.fill_(2)
    
    def forward(self, features: Tensor) -> Dict[str, Tensor]:
        # Classifier Branch
        class_features = self.classifier(features)
        logger.debug(f"After classifier: {class_features.shape}")
        
        # Size Branch
        length_features = self.size(features)
        logger.debug(f"After size: {length_features.shape}")
    
        # Recombination
        new_class_features = self.classifier_recombination(class_features, length_features)
        logger.debug(f"After classifier recombination: {new_class_features.shape}")
        
        new_length_features = self.size_recombination(length_features, class_features)
        logger.debug(f"After size recombination: {new_length_features.shape}")
    
        # Upsampling
        predicted_classes = self.classifier_up(new_class_features)
        logger.debug(f"After classifier upsampling: {predicted_classes.shape}")
    
        predicted_length = self.size_up(new_length_features)
        logger.debug(f"After size upsampling: {predicted_length.shape}")
    
        # Objectness Branch
        objectness_map = self.objectness(features)
        if objectness_map is None:
            logger.error("Objectness map is None.")
        else:
            logger.debug(f"Objectness map shape: {objectness_map.shape}")
    
        # Vessel and Fishing Branches
        vessel_map = predicted_classes[:, 0:1]
        fishing_map = predicted_classes[:, 1:2]
        logger.debug(f"Vessel map shape: {vessel_map.shape}")
        logger.debug(f"Fishing map shape: {fishing_map.shape}")
    
        # Size Branch
        size_map = predicted_length
        logger.debug(f"Size map shape: {size_map.shape}")
    
        # YOLO Head
        yolo_output = self.yolo_head(features)
        if yolo_output is None:
            logger.error("YOLOHead output is None.")
        else:
            logger.debug(f"YOLOHead output shape: {yolo_output.shape}")
    
        # Offset Branch (Optional)
        if self.offset is not None:
            offset = self.offset(features).clip(0, 1)
            if offset is None:
                logger.error("Offset output is None.")
            else:
                logger.debug(f"Offset shape: {offset.shape}")
            output = {
                CENTERNET_OUTPUT_OBJECTNESS_MAP: objectness_map,
                CENTERNET_OUTPUT_VESSEL_MAP: vessel_map,
                CENTERNET_OUTPUT_FISHING_MAP: fishing_map,
                CENTERNET_OUTPUT_SIZE: size_map,
                CENTERNET_OUTPUT_YOLO: yolo_output,
                CENTERNET_OUTPUT_OFFSET: offset,
            }
        else:
            output = {
                CENTERNET_OUTPUT_OBJECTNESS_MAP: objectness_map,
                CENTERNET_OUTPUT_VESSEL_MAP: vessel_map,
                CENTERNET_OUTPUT_FISHING_MAP: fishing_map,
                CENTERNET_OUTPUT_SIZE: size_map,
                CENTERNET_OUTPUT_YOLO: yolo_output,
            }
    
        # Debugging: Check for None in outputs
        for key, value in output.items():
            if value is None:
                logger.error(f"Output for {key} is None.")
                raise ValueError(f"Output for {key} is None.")
            else:
                logger.debug(f"Output {key} shape: {value.shape}")
    
        return output
b. Updated YOLOHead Implementation


# yolo_head.py

import torch.nn as nn
import logging

logger = logging.getLogger(__name__)

class YOLOHead(nn.Module):
    def __init__(
        self, 
        num_classes: int, 
        anchors: List[Tuple[int, int]], 
        input_image_size=(512, 512), 
        stride: int = 32
    ):
        super(YOLOHead, self).__init__()
        self.num_classes = num_classes
        self.anchors = anchors
        self.num_anchors = len(anchors)
        self.stride = stride
        self.input_image_size = input_image_size
        
        # Grid dimensions
        self.grid_height = input_image_size[0] // stride
        self.grid_width = input_image_size[1] // stride
        
        # Ensure the calculated grid dimensions are valid
        if self.grid_height <= 0 or self.grid_width <= 0:
            raise ValueError(
                f"Invalid grid dimensions: grid_height={self.grid_height}, grid_width={self.grid_width}.\n"
                "Ensure stride is a positive integer and input_image_size is divisible by stride."
            )
        
        self.out_channels = self.num_anchors * (5 + self.num_classes)  # (4 bbox + 1 confidence + classes)

        # Define the convolutional layers
        self.reg_conv = nn.Conv2d(256, 4 * self.num_anchors, kernel_size=1)  # 6 anchors * 4 = 24
        self.obj_conv = nn.Conv2d(256, self.num_anchors, kernel_size=1)         # 6 anchors * 1 = 6
        self.cls_conv = nn.Conv2d(256, self.num_classes * self.num_anchors, kernel_size=1)  # 6 anchors * 2 = 12
        
        # Validate Conv2d Layers' Output Dimensions
        assert self.reg_conv.out_channels == 4 * self.num_anchors, "reg_conv out_channels mismatch."
        assert self.obj_conv.out_channels == self.num_anchors, "obj_conv out_channels mismatch."
        assert self.cls_conv.out_channels == self.num_classes * self.num_anchors, "cls_conv out_channels mismatch."
    
    def forward(self, x: nn.Tensor) -> nn.Tensor:
        """Forward pass for YOLOHead."""
        logger.debug(f"YOLOHead input shape: {x.shape}")
        cls_pred = self.cls_conv(x)  # Shape: [B, num_classes * num_anchors, H, W]
        reg_pred = self.reg_conv(x)  # Shape: [B, 4 * num_anchors, H, W]
        obj_pred = self.obj_conv(x)  # Shape: [B, num_anchors, H, W]
        yolo_output = torch.cat([reg_pred, obj_pred, cls_pred], dim=1)  # Shape: [B, 5 * num_anchors + num_classes * num_anchors, H, W]
        logger.debug(f"YOLOHead output shape: {yolo_output.shape}")
        return yolo_output
Key Corrections:
Aligned Output Channels with 6 Anchors:
reg_conv outputs 24 channels (6 * 4).
obj_conv outputs 6 channels (6 * 1).
cls_conv outputs 12 channels (6 * 2).
Total yolo_output channels: 24 + 6 + 12 = 42.
Added Assertions:
Ensures that convolutional layers' out_channels match the expected values based on the number of anchors and classes.
Logging:
Uses logger.debug instead of print for better control over logging verbosity.
7. Performing a Controlled Forward Pass
After making these corrections, perform a forward pass with dummy data to ensure that all components function correctly.

# test_model_integration.py

import torch
from omegaconf import OmegaConf
from centernet_unet import EnhancedCenterNetUNet

def test_entire_model_integration():
    # Load configuration
    config = OmegaConf.load("path_to_your_config.yaml")
    
    # Instantiate the model
    model = EnhancedCenterNetUNet.from_config(config)
    model.eval()
    
    # Create dummy input (batch_size=2, num_channels=2, height=512, width=512)
    dummy_input = torch.randn(2, config.dataset.num_channels, 512, 512)
    
    # Forward Pass
    with torch.no_grad():
        outputs = model(dummy_input)
    
    # Print output shapes
    for key, value in outputs.items():
        if value is not None:
            print(f"{key}: {value.shape}")
        else:
            print(f"{key}: None")

test_entire_model_integration()
Expected Output:

CENTERNET_OUTPUT_OBJECTNESS_MAP: torch.Size([2, 1, 16, 16])
CENTERNET_OUTPUT_VESSEL_MAP: torch.Size([2, 1, 16, 16])
CENTERNET_OUTPUT_FISHING_MAP: torch.Size([2, 1, 16, 16])
CENTERNET_OUTPUT_SIZE: torch.Size([2, 1, 16, 16])
CENTERNET_OUTPUT_YOLO: torch.Size([2, 42, 16, 16])
CENTERNET_OUTPUT_OFFSET: torch.Size([2, 2, 16, 16])
fused_output: torch.Size([2, <head_output_channels>, 16, 16])
Notes:
<head_output_channels>: Replace with the actual number of channels produced by your head modules. For instance, if you have two heads producing 42 and 6 channels respectively, <head_output_channels> would be 48.
Handling Remaining Errors

If you encounter further errors such as:

TypeError: forward() got an unexpected keyword argument 'inplace'
AttributeError: 'ReLU' object has no attribute 'shape'
Potential Causes and Solutions:
Activation Module Misapplication:
Cause: Activation functions are being assigned to tensors instead of being applied.
Solution: Ensure that activation functions are applied to tensors and not assigned to variables. For example:

# Correct
out = self.activation1(out)

# Incorrect
out = self.activation1  # This assigns the module instead of applying it
Verification in Forward Methods:
Action: Review all forward methods to ensure activations are applied correctly.
Example Correction:

def forward(self, x: nn.Tensor) -> nn.Tensor:
    out = self.conv1(x)
    out = self.bn1(out)
    out = self.activation1(out)  # Correctly applying activation
    
    out = self.conv2(out)
    out = self.bn2(out)
    out = self.activation2(out)  # Correctly applying activation
    
    # Remaining forward pass...
    
    return out
Removing Redundant Assignments:
Cause: Assigning activation modules directly to tensors.
Solution: Avoid any line that reassigns activation modules to tensors. Always use activations as functions.
Logging Instead of Printing:
Benefit: Offers better control and integration with logging frameworks.
Implementation: Replace print statements with logger.debug(f"...").
Example:

# Before
print(f"[DEBUG] After activation1: {out.shape}")

# After
logger.debug(f"After activation1: {out.shape}")
8. Final Recommendations
Ensure All Activation Functions Are Correctly Applied:
Activation functions should always be applied to tensors.
Avoid assigning activation modules to variables intended for tensors.
Maintain Unique and Descriptive Layer Naming:
Prevent layer overwriting in OrderedDict by ensuring all layer names are unique.
Comprehensive Logging:
Utilize Python's logging module for better debugging and log management.
Replace all print statements with appropriate logger.debug/info/error calls.
Validate Configuration Alignment:
Double-check that your configuration file correctly specifies channel dimensions, number of anchors, and activation functions aligning with your model's architecture.
Monitor and Optimize Memory Usage:
Given the earlier MPS backend out of memory error, consider strategies like reducing batch sizes, enabling AMP, or simplifying the model architecture.
Implement Unit Tests Regularly:
Validate individual components to ensure they function correctly before integrating them into the broader model.
Use Gradient Checkpointing (Advanced):
Trades compute for memory by recomputing activations during the backward pass.
Caution: Can complicate debugging and might not be compatible with all layers.
Iterative Testing and Refinement:
Make incremental changes and test thoroughly to ensure stability and correctness.
By meticulously implementing the above corrections and following best practices, you should resolve the channel mismatch and activation function misapplication issues. This will help ensure that your vessel detection and fishing activity classification system functions correctly without runtime errors.
If, after applying these corrections, you continue to experience challenges, please provide updated error logs or specific code snippets, and I'll assist you further in troubleshooting.

(sam3) samerkharboush@Mac SAR-MPS-OptunaNasFbnYoloTransformers % HYDRA_FULL_ERROR=1 python train_multilabel3.py
Traceback (most recent call last):
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/train_multilabel3.py", line 21, in
from xview3.nas import OptunaNAS
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/init.py", line 1, in
from .centernet import *
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/init.py", line 4, in
from .models import *
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/models/init.py", line 2, in
from .heads import *
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/models/heads/init.py", line 1, in
from .decoupled_head import *
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/models/heads/decoupled_head.py", line 11, in
from ..unet_blocks import SqueezeExcitation, get_activation_block
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/models/unet_blocks.py", line 173, in
class UnetBlock(nn.Module):
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/models/unet_blocks.py", line 216, in UnetBlock
def forward(self, x: nn.Tensor) -> nn.Tensor:
AttributeError: module 'torch.nn' has no attribute 'Tensor'
(sam3) samerkharboush@Mac SAR-MPS-OptunaNasFbnYoloTransformers %
###also please ensure integration with my config
##config
defaults:
self
dataset: full_4fold
augs: flips_light
sampler: balanced
normalization: default
loss: centernet/rfl_bce_mse
model: centernet/b5_unet_s2
optimizer: adamw
scheduler: cos
train: default
runner: default
ema: disabled
torch: default
slug: b5_unet_s2
activation: silu
num_channels: ${dataset.num_channels}
num_extra_blocks: 0
model:
target: xview3.centernet.models.centernet_unet.EnhancedCenterNetUNet
slug: b5_unet_s2
weights: IMAGENET1K_V1
pretrained: True
output_stride: 32
image_size: [1024, 1024]
window_size:
- 4
- 4
num_layers: [2, 5]
num_filters: [64, 128, 256]
kernel_size: [3, 5]
activation: [relu, leaky_relu, silu]
dropout_rate: [0.0, 0.5]
dropout_rate_max: 0.5
optimizer: adamw
learning_rate: 0.001
transformer:
enabled: true
embed_dim: 96
depths: [2, 2, 6, 2]
num_heads: [3, 6, 12, 24]
window_size:
- 4
- 4
mlp_ratio: 4.0
qkv_bias: true
qk_scale: null
drop_rate: 0.0
attn_drop_rate: 0.0
drop_path_rate: 0.1
norm_layer: torch.nn.LayerNorm
use_dropout: false
use_custom_pos_embed: false
decoder:
target: xview3.centernet.models.custom_encoder.DebugUnetDecoder
encoder_channels: [24, 48, 64, 160, 272] #[32, 24, 40, 112, 320]
decoder_channels: [512, 384, 256, 128, 64]
unet_block:
target: xview3.centernet.models.unet_blocks.UnetBlock
partial: true
upsample_block: xview3.centernet.models.custom_encoder.UpsampleBlock
#scale_factor: 2
#mode: bilinear
center: False # Changed from 'bilinear' to 'nearest'
activation: silu
encoder:
target: pytorch_toolbelt.modules.encoders.TimmEfficientNetV2
model_name: efficientnetv2_rw_s
pretrained: True #features_only: True
activation: silu
drop_rate: 0.1
drop_path_rate: 0.1
layers:
- 1 # 4
- 2 # 8
- 3 # 16
- 4 # 32
nas:
enabled: true
search_space:
num_layers: [2, 5]
num_filters: [64, 128, 256]
kernel_size: [3, 5]
activation: [relu, leaky_relu, silu]
dropout_rate: [0.0, 0.5]
dropout_rate_max: 0.5
num_trials: 50
metric: "val_acc"
direction: "maximize"
num_channels: ${dataset.num_channels}
num_classes: 2
anchors: [[10, 13], [16, 30], [33, 23]]
input_upsample_factor: 1
neck_out_channels: 256
box_coder:
target: xview3.centernet.bboxer.multilabel_circle_coder.MultilabelCircleNetCoder
image_size:
- 1024
- 1024
max_objects: 512
heatmap_encoding: umich
labels_encoding: circle
ignore_value:
target: xview3.ignore_value
fixed_radius: 3
labels_radius: 1
ignore_low_confidence_detections: ${dataset.ignore_low_confidence_detections}
head:
decoupled_head:
target: xview3.centernet.models.heads.DecoupledHeadGroupNormLateShuffle
channels: 256
classifier_dim: 128
objectness_dim: 128
size_dim: 128
offset_dim: 16
dropout_rate: 0.1
activation: silu
num_blocks: 3
yolo_head:
target: xview3.centernet.models.heads.yolo_head.YOLOHead
num_classes: 2
anchors: [[10, 13], [16, 30], [33, 23], [30, 61], [62, 45], [59, 119]]
input_image_size: [1024, 1024]
stride: 32
num_anchors_per_scale: 3
fan:
target: xview3.centernet.models.necks.FeatureAggregationNetwork
input_channels: [272] #[32, 48, 64, 128] #in_channels: 272 #out_channels: 256 #num_layers: 2
out_channels: 256
num_layers: 2
upsample_mode: bilinear
use_attention: True
fusion_method: weighted_sum
expected_fan_input_channels: 256
Per-Channel Normalization
normalization:
channels:
vv:
target: xview3.SigmoidNormalization
midpoint: -20.0
temperature: 0.18
vh:
target: xview3.SigmoidNormalization
midpoint: -20.0
temperature: 0.18
bathymetry:
target: xview3.CubicRootNormalization
wind_direction:
target: albumentations.NoOp
wind_speed:
target: albumentations.NoOp
mask:
target: albumentations.NoOp
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]
Dataset Configuration
dataset:
normalization:
channels:
vv:
target: xview3.SigmoidNormalization
midpoint: -20.0
temperature: 0.18
vh:
target: xview3.SigmoidNormalization
midpoint: -20.0
temperature: 0.18
bathymetry:
target: xview3.CubicRootNormalization
wind_direction:
target: albumentations.NoOp
wind_speed:
target: albumentations.NoOp
mask:
target: albumentations.NoOp
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]
slug: full_4fold
splitter: full
fold: 0
num_folds: 4
Added the following parameters to address references in box_coder
ignore_low_confidence_detections: true
ignore_low_confidence_labels: false
filter_low_confidence_objects: False
num_channels: 2
input_channels:
- vh
- vv
DDP Stuff
local_rank: ${oc.env:LOCAL_RANK, 0}
world_size: ${oc.env:WORLD_SIZE, 0}
experiment: null
seed: 42
deterministic: False
detect_anomaly: False
find_unused: False
transfer: null
checkpoint: null
inference:
channels_last: False
use_traced_model: True
tile_size: 2048
tile_step: 1536
accumulate_on_gpu: False
batch_size: 1
fp16: True
Torch Configuration
torch:
amp:
enabled: false
cudnn:
allow_tf32: True
cuda_allow_tf32: True
detect_anomaly: False
deterministic: False
benchmark: True
Inference and Training Enhancements
activation_after: "after_model"
tta: "d4"
with_offset: True
yolo_decoders: []
Validation Configuration
valid:
image_size: [2048, 2048]
batch_size: 2
num_workers: 2
Training Configuration
train:
slug: default_training
epochs: 4
output_stride: 32
early_stopping:
enabled: true
patience: 2
mode: "max" # Options: 'min' for loss-based, 'max' for accuracy-based
metric: "loss" # The metric to monitor for early stopping
image_size: [512, 512]
batch_size: 2
num_workers: 2
num_samples: 8192
persistent_workers: False
model2:
target: xview3.centernet.models.centernet_unet.EnhancedCenterNetUNet
slug: b5_unet_s2
weights: IMAGENET1K_V1
pretrained: True
output_stride: 32
image_size: [1024, 1024]
window_size:
- 4
- 4
num_layers: ${model2.num_layers} # Ensure these are set up for overrides
num_filters: ${model2.num_filters}
kernel_size: ${model2.kernel_size}
activation: ${model2.activation}
dropout_rate: ${model2.dropout_rate}
dropout_rate_max: 0.5
optimizer: ${model2.optimizer}
learning_rate: ${model2.learning_rate}
transformer:
enabled: true
embed_dim: 96
depths: [2, 2, 6, 2]
num_heads: [3, 6, 12, 24]
window_size:
- 4
- 4
mlp_ratio: 4.0
qkv_bias: true
qk_scale: null
drop_rate: 0.0
attn_drop_rate: 0.0
drop_path_rate: 0.1
norm_layer: torch.nn.LayerNorm
use_dropout: false
use_custom_pos_embed: false
decoder:
target: xview3.centernet.models.custom_encoder.DebugUNetDecoder
encoder_channels: [24, 48, 64, 160, 272]
channels: [512, 384, 256, 128, 64]
unet_block:
target: xview3.centernet.models.unet_blocks.UnetBlock
partial: true
upsample_block:
target: torch.nn.Upsample
scale_factor: 2
mode: bilinear
encoder:
target: pytorch_toolbelt.modules.encoders.TimmEfficientNetV2
model_name: efficientnetv2_rw_s
pretrained: True
activation: silu
drop_rate: 0.1
drop_path_rate: 0.1
layers:
- 1 # 4
- 2 # 8
- 3 # 16
- 4 # 32
nas:
enabled: true
search_space:
num_layers: [2, 5]
num_filters: [64, 128, 256]
kernel_size: [3, 5]
activation: [relu, leaky_relu, silu]
dropout_rate: [0.0, 0.5]
dropout_rate_max: 0.5
num_trials: 50
metric: "val_loss"
direction: "minimize"
num_channels: ${dataset.num_channels}
num_classes: 2
anchors: [[10, 13], [16, 30], [33, 23]]
input_upsample_factor: 1
neck_out_channels: 256
box_coder:
target: xview3.centernet.bboxer.multilabel_circle_coder.MultilabelCircleNetCoder
image_size:
- 1024
- 1024
max_objects: 512
heatmap_encoding: umich
labels_encoding: circle
ignore_value:
target: xview3.ignore_value
fixed_radius: 3
labels_radius: 1
ignore_low_confidence_detections: ${dataset.ignore_low_confidence_detections}
head:
decoupled_head:
target: xview3.centernet.models.heads.DecoupledHeadGroupNormLateShuffle
channels: 256
classifier_dim: 128
objectness_dim: 128
size_dim: 128
offset_dim: 16
dropout_rate: 0.1
activation: silu
num_blocks: 3
yolo_head:
target: xview3.centernet.models.heads.yolo_head.YOLOHead
num_classes: 2
anchors: [[10, 13], [16, 30], [33, 23], [30, 61], [62, 45], [59, 119]]
input_image_size: [1024, 1024]
stride: 32
num_anchors_per_scale: 3
fan:
target: xview3.centernet.models.necks.FeatureAggregationNetwork
input_channels: [272] #[32, 48, 64, 128] #in_channels: 272 #out_channels: 256 #num_layers: 2
out_channels: 256
num_layers: 2
upsample_mode: bilinear
use_attention: True
fusion_method: weighted_sum

maybe better to see my config also before reading the error to understand how to correctly and deeply solve it
##config
defaults:
self
dataset: full_4fold
augs: flips_light
sampler: balanced
normalization: default
loss: centernet/rfl_bce_mse
model: centernet/b5_unet_s2
optimizer: adamw
scheduler: cos
train: default
runner: default
ema: disabled
torch: default
slug: b5_unet_s2
activation: silu
num_channels: ${dataset.num_channels}
num_extra_blocks: 0
model:
target: xview3.centernet.models.centernet_unet.EnhancedCenterNetUNet
slug: b5_unet_s2
weights: IMAGENET1K_V1
pretrained: True
output_stride: 32
image_size: [1024, 1024]
window_size:
- 4
- 4
num_layers: [2, 5]
num_filters: [64, 128, 256]
kernel_size: [3, 5]
activation: [relu, leaky_relu, silu]
dropout_rate: [0.0, 0.5]
dropout_rate_max: 0.5
optimizer: adamw
learning_rate: 0.001
transformer:
enabled: true
embed_dim: 96
depths: [2, 2, 6, 2]
num_heads: [3, 6, 12, 24]
window_size:
- 4
- 4
mlp_ratio: 4.0
qkv_bias: true
qk_scale: null
drop_rate: 0.0
attn_drop_rate: 0.0
drop_path_rate: 0.1
norm_layer: torch.nn.LayerNorm
use_dropout: false
use_custom_pos_embed: false
decoder:
target: xview3.centernet.models.custom_encoder.DebugUnetDecoder
encoder_channels: [24, 48, 64, 160, 272] #[32, 24, 40, 112, 320]
decoder_channels: [512, 384, 256, 128, 64]
unet_block:
target: xview3.centernet.models.unet_blocks.UnetBlock
partial: true
upsample_block: xview3.centernet.models.custom_encoder.UpsampleBlock
#scale_factor: 2
#mode: bilinear
center: False # Changed from 'bilinear' to 'nearest'
activation: silu
encoder:
target: pytorch_toolbelt.modules.encoders.TimmEfficientNetV2
model_name: efficientnetv2_rw_s
pretrained: True #features_only: True
activation: silu
drop_rate: 0.1
drop_path_rate: 0.1
layers:
- 1 # 4
- 2 # 8
- 3 # 16
- 4 # 32
nas:
enabled: true
search_space:
num_layers: [2, 5]
num_filters: [64, 128, 256]
kernel_size: [3, 5]
activation: [relu, leaky_relu, silu]
dropout_rate: [0.0, 0.5]
dropout_rate_max: 0.5
num_trials: 50
metric: "val_acc"
direction: "maximize"
num_channels: ${dataset.num_channels}
num_classes: 2
anchors: [[10, 13], [16, 30], [33, 23]]
input_upsample_factor: 1
neck_out_channels: 256
box_coder:
target: xview3.centernet.bboxer.multilabel_circle_coder.MultilabelCircleNetCoder
image_size:
- 1024
- 1024
max_objects: 512
heatmap_encoding: umich
labels_encoding: circle
ignore_value:
target: xview3.ignore_value
fixed_radius: 3
labels_radius: 1
ignore_low_confidence_detections: ${dataset.ignore_low_confidence_detections}
head:
decoupled_head:
target: xview3.centernet.models.heads.DecoupledHeadGroupNormLateShuffle
channels: 256
classifier_dim: 128
objectness_dim: 128
size_dim: 128
offset_dim: 16
dropout_rate: 0.1
activation: silu
num_blocks: 3
yolo_head:
target: xview3.centernet.models.heads.yolo_head.YOLOHead
num_classes: 2
anchors: [[10, 13], [16, 30], [33, 23], [30, 61], [62, 45], [59, 119]]
input_image_size: [1024, 1024]
stride: 32
num_anchors_per_scale: 3
fan:
target: xview3.centernet.models.necks.FeatureAggregationNetwork
input_channels: [272] #[32, 48, 64, 128] #in_channels: 272 #out_channels: 256 #num_layers: 2
out_channels: 256
num_layers: 2
upsample_mode: bilinear
use_attention: True
fusion_method: weighted_sum
expected_fan_input_channels: 256
Per-Channel Normalization
normalization:
channels:
vv:
target: xview3.SigmoidNormalization
midpoint: -20.0
temperature: 0.18
vh:
target: xview3.SigmoidNormalization
midpoint: -20.0
temperature: 0.18
bathymetry:
target: xview3.CubicRootNormalization
wind_direction:
target: albumentations.NoOp
wind_speed:
target: albumentations.NoOp
mask:
target: albumentations.NoOp
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]
Dataset Configuration
dataset:
normalization:
channels:
vv:
target: xview3.SigmoidNormalization
midpoint: -20.0
temperature: 0.18
vh:
target: xview3.SigmoidNormalization
midpoint: -20.0
temperature: 0.18
bathymetry:
target: xview3.CubicRootNormalization
wind_direction:
target: albumentations.NoOp
wind_speed:
target: albumentations.NoOp
mask:
target: albumentations.NoOp
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]
slug: full_4fold
splitter: full
fold: 0
num_folds: 4
Added the following parameters to address references in box_coder
ignore_low_confidence_detections: true
ignore_low_confidence_labels: false
filter_low_confidence_objects: False
num_channels: 2
input_channels:
- vh
- vv
DDP Stuff
local_rank: ${oc.env:LOCAL_RANK, 0}
world_size: ${oc.env:WORLD_SIZE, 0}
experiment: null
seed: 42
deterministic: False
detect_anomaly: False
find_unused: False
transfer: null
checkpoint: null
inference:
channels_last: False
use_traced_model: True
tile_size: 2048
tile_step: 1536
accumulate_on_gpu: False
batch_size: 1
fp16: True
Torch Configuration
torch:
amp:
enabled: false
cudnn:
allow_tf32: True
cuda_allow_tf32: True
detect_anomaly: False
deterministic: False
benchmark: True
Inference and Training Enhancements
activation_after: "after_model"
tta: "d4"
with_offset: True
yolo_decoders: []
Validation Configuration
valid:
image_size: [2048, 2048]
batch_size: 2
num_workers: 2
Training Configuration
train:
slug: default_training
epochs: 4
output_stride: 32
early_stopping:
enabled: true
patience: 2
mode: "max" # Options: 'min' for loss-based, 'max' for accuracy-based
metric: "loss" # The metric to monitor for early stopping
image_size: [512, 512]
batch_size: 2
num_workers: 2
num_samples: 8192
persistent_workers: False
model2:
target: xview3.centernet.models.centernet_unet.EnhancedCenterNetUNet
slug: b5_unet_s2
weights: IMAGENET1K_V1
pretrained: True
output_stride: 32
image_size: [1024, 1024]
window_size:
- 4
- 4
num_layers: ${model2.num_layers} # Ensure these are set up for overrides
num_filters: ${model2.num_filters}
kernel_size: ${model2.kernel_size}
activation: ${model2.activation}
dropout_rate: ${model2.dropout_rate}
dropout_rate_max: 0.5
optimizer: ${model2.optimizer}
learning_rate: ${model2.learning_rate}
transformer:
enabled: true
embed_dim: 96
depths: [2, 2, 6, 2]
num_heads: [3, 6, 12, 24]
window_size:
- 4
- 4
mlp_ratio: 4.0
qkv_bias: true
qk_scale: null
drop_rate: 0.0
attn_drop_rate: 0.0
drop_path_rate: 0.1
norm_layer: torch.nn.LayerNorm
use_dropout: false
use_custom_pos_embed: false
decoder:
target: xview3.centernet.models.custom_encoder.DebugUNetDecoder
encoder_channels: [24, 48, 64, 160, 272]
channels: [512, 384, 256, 128, 64]
unet_block:
target: xview3.centernet.models.unet_blocks.UnetBlock
partial: true
upsample_block:
target: torch.nn.Upsample
scale_factor: 2
mode: bilinear
encoder:
target: pytorch_toolbelt.modules.encoders.TimmEfficientNetV2
model_name: efficientnetv2_rw_s
pretrained: True
activation: silu
drop_rate: 0.1
drop_path_rate: 0.1
layers:
- 1 # 4
- 2 # 8
- 3 # 16
- 4 # 32
nas:
enabled: true
search_space:
num_layers: [2, 5]
num_filters: [64, 128, 256]
kernel_size: [3, 5]
activation: [relu, leaky_relu, silu]
dropout_rate: [0.0, 0.5]
dropout_rate_max: 0.5
num_trials: 50
metric: "val_loss"
direction: "minimize"
num_channels: ${dataset.num_channels}
num_classes: 2
anchors: [[10, 13], [16, 30], [33, 23]]
input_upsample_factor: 1
neck_out_channels: 256
box_coder:
target: xview3.centernet.bboxer.multilabel_circle_coder.MultilabelCircleNetCoder
image_size:
- 1024
- 1024
max_objects: 512
heatmap_encoding: umich
labels_encoding: circle
ignore_value:
target: xview3.ignore_value
fixed_radius: 3
labels_radius: 1
ignore_low_confidence_detections: ${dataset.ignore_low_confidence_detections}
head:
decoupled_head:
target: xview3.centernet.models.heads.DecoupledHeadGroupNormLateShuffle
channels: 256
classifier_dim: 128
objectness_dim: 128
size_dim: 128
offset_dim: 16
dropout_rate: 0.1
activation: silu
num_blocks: 3
yolo_head:
target: xview3.centernet.models.heads.yolo_head.YOLOHead
num_classes: 2
anchors: [[10, 13], [16, 30], [33, 23], [30, 61], [62, 45], [59, 119]]
input_image_size: [1024, 1024]
stride: 32
num_anchors_per_scale: 3
fan:
target: xview3.centernet.models.necks.FeatureAggregationNetwork
input_channels: [272] #[32, 48, 64, 128] #in_channels: 272 #out_channels: 256 #num_layers: 2
out_channels: 256
num_layers: 2
upsample_mode: bilinear
use_attention: True
fusion_method: weighted_sum
##error
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/train_multilabel3.py", line 21, in
from xview3.nas import OptunaNAS
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/init.py", line 1, in
from .centernet import *
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/init.py", line 4, in
from .models import *
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/models/init.py", line 2, in
from .heads import *
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/models/heads/init.py", line 1, in
from .decoupled_head import *
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/models/heads/decoupled_head.py", line 11, in
from ..unet_blocks import SqueezeExcitation, get_activation_block
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/models/unet_blocks.py", line 173, in
class UnetBlock(nn.Module):
File "/Volumes/Crucial/SAR-MPS-OptunaNasFbnYoloTransformers/xview3/centernet/models/unet_blocks.py", line 216, in UnetBlock
def forward(self, x: nn.Tensor) -> nn.Tensor:
AttributeError: module 'torch.nn' has no attribute 'Tensor'
(sam3) samerkharboush@Mac SAR-MPS-OptunaNasFbnYoloTransformers %