Mask r-cnn training runs infinitely without output or error

Gladius_monta: <p>Here’s a brief overview of my process:</p>
<ol>
<li>I generated a dataset using PyTorch by applying the SAM mask from bounding boxes to my images.</li>
<li>After creating the dataset, I split it into training and testing sets.</li>
<li>I loaded both sets using torch.utils.data.DataLoader.</li>
<li>I’m using a pre-trained model with 11 classes.</li>
</ol>
<p>However, I’m encountering an issue during training. The process seems to take an unusually long time, and I’m not seeing any progress or error messages to troubleshoot from.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/b/eb7dfdbbd95afa602889fe174241e0c88e998e73.png" data-download-href="https://discuss.pytorch.org/uploads/default/eb7dfdbbd95afa602889fe174241e0c88e998e73" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/b/eb7dfdbbd95afa602889fe174241e0c88e998e73.png" alt="image" data-base62-sha1="xBg8HtZ0Aaw3J6RhRG8GE9VzSy7" width="690" height="455" data-dominant-color="212121"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1250×825 32.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>What might be going wrong or how to improve my training process?</p>

ptrblck: <p>At the bottom of the notebook it states: “Output is truncated. View as…”. Are you sure the code is stuck or is your notebook not showing the outputs according to this message?</p>

Gladius_monta: <p>I modified the code and I tried on google colab<br>
in this cell I got different error</p>
<ul>
<li>First error</li>
</ul>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/6/66bc58d9ef1967f6771f3b2ff11f20a707a07f30.png" data-download-href="https://discuss.pytorch.org/uploads/default/66bc58d9ef1967f6771f3b2ff11f20a707a07f30" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/6/66bc58d9ef1967f6771f3b2ff11f20a707a07f30.png" alt="image" data-base62-sha1="eEQeutj8haMWC4OFfzqCpPIfLSU" width="690" height="443" data-dominant-color="373738"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">926×595 32.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<ul>
<li>Second error</li>
</ul>
<p>SystemExit                                Traceback (most recent call last)<br>
 in &lt;cell line: 3&gt;()<br>
6     num_epochs = 10<br>
7     for epoch in range(num_epochs):<br>
----&gt; 8         train_one_epoch(model_ft, optimizer, data_loader_train, device, epoch, print_freq=100)<br>
9         lr_scheduler.step()<br>
10         evaluate(model, data_loader_test, device=device)</p>
<p>/content/engine.py in train_one_epoch(model, optimizer, data_loader, device, epoch, print_freq, scaler)<br>
41             print(f"Loss is {loss_value}, stopping training")<br>
42             print(loss_dict_reduced)<br>
—&gt; 43             sys.exit(1)<br>
44<br>
45         optimizer.zero_grad()</p>
<p>SystemExit: 1</p>

ptrblck: <p>If you are using an older PyTorch release, could you update to the latest stable release and check if you are still seeing the error?<br>
If not, could you post a minimal and executable code snippet reproducing the issue?</p>

Gladius_monta: <p>I used PyTorch version 2.3.0, other files like engine.py utils.py… get from github<br>
I don’t know if the problem in the form of the dataset that train MaskRCNN or what?<br>
but I think that the structure of dataset is good</p>
<p>this is my dataset.py thar generate the data</p>
<pre><code class="lang-auto"># dataset.py

import torch
import torch.utils.data
import torchvision.transforms as transforms
from PIL import Image
import os
import numpy as np

class Dataset(torch.utils.data.Dataset):
    
    def __init__(self, grouped_df, df, image_folder, processor, model, device): 
        self.df = df
        self.image_folder = image_folder
        self.grouped_df = grouped_df
        self.processor = processor
        self.model = model
        self.device = device
        
        formatted_array = []
        for num in df["imageid"].unique():
            num_str = str(num)
            if len(num_str) == 1:
                formatted_array.append("00" + num_str)
            elif len(num_str) == 2:
                formatted_array.append("0" + num_str)
            else:
                formatted_array.append(num_str)
        self.image_ids = formatted_array
        self.classes = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
        
        self.masks = []
        for idx in range(len(self.image_ids)):
            image_name = self.image_ids[idx]
            image_path = os.path.join(self.image_folder, image_name + '.jpg')
            image = Image.open(image_path)
            resized_image = image.resize((350, 350))
            resized_array = np.array(resized_image).astype(np.uint8)
            
            # SAM to find mask
            inputs = self.processor(resized_array, return_tensors="pt").to(self.device)
            image_embeddings = self.model.get_image_embeddings(inputs["pixel_values"])
            inputs.pop("pixel_values", None)
            inputs.update({"image_embeddings": image_embeddings})
            with torch.no_grad():
                outputs = self.model(**inputs, multimask_output=False)
            masks = self.processor.image_processor.post_process_masks(outputs.pred_masks.cpu(), inputs["original_sizes"].cpu(), inputs["reshaped_input_sizes"].cpu())
            masks = torch.as_tensor(np.array(list(map(np.array, masks)), dtype=np.uint8))
            masks = masks.squeeze(dim=0).squeeze(dim=1)
            self.masks.append(masks)
        
    def __getitem__(self, idx):
        image_name = self.image_ids[idx]
        image_path = os.path.join(self.image_folder, image_name + '.jpg')
        image = Image.open(image_path)
        image_id = torch.tensor([idx])
        # Resize image
        resized_image = image.resize((350, 350))
        resized_array = np.array(resized_image).astype(np.uint8)
        
        # Get classes
        labels = torch.as_tensor(self.grouped_df["classid"][idx], dtype=torch.int64)
        
        # Get boxes
        boxes = torch.as_tensor(self.grouped_df["resized_bbox"][idx], dtype=torch.float32)
        
        # Calculate the area
        area = (boxes[:, 3] - boxes[:, 1]) * (boxes[:, 2] - boxes[:, 0])
        
        # Get mask
        masks = self.masks[idx]
        
        # Suppose all instances are not crowd
        iscrowd = torch.zeros((boxes.shape[0],), dtype=torch.int64)
        
        # Create target
        target = {}
        target["iscrowd"] = iscrowd
        target['boxes'] = boxes
        target["labels"] = labels
        target["masks"] = masks
        target["area"] = area
        target["image_id"] = image_id
        return transforms.ToTensor()(resized_array), target
    
    def __len__(self):
        return len(self.image_ids)
</code></pre>
<p>I save it as .pkl then loaded to split the data as you can see here</p>
<pre><code class="lang-auto">from torch.utils.data import Subset, DataLoader

torch.manual_seed(1)
indices = torch.randperm(len(loaded_dataset)).tolist()

test_split = 0.2
size = int(len(loaded_dataset) * test_split)
dataset_train = Subset(loaded_dataset, indices[:-size])
dataset_test = Subset(loaded_dataset, indices[-size:])

data_loader_train = DataLoader(
    dataset_train, batch_size=4, shuffle=True, num_workers=4,
    collate_fn=utils.collate_fn)

data_loader_test = DataLoader(
    dataset_test, batch_size=4, shuffle=False, num_workers=4,
    collate_fn=utils.collate_fn)
</code></pre>
<p>load MaskRCNN model and train it</p>
<pre><code class="lang-auto">
device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')

num_classes = 11

model_ft = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True)

in_features = model_ft.roi_heads.box_predictor.cls_score.in_features

model_ft.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)

in_features_mask = model_ft.roi_heads.mask_predictor.conv5_mask.in_channels

hidden_layer = 256

model_ft.roi_heads.mask_predictor = MaskRCNNPredictor(in_features_mask, hidden_layer, num_classes)

model_ft.to(device)
params = [p for p in model_ft.parameters() if p.requires_grad]
optimizer = torch.optim.SGD(params, lr=0.0005, momentum=0.9, weight_decay=0.0005)
lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer,
                                               step_size=5,
                                               gamma=0.1)
</code></pre>
<pre><code class="lang-auto">import torch.multiprocessing as mp
if __name__ == "__main__":
    
    mp.set_start_method('spawn')
    num_epochs = 10
    for epoch in range(num_epochs):
        train_one_epoch(model_ft, optimizer, data_loader_train, device, epoch, print_freq=100)
        lr_scheduler.step()
        evaluate(model, data_loader_test, device=device)
</code></pre>