Detection incredibly slow

egbertn: <p>Hi All,</p>
<p>I created a script which should do a simple task. Open a film fragment 1280wx720h and perform detection on it for each frame.<br>
On a jetson orin (6GB memory), with Pytorch 2.1 + 0.16 vision which has all optimizations ok, it performs very slow though.<br>
Any hints??</p>
<p>import hashlib<br>
from pathlib import Path<br>
import numpy as np<br>
import torch<br>
import cv2<br>
from torchvision.models import resnet50</p>
<p>from torchvision.models.detection import fasterrcnn_resnet50_fpn_v2<br>
from torchvision.models.detection.faster_rcnn import FasterRCNN_ResNet50_FPN_V2_Weights</p>
<p>import cv2<br>
import numpy as np</p>
<h1><a name="load-resnet50-model-ideally-for-extracting-features-1" class="anchor" href="#load-resnet50-model-ideally-for-extracting-features-1"></a>Load ResNet50 model ideally for extracting features</h1>
<p>class_labels = {<br>
1: ‘person’,<br>
2: ‘car’,<br>
3: ‘dog’,<br>
# Add more class labels as needed<br>
}</p>
<h1><a name="specify-the-nodes-to-extract-features-from-2" class="anchor" href="#specify-the-nodes-to-extract-features-from-2"></a>Specify the nodes to extract features from</h1>
<p>return_nodes = {<br>
‘layer1’: ‘layer1’,<br>
‘layer2’: ‘layer2’,<br>
‘layer3’: ‘layer3’,<br>
‘layer4’: ‘layer4’,<br>
}</p>
<h1><a name="create-the-feature-extractor-3" class="anchor" href="#create-the-feature-extractor-3"></a>Create the feature extractor</h1>
<p>resnetModel = resnet50()<br>
resnetModel.eval()</p>
<h1><a name="function-to-draw-bounding-box-on-image-4" class="anchor" href="#function-to-draw-bounding-box-on-image-4"></a>Function to draw bounding box on image</h1>
<p>def draw_box(image, box, label):<br>
x1, y1, x2, y2 = box<br>
image_with_box = cv2.rectangle(image.copy(), (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 0), 2)<br>
cv2.putText(image_with_box, label, (int(x1), int(y1) - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2)<br>
return image_with_box</p>
<h1><a name="load-a-pre-trained-faster-r-cnn-model-5" class="anchor" href="#load-a-pre-trained-faster-r-cnn-model-5"></a>Load a pre-trained Faster R-CNN model</h1>
<p>def load_model():<br>
model = fasterrcnn_resnet50_fpn_v2()<br>
state_dict = torch.hub.load_state_dict_from_url(FasterRCNN_ResNet50_FPN_V2_Weights.COCO_V1.url)<br>
model.load_state_dict(state_dict)<br>
model.eval()<br>
return model</p>
<h1><a name="main-function-to-perform-object-detection-and-tracking-6" class="anchor" href="#main-function-to-perform-object-detection-and-tracking-6"></a>Main function to perform object detection and tracking</h1>
<p>def detect_and_track_objects(video_path, use_visdom=False):</p>
<pre><code># Load the video file using OpenCV
cap = cv2.VideoCapture(video_path)

# Get width and height of the video frames
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

# Create VideoWriter for output video
out = cv2.VideoWriter('output.mp4', cv2.VideoWriter_fourcc(*'mp4v'), 25, (width, height))

# Load the model
model = load_model()

frame_count = 0
while True:
    ret, frame = cap.read()
    if not ret:
        break

    frame_count += 1

    # Perform object detection
    # Convert the frame to tensor
    frame_tensor = torch.tensor(frame, dtype=torch.float32) / 255.0
    frame_tensor = frame_tensor.permute(2, 0, 1).unsqueeze(0)

    # Forward pass through the model
    with torch.no_grad():
        outputs = model(frame_tensor)
    frame_with_bbox = frame
    # Get the predicted bounding box from the output
    tracker_id = 0

    for bbox, score, label in zip(outputs[0]['boxes'], outputs[0]['scores'], outputs[0]['labels']):
        bbox = bbox.cpu().numpy()
        score = score.cpu().numpy()

        # Get human-readable label
        label_name = class_labels.get(label.item(), f'Unknown {label.item()}')
        #hash_code = generate_hash(label_name, bbox, feature)
        if score &gt; 0.5:
            tracker_id +=1
            frame_with_bbox = draw_box(frame_with_bbox, bbox, label_name)

    # Write frame to output video
    out.write(frame_with_bbox)

# Release video capture and close output video
cap.release()
out.release()
</code></pre>
<h1><a name="main-function-call-7" class="anchor" href="#main-function-call-7"></a>Main function call</h1>
<p>if <strong>name</strong> == “<strong>main</strong>”:<br>
# Video file path<br>
video_path = str(Path(“~/001.avi”).expanduser())</p>
<pre><code># Detect and track objects
detect_and_track_objects(video_path, use_visdom=True)
</code></pre>

egbertn: <p>the load_model did not have a .to(‘cuda’/‘cpu’)<br>
Still, inference is very slow. Talking about 1,5 frames per second. Any tips welcome.</p>
<pre><code class="lang-auto">def load_model():
    model = fasterrcnn_resnet50_fpn_v2()
    state_dict = torch.hub.load_state_dict_from_url(FasterRCNN_ResNet50_FPN_V2_Weights.COCO_V1.url)
    model.load_state_dict(state_dict)
    model = model.to(device)
    model.eval()
    return model
</code></pre>

JamesTrick: <p>I don’t believe you move your tensors to the GPU either. <code>frame_tensor = torch.tensor(frame, dtype=torch.float32) / 255.0</code></p>