A question about model output

zelvet-velvet: <p>Hi.<br>
Recently, I want to show tello stream with image detection. My question is how to convert the output to data type that cv2.imshow() can work. Is this possible?  My first thought is to save model’s output to my local with save() method,  and show it by cv2.imshow() method. It works but the stream with objects detected will have a delay about 4~5 second.<br>
Does anyone have ideas to convert model’ output?<br>
Appreciate.</p>

ptrblck: <p><code>cv2.imshow</code> expects numpy arrays and you can transform PyTorch tensors to an <code>np.array</code> via <code>tensor.numpy()</code>. Depending on your use case you might need to push the tensor to the CPU first and maybe detach it additionally:</p>
<pre><code class="lang-python">arr = tensor.detach().cpu().numpy()
</code></pre>
<p>Once this is done, make sure the expected <code>dtype</code> and shape is passed to <code>imshow</code>, so you might need to <code>permute</code> the tensor to the channels-last memory layout and also swap the color channel order from RGB to BGR if needed.</p>

zelvet-velvet: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>,<br>
Thanks for replay!.</p>
<p>I have edit my code like this:</p>
<pre><code class="lang-auto">from threading import Thread
from djitellopy import Tello
import cv2, math, time
import torch
import os
import numpy as np
import asyncio
import imutils
from PIL import Image


path = r'C:\yolov5-master'
model = torch.hub.load(path, 'yolov5s',source='local', pretrained=True)
tello = Tello()
tello.connect()
tello.streamon()

frame_read = tello.get_frame_read()
    
class VideoStreamWidget(object):
    def __init__(self, src=0):
        # Start the thread to read frames from the video stream
        self.thread = Thread(target=self.update, args=())
        self.thread.daemon = True
        self.thread.start()

    def update(self):
        # Read the next frame from the stream
        global frame
        while True:
            self.frame = cv2.cvtColor(frame_read.frame,cv2.COLOR_RGB2BGR)
            time.sleep(.01)

    def show_frame(self): 
        # Display frames in main program
        wee = model(self.frame)
        arr = wee.datah().cpu().numpy()
        img = Image.fromarray.fromarray(arr, 'RGB')
        result = cv2.cvtColor(img,cv2.COLOR_RGB2BGR)
        cv2.imshow('frame', result)

        key = cv2.waitKey(1)

if __name__ == '__main__':
    video_stream_widget = VideoStreamWidget()
    while True:
        try:
            video_stream_widget.show_frame()
        except AttributeError:
            pass
</code></pre>
<p>I noticed after I put self.frame into model( ) method in line 35. It will jump out show_frame(self).</p>
<p>I’m wondering what’s going on here. I’m also wondering what type is the output of model( ). Since the issue above, I can’t check if my convert progress before imshow is correct or not ( I think it is not.).</p>

ptrblck: <aside class="quote no-group" data-username="zelvet-velvet" data-post="3" data-topic="152272">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zelvet-velvet/48/49837_2.png" class="avatar"> zelvet-velvet:</div>
<blockquote>
<p>I noticed after I put self.frame into model( ) method in line 35. It will jump out show_frame(self).</p>
</blockquote>
</aside>
<p>Python shouldn’t “jump out” of functions without a return statement so is a runtime error or so raised?</p>
<aside class="quote no-group" data-username="zelvet-velvet" data-post="3" data-topic="152272">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zelvet-velvet/48/49837_2.png" class="avatar"> zelvet-velvet:</div>
<blockquote>
<p>I’m also wondering what type is the output of model( ).</p>
</blockquote>
</aside>
<p>You can check your model definition and what type is returned in the <code>forward</code> method.<br>
I would guess it’s a tensor.</p>