Torch.from_numpy(im.copy()).to(device) takes time

swharaday: <p>Hi,</p>
<p>I’ve been using YOLOv5s model for several years.<br>
I converted the model to TensorRT (engine) to increase the detection speed.<br>
It works.<br>
Although the inference time on GPU was halved,<br>
it took 5-6 milliseconds to convert an image array im(640×640×3) for GPU with the script;<br>
im = torch.from_numpy(im.copy()).to(device)<br>
before feeding input to model.</p>
<p>Is this normal for a PC with the following specs?</p>
<p>CPU:11th Gen Intel(R) Core™ i7-1185G7 <span class="mention">@3.00GHz</span> 3.00 GHz<br>
RAM：16.0 GB<br>
Win10+Anaconda3<br>
Python 3.9.18<br>
pytorch 1.11.0<br>
cudatoolkit 11.3.1<br>
eGPU:RTX3060Ti in RAZER CORE X</p>
<p>Is there a way to shorten this pre-processing time?</p>
<p>Thanks in advance.</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/swharaday">@swharaday</a>,</p>
<aside class="quote no-group" data-username="swharaday" data-post="1" data-topic="202250">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/swharaday/48/64640_2.png" class="avatar"> swharaday:</div>
<blockquote>
<p>im = torch.from_numpy(im.copy()).to(device)</p>
</blockquote>
</aside>
<p>Another way to do this would be to initialize the tensor on the GPU directly (rather than mounting on the CPU and moving it to the GPU). You could try something like,</p>
<pre><code class="lang-auto">im = torch.as_tensor(im, device=device)
</code></pre>
<p>Is there a reason why you copy the <code>im</code> tensor before moving to the GPU?</p>

swharaday: <p>Hi AlphaBetaGamma96,</p>
<p>Thank you for your reply.<br>
I’ll try what you suggest and let you know the results.</p>
<p>There is no reason to copy and move. Just following the author’s description.</p>
<p>Best Regards, Yuji</p>

swharaday: <p>Hi AlphaBetaGamma96,</p>
<p>Just tried and got error as below. I’ll try to find the reason.<br>
Here it is:</p><aside class="quote quote-modified" data-post="1" data-topic="134287">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/samuel_bachorik/48/56232_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/negative-strides-in-tensor-error/134287">Negative strides in tensor error</a> 
  </div>
  <blockquote>
    Hi, when I use numpy.flip()  on my array and then i try to create from this array torch tensor with torch.from_numpy(x) 
It says 
ValueError: At least one stride in the given numpy array is negative, and tensors with negative strides are not currently supported. (You can probably work around this by making a copy of your array  with array.copy().) 

Why is this happening ? I see there are some solutions on internet but this never happened to me yet and I was using numpy.flip() a lot. 
When I use…
  </blockquote>
</aside>

<pre><code>im = torch.as_tensor(im, device=model.device)
</code></pre>
<p>ValueError: At least one stride in the given numpy array is negative, and tensors with negative strides are not currently supported. (You can probably work around this by making a copy of your array  with array.copy().)</p>
<p>Best Regards, Yuji</p>

AlphaBetaGamma96: <p>Try,</p>
<pre><code class="lang-auto">im = torch.as_tensor(im.copy(), device=model.device)
</code></pre>

swharaday: <p>Hi AlphaBetaGamma96,</p>
<p>Thank you for your response.</p>
<p>Here is results of trial after 2,000 iteration.</p>
<ul>
<li>“from_numpy”   ave. 5.509ms  stdev. 0.797</li>
<li>“as_tensor”       ave. 5.562ms  stdev. 0.762</li>
</ul>
<p>Unfortunately there seems to be no significant difference.<br>
If you have more idea, please kindly advise.</p>
<p>Best Regards, Yuji</p>