Strange behavior of torch::Stack

Duriandyl: <p>Hi everyone, I’m using Pytorch for a project that involves OpenGL/Cuda interop and OpenCV (for visualization).</p>
<p>I’m quite new to Pytorch so I might be missing something but I’m having a problem with the stack function.</p>
<p>Basically, I’m rendering an image (640X640X4) with OpenGL and I’m loading it into a tensor on GPU.</p>
<p>I managed to visualize it properly by converting it to a BGRA matrice with OpenCV using data_ptr.</p>
<p>Then I split the image into 100 non-overlapping patches of (64X64X4) with index /slice operations :</p>
<pre><code class="lang-auto">torch::Tensor tensors_stacked; 
std::vector&lt;torch::Tensor&gt; patches;
for (int i = 0; i &lt; numPatchesY; ++i) {
    for (int j = 0; j &lt; numPatchesX; ++j) {
        torch::Tensor patch = image_tensor.index({0, 
                              torch::indexing::Slice(),
                              torch::indexing::Slice(i * patchSize, (i + 1) * patchSize),
                              torch::indexing::Slice(j * patchSize, (j + 1) * patchSize)}).clone(); 
         //Visualizing the patch tensor here give expected results.
        patches.push_back(patch);
    }
}
//Then I stack each patch into a higher dimension tensor.
tensors_stacked = torch::stack(patches);

//Visualizing the patch tensor tensors_stacked[0] here gives weird patterns.


</code></pre>
<p>My problem is that the patch resulting from the stacking operation doesn’t seem to be the same from the slicing as you can see now :<br>
Visualizing the patch after slicing =&gt; (new user restrictions - can only put one image) Everything is fine !<br>
Visualizing the corresponding patch after stacking =&gt;<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/e/bef7dfd3b11a1f9f6e8875fe3ec4d898ff260553.png" alt="model_25_view_-1" data-base62-sha1="rfnMPP116lRffpZneB6gtWnSnpF" width="64" height="64"></p>
<p>I’ve checked min/max value and the Dtype of the Input  image_tensor / patch and tensor_stacked[0] which are the same.<br>
I’ve tried to change the memory layout (contiguous / channel_first etc but nothing did the trick).</p>
<p>Regarding the strides and sizes :</p>
<ul>
<li>My input image/tensor image_tensor has sizes of [1,4,640,640] and strides [1638400,1,2560,4]</li>
<li>The patch created after the slice operation has sizes of [4,64,64] and strides [4096,64,1]</li>
<li>Tensor_stacked has sizes of [100,4,64,64] and strides [64,1,4096]</li>
<li>Finally, Tensor_stacked[0] has sizes [4,64,64] and strides [4096,64,1]</li>
</ul>
<p>Like I said, after the stacking operation I’m not able to display the patch properly and I don’t understand why this is not working.</p>
<p>I gladly thank you if you have any explanation for what going on here!</p>

ptrblck: <p>The output looks interleaved and might be caused by the actual code to visualize the patch so could you share it, please?<br>
A quick test in Python shows that your general slicing is correct:</p>
<pre data-code-wrap="python"><code class="lang-python">x = torch.randn(1, 4, 640, 640)
stride = 64

patches = []
for i in range(10):
    for j in range(10):
        patch = x[:, :, i*stride : (i+1)*stride, j*stride : (j+1)*stride]
        patches.append(patch)

stacked = torch.stack(patches)
print(stacked.shape)
# torch.Size([100, 1, 4, 64, 64])

# test
print((stacked[0] == x[:, :, :stride, :stride]).all())
# tensor(True)
print((stacked[-1] == x[:, :, x.size(2)-stride:, x.size(3)-stride:]).all())
# tensor(True)
</code></pre>

Duriandyl: <p>Thank you for the test.</p>
<p>Operations done on tensor before visualizing :</p>
<pre><code class="lang-auto">
for(int i = 0; i &lt; tensors_stacked.sizes()[0];i++) {
    //Tensor values are between [-1,1] for model purpose. Converting back to [0,1] 
    torch::Tensor debug_print = (tensors_stacked[i].clone()*0.5)+0.5;
    //Going from [4,64,64] to [64,64,4]
    debug_print = debug_print.permute({ 1,2,0 });
    //Adding one extra dim [1,64,64,4] to match function input format
    debug_print = debug_print.view({1, 64, 64, 4});
    saveTensorToImage( debug_print, "debug/",i ,-1);  
}
</code></pre>
<p>Note : Patches from the slicing operation (before stacking) are displayed well using the same transformations. So I assume the normalisation / permutation / view are safe.</p>
<p>Now the part related to visualising :</p>
<pre><code class="lang-auto">void saveTensorToImage(const torch::Tensor &amp;texture_tensor, std::string folder, int current_token, int counter_frames) {
    //Sending data to the CPU and removing the extra dim 
    torch::Tensor tensor_cpu = texture_tensor.clone().to(torch::kCPU)[0];	
    
    cv::Mat save_img;
    save_img = ToCvImage(tensor_cpu);
    if(save_img.empty())
    {
      std::cerr &lt;&lt; "Something is wrong with the image, could not get frame." &lt;&lt; std::endl;
    }

    // Save the frame into a file
    char tmp[4096];
    sprintf(tmp,"%s%s%02d_view_%02d.png",folder.c_str(),"/model_",current_token,counter_frames);
    cv::Mat bgra8;
    save_img.convertTo(bgra8, CV_8UC4, 255); // [0,1] float image to [0,255] int
    cv::imwrite(tmp, bgra8); //saving a png file
}

//Convert and RGBA float tensor to an BGRA image ! 
cv::Mat ToCvImage(const torch::Tensor &amp;tensor){
    int height = tensor.size(0);
    int width = tensor.size(1);
try {
    // Assuming the tensor has 4 channels (RGBA)
    int channels = tensor.size(2);

    // Check the data type of the tensor
    at::ScalarType dtype = tensor.scalar_type();
    cv::Mat output_mat;
    if (dtype == torch::kFloat) {
        if (channels == 4) {
            // Four channels (RGBA)
            output_mat = cv::Mat(height, width, CV_32FC4, tensor.data_ptr&lt;float&gt;());
            std::vector&lt;cv::Mat&gt; channels(4);
	    cv::split(output_mat, channels);

	    // Permute the channels (swap Red and Blue) RGB =&gt; BGR
	    std::swap(channels[0], channels[2]);

	    // Merge the channels back to form the BGR Mat
	    cv::Mat bgrMat;
	    cv::merge(channels, bgrMat);
	    output_mat = bgrMat.clone();
            } else {
                // Unsupported number of channels
                std::cerr &lt;&lt; "Unsupported number of channels: " &lt;&lt; channels &lt;&lt; std::endl;
                return cv::Mat();
            }
        } else {
            // Unsupported data type
            std::cerr &lt;&lt; "Unsupported data type: " &lt;&lt; dtype &lt;&lt; std::endl;
            return cv::Mat();
        }
    return output_mat;
} catch (const c10::Error&amp; e) {
    std::cout &lt;&lt; "An error has occurred: " &lt;&lt; e.msg() &lt;&lt; std::endl;
}
    return cv::Mat();  // Return an empty Mat on failure
}
</code></pre>
<p>Again, thank you for your time !</p>