How do I shade a patch

username: binbbaz
cooked: <p>I am dividing an image into patches with the function below:</p>
<pre><code class="lang-auto">def extract_image_patches(x, kernel, stride=16, dilation=1):
    # Do TF 'SAME' Padding
    b, c, h, w = x.shape
    h2 = math.ceil(h / stride)
    w2 = math.ceil(w / stride)
    pad_row = (h2 - 1) * stride + (kernel - 1) * dilation + 1 - h
    pad_col = (w2 - 1) * stride + (kernel - 1) * dilation + 1 - w
    x = F.pad(x, (pad_row // 2, pad_row - pad_row // 2, pad_col // 2, pad_col - pad_col // 2))

    # Extract patches
    patches = x.unfold(2, kernel, stride).unfold(3, kernel, stride)
    patches = patches.permute(0, 4, 5, 1, 2, 3).contiguous()

    return patches.view(b, -1, patches.shape[-2], patches.shape[-1])
</code></pre>
<p>For a <code>224 x 224</code> image, the above function returns a tensor of shape <code>(1, 768, 14, 14)</code>. <code>768 = (3 x 16 x 16)</code> for one patch and <code>(14, 14)</code> is the height and width. Meaning there are <code>14</code> rows and columns of these patches.</p>
<p>Is there a way to change the pixel values of one patch with another function? Below is what I’ve tried so far</p>
<pre><code class="lang-auto">def shade_patch(patch_list, image):
    batch = len(patch_list)
    patches = extract_image_patches(image.unsqueeze(0), 16)
    pa = patches.repeat(batch, 1, 1, 1)
    
    count = 0 

    for x in patch_list:
       my_patches = np.split(x, len(x)/5

       for patch in my_patches:
           x_pos, y_pos, r, g, b = patch
           pa[count, 0, x_pos, y_pos] = (r / 255.0 - 0.4914) / 0.2023
           pa[count, 1, x_pos, y_pos] = (g / 255.0 - 0.4822) / 0.1994
           pa[count, 2, x_pos, y_pos] = (b / 255.0 - 0.4465) / 0.2010
       count += 1

    return pa
</code></pre>
<p>Where <code>patch_list</code> is a numpy array of 5 elements <code>([x, y, r, g, b])</code>. <code>x</code> and <code>y</code> are the positions of the patch to shade. The positions would range from <code>0</code> to <code>13</code> in my case since I have <code>14 x 14</code> patches. <code>(r, g, b)</code> would be the color channels</p>
<p>So if I expect the patch at cordinate <code>(3, 5)</code> to have been shaded with <code>(0, 121, 256)</code> color combination if I do the following:</p>
<pre><code class="lang-auto">x = np.array([3, 5, 0, 121, 256])
s = shade_patch(x, img)
</code></pre>

username: binbbaz
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> , could you please help me this. Thank you</p>

username: ptrblck
cooked: <p>Your general explanation sounds alright and I’m unsure where you are stuck?<br>
Do you have trouble indexing the patches or do you see an erro running your code?</p>

username: binbbaz
cooked: <p>Thank you for your reply.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/f/afbd3d7e0653124f406df681fc453b3f2bb587f7.png" alt="tt" data-base62-sha1="p4F09YLfHSdGRchpi6c4MgtXY8v" width="238" height="238"></p>
<p>Here is an example of what I want. One pixel has been shaded here. They did this by specifying the x,y cordinate and the colour channels to change to ([x, y, r, g, b]).</p>
<p>In my case, I want to do the same for a patch (16 x16) where I can just specify the patch location (x, y) and replace their color channels (r,g,b) with what I want.</p>
<p>I am not getting any error with the code above. I get a tensor of size [1, 768, 14, 14] which makes sense. However, I tired visualizing if specified patch has been shaded. To do so, I reshaped to (196, 3, 16, 16) but got the following output</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/c/dc8cc07afa9ee688066dc925dcfe799a26a73cb0.png" alt="F" data-base62-sha1="vt4BnWBp5rtucZ3gkDsKCAhVDjO" width="175" height="418"></p>

username: ptrblck
cooked: <p>Your reshape method is most likely wrong as the output looks interleaved (i.e. the actual indexing might work, but you can’t verify it currently).<br>
Try to reshape the output of your <code>unfold</code> method to its original form first without any image manipulations and make sure you are getting the same tensor back.</p>

username: binbbaz
cooked: <p>You are very right sir. Below is the output of the extract_image_patches() function above where the image is being unfolded. This is before any manipulation. It’s probably worth mentioning that I visualized the first one with pytorch’s make_grid function</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/2/1287d63ea243a93ca122acb82d45278832e56331.png" alt="r" data-base62-sha1="2DVCFnVnfqzEPwUbv6qvFIr55yp" width="425" height="417"></p>

username: ptrblck
cooked: <p>Maybe <a href="https://discuss.pytorch.org/t/patch-making-does-pytorch-have-anything-to-offer/33850/11">this post</a> is helpful, which shows a code snippet how to reshape the unfolded patches back to their original.</p>

username: binbbaz
cooked: <p>Many thanks sir. I’ll check it out</p>

username: binbbaz
cooked: <p>Thank you sir</p>
<p>The post here <a href="https://discuss.pytorch.org/t/fold-patches-of-images-back-to-single-image/61395/3" class="inline-onebox">Fold patches of images back to single image - #3 by agadetsky</a> works for me</p>