Custom color mapping in data loader for UNET image segmentation

username: hanoody
cooked: <p>Hi,</p>
<p>I have a 1000x1000 image and a mask of the same size with 5 color-coded classes (blue, red, dark green, light green, and pink).</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/a/ca933445e89256b8d643d14c4dbc9da9b26c34de.png" data-download-href="https://discuss.pytorch.org/uploads/default/ca933445e89256b8d643d14c4dbc9da9b26c34de" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/c/a/ca933445e89256b8d643d14c4dbc9da9b26c34de.png" alt="image" data-base62-sha1="sU3PClCYOwjBUjv5saPMAPjsE9w" width="689" height="438" data-dominant-color="F1F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1106×703 34.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I extracted this snippet of my code onto Jupyter notebook for debugging.</p>
<p>As we see below, the image shape is (1000, 1000, 3) and same thing for the mask shape. Yet len(colors) is 3809 instead of 5. Therefore my color map has 3809 key-value pairs instead of 5. Any reason why this might be happening?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/9/390af335e23816d2d75976bf66dabb5298c4b19d.png" data-download-href="https://discuss.pytorch.org/uploads/default/390af335e23816d2d75976bf66dabb5298c4b19d" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/9/390af335e23816d2d75976bf66dabb5298c4b19d.png" alt="image" data-base62-sha1="88CFVch4imIcRYtKJEMlTCkBBPn" width="690" height="116" data-dominant-color="F8EAEA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1100×186 6.76 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Thank you!</p>

username: ptrblck
cooked: <p>Could you print (some) values of the unique colors and check, if they might be noisy e.g. due to a resizing operation?</p>
<p>PS: It’s better to post code snippets by wrapping them into three backticks ```, as is makes debugging easier and also allows the search to index the code.</p>

username: hanoody
cooked: <p>Hi, my apologies for the late response.</p>
<p>I think you are exactly right. What I did was I added the masks using GIMP, then I scaled down the images also using GIMP. I am going to try reversing the order and will let you know if that fixes things. Thank you.</p>

username: hanoody
cooked: <p>Unfortunately resizing first and then adding in the colored masks did not change anything (unless I did something wrong in these steps).</p>
<p>I printed the list of unique colors and the length and this is what I get:</p>
<pre><code class="lang-auto">print(colors)
[[  7. 133.   0.]
 [  8. 132.   0.]
 [  8. 132.   2.]
 ...
 [255. 254. 255.]
 [255. 255. 253.]
 [255. 255. 255.]]
</code></pre>
<p>len(colors) = 3809 but that is strange because it should be 5 (or maybe 6 if it counts the black background).</p>

username: ptrblck
cooked: <p>Based on the output it seems that some colors (e.g. white, which is given as <code>[255, 255, 255]</code>) have some “neightbors” with a small difference (e.g. <code>[255, 254, 255]</code>), which could be introduced by an interpolation method in e.g. resizing.<br>
If you are resizing the image, I would recommend to use the nearest neighbor interpolation method, as it would keep the pixel values without adding new unique ones.<br>
In case you cannot isolate the operation, which is adding these values, you could try to cluster the values afterwards into the desired 6 classes.</p>

username: hanoody
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/3/8323cd9c7b59e9fa14d03a2e133c7e187b9e2038.png" data-download-href="https://discuss.pytorch.org/uploads/default/8323cd9c7b59e9fa14d03a2e133c7e187b9e2038" title="test2"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/3/8323cd9c7b59e9fa14d03a2e133c7e187b9e2038_2_690x388.png" alt="test2" data-base62-sha1="iI7fUewpE0AZbFexbQDfnW0NUg0" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/3/8323cd9c7b59e9fa14d03a2e133c7e187b9e2038_2_690x388.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/3/8323cd9c7b59e9fa14d03a2e133c7e187b9e2038_2_1035x582.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/3/8323cd9c7b59e9fa14d03a2e133c7e187b9e2038_2_1380x776.png 2x" data-dominant-color="EEEDEC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">test2</span><span class="informations">1920×1080 12.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>So I did this test with a 1920x1080 image consisting of my 5 masks and white background. When I inputted this image into the program above, the colors array consists of the following</p>
<pre><code class="lang-auto">[[ 10. 133.   1.]
 [ 14.   1. 133.]
 [ 32. 255.   0.]
 [182. 182. 202.]
 [182. 202. 182.]
 [183. 255. 182.]
 [187. 206. 187.]
 [188. 187. 206.]
 [188. 188. 206.]
 [188. 206. 188.]
 [188. 255. 187.]
 [189. 255. 188.]
 [193. 193. 210.]
 [193. 210. 193.]
 [194. 255. 193.]
 [243.   5. 247.]
 [249. 182. 251.]
 [249. 187. 251.]
 [249. 188. 251.]
 [249. 193. 251.]
 [255.   0.   0.]
 [255. 182. 182.]
 [255. 187. 187.]
 [255. 188. 188.]
 [255. 193. 193.]
 [255. 255. 255.]]
</code></pre>
<p>With len(colors) = 26. I did not resize the image so I am quite confused why there are 26 unique colors as opposed to 6. Perhaps I am doing something wrong with my preprocessing but I am lost with regards to that.</p>

username: ptrblck
cooked: <p>It seems the image itself contains these additional values and you can check the first 16 unique values via:</p>
<pre data-code-wrap="python"><code class="lang-python">img = PIL.Image.open("./color_test.png")
a = np.array(img)
u = np.unique(a)

f, axarr = plt.subplots(4, 4)
for idx, ax in enumerate(axarr.reshape(-1)):
    if idx == 16:
        break
    ax.imshow(a == u[idx], cmap='gray')
</code></pre>
<p>Which shows the expected blobs as well as the (unwanted) borders:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/9/9/994482894a86052c21bdd3fa3d72188ea295442e.png" alt="image" data-base62-sha1="lRRPKT3oxMiNDwgBsjphgLFwnj8" width="375" height="246"></p>
<p>so you would have to make sure to either filter them after loading the image or during the creation of it.</p>

username: hanoody
cooked: <p>Thank you very much! I really appreciate it.</p>
<p>I realized that the tool I was using on GIMP to create masks added additional pixel values to the edges of the fills.</p>

username: hanoody
cooked: <p>Hi, do you have any recommended software or editing tools I can use to mask my data? The tools that I use on GIMP always seem to introduce additional pixel values to my masks and it interferes with the unique-color-picking code. Thank you very much.</p>

username: ptrblck
cooked: <p>I guess the additional pixels might come from an interpolation step and/or quality reduction during the saving/export of the annotated mask. Which format are you currently using and are you able to define some kind of quality level while storing the mask or use a lossless format?</p>

username: hanoody
cooked: <p>I export my masks as a JPG with the following settings on GIMP.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/8/e8ef27c326ebaaebced844f9d4c8401422e7bd3b.png" data-download-href="https://discuss.pytorch.org/uploads/default/e8ef27c326ebaaebced844f9d4c8401422e7bd3b" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/8/e8ef27c326ebaaebced844f9d4c8401422e7bd3b.png" alt="image" data-base62-sha1="xeD9Z6v7CfhRsqHm7KRW2CGN59x" width="512" height="500" data-dominant-color="4A4A4A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">712×694 18.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>So for example, with the following mask saved as a JPG, the Python program above reads 85 unique color values as opposed to 2 (green and white).</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/a/da955485ce2852284788a4f03652e1f3042e92e8.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/da955485ce2852284788a4f03652e1f3042e92e8" title="test13_mask"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/a/da955485ce2852284788a4f03652e1f3042e92e8_2_500x500.jpeg" alt="test13_mask" data-base62-sha1="vbG1FEoQLmqGbzL1voZW4KmPNcQ" width="500" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/a/da955485ce2852284788a4f03652e1f3042e92e8_2_500x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/d/a/da955485ce2852284788a4f03652e1f3042e92e8_2_750x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/d/a/da955485ce2852284788a4f03652e1f3042e92e8_2_1000x1000.jpeg 2x" data-dominant-color="F9FFF8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">test13_mask</span><span class="informations">1000×1000 24.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: hanoody
cooked: <p>Figured it out! The fill tool I was using on GIMP had anti-aliasing which added additional pixel values to the mask boundary in order to smooth the edges. Thank you.</p>