Semantic segmentation on very big images

username: Sylvain_Ard
cooked: <p>Re,<br>
the last program I am looking for is a semantic segmentation program for very big images (6000 x 4000 pixels) without resizing before enter into the neural network because objects are very thin (about 1 to 4 pixels large).<br>
Thank you<br>
Best regards</p>

username: KFrank
cooked: <p>Hi Sylvain!</p>
<aside class="quote no-group" data-username="Sylvain_Ard" data-post="1" data-topic="152522" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sylvain_ard/48/30714_2.png" class="avatar"> Sylvain_Ard:</div>
<blockquote>
<p>semantic segmentation program for very big images (6000 x 4000 pixels) without resizing</p>
</blockquote>
</aside>
<p>One reasonable approach would be to use a U-Net (a common and<br>
solid semantic-segmentation network architecture) with the so-called<br>
“overlap-tile strategy” (that is, apply the U-Net to patches of the large<br>
input images).  This is discussed in detail in the <a href="https://arxiv.org/abs/1505.04597" rel="noopener nofollow ugc">original U-Net paper.</a></p>
<p>Best.</p>
<p>K. Frank</p>

username: Sylvain_Ard
cooked: <p>but the recognition of the objects require the entire image because objects are thin but long, I have a GPU of 24GB of memory, is it sufficient to work with entire 6000x4000 images ?</p>

username: KFrank
cooked: <p>Hi Sylvain!</p>
<aside class="quote no-group" data-username="Sylvain_Ard" data-post="3" data-topic="152522" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sylvain_ard/48/30714_2.png" class="avatar"> Sylvain_Ard:</div>
<blockquote>
<p>recognition of the objects require the entire image because objects are thin but long,</p>
</blockquote>
</aside>
<p>Could a person looking at your image perform semantic segmentation<br>
of one end of the object without being able to see the other end?</p>
<p>For example, in an image of, say, pencils laying on a table, one could<br>
easily segment the pencils on a patch-wise basis – either by hand or<br>
with a network.</p>
<p>If so, “overlap-tile” U-Net would likely work.</p>
<p>In the special case that your thin objects are oriented approximately<br>
horizontally (or vertically) in your image, you could design an anisotropic<br>
U-Net whose “field of view” is a skinny rectangle that is long enough to<br>
fit your entire object (rather than a square) and then use the “overlap-tile”<br>
strategy.</p>
<aside class="quote no-group">
<blockquote>
<p>I have a GPU of 24GB of memory, is it sufficient to work with entire 6000x4000 images ?</p>
</blockquote>
</aside>
<p>You would have to try it.  My <em>guess</em> is that a typical U-Net (The original<br>
paper describes a U-Net with a “depth” of five.) would not be able to<br>
process a 6000x4000 image with 24GB of memory, especially if you<br>
plan to train, and therefore perform backward passes.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Sylvain_Ard
cooked: <p>in fact this program is made to recognize fine primary and secondary veins of tree leaves, to be able to recognize that they are primary or secondary veins the program needs to see the whole leaf, but it is true that in most cases the leaf is not of the size of the image, i.e. 6000x4000 pixels but less</p>

username: Sylvain_Ard
cooked: <p>I think that I can crop the leaf and resize it in for example 2000x1500 pixels</p>

username: KFrank
cooked: <p>Hi Sylvain!</p>
<aside class="quote no-group" data-username="Sylvain_Ard" data-post="5" data-topic="152522" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sylvain_ard/48/30714_2.png" class="avatar"> Sylvain_Ard:</div>
<blockquote>
<p>to be able to recognize that they are primary or secondary veins the program needs to see the whole leaf</p>
</blockquote>
</aside>
<p>You’re the domain expert, but let me go out on a limb and challenge<br>
this assumption.</p>
<p>I could well believe that seeing the base of the leaf could let you<br>
segment primary veins, seeing the junctions where primary veins<br>
branch into secondary veins could let you segment both, and seeing<br>
the tip or edges of the leaf could let you segment secondary veins.</p>
<p>Have you tried semantic segmentation on partial images?</p>
<p>Responding to your subsequent post, I expect that if you can crop<br>
your images down to 2000x1500, 24 GB should be enough to process<br>
entire (cropped) images with a network such as U-Net.</p>
<p>Have you tried running U-Net on full or cropped images?</p>
<p>Best.</p>
<p>K. Frank</p>