Segmentation of any image

username: Aldimir
cooked: <p>I have an image segmentation (Learned Random Walker), but I would like to generalize it.</p>
<pre><code class="lang-auto"># Init the random walker modules
rw = RandomWalker(1000, max_backprop=True)

# Load data and init
target = init_image(imagem)
seeds = init_seeds(torch.zeros(1, 60, 59).long(), target)  
diffusivities = torch.zeros(1, 2, 60, 59, requires_grad=True)

# Init optimizer
optimizer = torch.optim.Adam([diffusivities], lr=0.5)

loss = torch.nn.NLLLoss()

# Main overfit loop
for it in range(iterations + 1):
    optimizer.zero_grad()

    # Diffusivities must be positive
    net_output = torch.sigmoid(diffusivities)

    # Random walker
    output = rw(net_output, seeds)

    # Loss and diffusivities update
    m = torch.nn.LogSoftmax(dim=1)
    output_log = [m(o) for o in output]

    # ls = loss(output_log, target)
    ls = loss(output_log[0], target[0])
    ls.backward()
    optimizer.step()
</code></pre>
<p>Given <code>target</code> of dimension [1, 1, 60, 59], <code>output</code> will be of dimension [1, nClass, 60, 59], that is, it will be the <code>nClass</code> probability matrices for each region.<br>
But for example, with this same dimension, I have the image <code>target</code> in the range [0, 255] with 3 regions, <code>NLLLoss</code> does not allow me to do this, because I would have <code>output</code> with dimension [1, 3, 60, 59] and several pixels greater than 3.<br>
Is there anything you can do to get around this, or some loss function that solves this problem?</p>
<p>Any help is welcome, thanks.</p>

username: ptrblck
cooked: <p>I’m not familiar with the use case, but based on the description it seems you are using <code>256</code> classes for the target?<br>
If that’s not the case, could you explain how the <code>target</code> is constructed?</p>

username: Aldimir
cooked: <p><code>target</code> is any gray scale image, I will use this as an example: <img src="https://discuss.pytorch.org/uploads/default/original/3X/3/6/3613ab2e3d7ee2ab3ee56931b54fb243e263b919.png" alt="yin" data-base62-sha1="7InVfkSwcMRSqf3swNGKp4hXPOV" width="225" height="225"></p>
<p>Visually target contains 4 regions (classes), the fund counts. But because <code>target</code> has pixels of various values (including 255), NLLLoss requires that<code> output</code> be [1, 256, 60, 59] in size, but I would like it to be [1, 4, 60, 59]. (The loss function is not required to be NLLLoss.)</p>

username: ptrblck
cooked: <p>If your target only contains 4 classes, you should be able to map the values to just these 4 class indices <code>[0, 1, 2 ,3]</code>.<br>
Does your target contain 4 discrete values in the range <code>[0, 255]</code>? If not, you might need to create bins for the noisy values to map them to the needed class indices.</p>

username: Aldimir
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="82017">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>you might need to create bins for the noisy values</p>
</blockquote>
</aside>
<p>But isn’t it an approximation of the regions?</p>

username: ptrblck
cooked: <p>It depends on your input and how noisy the data is.<br>
Assuming you have 4 classes, but these classes have a certain grayscale value with a certain amount of noise.<br>
As long as you see clear cuts in the histogram of these images, you might just use thresholds.<br>
If you are seeing a wide spread of the noise such that classes are overlapping, you would have to think about either reducing the noise in your image or a smoothing strategy, which tries to keep the pixels in a certain area consistent.</p>
<p>Based from the output you’ve posted it seems that the noise is rather small (but check it with histograms).</p>