How to make U-net to better quality of RGB images

verku: <p>Hi,<br>
I’m quite new to ML and I’m trying to modify my current U-net image segmentation model (takes RGB input returns binary classification map) to give RGB output. I have made changes to the number of output channels, changed the loss function to cross entropy loss and removed thresholding as I want a continuous brightness level for the color channels instead of having just 0 or 1. The output images seem gray. All pixel values are the same in a channel. Now, in my R and B channel all values are 220 and in G channel all values are 0. What should I still change?</p>

KFrank: <p>Hi Veera!</p>
<aside class="quote no-group" data-username="verku" data-post="1" data-topic="183207" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/v/919ad9/48.png" class="avatar"> verku:</div>
<blockquote>
<p>I’m trying to modify my current U-net image segmentation model (takes RGB input returns binary classification map) to give RGB output.</p>
</blockquote>
</aside>
<p>Let me speculate that instead of having your U-Net model perform<br>
semantic segmentation (e.g., foreground vs. background), you would<br>
like your model to do something like denoise or deblur your input image.</p>
<p>In such a case, you don’t want the pixels of your output image to be<br>
predicted class labels, but rather predicted RGB pixel values.</p>
<aside class="quote no-group">
<blockquote>
<p>changed the loss function to cross entropy</p>
</blockquote>
</aside>
<p>This is incorrect for what I assume is your use case.</p>
<aside class="quote no-group">
<blockquote>
<p>I want a continuous brightness level for the color channels instead of having just 0 or 1.</p>
</blockquote>
</aside>
<p>To train a model to predict continuous values (that don’t somehow<br>
represent probabilities) you would typically use <code>MSELoss</code>.</p>
<p>So if your training data consists of input images with noise added to<br>
them and target images that are the original images without the added<br>
noise, you would want the (per-pixel) output of your U-Net to have<br>
three channels (the RGB) and you would compute the <code>MSELoss</code> of<br>
the output image with respect to the (three-channel, RGB) target image.</p>
<p>This is simply saying that you want the denoised image produced by<br>
the model to be as close as possible to the original no-noise image.</p>
<p>Best.</p>
<p>K. Frank</p>

verku: <p>Hi,</p>
<p>thanks for your answer. I tried MSELoss and also L1Loss, but the images are still grayish. I checked that the result images have around 30-70 different values (instead of 256) all nearing the maximum brightness (getting values from 200-255). I wonder if I should somehow normalize the images during training or something. Do you have any idea what is going wrong?</p>
<p>Br. Veera</p>

KFrank: <p>Hi Veera!</p>
<aside class="quote no-group" data-username="verku" data-post="3" data-topic="183207" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/v/919ad9/48.png" class="avatar"> verku:</div>
<blockquote>
<p>I tried MSELoss and also L1Loss, but the images are still grayish.</p>
</blockquote>
</aside>
<p>What sorts of values do you get for <code>MSELoss</code> and <code>L1Loss</code>?  Do they<br>
seem consistent with the fact that your “result images” aren’t as good<br>
as you would like?</p>
<p>Both <code>MSELoss</code> and <code>L1Loss</code> should become zero as your result images<br>
becomes exactly the same as your target images.  Do you get zero when<br>
you apply your loss function to <code>(result, result)</code> and <code>(target, target)</code>?</p>
<aside class="quote no-group">
<blockquote>
<p>I checked that the result images have around 30-70 different values (instead of 256) all nearing the maximum brightness (getting values from 200-255).</p>
</blockquote>
</aside>
<p>Is it possible to <em>overfit</em> your model by training (a lot) on a small subset<br>
of your training data?  You would like to be able to get very good “result<br>
images” for just a small set of training samples by training a lot on just<br>
those samples.  If you can’t, there is probably something the matter with<br>
your model or your training algorithm.</p>
<aside class="quote no-group">
<blockquote>
<p>I wonder if I should somehow normalize the images during training or something.</p>
</blockquote>
</aside>
<p>Normalizing your images – both the inputs and targets – could potentially<br>
improve training.  However, unless different samples within your training<br>
set differ a lot in brightness or contrast, or skew towards being unusually<br>
bright, I wouldn’t expect normalization to fix the problem that you are<br>
describing.</p>
<p>Best.</p>
<p>K. Frank</p>