image segmentation with cross-entropy loss

neoSilex: <p>I am a new user of Pytorch.<br>
I’d like to use the cross-entropy loss function</p>
<p>number of classes=2<br>
output.shape=[4,2,224,224]<br>
output_min=tensor(-1.9295)]<br>
output_max=tensor(2.6400)]</p>
<p>number of channels=3<br>
target.shape=[4,3,224,224]<br>
targets_max=tensor(-2.1008)]<br>
targets_min=tensor(-2.1179)]</p>
<p>how to evaluate:<br>
loss = criterion(output, target)?<br>
Thanks.</p>

KFrank: <p>Hello Neo!</p>
<aside class="quote no-group" data-username="neoSilex" data-post="1" data-topic="79138" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/e9a140/48.png" class="avatar"> neoSilex:</div>
<blockquote>
<p>I’d like to use the cross-entropy loss function</p>
<p>number of classes=2<br>
output.shape=[4,2,224,224]</p>
</blockquote>
</aside>
<p>As an aside, for a two-class classification problem, you will be<br>
better off treating this explicitly as a binary problem, rather than<br>
as a two-class instance of the more general multi-class problem.<br>
To do so you would use <code>BCEWithLogitsLoss</code> (“Binary Cross<br>
Entropy”), rather than the multi-class <code>CrossEntropyLoss</code>.</p>
<p>But you can certainly treat this as a general multi-class problem,<br>
and I will answer your question in this context.</p>
<aside class="quote no-group">
<blockquote>
<p>output_min=tensor(-1.9295)]<br>
output_max=tensor(2.6400)]</p>
</blockquote>
</aside>
<p>So your outputs are raw-score <em>logits</em>, rather than probabilities<br>
that lie between <code>0.0</code> and <code>1.0</code>.  Good.</p>
<aside class="quote no-group">
<blockquote>
<p>number of channels=3<br>
target.shape=[4,3,224,224]</p>
</blockquote>
</aside>
<p>I assume that your first dimension, <code>4</code>, is your batch size.  This is<br>
fine.</p>
<p>It probably does not make sense to have a channels dimension in<br>
your  target.  (If you think it does, you should further explain your<br>
use case.)  In any event, as it stands, this target shape won’t match<br>
your output shape.</p>
<p>If your output shape is <code>[nBatch, nClass, height, width]</code>,<br>
then (for <code>CrossEntropyLoss</code>) your target shape must be<br>
<code>[nBatch, height, width]</code>, with no <code>nClass</code> dimension.</p>
<aside class="quote no-group">
<blockquote>
<p>targets_max=tensor(-2.1008)]<br>
targets_min=tensor(-2.1179)]</p>
</blockquote>
</aside>
<p>This is wrong (for <code>CrossEntropyLoss</code>).  Your target values must<br>
be integer (<code>long</code>) class labels that run from <code>0</code> to <code>nClass - 1</code>,<br>
so in your two-class case, that take on the values <code>0</code> and <code>1</code>.</p>
<p>If you could explain a little more where <code>target</code> comes from, and<br>
what the numbers actually mean, we can help sort this out.</p>
<p>Best.</p>
<p>K. Frank</p>

neoSilex: <p>I’m sorry for the delay i had problems<br>
and I’m sorry for my English</p>
<p>my model is:</p>
<h2>number of classes=2 or 3 or 10</h2>
<p>And the output dimension of the model is [No x Co x Ho x Wo]<br>
where,</p>
<pre><code>No -&gt; is the batch size (same as Ni)
Co -&gt; is the number of classes that the dataset have!
Ho -&gt; the height of the image (which is the same as Hi in almost all cases)
Wo -&gt; the width of the image (which is the same as Wi in almost all cases)
</code></pre>
<hr>
<p>number of channels=1 or 3</p>
<p>the target dimension is [Ni x Ci x Hi x Wi]<br>
where,</p>
<pre><code>Ni -&gt; the batch size
Ci -&gt; the number of channels (which is 3 or 1)
Hi -&gt; the height of the image
Wi -&gt; the width of the image 
</code></pre>
<hr>
<p>and thanks for your reply</p>
<p>I apply train transformation into image and mask:</p>
<p>train_transform = et.ExtCompose([<br>
<span class="hashtag">#et</span>.ExtResize(size=opts.crop_size),<br>
et.ExtRandomScale((0.5, 2.0)),<br>
et.ExtRandomCrop(size=(opts.crop_size, opts.crop_size), pad_if_needed=True),<br>
et.ExtRandomHorizontalFlip()<br>
et.ExtToTensor(),<br>
et.ExtNormalize(mean=[0.485, 0.456, 0.406],<br>
std=[0.229, 0.224, 0.225]),<br>
])</p>
<p>mask  is rgb or gray with values ​​of 0 background and 1class1 2class2</p>

KFrank: <p>Hello Neo!</p>
<aside class="quote no-group" data-username="neoSilex" data-post="3" data-topic="79138" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/e9a140/48.png" class="avatar"> neoSilex:</div>
<blockquote>
<hr>
<p>And the output dimension of the model is [No x Co x Ho x Wo]<br>
where,</p>
<pre><code>No -&gt; is the batch size (same as Ni)
Co -&gt; is the number of classes that the dataset have!
Ho -&gt; the height of the image (which is the same as Hi in almost all cases)
Wo -&gt; the width of the image (which is the same as Wi in almost all cases)
</code></pre>
<hr>
</blockquote>
</aside>
<p><code>Ho</code> and <code>Wo</code> must be the same as <code>Hi</code> and <code>Wi</code> in <em>all</em> cases, not<br>
just in “almost all” cases.</p>
<aside class="quote no-group">
<blockquote>
<p>number of channels=1 or 3</p>
<p>the target dimension is [Ni x Ci x Hi x Wi]<br>
where,</p>
<pre><code>Ni -&gt; the batch size
Ci -&gt; the number of channels (which is 3 or 1)
Hi -&gt; the height of the image
Wi -&gt; the width of the image 
</code></pre>
<hr>
</blockquote>
</aside>
<p>This might be what you have, but it simply won’t work.<br>
<code>CrossEntropyLoss</code> requires that, for a model output of shape<br>
<code>[No, Co, Ho, Wo]</code>, the target have shape <code>[No, Ho, Wo]</code><br>
(and that the values of the target are integer class labels that<br>
run from <code>0</code> to <code>Co - 1</code>).</p>
<p>If your target has this extra “channel” dimension (<code>Ci</code>), it won’t<br>
work (and <code>Hi</code> and <code>Wi</code> must match <code>Ho</code> and <code>Wo</code>, as well).  (Just<br>
to be clear, you can’t have the <code>Ci</code> dimension at all, even if<br>
<code>Ci = 1</code>.)</p>
<p>Good luck.</p>
<p>K. Frank</p>

neoSilex: <p>thanks Frank for your reply<br>
I’m going to look for another way</p>

neel_g: <p>Hey <img src="https://discuss.pytorch.org/images/emoji/apple/hugs.png?v=12" title=":hugs:" class="emoji" alt=":hugs:" loading="lazy" width="20" height="20"> sorry to disturb you, just wanted to confirm -</p>
<ol>
<li>the raw logits are supposed to be one-hot encoded - say as a sample shape of <code>(1, 6, 256, 256)</code> if one has multi-class classification w/ <code>6</code> labels</li>
<li>then the target to the loss function <em>has</em> to be to the <strong>non-onehot-encoded</strong>, the true integer labels in their pristine form.</li>
</ol>
<p>I am confused why pytorch doesn’t do this implicitly <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=12" title=":thinking:" class="emoji" alt=":thinking:" loading="lazy" width="20" height="20"> though was my understanding correct?</p>

Hillary: <p>So How do I get rid off  “Channel” dimension in my case</p>
<pre data-code-wrap="class"><code class="lang-nohighlight">    def __init__(self):
        super(Dir_VAE, self).__init__()
        self.encoder = nn.Sequential(
            # input is (nc) x 28 x 28
            nn.Conv2d(nc, ndf, 4,4, 0,bias=False),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf) x 14 x 14
            nn.Conv2d(ndf, ndf * 2, 4,4, 0,bias=False),
            nn.BatchNorm2d(ndf * 2),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf*2) x 7 x 7
            nn.Conv2d(ndf * 2, ndf * 4, 4, 4, 0,bias=False),
            nn.BatchNorm2d(ndf * 4),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf*4) x 4 x 4
            nn.Conv2d(ndf * 4, 512, 4, 2,0,bias=False),
            # nn.BatchNorm2d(1024),
            nn.LeakyReLU(0.2, inplace=True),
            # nn.Sigmoid()
        )

        self.decoder = nn.Sequential(
            # input is Z, going into a convolution
            nn.ConvTranspose2d(512, ngf * 4, 4, 2, 0, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(True),
            # state size. (ngf*8) x 4 x 4
            nn.ConvTranspose2d(ngf * 4, ngf * 2, 4, 4, 0, bias=False),
            nn.BatchNorm2d(ngf * 2),
            nn.ReLU(True),
            # state size. (ngf*4) x 8 x 8
            nn.ConvTranspose2d(ngf * 2, ngf * 2, 4, 4, 0, bias=False),
            nn.BatchNorm2d(ngf * 2),
            nn.ReLU(True),
            # state size. (ngf*2) x 16 x 16
            nn.ConvTranspose2d(ngf * 2, nc, 4, 4, 0, bias=False),
            # nn.BatchNorm2d(ngf),
            # nn.ReLU(True),
            # state size. (ngf) x 32 x 32
            # nn.ConvTranspose2d(    ngf,      nc, 4, 2, 1, bias=False),
            # nn.Tanh()
            nn.Sigmoid()
            # state size. (nc) x 64 x 64
        )
        self.fc1 = nn.Linear(512, 256)
        self.fc21 = nn.Linear(256, 10)
        self.fc22 = nn.Linear(256, 10)

        self.fc3 = nn.Linear(10, 256)
        self.fc4 = nn.Linear(256, 512)

        self.lrelu = nn.LeakyReLU()
        self.relu = nn.ReLU()

        # Dir prior
        self.prior_mean, self.prior_var = map(nn.Parameter, prior(10, 0.3)) # 0.3 is a hyper param of Dirichlet distribution
        self.prior_logvar = nn.Parameter(self.prior_var.log())
        self.prior_mean.requires_grad = False
        self.prior_var.requires_grad = False
        self.prior_logvar.requires_grad = False


    def encode(self, x):
        conv = self.encoder(x);
        print('Size', conv.shape)
        h1 = self.fc1(conv.view(-1, 512))
        return self.fc21(h1), self.fc22(h1)

    def decode(self, gauss_z):
        dir_z = F.softmax(gauss_z,dim=1) 
        # This variable (z) can be treated as a variable that follows a Dirichlet distribution (a variable that can be interpreted as a probability that the sum is 1)
        # Use the Softmax function to satisfy the simplex constraint
        # シンプレックス制約を満たすようにソフトマックス関数を使用
        h3 = self.relu(self.fc3(dir_z))
        deconv_input = self.fc4(h3)
        print('Deconv ', deconv_input.shape)
        deconv_input = deconv_input.view(-1,512,1,1)
        return self.decoder(deconv_input)

    def reparameterize(self, mu, logvar):
        std = torch.exp(0.5*logvar)
        eps = torch.randn_like(std)
        return mu + eps*std


    def forward(self, x):
        mu, logvar = self.encode(x)
        gauss_z = self.reparameterize(mu, logvar) 
        # gause_z is a variable that follows a multivariate normal distribution
        # Inputting gause_z into softmax func yields a random variable that follows a Dirichlet distribution (Softmax func are used in decoder)
        dir_z = F.softmax(gauss_z,dim=1) # This variable follows a Dirichlet distribution
        return self.decode(gauss_z), mu, logvar, gauss_z, dir_z

    # Reconstruction + KL divergence losses summed over all elements and batch
    def loss_function(self, recon_x, x, mu, logvar, K):
    
        print('Recon ',recon_x.shape)
        print('Data ' ,x.shape)
        BCE = F.binary_cross_entropy(recon_x.view(-1, 65536), x.view(-1, 65536), reduction='sum')
        # ディリクレ事前分布と変分事後分布とのKLを計算
        # Calculating KL with Dirichlet prior and variational posterior distributions
        # Original paper:"Autoencodeing variational inference for topic model"-https://arxiv.org/pdf/1703.01488
        ''' prior_mean = self.prior_mean.expand_as(mu)
        prior_var = self.prior_var.expand_as(logvar)
        prior_logvar = self.prior_logvar.expand_as(logvar)
        var_division = logvar.exp() / prior_var # Σ_0 / Σ_1
        diff = mu - prior_mean # μ_１ - μ_0
        diff_term = diff *diff / prior_var # (μ_1 - μ_0)(μ_1 - μ_0)/Σ_1
        logvar_division = prior_logvar - logvar # log|Σ_1| - log|Σ_0| = log(|Σ_1|/|Σ_2|)
        # KL
        KLD = 0.5 * ((var_division + diff_term + logvar_division).sum(1) - K) '''
        KLD = -0.5 * torch.sum(1+logvar - mu**2 - torch.exp(logvar), axis=1)
        return BCE + KLD
</code></pre>
<pre><code class="lang-auto">Deconv  torch.Size([1, 512])
Recon  torch.Size([1, 1, 256, 256])
Data  torch.Size([1, 1, 256, 256])
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-33-5c58cc28c4e5&gt; in &lt;module&gt;
    266     # 学習(Train)
    267     for epoch in range(1, 10):
--&gt; 268         train(epoch)
    269         test(epoch)
    270         with torch.no_grad():

&lt;ipython-input-33-5c58cc28c4e5&gt; in train(epoch)
    226         optimizer.zero_grad()
    227         recon_batch, mu, logvar, gauss_z, dir_z = model(data)
--&gt; 228         loss = model.loss_function(recon_batch, data, mu, logvar, 10)
    229         loss = loss.mean()
    230         loss.backward()

&lt;ipython-input-33-5c58cc28c4e5&gt; in loss_function(self, recon_x, x, mu, logvar, K)
    198         print('Recon ',recon_x.shape)
    199         print('Data ' ,x.shape)
--&gt; 200         BCE = F.binary_cross_entropy(recon_x.view(-1, 65536), x.view(-1, 65536), reduction='sum')
    201         # ディリクレ事前分布と変分事後分布とのKLを計算
    202         # Calculating KL with Dirichlet prior and variational posterior distributions

C:\Conda5\lib\site-packages\torch\nn\functional.py in binary_cross_entropy(input, target, weight, size_average, reduce, reduction)
   2760         weight = weight.expand(new_size)
   2761 
-&gt; 2762     return torch._C._nn.binary_cross_entropy(input, target, weight, reduction_enum)
   2763 
   2764 

RuntimeError: all elements of input should be between 0 and 1```</code></pre>