MinMax Adversarial Loss

shakeel608: <p>I have a multi-task learning model with two multi classification tasks. One part of the model creates a shared feature representation that is fed into two subnets in parallel. The loss function for each subnet at the moment is cross entropy.<br>
I want to minimize CE in one task and to maximise the cross entropy in one task so the model doesn’t/can’t learn anything about that one task, and then I think the resulting accuracy for that task should be <code>1/numoflabels</code>  i.e no better than random guess</p>
<p>In my network. I have two classifiers as:</p>
<pre><code class="lang-auto">class Adversarial(nn.Module):
    def __init__(self):
        super(Adversarial, self).__init__()       
        self.cnn = CNN() 
        self.emotion_classfier = EmotionClassfier()
        self.speaker_invariant = SpeakerInvariant()

    def forward(self, input):
        input         = self.cnn(input)
        input_emotion = self.emotion_classfier(input)
        input_speaker = self.speaker_invariant(input) 
       """First confusion is here, how shall handle it here and then how to implemenet this loss function  """ 
        return input_stutter, input_speaker

class EmotionClassfier(nn.Module):
    def __init__(self):
        super(EmotionClassfier, self).__init__() 
        self.fc1 = nn.Linear(128, 64)
        self.fc2 = nn.Linear(64, 5)     #5 here is number of emotion classes 
        self.relu = nn.ReLU()

    def forward(self, input):
        input = input.view(input.size(0), -1)
        input = self.fc1(input)
        input = self.relu(input)
        input = self.fc2(input)
        return input

class SpeakerInvariant(nn.Module):
    def __init__(self):
        super(SpeakerInvariant, self).__init__()
        self.fc1 = nn.Linear(128, 64)
        self.fc2 = nn.Linear(64, 11)   #11 here is number of speakers/labels
        self.relu = nn.ReLU()

    def forward(self, input):
        input = input.view(input.size(0), -1)
        input = self.fc1(input)
        input = self.relu(input)
        input = self.fc2(input)       
        return input
</code></pre>
<p>and the loss function I want to implement is as follows:</p>
<pre><code class="lang-auto">criterion = nn.CrossEntropyLoss()  #For EmotionClassfier as well as SpeakerClassfier
loss_motion   = criterion(output, emotion_label)  #Want to Minimise this loss
loss_speaker = criterion(output, speaker_label)
"""Here I want to maximize the speakerloss,(Suppose there are five speaker labels and I want to learn the speaker independent features)
i.e I want to apply Gradient Reversal for SpeakerInvariant Network"""
loss      = loss+emotion + lambda * loss_speaker 

</code></pre>
<p><em><strong>Could you please help it out here</strong></em></p>
<p>From paper:</p>
<blockquote>
<p><a href="https://www.microsoft.com/en-us/research/uploads/prod/2018/04/ICASSP2018_Speaker_Invariant_Training.pdf" rel="noopener nofollow ugc">https://www.microsoft.com/en-us/research/uploads/prod/2018/04/ICASSP2018_Speaker_Invariant_Training.pdf</a></p>
</blockquote>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/8/18a5712f23fdb9074be0b81f837d08e8ff75d864.png" data-download-href="https://discuss.pytorch.org/uploads/default/18a5712f23fdb9074be0b81f837d08e8ff75d864" title="Screenshot from 2020-07-20 12-02-57"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/8/18a5712f23fdb9074be0b81f837d08e8ff75d864_2_405x500.png" alt="Screenshot from 2020-07-20 12-02-57" data-base62-sha1="3w1USwv2uXNfvGh7hHmmi8CPzPS" width="405" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/8/18a5712f23fdb9074be0b81f837d08e8ff75d864_2_405x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/8/18a5712f23fdb9074be0b81f837d08e8ff75d864_2_607x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/8/18a5712f23fdb9074be0b81f837d08e8ff75d864.png 2x" data-dominant-color="E8E8E7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2020-07-20 12-02-57</span><span class="informations">661×815 140 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

KFrank: <p>Hi Shakeel!</p>
<aside class="quote no-group quote-modified" data-username="shakeel608" data-post="1" data-topic="89865" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/a587f6/48.png" class="avatar"> shakeel608:</div>
<blockquote>
<p>I want to minimize CE in one task and to maximise the cross entropy in one task so the model doesn’t/can’t learn anything about that one task, .<br>
…</p>
<pre><code class="lang-auto">"""Here I want to maximize the speakerloss,
...
i.e I want to apply Gradient Reversal for SpeakerInvariant Network"""
</code></pre>
</blockquote>
</aside>
<p>I don’t know that much about adversarial networks, and I haven’t<br>
looked at the paper to which you linked, but let me outline some<br>
details that might be relevant to what you are trying to do:</p>
<p>Your network comes in three pieces, the front end that produces your<br>
“shared feature representation,” let’s call it <code>FeatureNet</code>, the one half<br>
of your back end that predicts emotions, call it <code>EmotionNet</code>, and the<br>
other half that predicts the speakers, `SpeakerNet’.</p>
<p>To be clear, you <em>don’t</em> want to train <code>SpeakerNet</code> to do poorly<br>
predicting the speakers.  (That would be trivial.)</p>
<p>The core goal of your adversarial network is to train <code>FeatureNet</code><br>
to generate features that that can be used to successfully predict<br>
emotions, but that are not useful for predicting the speakers.</p>
<p>The key idea is that we actively train <code>SpeakerNet</code> to do the best<br>
job it can predicting the speakers using the output of <code>FeatureNet</code>,<br>
but train <code>FeatureNet</code> so that <code>SpeaketNet</code> does poorly (while<br>
<code>EmotionNet</code> does well).</p>
<p>We introduce two loss functions, <code>emotionLoss</code> and <code>speakerLoss</code>, and<br>
the combined loss, <code>loss = emotionLoss + lambda * speakerLoss</code>.<br>
(In your case the two losses are both <code>CrossEntropyLoss</code>.)</p>
<p>We backpropagate / optimize <code>loss</code> normally through both <code>EmotionNet</code><br>
and <code>SpeakerNet</code>, so that we train the both to make their predictions as<br>
successfully as they can, <em>given the features produced by</em> <code>FeatureNet</code>.<br>
(Notice that <code>emotionLoss</code> doesn’t depend on <code>SpeakerNet</code>, so<br>
backpropagating the gradient of <code>emotionLoss</code> through <code>SpeakerNet</code><br>
doesn’t affect how <code>SpeakerNet</code>’s weights are updated.  And vice<br>
versa.)</p>
<p>But we keep track of the <code>emotionLoss</code> and <code>speakerLoss</code> gradients<br>
separately so that before we backpropagate through <code>FeatureNet</code><br>
we can flip the sign of the <code>speakerLoss</code> gradient.  (This would be<br>
your “Gradient Reversal.”)</p>
<p>You are now training <code>FeatureNet</code> to produces features that let<br>
<code>EmotionNet</code> be trained to do well predicting emotions, but also<br>
<em>prevent</em> <code>SpeakerNet</code> from doing well predicting the speakers,<br>
<em>even though</em> the <code>SpeakerNet</code> portion of your whole network is<br>
being trained to do as well as it can, given the features it gets<br>
from <code>FeatureNet</code>.</p>
<p>(I’m not aware of anything built into pytorch that will do “Gradient<br>
Reversal” for you, and I’m not really sure how best to implement<br>
something like this.  In any event, at the point where the gradients<br>
are being backpropagated from <code>EmotionNet</code> and <code>SpeakerNet</code> back<br>
through <code>FeatureNet</code>, you somehow have to get your hands on the<br>
<code>speakerLoss</code> / <code>SpeakerNet</code> gradient and flip its sign, before passing<br>
it on back through <code>FeatureNet</code>.)</p>
<p>Good luck.</p>
<p>K. Frank</p>

shakeel608: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>  Can you please help here.</p>

ptrblck: <p>I think <a class="mention" href="/u/kfrank">@KFrank</a> provided a great answer. Do you have specific questions and did you try to follow the suggestions?</p>

shakeel608: <p><a class="mention" href="/u/ptrblck">@ptrblck</a><br>
Yes the specific question is regarding gradient reversal<br>
How do we flip the gradients here ?<br>
Can you please provide an example with SpeakerNet</p>

shakeel608: <p><a class="mention" href="/u/ptrblck">@ptrblck</a><br>
I am trying to apply the Gradient reversal layer (GRL) from  <a href="https://arxiv.org/pdf/1903.09606.pdf" rel="noopener nofollow ugc">Speaker Invariant Training using GRL</a> and <a href="https://arxiv.org/pdf/1409.7495.pdf" rel="noopener nofollow ugc">Unsupervised Domain Adaptation by Backpropagation</a> in the SpeakerNet as mentioned below.<br>
The idea is to learn speaker invariant emotion features by adversarial training using GRL i.e to unlearn speaker information.</p>
<p>Is my implementation correct for GRL here and in main training function, do I have to simply sum the losses of EmotionNet and SpeakerNet and then back-propagate ?</p>
<pre><code class="lang-auto">class EncoderNet(nn.Module):
    def __init__(self):
        super(Multitask, self).__init__()       
        self.cnn = TDNN()   #Assume Shared Encoder Here 

    def forward(self, input):
       return self.cnn(input)

        
class Multitask(nn.Module):
    def __init__(self):
        super(Multitask, self).__init__()       
        self.enc_net               =  EncoderNet()
        self.emotion_classfier = EmotionClassfier()
        self.speaker_invariant = SpeakerInvariant()

    def forward(self, x):
        input = self.enc_net(x)
        input_emotion = self.emotion_classfier(input)
        input_speaker = self.speaker_invariant(input) 
       """First confusion is here, how shall handle it here and then how to implemenet this loss function  """ 
        return [input_emotion, input_speaker]

class EmotionClassfier(nn.Module):
    def __init__(self):
        super(EmotionClassfier, self).__init__() 
        self.fc1 = nn.Linear(128, 64)
        self.fc2 = nn.Linear(64, 5)     #5 here is number of emotion classes 
        self.relu = nn.ReLU()

    def forward(self, input):
        input = input.view(input.size(0), -1)
        input = self.fc1(input)
        input = self.relu(input)
        input = self.fc2(input)
        return input

class SpeakerInvariant(nn.Module):
    def __init__(self):
        super(SpeakerInvariant, self).__init__()
        self.fc1 = nn.Linear(128, 64)
        self.fc2 = nn.Linear(64, 10)   #10 here is number of speakers/labels
        self.relu = nn.ReLU()
   
    def forward(self, input):
        input_reverse_gradients = ReverseGradients.apply(input, 0.5)  #alpha is 0.5
         input = self.relu(self.fc1(input_reverse_gradients))
         return self.fc2(input)
"""For GRadient Reversal Layer """
from torch.autograd import Function

class ReverseGradients(Function):
    @staticmethod
    def forward(ctx, x, lamda):
        ctx.lamda = lamda
        return x.view_as(x)

    @staticmethod
    def backward(ctx, grad_output):
        output = grad_output.neg() * ctx.lamda
        return output, None

</code></pre>