Why is my generator model M_gen not training when optimizing based on the classifier model M_cls?

Brayn_O_Conner: <p>I’m trying to train a generator model <code>M_gen</code> whose output serves as input for a classifier model <code>M_cls</code>. However, <code>M_cls</code> is also trained with additional data that doesn’t come from <code>M_gen</code>. To handle this, I combine both datasets into a single <code>DataLoader</code>. The issue is that <code>M_gen</code> doesn’t seem to be training properly, the evaluation metric <code>avg_f1</code> remains the same across all epochs. This is my code to train <code>M_gen</code>:</p>
<pre><code class="lang-auto">while epoch &lt;= num_epochs:
    for phase in ['train', 'val']:
        print(phase + "-Phase")
        if phase == 'train':
            model.train()  # Set model to training mode
            dataloader = train_dataloader
        else:
            model.eval()  # Set model to evaluate mode
            dataloader = val_dataloader

        
        generated_data_list = []
        for idx, inputs in tqdm(enumerate(dataloader), total=len(dataloader)):
            inputs = inputs.to(device).float()

            regression_output, classification_output = model(inputs)
            generated_data_list.append((regression_output, classification_output))

        f1 = main_M_cls.main(seed=2025, generated_data_list=generated_data_list)
        loss = torch.tensor(100-(f1*100), requires_grad=True, device=device)

        if phase == 'train':
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
</code></pre>
<p>In main_M_cls.main() I merge the samples from <code>M_gen</code> and the additional data saved in <code>train_dataset</code></p>
<pre><code class="lang-auto">dataset2 = [(x1, y1) for (regression_output, classification_output) in generated_data_list for x1, y1 in zip(regression_output, classification_output)]
train_merged_dataset = ConcatDataset([train_dataset, dataset2])
train_loader = DataLoader(train_merged_dataset, batch_size=batch_size, shuffle=True)
</code></pre>
<p>Then I start the training and evaluation of <code>M_cls</code>:</p>
<pre><code class="lang-auto">for epoch in range(3):
    train_loss, val_loss = [], []

    # Start Training
    model.train()
    for i, (sample, label) in tqdm(enumerate(train_loader), total=len(train_loader)):
        sample = sample.to(device=device)
        label = label.to(device=device)

        output = model(sample)
        loss = criterion(output, label)

        optimizer.zero_grad()
        loss.backward(retain_graph=True)
        optimizer.step()
        
    running_val_loss = 0

    # Start Validation
    model.eval()
    true_labels, pred_labels = [], []
    for i, (sample, label) in enumerate(val_loader):
        sample = sample.to(device=device, dtype=torch.float)
        label = label.to(device=device, dtype=torch.long)

        output = model(sample)
        loss = criterion(output, label)

        true_labels.append(vote_label.detach().cpu().numpy())
        pred_labels.append(output.detach().cpu().numpy())

   
    y_true = np.concatenate(true_labels, axis=0)
    y_prob = np.concatenate(pred_labels, axis=0)
    y_pred = np.argmax(y_prob, axis=1)
    val_f1 = f1_score(y_true, y_pred, average='macro')    # this function comes from sklearn.metrics
    return val_f1
</code></pre>
<p>I first thought that the f1-Score might be a problem, because the computation graph might not be able to compute and backpropagate the gradients, since the f1-Score is a Python scaler. However, I also tested my code by using the validation loss instead of f1-Score but still, the model <code>M_gen</code> is not learning anything, the losses for <code>M_gen</code> are always the same.</p>
<p>I appreciate any help or suggestions. Thanks in advance!</p>

KFrank: <p>Hi Brayn!</p>
<aside class="quote no-group" data-username="Brayn_O_Conner" data-post="1" data-topic="216263" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/brayn_o_conner/48/75477_2.png" class="avatar"> Brayn_O_Conner:</div>
<blockquote>
<pre><code class="lang-auto">        loss = torch.tensor(100-(f1*100), requires_grad=True, device=device)
</code></pre>
</blockquote>
</aside>
<p><code>torch.tensor()</code> creates a new tensor that is not connected to any previous<br>
computation graph.  Calling it with <code>requires_grad = True</code> doesn’t change that.<br>
(The new tensor would then be a leaf of a new branch of a computation graph.)</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">   loss.backward()
</code></pre>
</blockquote>
</aside>
<p>So this call to <code>.backward()</code> doesn’t actually backpropagate anything.</p>
<p>More generally, I can’t really follow the details of what you are doing here.  Here<br>
is a <a href="https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html" rel="noopener nofollow ugc">pytorch GAN tutorial</a> that you might find helpful.</p>
<p>Best.</p>
<p>K. Frank</p>

Brayn_O_Conner: <p>Hi Frank,<br>
Thanks for your response and your help! I was already afraid that using <code>torch.tensor(...)</code> wouldn’t work in this case. But what would be a better approach instead?</p>
<p>What I’m trying to do is not a traditional GAN. In a typical GAN setup, we have a generator that produces samples (or a batch of samples) and a discriminator that learns to distinguish between real and generated samples.</p>
<p>However, in my case, the “discriminator” is actually a classifier that I want to improve using the data generated by the generator. To evaluate whether the classifier benefits from the generated data, it needs to be trained to completion, so let’s say for 30 epochs.</p>
<p>What would be the best way to implement this kind of setup in PyTorch? Any suggestions would be greatly appreciated!</p>
<p>Best regards<br>
Brayn</p>

KFrank: <p>Hi Brayn!</p>
<aside class="quote no-group" data-username="Brayn_O_Conner" data-post="3" data-topic="216263" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/brayn_o_conner/48/75477_2.png" class="avatar"> Brayn_O_Conner:</div>
<blockquote>
<p>However, in my case, the “discriminator” is actually a classifier that I want to improve using the data generated by the generator.</p>
</blockquote>
</aside>
<p>I don’t really understand what you are trying to do here.</p>
<p>As an aside, people do sometimes use generated data to train models, so it can<br>
make sense.</p>
<aside class="quote no-group">
<blockquote>
<p>To evaluate whether the classifier benefits from the generated data, it needs to be trained to completion, so let’s say for 30 epochs.</p>
</blockquote>
</aside>
<p>If I understand you correctly, you want to generate data with your generator, use<br>
the generated data to train the classifier, compute a figure of merit for how well<br>
your trained classifier performs, and use that figure of merit as a loss value that<br>
you backpropagate in order to train your generator to generate data that does a<br>
better job of training your classifier.  Is this correct?</p>
<p>The problem is that pytorch (for good reasons, if you dig down into it) does not<br>
support backpropagating through the optimizer step that updates the parameters<br>
of the classifier.  That is, autograd won’t give you the gradients of your figure of<br>
merit with respect to the parameters of the generator.</p>
<p>(Whether something else along the lines of what you want could work would depend<br>
on the details of what you actually want to do.)</p>
<p>Best.</p>
<p>K. Frank</p>

Brayn_O_Conner: <p>Hello Frank,<br>
thanks a lot for your answer.</p>
<blockquote>
<p>If I understand you correctly, you want to generate data with your generator, use<br>
the generated data to train the classifier, compute a figure of merit for how well<br>
your trained classifier performs, and use that figure of merit as a loss value that<br>
you backpropagate in order to train your generator to generate data that does a<br>
better job of training your classifier. Is this correct?</p>
</blockquote>
<p>Yes, that’s exactly what I want to do <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<blockquote>
<p>The problem is that pytorch (for good reasons, if you dig down into it) does not<br>
support backpropagating through the optimizer step that updates the parameters<br>
of the classifier. That is, autograd won’t give you the gradients of your figure of<br>
merit with respect to the parameters of the generator.</p>
</blockquote>
<p>OK, too bad. Then I’ll have to find another way to do this, maybe Reinforcement Learning? Though I’d need to think about that for a while, how this can work. If I understood you correctly, the problem is that the classification model is fully trained (and the gradients get reset every iteration, therefore there are no gradients after the classifier training that could be used), which makes it different from GAN training, right?</p>
<p>Maybe some background on how I came up with this idea: I think I once heard that some companies optimize aircraft turbines by modifying their geometry to improve efficiency, for example. These modified turbines are then tested and evaluated in a simulator.</p>
<p>Now I thought: in my case, a generative model could produce data, and my classifier could take on the role of the simulator. However, my context is entirely different, I’m working with accelerometer data and simply want to generate virtual data to improve classification.</p>
<p>Thanks a lot for your help. Do you have some ideas that I could try?</p>
<p>Best regards<br>
Brayn</p>

KFrank: <p>Hi Brayn!</p>
<aside class="quote no-group quote-modified" data-username="Brayn_O_Conner" data-post="5" data-topic="216263" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/brayn_o_conner/48/75477_2.png" class="avatar"> Brayn_O_Conner:</div>
<blockquote>
<p>the problem is that the classification model is fully trained<br>
…<br>
which makes it different from GAN training, right?</p>
</blockquote>
</aside>
<p>Yes, in this regard what you describe is different than typical GAN training.  If there<br>
were some way you could measure the future benefit to your classifier due a single<br>
sample (or batch) produced by your generator without taking an (official) optimizer<br>
step, you might be able to make your training look enough like a GAN that typical<br>
GAN techniques would  work.</p>
<aside class="quote no-group">
<blockquote>
<p>simply want to generate virtual data to improve classification.<br>
…<br>
Do you have some ideas that I could try?</p>
</blockquote>
</aside>
<p>I don’t really understand your use case.  What it seems like to me is:</p>
<p>You want to use fake (generated) data to train your classifier to do a better job of<br>
classifying …   Fake data?</p>
<p>If so, you could simply have your generator spit out a vector of <code>7.2</code>s.  Then your<br>
classifier could classify it as a <code>7.2</code> sample.  Presumably this is not what you mean.</p>
<p>There must be some interplay between your fake data and the real-world purpose<br>
of your classifier.  It’s the details of that interplay that will determine how or whether<br>
you might train your generator to generate better fake data.</p>
<p>Best.</p>
<p>K. Frank</p>

Brayn_O_Conner: <p>Dear Frank,<br>
thanks again for your answer.</p>
<blockquote>
<p>I don’t really understand your use case. What it seems like to me is:</p>
<p>You want to use fake (generated) data to train your classifier to do a better job of<br>
classifying … Fake data?</p>
<p>If so, you could simply have your generator spit out a vector of <code>7.2</code>s. Then your<br>
classifier could classify it as a <code>7.2</code> sample. Presumably this is not what you mean.</p>
<p>There must be some interplay between your fake data and the real-world purpose<br>
of your classifier. It’s the details of that interplay that will determine how or whether<br>
you might train your generator to generate better fake data.</p>
</blockquote>
<p>Oh, I’m so sorry. I thought I made this clear earlier. We actually have some real IMU data for Human Activity Recognition, but our data is very limited. We train a classifier on this data and achieve an F1-score of, let’s say, 50%. Now, we want to implement a generator that produces synthetic (or fake) data, which we will use alongside some real data to improve the classifier.</p>
<p>Generating <code>7.2</code> samples is probably not very helpful because <code>7.2</code>s might never occur in the real training, validation, or test data. Moreover, the goal is to generate sliding windows of IMU data, including the corresponding label for each window. This means that simply generating a feature matrix of <code>7.2</code>s along with a (random?) label would be most likely useless for improving the classifier.</p>
<p>We have already tried Conditional GANs and VAEs, but neither approach has significantly improved the classifier’s performance. Therefore I just wanted to try a different approach.</p>
<p>I hope this clarifies my use case.</p>
<p>Thanks again for your time and help.</p>
<p>Best regards<br>
Brayn</p>