RuntimeError: bool value of Tensor with more than one value is ambiguous

rajasekhar: <p>I’m trying to incorporate this tutorial: <a href="https://adversarial-ml-tutorial.org/adversarial_examples/" rel="nofollow noopener">https://adversarial-ml-tutorial.org/adversarial_examples/</a></p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.optim as optim
import matplotlib.pyplot as plt
import os


from torchvision import datasets, transforms, models
from torch.utils.data import DataLoader


device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

model = models.googlenet(num_classes=5)

num_ftrs = model.aux1.fc2.in_features
model.aux1.fc2 = nn.Linear(num_ftrs, 5)
        
num_ftrs = model.aux2.fc2.in_features
model.aux2.fc2 = nn.Linear(num_ftrs, 5)
        # Handle the primary net
        #num_ftrs = model_ft.fc.in_features
        #model_ft.fc = nn.Linear(num_ftrs,num_classes)
model.load_state_dict(torch.load("standard_googlenet.pth"))

#phase_transforms = transforms.Compose(transform)
vtype_train = datasets.ImageFolder(os.path.join("data/train"), transforms.Compose([transforms.RandomResizedCrop(224), transforms.RandomHorizontalFlip(), transforms.ToTensor(),  transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])]))
vtype_test = datasets.ImageFolder(os.path.join("data/val"), transforms.Compose([transforms.Resize(224), transforms.CenterCrop(224), transforms.ToTensor(), transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])]))
train_loader = DataLoader(vtype_train, batch_size = 96, shuffle=True)
test_loader = DataLoader(vtype_test, batch_size = 16, shuffle=False)

def fgsm(model, X, y, epsilon):
    """ Construct FGSM adversarial examples on the examples X"""
    delta = torch.zeros_like(X, requires_grad=True)
    loss = nn.CrossEntropyLoss()(model(X + delta), y)
    loss.backward()
    return epsilon * delta.grad.detach().sign()
for X,y in test_loader:
    X,y = X.to(device), y.to(device)
    break
    
def plot_images(X,y,yp,M,N):
    f,ax = plt.subplots(M,N, sharex=True, sharey=True, figsize=(N,M*1.3))
    for i in range(M):
        for j in range(N):
            ax[i][j].imshow(1-X[i*N+j][0].cpu().numpy())
            title = ax[i][j].set_title("Pred: {}".format(yp[i*N+j].max(dim=0)[1]))
            plt.setp(title, color=('g' if yp[i*N+j].max(dim=0)[1] == y[i*N+j] else 'r'))
            ax[i][j].set_axis_off()
    plt.tight_layout()

yp = model(X)

#yp = model_dnn_2(X)
plot_images(X, y, yp, 3, 6)
def epoch_adversarial(model, loader, attack, *args):
    total_loss, total_err = 0.,0.
    for X,y in loader:
        X,y = X.to(device), y.to(device)
        delta = attack(model, X, y, *args)
        yp = model(X+delta)
        loss = nn.CrossEntropyLoss()(yp,y)
        
        total_err += (yp.max(dim=1)[1] != y).sum().item()
        total_loss += loss.item() * X.shape[0]
    return total_err / len(loader.dataset), total_loss / len(loader.dataset)
#print(" GoogLeNet CNN:", epoch_adversarial(model, test_loader, fgsm, 0)[0])
</code></pre>
<p>Got this error, while I’m trying to plot:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "pgd.py", line 55, in &lt;module&gt;
    plot_images(X, y, yp, 3, 6)
  File "pgd.py", line 48, in plot_images
    plt.setp(title, color=('g' if yp[i*N+j].max(dim=0)[1] == y[i*N+j] else 'r'))
RuntimeError: bool value of Tensor with more than one value is ambiguous

</code></pre>
<p>And when I comment the plot proceed to <code>epoch_adversarial</code>, got one more error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "pgd.py", line 67, in &lt;module&gt;
    print(" GoogLeNet CNN:", epoch_adversarial(model, test_loader, fgsm, 0)[0])
  File "pgd.py", line 60, in epoch_adversarial
    delta = attack(model, X, y, *args)
  File "pgd.py", line 35, in fgsm
    loss = nn.CrossEntropyLoss()(model(X + delta), y)
  File "/home/yolo/Documents/raja/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/yolo/Documents/raja/anaconda3/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 904, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "/home/yolo/Documents/raja/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py", line 1970, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/home/yolo/Documents/raja/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py", line 1295, in log_softmax
    ret = input.log_softmax(dim)
AttributeError: 'GoogLeNetOuputs' object has no attribute 'log_softmax'

</code></pre>
<p>corresponding GoogLeNet: <a href="https://github.com/pytorch/vision/blob/master/torchvision/models/googlenet.py" rel="nofollow noopener">https://github.com/pytorch/vision/blob/master/torchvision/models/googlenet.py</a></p>
<p>any thoughts to solve?</p>

rajasekhar: <p>should place <code>model.eval()</code>, then it works.</p>

pinocchio: <p>related posts:</p>
<ul>
<li><a href="https://stackoverflow.com/questions/52946920/bool-value-of-tensor-with-more-than-one-value-is-ambiguous-in-pytorch" rel="nofollow noopener">https://stackoverflow.com/questions/52946920/bool-value-of-tensor-with-more-than-one-value-is-ambiguous-in-pytorch</a></li>
<li><a href="https://discuss.pytorch.org/t/runtimeerror-bool-value-of-variable-objects-containing-non-empty-torch-longtensor-is-ambiguous/7310" class="inline-onebox">RuntimeError: bool value of Variable objects containing non-empty torch.LongTensor is ambiguous</a></li>
<li><a href="https://discuss.pytorch.org/t/runtimeerror-bool-value-of-tensor-with-more-than-one-value-is-ambiguous/43800" class="inline-onebox">RuntimeError: bool value of Tensor with more than one value is ambiguous</a></li>
<li><a href="https://discuss.pytorch.org/t/why-cant-one-pass-data-through-a-torch-relu-module-directly">https://discuss.pytorch.org/t/why-cant-one-pass-data-through-a-torch-relu-module-directly</a></li>
</ul>

pinocchio: <p>why does that weird solution works? Do you know?</p>