Model outputs a tuple which does not match the output dimension

username: nimning
cooked: <p>I first load a pretrained Inception V3. I only list the information of last layer to pass the character limit.</p>
<pre><code class="lang-auto">model = models.inception_v3(pretrained = True)
model
</code></pre>
<pre><code class="lang-auto">Inception3(
....
....
....
  (fc): Linear(in_features=2048, out_features=1000, bias=True)
)
</code></pre>
<p>I defined my dataloader as the following</p>
<pre><code class="lang-auto">data_transforms = {
    'training': transforms.Compose([
        transforms.Resize(300),
        transforms.CenterCrop(299),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ]),
    'validation': transforms.Compose([
        transforms.Resize(300),
        transforms.CenterCrop(299),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ]),
}

data_dir = './data/images/'
batch_size = 32

image_datasets = {x: datasets.ImageFolder(os.path.join(data_dir, x),
                                          data_transforms[x])
                  for x in ['training', 'validation']}

dataloaders = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=batch_size,
                                             shuffle=True if x == 'training' else False, num_workers=4) 
               for x in ['training', 'validation']}

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
</code></pre>
<p>There are 8 images in the training folder. So, I try to get the output of model by running</p>
<pre><code class="lang-auto">for inputs, labels in dataloaders['training']:
    outputs = model(inputs)
    print(outputs)
</code></pre>
<p>Then it returns a tuple</p>
<pre><code class="lang-auto">(tensor([[-0.5366, -0.4652,  0.3395,  ...,  0.9199, -0.8712,  1.5646],
        [ 0.5458,  0.2556,  1.4850,  ..., -0.5150,  0.1431, -0.3381],
        [ 0.7030,  0.1582,  0.5104,  ..., -1.6457,  0.8637, -0.1510],
        ...,
        [-0.0278,  0.3151,  0.1143,  ..., -1.6702,  1.7972,  1.3774],
        [-0.3401,  0.3745, -0.6404,  ...,  2.6012, -0.2325,  0.6556],
        [ 1.5351, -0.8767, -0.4649,  ..., -1.0163,  0.0899, -2.0677]],
       grad_fn=&lt;ThAddmmBackward&gt;), tensor([[-0.5303, -0.6241,  2.4938,  ..., -1.4214, -3.4095,  0.9604],
        [ 0.5581,  0.4177,  0.4887,  ...,  1.8613,  1.5896, -1.2584],
        [-0.1217, -0.8219, -1.8897,  ..., -1.4574,  1.5608,  1.4512],
        ...,
        [-0.1217, -0.8219, -1.8897,  ..., -1.4574,  1.5608,  1.4512],
        [-0.6349, -1.2075, -0.7198,  ...,  2.6083, -0.1592, -0.8818],
        [ 0.0503,  1.1492, -0.8055,  ..., -0.6040,  0.5480, -2.1857]],
       grad_fn=&lt;ThAddmmBackward&gt;))
</code></pre>
<p>Then, I replace the last full connected layer with a new layer</p>
<pre><code class="lang-auto">num_ftrs = model.fc.in_features
model.fc = nn.Linear(num_ftrs, 2)
model
</code></pre>
<p>I get an Inception-V3 with</p>
<pre><code class="lang-auto">Inception3(
......
......
......
  (fc): Linear(in_features=2048, out_features=2, bias=True)
)
</code></pre>
<p>When I run the model, it gives a tuple. However, only the first part has output with dimension 2 the other has dimension 1000. It seems very weird as the new model’s output layer has output dimension 2. Why I will get an output with old dimension 1000?</p>
<pre><code class="lang-auto">for inputs, labels in dataloaders['training']:
    outputs = model(inputs)
    print(outputs)
</code></pre>
<pre><code class="lang-auto">(tensor([[ 0.2003, -0.2622],
        [ 0.2755,  0.4227],
        [ 0.3666,  0.3791],
        [ 0.1673,  0.1446],
        [-0.0471, -0.0233],
        [ 0.3193,  0.0048],
        [ 0.2377,  0.2360],
        [ 0.1462,  0.5231]], grad_fn=&lt;ThAddmmBackward&gt;), tensor([[-1.1838,  1.5070,  0.9930,  ..., -0.8319,  0.4863,  3.8230],
        [-0.4090, -0.7720, -1.1981,  ..., -1.3736,  1.3311,  1.1581],
        [-0.4724, -1.0721, -0.5711,  ...,  2.1945, -0.4901, -0.6309],
        ...,
        [ 0.5163,  0.3829,  0.6088,  ...,  1.7575,  1.4754, -1.1107],
        [ 0.9938, -0.9067, -2.3349,  ..., -0.8603,  1.8640,  0.2998],
        [-0.3151, -0.3183,  2.8335,  ..., -0.7222, -2.9774,  0.7192]],
       grad_fn=&lt;ThAddmmBackward&gt;))
</code></pre>

username: kaixin
cooked: <p>Hi,<br>
I tried with a random input tensor (shape <code>(8, 3, 299, 299)</code>), it works fine and outputs a tensor with shape <code>(8, 2)</code>. Could you give more information about your inputs?</p>

username: nimning
cooked: <p>Hi, Kaixin</p>
<p>My input also has shape (8,3,299,299).  I look at the source code of Inception-V3 implementation and find that the Inception-V3 seems returning a tuple as output. See <a href="https://github.com/pytorch/vision/blob/master/torchvision/models/inception.py" rel="nofollow noopener">Inception-V3 pytorch</a></p>

username: kaixin
cooked: <p>I see why I got a tensor output. I set the model to <code>eval</code> mode.<br>
The source file is pretty clear, I assume you have figured out what the output tuple contains are and how to disable outputting tuple.</p>
<p>Thanks.</p>

username: nimning
cooked: <p>Yes, thanks! I have figured out.</p>

username: Kaiwkh
cooked: <p>Hi Kaixin,<br>
I met the exact same problem but I don’t understand how to set model to eval mode or how to disable outputting tuple. Would you please explain more about it?</p>