Torch.view() dimensions constraint

username: alex_gilabert
cooked: <p>Hello</p>
<p>I am  doing some research on CNN and i want to extract manually features through a function.</p>
<p>The  problem i am facing is that i want to change the dimension of the output tensor of an image batch to treat it in a more lineal way.</p>
<p>I am using VGG16 and AlexNet. In VGG16 in the Conv_5_3 we receive a matrix with dimensions (10,512,14,14). I want to keep the first dimension for  the number of images that are in the minibatch and want a result like (10, 100352). For this i have no issue.</p>
<p>If i want to go to Conv4_3 for example, the matrix that i want to convert is (10, 512, 28 28) what would make it a matrix (10, 4014080) but i get an error message like this “shape ‘[10, 4014080]’ is invalid for input of size 4014080”.</p>
<p>I am pretty sure that its because it goes over the limit of dimensions. But i can not asure this and have no clue what is really going on.</p>
<p>If this is the problem, anyone knows a way that i can change this matrix dimensions for the code below???</p>
<pre><code class="lang-auto">net = models.alexnet(pretrained=True)
num_features = net.classifier[6].in_features      #4096
features = list(net.classifier.children())[:-2] # Remove last layer
print(features)
net_.classifier = nn.Sequential(*features) # Replace the model classifier
modified_pretrained = nn.Sequential(*list(net.features.children())[:-2]) # to relu5_3 (:-2)-------relu4_3 (:-9)

def feature_extr_cdist_train (model, dataloader):
    features_list = torch.empty( (0, 432640), dtype=torch.float ).cuda()  #Habrá que cambiarlo en funcion de las características #1000 #4096 #100352 #4014080   AlexNet 432640
    image_album = torch.empty( [1, 3, 224, 224], dtype=torch.float ).cuda()
    counter = 0  
    model.cuda()
    for i, data in enumerate(dataloader, 0):
        input, label = data                                             
        input, label = input.to(device), label.to(device)               
        n,c ,h,w = input.size()
        outputs = model(input)
        outputs = outputs.view(10, 432640)   #8028160
        

        if (i == 0):                                                    
            features_list = torch.cat( (features_list, outputs[0].view(1,432640)), 0) #view(1,-1)
            image_album[0] = input[0]
        
        dist_tensores = torch.cdist(outputs, features_list, p=2.0) 
        activation = torch.gt(dist_tensores, AVG, out=torch.cuda.FloatTensor(len(outputs), len(features_list)))
        counter = len(features_list)
        idx = activation.sum(1) == counter
        features_list = torch.cat((features_list, outputs[idx]), 0)
        image_album = torch.cat((image_album, input[idx,:,:,:]), 0)

    return features_list, image_album

</code></pre>
<p>My issue is in the part i associate the input with the output of the CNN and the conversion with view().</p>
<p>Thanks in advance</p>

username: Nikronic
cooked: <p>Hi,</p>
<p>Could you add stack trace of error? The dimensions you have mentioned in your explanation do not exist in your code so I cannot find where it happens.</p>
<p>But I would like to mention that</p>
<aside class="quote no-group" data-username="alex_gilabert" data-post="1" data-topic="88044">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alex_gilabert/48/25808_2.png" class="avatar"> alex_gilabert:</div>
<blockquote>
<p>(10, 512, 28 28) what would make it a matrix (10, 4014080) but i get an error message like this “shape ‘[10, 4014080]’ is invalid for input of size 4014080”.</p>
</blockquote>
</aside>
<p><code>512*28*28 = 401408</code> and <code>10*512*28*28 = 4014080</code> (note the zero at the end). So the error says that you cannot reshape <code>4014080</code> into <code>[10, 4014080]</code>.</p>
<p>Bests</p>

username: alex_gilabert
cooked: <p>Yes sure!  Tou were right, there was a mistake writing the issue. Hope this helps you more <a class="mention" href="/u/nikronic">@Nikronic</a> !</p>
<p>Output before redimension torch.Size([10, 256, 13, 13])<br>
RuntimeError                              Traceback (most recent call last)<br>
 in ()<br>
2 for param in modified_pretrained.parameters():<br>
3    param.requires_grad = False<br>
----&gt; 4 (tensor_train, image_album) = feature_extr_cdist_train(modified_pretrained, load_train)</p>
<p> in feature_extr_cdist_train(model, dataloader)<br>
14         print(“Output before redimension”,outputs.size())<br>
15         <span class="hashtag">#print</span>(“antes”, outputs.size())<br>
—&gt; 16         outputs = outputs.view(10, 432640)   <span class="hashtag">#8028160</span><br>
17<br>
18         <span class="hashtag">#print</span>(outputs.size())</p>
<p>RuntimeError: shape ‘[10, 432640]’ is invalid for input of size 432640</p>

username: Nikronic
cooked: <p>I think you have same issue hero.<br>
Are your feature maps in size <code>[10, 208, 208]</code>? In this case again you have multiplied by 10 so you need to use</p>
<pre><code class="lang-auto">outputs = outputs.view(10, 43264) #8028160
</code></pre>
<p>instead of:</p>
<aside class="quote no-group" data-username="alex_gilabert" data-post="3" data-topic="88044">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alex_gilabert/48/25808_2.png" class="avatar"> alex_gilabert:</div>
<blockquote>
<p>—&gt; 16 outputs = outputs.view(10, 432640) <span class="hashtag-raw">#8028160</span></p>
</blockquote>
</aside>