Problem with extracting the feature

mathwseg: <p>I need to extract features for the images before classifying them… or removing the last layer for the classification model using vit-PyTorch</p>
<aside class="onebox githubblob" data-onebox-src="https://github.com/lucidrains/vit-pytorch/blob/main/vit_pytorch/max_vit.py">
  <header class="source">

      <a href="https://github.com/lucidrains/vit-pytorch/blob/main/vit_pytorch/max_vit.py" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/lucidrains/vit-pytorch/blob/main/vit_pytorch/max_vit.py" target="_blank" rel="noopener nofollow ugc">lucidrains/vit-pytorch/blob/main/vit_pytorch/max_vit.py</a></h4>


      <pre><code class="lang-py">from functools import partial

import torch
from torch import nn, einsum

from einops import rearrange, repeat
from einops.layers.torch import Rearrange, Reduce

# helpers

def exists(val):
    return val is not None

def default(val, d):
    return val if exists(val) else d

def cast_tuple(val, length = 1):
    return val if isinstance(val, tuple) else ((val,) * length)

# helper classes
</code></pre>



  This file has been truncated. <a href="https://github.com/lucidrains/vit-pytorch/blob/main/vit_pytorch/max_vit.py" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>I tried to ignore the classification layer by<br>
<code>        self.mlp_head = nn.Identity()</code></p>
<p>then do that code</p>
<pre><code class="lang-auto">from max_vit import MaxViT
from extractor import Extractor
model = MaxViT(
    num_classes = 0,
    dim = 192,                        
    depth = (2, 6, 14, 2)
    )          
feature= model(img)
</code></pre>
<p>but got the shape for each image <code>torch.Size([1, 1536, 7, 7])</code><br>
and got the file size 300MB for 1000 images ! … is there anything wrong with the code, please</p>

mathwseg: <p>can someone help please</p>

ptrblck: <p>The feature dimension looks reasonable as the <code>mlp_head</code> would not be applied as seen <a href="https://github.com/lucidrains/vit-pytorch/blob/f86e052c053f867b6556bf0ee2481314eb46ba91/vit_pytorch/max_vit.py#L276-L280">here</a>. I haven’t checked the expected shape as the <code>einops</code> reductions don’t show the actual values, but I would expect to see this or a similar shape.<br>
Could you explain what the issue is or why this shape would not be expected?</p>

mathwseg: <p>is there any help please?</p>

ptrblck: <p>It’s unclear to me where you are currently stuck. You didn’t follow up from my previous post but are asking for help again so I guess you are hitting a different issue?</p>

mathwseg: <p>I wrote more details about the problem but after time I feel disappointed to find a solution so I deleted it … if you can help I post it again … hope to find help … Thanks <a href="https://discuss.pytorch.org/t/problem-with-extracting-the-feature/157323/4" class="inline-onebox">Problem with extracting the feature - #4 by mathwseg</a></p>

ptrblck: <p>Your model seems to be overfitting on the training set and I don’t think that your feature extraction is necessarily wrong.<br>
Overfitting can have different reasons, e.g. the model capacity might be too large for the given data and your model is thus able to easily learn all training samples.</p>

mathwseg: <p>doe the code i wrote for extracting the feature is right, please?</p>
<pre><code class="lang-auto">model = MaxViT(
    num_classes = 0,
    dim = 128,                         
    depth = (2, 6, 14, 2) 
    )

model.mlp_head = model.mlp_head[0]
</code></pre>
<p>from this class <a href="https://github.com/lucidrains/vit-pytorch/blob/main/vit_pytorch/max_vit.py" class="inline-onebox" rel="noopener nofollow ugc">vit-pytorch/max_vit.py at main · lucidrains/vit-pytorch · GitHub</a></p>

ptrblck: <p>It depends which features you want to use. Initially, you’ve replaced the entire <code>mlp_head</code> with an <code>nn.Identity</code> layer, now you are using the <code>Reduce</code> layer. Both sound reasonable as they are applied before the final linear layer which would act as the classifier.</p>

mathwseg: <p>Appreciate your reply so both ways means the features before classification or there is something i should do it else</p>

ptrblck: <p>I think both are valid approaches to extract the features from the model and you would need to check which ones would work better for your use case.</p>