Extract features from CNN

Mike2004: <p>Hello everyone, I’m doing a research project and I have a CNN model already trained. Now I want to extract features from this CNN to apply conventional Machine Learning algorithms.</p>
<p>I have saved the CNN:</p>
<pre><code class="lang-python">torch.save(model.state_dict(), './cnn.pth')
</code></pre>
<p>Now, how do I extract features from this model, to apply conventional ML algorithms?</p>
<p>Thanks.</p>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a> any idea?</p>

caonv: <p>It’s straight forward. Let’s assume that your model contains two part: <code>model.extractor</code> and <code>model.classifier</code>.<br>
Just pass an input batch to <code>model.extractor</code>, and you’ll get the desired features.</p>

Mike2004: <p>Sorry, I’m very noobie in this yet, so I didn´t understand what you are suggesting.</p>
<pre><code class="lang-auto">model = CNN()

model.load_state_dict(torch.load(path))
</code></pre>
<p>To extract the features, what do you suggest to do after load the model?</p>

caonv: <p>Let’s assume that your <code>CNN</code> class looks like this:</p>
<pre><code class="lang-auto">class CNN(nn.Module): 
    def __init__(self, *args, **kwargs):
        super(CNN, self).__init__()
        self.feature_extractor = ... # CNN layers or whatever
        self.classifier = .... # Linear layers
    
    def forward(self, x):
        x = self.feature_extractor(x)
        x = self.classifier(x.view(x.shape[0], -1) # depend on your architecture
        return x
</code></pre>
<p>After loading your trained state dict, you can extract the feature from an input by just <code>features = model.feature_extractor(input)</code>.</p>

Mike2004: <p>Thank you, that’s what I was looking for.</p>

Mike2004: <p>I already did what you have suggested.</p>
<pre><code class="lang-auto">def forward(self, x):
        out = self.cnn1(x)
        out = self.relu1(out)
       
        out = self.maxpool1(out)
       
        out = self.cnn2(out)
        out = self.relu2(out)

        out = self.maxpool2(out)

        out = out.view(out.size(0), -1)

        out = self.fc1(out)
       
        return out
</code></pre>
<p>This is my architecture. Now (using what you have suggested), I will extract the features from the layer previous to the last (which is what I wanted):</p>
<pre><code class="lang-auto">out = out.view(out.size(0), -1)
</code></pre>
<p>The output shape is: torch.Size([1, 1568])<br>
Converting to numpy array: (1, 1568)</p>
<p>Using this features, do you have any idea of what can I do now to apply traditional ML methods and get an output of ( -918.5343,  -532.8511,  -771.1676,  -722.4738,  -863.3235,  -838.9160,<br>
-1015.9191,  -593.9412,  -680.5461,  -670.0557) (Class Activations).</p>
<p>How can I apply this array of (1,1568) to get a 10 class output?</p>
<p>Thank you for your answers and time <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

caonv: <p>Why don’t you directly use an additional Linear layer to produce your desired outputs.</p>

Mike2004: <p>I did that, now I want to apply ML traditional techniques, just to compare the performance of methodologies totally based on CNNs. But I’m not sure how to use (1, 1568) features, to get an 10 class output using ML methodologies.<br>
I’m not sure if I am expressing myself correctly.</p>

caonv: <p>As far as I know, you should save all the outputs into a numpy array, then use that array as data for a machine learning model in scikit-learn or other libraries.</p>

Mike2004: <p>That’s what I did so far, I just don’t understand how will I get a 10 class output with 1568 input… But I will search more, if I found a solution, I will put here.<br>
In time, if anyone knows what I should do, you are more than welcome to give some tips. I think that with one concrete example I will be able to apply more techniques and compare performances! Thank you <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

albanD: <p>I think it all depends which “ML traditional techniques” you want to use. And if you use a reference implementation, from which lib you get that implementation.<br>
But as <a class="mention" href="/u/caonv">@caonv</a> said, you are most likely going for scikit-learn like APIs, you will have to extract all the features in one big Tensor (or numpy array) and then feed that to scikit learn’s API</p>

Mike2004: <p>Thank you, I think I’m understading what you are saying. In this case, I have a CNN already trained with MNIST.<br>
Now, if I pass an image as input to the trained model, the model will give me the activations of the classes (MNIST = 10 activations). But what I really want now is, with this information (which I already converted from tensor to numpy array):</p>
<pre><code class="lang-auto">out = out.view(out.size(0), -1)
</code></pre>
<p>For example, apply ridge regression (<a href="https://scikit-learn.org/stable/modules/linear_model.html#ridge-regression-and-classification" class="inline-onebox" rel="noopener nofollow ugc">1.1. Linear Models — scikit-learn 0.24.0 documentation</a>) to the array with this features extracted from the image, with shape = (1,1568). I just have to do this?</p>
<pre><code class="lang-auto">from sklearn import linear_model

reg = linear_model.RidgeCV(alphas=np.logspace(-6, 6, 13))

reg.fit(arr)
</code></pre>
<p>arr is the numpy array extracted from the layer previous to the last.<br>
I don’t know if I made myself clear, but if you want I can provide more code in order to you understand better what I’m trying to say <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> .</p>

albanD: <p>This looks like the right thing to do yes.</p>

Mike2004: <p>Ok, very nice. It gives me this</p>
<pre><code class="lang-auto">TypeError: fit() missing 1 required positional argument: 'y'
</code></pre>
<p>y should be the label right?</p>

albanD: <p>It depends on which function you are using. For the RidgeCV from sklearn, I think it is the label yes. But you should check the functions doc to know what the argument is expected to contain.</p>

Mike2004: <p>Alright, thank you! I will try other implementations of RidgeCV or other methods.</p>

Mike2004: <p>Update: I already did it! In the future if someone needs I can provide the code to do this! Thank you all.</p>

caonv: <p>Great to hear that, I think a summary of your key observations will help others to save a lot of time when facing the same situation.</p>