Which pre-trained model to be used for Binary Image document classifier?

username: Aarya
cooked: <p>I want to train a model ( <strong>Binary classifier</strong>) which should classify the image documents into respective classes like ( child-optic / not-child-optic ).</p>
<p><strong>Dataset</strong> : <strong>Image documents</strong> (child optic data)</p>
<p>Which pre-trained model can be used in this case ? and where can i find it ?</p>

username: islomjon
cooked: <p>Hello Aarya, can you elaborate more about your dataset?</p>
<p>Maybe my answer could be redundant, but so far, you can use pretrained model with transfer learning, for example ResNet. An example you can find here: <a href="https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Transfer Learning for Computer Vision Tutorial — PyTorch Tutorials 1.12.1+cu102 documentation</a></p>

username: Aarya
cooked: <p>Thank you for your response <a class="mention" href="/u/islomjon">@islomjon</a><br>
Dataset : Binary classes:<br>
KO_false - 1800 images<br>
KO_true - 1600 images</p>
<p>Should the dataset be balanced, so should be fine ?</p>

username: Aarya
cooked: <p><a class="mention" href="/u/islomjon">@islomjon</a> Which one do you think will best suit my situation with the dataset ?</p>
<p>These two major transfer learning scenarios look as follows:</p>
<ul>
<li>
<strong>Finetuning the convnet</strong>: Instead of random initialization, we initialize the network with a pretrained network, like the one that is trained on imagenet 1000 dataset. Rest of the training looks as usual.</li>
<li>
<strong>ConvNet as fixed feature extractor</strong>: Here, we will freeze the weights for all of the network except that of the final fully connected layer. This last fully connected layer is replaced with a new one with random weights and only this layer is trained.</li>
</ul>

username: srishti-git1110
cooked: <p><a class="mention" href="/u/aarya">@Aarya</a> Both approaches are fine to use, but in my experience (in both vision and NLP domains) finetuning the whole network (Scenario 1) outperforms the other way where only the top layer is fine-tuned. However, more resources in terms of time and cost are required in the former.<br>
Reason is kind of obvious - a larger no. of weights get fine-tuned according to the dataset at hand in the former.</p>
<p>Some pointers -<br>
Generally, 2nd scenario works well only when the dataset at hand comes from atleast a similar domain as the dataset that was used to pre-train the network.<br>
Remember to work with a decreased learning rate here.</p>
<p>In case, domains are entirely different and you need to do what is called “domain adaptation” according to your dataset, 1st scenario should give better results.</p>
<p>Also note the fact that even for different domains, the layers close to the input layer have been shown to learn nearly the same set of (low-level) features and this knowledge is hence transferable across domains. So, you can maybe put it like – The no.of layers to keep frozen is inversely related to the similarity between the dataset used to pre-train &amp; the dataset at hand.</p>
<p>Additionally, other factors like the size of dataset might also play a role in which scenario to choose.</p>
<p>Hope this helps.</p>