Is it possible to recognize images with very large intra-class variation(such as one species within different life cycles)?

Hoodythree: <p>Is it possible to recognize images with very large intra-class variation(such as one species within different life cycles)?<br>
If it’s possible, then what CNN learned from this.<br>
Thanks in advance.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/3/63abed76313cb096239ee00717da8246c539d2ac.png" data-download-href="https://discuss.pytorch.org/uploads/default/63abed76313cb096239ee00717da8246c539d2ac" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/3/63abed76313cb096239ee00717da8246c539d2ac_2_441x375.png" alt="image" data-base62-sha1="edJCBL14XRqsbiOXeC8jcV8dQPa" width="441" height="375" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/3/63abed76313cb096239ee00717da8246c539d2ac_2_441x375.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/3/63abed76313cb096239ee00717da8246c539d2ac_2_661x562.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/3/63abed76313cb096239ee00717da8246c539d2ac.png 2x" data-dominant-color="C6CFBB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">846×718 422 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

KFrank: <p>Hello Hoody!</p>
<aside class="quote no-group" data-username="Hoodythree" data-post="1" data-topic="95523" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hoodythree/48/24384_2.png" class="avatar"> Hoodythree:</div>
<blockquote>
<p>Is it possible to recognize images with very large intra-class variation(such as one species within different life cycles)?</p>
</blockquote>
</aside>
<p>Yes, certainly (given good training data, of course).</p>
<p>As an example, consider caterpillars / butterflies.  Let’s say you have<br>
monarch caterpillars and  butterflies and swallowtail caterpillars and<br>
butterflies.</p>
<p>It would be easy to train a <em>four-class</em> classifier – monarch caterpillar,<br>
monarch butterfly, swallowtail caterpillar, swallowtail butterfly.  So all<br>
the network has to do is “learn” to collapse each pair of caterpillar /<br>
butterfly classes together to form a two-class monarch / swallowtail<br>
classifier.</p>
<p>When you train your two-class monarch / swallowtail classifier, I don’t<br>
think that the butterfly training data is really going to help with the<br>
caterpillar classification (and vice versa) (Well, actually in this example,<br>
there are some coloration clues, but ignore that …), so, under the hood,<br>
the network is, more or less, learning four classes.</p>
<p>That is, after training, the four classes are largely represented as “latent<br>
features” inside of the network.  To give some concrete meaning to this<br>
verbiage, I bet that, starting with your well-trained monarch / swallowtail<br>
classifier, you could, with only a little bit of “fine-tuning” training, produce<br>
the four-class classifier mentioned above or a two-class caterpillar /<br>
butterfly classifier.</p>
<p>Or to say this a slightly different way, let’s say you train your monarch /<br>
swallowtail classifier with lots butterfly data, but hardly any caterpillar<br>
data.  Your classifier will work well on butterfly images, but will hardly<br>
work at all on caterpillar images.  Again, under the hood, you’re training<br>
a four-class classifier, and you didn’t give it enough caterpillar data to<br>
learn the caterpillar classes.</p>
<p>Best.</p>
<p>K. Frank</p>

Hoodythree: <aside class="quote no-group" data-username="KFrank" data-post="2" data-topic="95523">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>Frank</p>
</blockquote>
</aside>
<p>Hi, Frank, thanks for your patience and kindness. Sorry for making my question confusing. Actually I mean treating these four classes in the above image(a) as one, cause they belongs to the same category semantically.</p>

KFrank: <p>Hello Hoody!</p>
<aside class="quote no-group" data-username="Hoodythree" data-post="3" data-topic="95523" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hoodythree/48/24384_2.png" class="avatar"> Hoodythree:</div>
<blockquote>
<p>Actually I mean treating these four classes in the above image(a) as one, cause they belongs to the same category semantically.</p>
</blockquote>
</aside>
<p>I believe that I did answer the question you asked (with a somewhat<br>
simplified example).</p>
<aside class="quote no-group" data-username="Hoodythree" data-post="1" data-topic="95523">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hoodythree/48/24384_2.png" class="avatar"> Hoodythree:</div>
<blockquote>
<p>Is it possible to recognize images with very large intra-class variation</p>
</blockquote>
</aside>
<p>In my example, a class showing “very large intra-class variation” would<br>
be “monarch”, with the intra-class variation being caterpillar vs. butterfly.<br>
(“Swallowtail” would be another class showing large intra-class variation<br>
for the same reason.)</p>
<aside class="quote no-group" data-username="Hoodythree" data-post="1" data-topic="95523">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hoodythree/48/24384_2.png" class="avatar"> Hoodythree:</div>
<blockquote>
<p>(such as one species within different life cycles)?</p>
</blockquote>
</aside>
<p>To be very specific, in my example “one species” would be monarch,<br>
and the “different life cycles” would be caterpillar and butterfly.</p>
<p>(If your asking about building a binary classifier that recognizes<br>
“monarch” vs. “not monarch” the logic of my previous post still applies.<br>
You can imagine training a three-class classifier, “monarch caterpillar”<br>
“monarch butterfly”, “neither”, that then learns to collapse the two<br>
classes “monarch caterpillar” and “monarch butterfly” into a single<br>
“monarch” class, leaving you with a two-class (binary) “monarch”<br>
vs. “not monarch” classifier.)</p>
<p>Best.</p>
<p>K. Frank</p>