Searching a project in Deep Learning

LsTam91: <p>Hello,</p>
<p>I have to find and realize a project in deep learning for my master degree, but I don’t know what to choose. We will have 20 days to realize it, and my partner and I are beginner in deep learning, we have already done a project in image classification using transfer learning. Do you have some idea for interesting beginner project?</p>

Ivan_A: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> <a class="mention" href="/u/alband">@albanD</a> <a class="mention" href="/u/deeply">@Deeply</a><br>
Hello, I hope can they help us, we are really interested in hearing your opinion on this topic, I think that will be really illustrative for all of us.</p>

ptrblck: <p>20 days is not a lot of time and I would suggest to use at least an already available (and clean) dataset.<br>
Given this requirement I would then look for projects, which both of you find interesting from a personal point of view, and try to make sure this topic meets the requirements from your university.<br>
Do you have any recommended projects, topics, or any other information or are you completely free to pick your topic for 20 days?</p>

LsTam91: <p>They proposed us to participate to a challenge or take a dataset and test new algortihm on it, but we should bring a new/interesting approach to the implementation, and we haven’t find good or rechable idea to start with.</p>

ptrblck: <p>I don’t know if this would fit the criteria (so you should make sure your advisor would be OK with it), but you could take a look at (past) competitions from e.g. <a href="https://www.drivendata.org/">DrivenData</a> and see if new approaches could yield any new results/findings.</p>

LsTam91: <p>Thanks, I will take a look</p>

Deeply: <aside class="quote no-group" data-username="LsTam91" data-post="1" data-topic="103561">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/lstam91/48/31319_2.png" class="avatar"> LsTam91:</div>
<blockquote>
<p>done a project in image classification using transf</p>
</blockquote>
</aside>
<p>Here is an idea that occurred to me but was not able to try due to lack of time. I’m not sure if it has been tackled before! My guess is not, but you can do some search to find out.</p>
<p>We report the classification accuracy in any classification problem. This metric, however, might be deceptive due to lots of reasons; data imbalance, the percentage of incorrectly labeled samples (especially in the test set, if any) and the number of classes (the latter would be interesting to tackle).</p>
<p>Now, the simplest problems to consider are CIFAR-10 and CIFAR-100.  I think the best reported performance is by <a href="https://paperswithcode.com/paper/sharpness-aware-minimization-for-efficiently-1" rel="noopener nofollow ugc">this work</a> and the code is available in PyTorch (EffNet-L2 achieved 99.70% and 96.08 on CIFAR-10 and CIFAR-100, respectively).<br>
<strong>The research question</strong>: Is EffNet-L2 doing a better job at CIFAR-10 or CIFAR-100?</p>
<p>The problem hence is how to statistically quantify the Model Performance when the number of classes/categories increases? This should be an additional metric that complements the classification accuracy. One dilemma that might affect our judgement is the 10% of samples (which I consider a bit low) used in the testing phase, as it might affect model generalization.</p>