Image segmentation advice

Rick1989: <p>Hi all,</p>
<p>I’m very new to anything related to Machine Learning/AI etc and hope to get some advice.<br>
I’m currently working on a small program with the goal to identify particles on a background substrate.</p>
<p>The images are all gray scale, unfortunately, easier methods that only involve thresholding, canny edge, or hough transforms don’t provide results that are good enough. Therefore, I’m currently using these methods for a first approximation to assist in drawing masks over these particles.</p>
<p>My question is: would you advice on using the original image + masks of let’s assume 1-20 particles in each image. Or should I use cropped-out version with or withour padding to ensure enough background information is present as well? The current program I have apparently accepts both, but I’m not sure if mixing these would create new problems.</p>

KFrank: <p>Hi Rick!</p>
<aside class="quote no-group quote-modified" data-username="Rick1989" data-post="1" data-topic="218616">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rick1989/48/76489_2.png" class="avatar"> Rick1989:</div>
<blockquote>
<p>I’m currently working on a small program with the goal to identify particles on a background substrate.<br>
…<br>
I’m currently using these methods for a first approximation to assist in drawing masks over these particles.</p>
</blockquote>
</aside>
<p>I’m not sure what you’re asking.</p>
<p>What is your use case?  Are you intending to train a model to identify your particles?  Are<br>
you asking how to better use non-machine-learning techniques to perform semi-automated<br>
labelling of the ground-truth annotated data you need for training?</p>
<p>Best.</p>
<p>K. Frank</p>

Rick1989: <p>Dear K Frank,</p>
<p>I’m intending to train a model to identify the particles.<br>
My main question is, what would be the best input data to train the model?</p>
<p>Best,</p>
<p>Rick</p>

Ayush_Saini: <p>can you explain your both approaches in detail with images of both?<br>
I understand the first approach but not the second one.</p>

KFrank: <p>Hi Rick!</p>
<aside class="quote no-group" data-username="Rick1989" data-post="3" data-topic="218616">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rick1989/48/76489_2.png" class="avatar"> Rick1989:</div>
<blockquote>
<p>I’m intending to train a model to identify the particles.<br>
My main question is, what would be the best input data to train the model?</p>
</blockquote>
</aside>
<p>You haven’t said much about your use case, so I will have to speculate to give a<br>
concrete answer.</p>
<p>Let me assume that your individual particles are separated from one another, that is,<br>
they don’t overlap.  If so, you could use <em>semantic segmentation.</em>  This means that you<br>
would classify each pixel as particle vs. background.  If you then perform <em>connected<br>
components</em> on the particle pixels, each connected component will be a particle,</p>
<p>Let me assume that your particles are smallish so that each particle will be able to fit in<br>
the <em>field of view</em> of a reasonably-sized U-Net.  Then you can use a U-Net – a workhorse<br>
for semantic segmentation – as your model.</p>
<p>For training data, you want training images that reasonably represent the images that<br>
you will use your trained model to segment.  (Ideally, you would take your training images<br>
from the same population that you will apply your model to.)  You then need annotated<br>
ground-truth mask images.  These images will be the same shape as the input training<br>
images, but with each pixel labelled as either “background” or “particle.”  The simplest<br>
approach – but the most labor intensive – will be to annotate your training data by hand.<br>
In general, the more training data you have, the better.  (Unless your training images<br>
are very large and contain a lot of particles, you will need quite a few annotated images<br>
to train your model effectively.)</p>
<p>If this is a practical problem and you don’t have any requirement that you train your own<br>
model, you could look into using a pre-packaged solution.  In particular, you might look<br>
at <a href="https://arxiv.org/pdf/2304.02643" rel="noopener nofollow ugc">Segment Anything</a> and its associated <a href="https://segment-anything.com/" rel="noopener nofollow ugc">web site.</a></p>
<p>Good luck!</p>
<p>K. Frank</p>