Beyond Classification

Sally_Goldin: <p>It appears that the most common task that CNN models perform is classification. Given enough training data, CNNs excel at taking an unknown data item (e.g. an image) and assigning it to one of a set of predefined categories.</p>
<p>I’m looking for pointers, suggestions or guidance for moving beyond classification to a more generalized response based on image features. Specifically, I have images of gravel with a mix of particle sizes. My output is a 5 element vector of values that represent the relative proportion of particles in different size categories. My goal is to use ML to “measure” the proportions of different sizes in unknown images.</p>
<p>I’ve trained a fairly simple CNN (4 conv layers, 3 fully connected) to a quite high level of accuracy - if the input image more or less matches one of the particle size distributions in the training data set. However, if I test an image with a different distribution than any the network has seen during training, the results are typically poor. That is, the results are neither an accurate representation of the true particle distribution (which is what is really desired) nor the closest match from among the distributions that the network has seen during training.</p>
<p>It seems that what my model has learned is basically a classification with a five-element label. <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_frowning_face.png?v=10" title=":slightly_frowning_face:" class="emoji" alt=":slightly_frowning_face:"></p>
<p>Can anyone suggest a way to proceed? With data augmentation, I currently have about 22,000 input images, in 25 different distribution groups. I am considering adding more images, from additional, different distributions, but acquiring this data will be time-consuming. Do you think it’s likely to improve the performance?</p>
<p>I’ve considered transfer learning. For instance, some papers have claimed that ResNet learns many texture features (which I believe is likely the main type of information needed for my task). However, ResNet was designed for classification. Can I somehow adapt it to my more general objectives?</p>
<p>I am a relative newcomer to machine learning, so I apologize in advance if the answer to this question is obvious.</p>
<p>Thanks!</p>

KFrank: <p>Hi Sally!</p>
<aside class="quote no-group" data-username="Sally_Goldin" data-post="1" data-topic="131110" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sally_goldin/48/18108_2.png" class="avatar"> Sally_Goldin:</div>
<blockquote>
<p>I have images of gravel with a mix of particle sizes.</p>
</blockquote>
</aside>
<p>May I ask some clarifying questions about your use case?</p>
<p>Leaving aside your current network and the kind of annotations you<br>
have for your data, ideally, what would you like the output of your<br>
network to be?  Would you like the full probability distribution of<br>
particle sizes – that is, roughly speaking, a histogram of particle sizes<br>
with lots of bins?</p>
<p>Is it important that you use neural networks / machine learning for this<br>
task, or would you be open to using other computer-vision techniques?</p>
<aside class="quote no-group">
<blockquote>
<p>My output is a 5 element vector of values that represent the relative proportion of particles in different size categories.</p>
</blockquote>
</aside>
<p>What, in more detail, is the meaning of these five numbers?  Are your<br>
ground-truth annotations analogous length-5 vectors that have the<br>
same meaning?</p>
<p>What loss function are you using to compare your network outputs<br>
to your annotations?</p>
<p>What does the final layer of your network look like – that is what is<br>
the layer that produces this length-5 output?</p>
<aside class="quote no-group">
<blockquote>
<p>if the input image more or less matches one of the particle size distributions in the training data set. However, if I test an image with a different distribution than any the network has seen during training, the results are typically poor.</p>
</blockquote>
</aside>
<p>Are the test images that your network does poorly on in any way<br>
different in character than the images you used to train your network,<br>
or are they basically the same?  For example, did you randomly<br>
select some test images from the same larger annotated set from<br>
which you selected your training set?</p>
<p>In general, if you are getting significantly worse results on test images<br>
that are of the same character as you training images, you are seeing<br>
signs of <em>overfitting.</em>  That is, your network has been trained enough<br>
times on the same images that in addition to “learning” the properties<br>
you want it to learn, it is also learning recognize individual images<br>
by their individual peculiarities.  Having recognized a specific image,<br>
the network now “parrots back” the “right” answer.</p>
<p>The “rigorous” approach is to split your annotated data set into three<br>
pieces (usually randomly) – a <em>training set,</em> a <em>validation set,</em> and a<br>
<em>test set.</em>  You use your training set to train with backpropagation and<br>
optimizer parameter updates.  At times of your choosing (for example,<br>
once every epoch), you calculate your loss and potentially other<br>
evaluation metrics for your validation set, and use this information to<br>
decide when to stop, perhaps because the results are good enough,<br>
or training has slowed down, or overfitting has set in.</p>
<p>Only after you have settled on your “final” trained network, do you<br>
apply it to your test set to see how well it actually performs.  (The<br>
purist notion of rigor here is that using your validation set to evaluate<br>
the performance of your final network would constitute a “data leak”<br>
because you did use the validation set as part of your training process.)</p>
<aside class="quote no-group">
<blockquote>
<p>With data augmentation, I currently have about 22,000 input images</p>
</blockquote>
</aside>
<p>How many <em>unaugmented</em> annotated images do you have?  How big<br>
are the images, and are different regions of a given images  relatively<br>
independent so that it might be reasonable to consider different regions<br>
of a single image as independent data samples?</p>
<aside class="quote no-group">
<blockquote>
<p>in 25 different distribution groups.</p>
</blockquote>
</aside>
<p>What is a “distribution group?”  What is the relationship of a distribution<br>
group and the 5-element output of your network?</p>
<p>Related to the question of whether your ground-truth annotations have<br>
the same structure as your network output, what is the relationship of<br>
a distribution group to your annotations?</p>
<aside class="quote no-group">
<blockquote>
<p>I’ve considered transfer learning. For instance, some papers have claimed that ResNet learns many texture features<br>
…<br>
Can I somehow adapt it to my more general objectives?</p>
</blockquote>
</aside>
<p>My gut feeling is that ResNet is unlikely to be helpful for you (but I<br>
could well be wrong).</p>
<p>My intuition – not based on any real experience or evidence – about<br>
transfer learning goes something like this:</p>
<p>The upstream layers of something like ResNet “learn” low-level<br>
“features” – things like edges and arcs and maybe blobs.  In the<br>
middle these get aggregated together into mid-level features like<br>
nose-like blobs and eye-like blobs.  In the final, down-stream layers<br>
they get aggregated into faces, cat-like faces, dog-like faces, and<br>
ultimately, into cat-vs-dog predictions.</p>
<p>As you say, upstream ResNet layers recognize “textures” so it’s<br>
plausible that pre-trained, upstream ResNet layers could be a useful<br>
starting point for your training.  But your ultimate task (as I understand<br>
it) is really quite different than what ResNet was trained for, so I’m not<br>
convinced that you would get that much benefit using it as a starting<br>
point.</p>
<p>Best.</p>
<p>K. Frank</p>

Sally_Goldin: <p>Hello, KFrank,</p>
<p>Thank you for your detailed and thoughtful reply.</p>
<blockquote>
<p>Leaving aside your current network and the kind of annotations you<br>
have for your data, ideally, what would you like the output of your<br>
network to be?  Would you like the full probability distribution of<br>
particle sizes – that is, roughly speaking, a histogram of particle sizes<br>
with lots of bins?</p>
</blockquote>
<p>Five bins is standard for this application. But yes, I’m basically looking for a histogram, aggregated to these five bins and normalized so the total of all bins is 1.0.</p>
<blockquote>
<p>Is it important that you use neural networks / machine learning for this<br>
task, or would you be open to using other computer-vision techniques?</p>
</blockquote>
<p>We’ve actually tried some analytical machine vision to identify individual particles. Results were quite poor. I am hoping ML can do better.</p>
<aside class="quote no-group">
<blockquote>
<p>My output is a 5 element vector of values that represent the relative proportion of particles in different size categories.</p>
</blockquote>
</aside>
<blockquote>
<p>What, in more detail, is the meaning of these five numbers?  Are your<br>
ground-truth annotations analogous length-5 vectors that have the<br>
same meaning?</p>
</blockquote>
<p>Yes, ground truth labels mean the same thing. These images were manually acquired from lab samples prepared to have different distributions.</p>
<blockquote>
<p>What loss function are you using to compare your network outputs<br>
to your annotations?</p>
</blockquote>
<p>RMSE</p>
<blockquote>
<p>What does the final layer of your network look like – that is what is<br>
the layer that produces this length-5 output?</p>
</blockquote>
<p>nn.linear(32,5), followed by softmax to normalize to 0-1 range.</p>
<blockquote>
<p>Are the test images that your network does poorly on in any way<br>
different in character than the images you used to train your network,<br>
or are they basically the same?  For example, did you randomly<br>
select some test images from the same larger annotated set from<br>
which you selected your training set?</p>
</blockquote>
<p>I have randomly selected 20% of the images as validation images. These do reasonably well though the average loss is higher than for the training images (as I’d expect). I then tried other images which have the same type of annotation but which are not from the original data set. Our goal is transfer to other images.</p>
<blockquote>
<p>In general, if you are getting significantly worse results on test images<br>
that are of the same character as you training images, you are seeing<br>
signs of <em>overfitting.</em>  That is, your network has been trained enough<br>
times on the same images that in addition to “learning” the properties<br>
you want it to learn, it is also learning recognize individual images<br>
by their individual peculiarities.  Having recognized a specific image,<br>
the network now “parrots back” the “right” answer.</p>
</blockquote>
<p>Yes, I have come to that conclusion also. However, I realized that even without the overfitting issue I see no sign of the generalization that I am trying for.</p>
<blockquote>
<p>The “rigorous” approach is to split your annotated data set into three<br>
pieces (usually randomly) – a <em>training set,</em> a <em>validation set,</em> and a<br>
<em>test set.</em>  You use your training set to train with backpropagation and<br>
optimizer parameter updates.  At times of your choosing (for example,<br>
once every epoch), you calculate your loss and potentially other<br>
evaluation metrics for your validation set, and use this information to<br>
decide when to stop, perhaps because the results are good enough,<br>
or training has slowed down, or overfitting has set in.</p>
<p>Only after you have settled on your “final” trained network, do you<br>
apply it to your test set to see how well it actually performs.  (The<br>
purist notion of rigor here is that using your validation set to evaluate<br>
the performance of your final network would constitute a “data leak”<br>
because you did use the validation set as part of your training process.)</p>
</blockquote>
<p>Yes, of course… but I’m in exploration mode at the moment. I’ve actually trained several different networks with variations in input image size and augmentation techniques. The results vary somewhat but the pattern is more or less the same.</p>
<aside class="quote no-group">
<blockquote>
<p>With data augmentation, I currently have about 22,000 input images</p>
</blockquote>
</aside>
<blockquote>
<p>How many <em>unaugmented</em> annotated images do you have?  How big<br>
are the images, and are different regions of a given images  relatively<br>
independent so that it might be reasonable to consider different regions<br>
of a single image as independent data samples?</p>
</blockquote>
<p>About 2600 different images. They are about 1500x1500 to start. My latest attempt downsamples them to 224x224 (earlier versions used 64x64 and 128x128).</p>
<p>The entire image area is relevant to the answer, because the particles are not uniformly distributed in the image area. One of my earlier attempts used subsetting for augmentation, but I realized the subsets might in fact reflect a significantly different distribution.</p>
<blockquote>
<p>What is a “distribution group?”  What is the relationship of a distribution<br>
group and the 5-element output of your network?</p>
</blockquote>
<p>As noted above, these images were synthetically created by preparing a gravel sample with a particular distribution of particle sizes, spreading it in a frame, and taking a series of photos. There were 25 different samples prepared, each one with a different distribution of particle sizes. For each distribution, the students took about 1000 images, from different locations. After a few images, they would “disturb” the gravel, moving it around to change the patterns, but not the underlying distribution.</p>
<p>What I mean to convey with the term “distribution group” is a set of images all of which have the same 5-element histogram.</p>
<blockquote>
<p>As you say, upstream ResNet layers recognize “textures” so it’s<br>
plausible that pre-trained, upstream ResNet layers could be a useful<br>
starting point for your training.  But your ultimate task (as I understand<br>
it) is really quite different than what ResNet was trained for, so I’m not<br>
convinced that you would get that much benefit using it as a starting<br>
point.</p>
</blockquote>
<p>That is more or less the conclusion I’ve come to. I’ve been looking for papers about using texture in ML. However, pretty much every one is considering a classification task – rather than a “characterization” or “description” task.</p>
<p>Any suggestions would be welcome! (Including your thoughts about non-ML methods.)</p>
<p>Thank you!</p>

KFrank: <p>Hi Sally!</p>
<aside class="quote no-group" data-username="Sally_Goldin" data-post="3" data-topic="131110" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sally_goldin/48/18108_2.png" class="avatar"> Sally_Goldin:</div>
<blockquote>
<p>Five bins is standard for this application. But yes, I’m basically looking for a histogram, aggregated to these five bins and normalized so the total of all bins is 1.0.</p>
</blockquote>
</aside>
<p>So just to confirm:  Regardless of any intermediate steps taken in your<br>
overall workflow (e.g., producing a fine-grained histogram or identifying<br>
each individual gravel particle), your final desired goal is to infer a<br>
five-fixed-bin probability distribution.  Correct?</p>
<aside class="quote no-group">
<blockquote>
<p>My output is a 5 element vector of values that represent the relative proportion of particles in different size categories.</p>
<blockquote>
<p>What loss function are you using to compare your network outputs<br>
to your annotations?</p>
</blockquote>
<p>RMSE</p>
<blockquote>
<p>What does the final layer of your network look like – that is what is<br>
the layer that produces this length-5 output?</p>
</blockquote>
<p>nn.linear(32,5), followed by softmax to normalize to 0-1 range.</p>
</blockquote>
</aside>
<p>This strikes me as <em>not</em> the best way to go.</p>
<p>Your ground-truth is a (discrete) probability distribution and you wish<br>
to predict a matching probability distribution.  Cross-entropy is the<br>
natural loss function for such a task.  It is a natural measure of how<br>
much two probability distributions differ, and root-mean-squared-error,<br>
while not unreasonable, just isn’t as natural (and the lore is that it<br>
doesn’t work as well for this kind of use case).</p>
<p>Note that pytorch’s <code>CrossEntropyLoss</code> was designed with classification<br>
in mind where you have an integer class label as your <code>target</code> so it<br>
doesn’t apply to your use case.  But cross-entropy makes perfect sense<br>
with a probabilistic <code>target</code> (sometimes called “soft labels”).</p>
<p>You can easily write your own probabilistic cross-entropy, as outlined<br>
in this post:</p>
<aside class="quote quote-modified" data-post="2" data-topic="69501">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/soft-cross-entropy-loss-tf-has-it-does-pytorch-have-it/69501/2">Soft Cross Entropy Loss (TF has it does Pytorch have it)</a> 
  </div>
  <blockquote>
    Hello Raaj! 

I do not believe that pytorch has a “soft” cross-entropy function built in. 
But you can implement it using pytorch tensor operations, so you should 
get the full benefit of autograd and gpu acceleration. 
See this (pytorch version 0.3.0) script: 
import torch
torch.__version__

# define "soft" cross-entropy with pytorch tensor operations
def softXEnt (input, target):
    logprobs = torch.nn.functional.log_softmax (input, dim = 1)
    return  -(target * logprobs).sum() / input.shap…
  </blockquote>
</aside>

<p>If you go this route, using <code>Linear (32, 5)</code> as your final layer is the<br>
right thing to do, but you will <em>not</em> want to follow it with a <code>softmax()</code><br>
layer.  (Both pytorch’s <code>CrossEntropyLoss</code> and the probabilistic<br>
version I linked to have, for reasons of numerical stability, <code>softmax()</code><br>
built into them in the form of <code>log_softmax()</code>.)  That is, you want your<br>
network to output the <em>logits</em> that correspond to your predicted<br>
probabilities, rather than the probabilities themselves.</p>
<p>I can’t say that changing to a cross-entropy loss will necessarily work<br>
great, but it is the first thing I would try.</p>
<aside class="quote no-group">
<blockquote>
<p>I have randomly selected 20% of the images as validation images. These do reasonably well though the average loss is higher than for the training images (as I’d expect). I then tried other images which have the same type of annotation but which are not from the original data set. Our goal is transfer to other images.</p>
</blockquote>
</aside>
<p>Training a network that generalizes to inputs that are not entirely of<br>
same character as the training set is an admirable goal, but is harder,<br>
and not always achievable.  Regularization techniques (e.g., weight<br>
decay or dropout) do help reduce overfitting, and can help with<br>
generalization to fundamentally different inputs, but are not a panacea.</p>
<p>I would recommend increasing the diversity of your training set to<br>
more completely represent the character of the real-world images<br>
that you will want to analyze “in the wild.”</p>
<p>(More, and better, and more fully representative data, although<br>
potentially expensive, is usually the best first step towards improving<br>
the practical performance of a network.)</p>
<p>Just to confirm, because your network works reasonably well on your<br>
validation images, this is not an issue of overfitting, per se, but one<br>
of generalization.</p>
<aside class="quote no-group">
<blockquote>
<blockquote>
<p>How many <em>unaugmented</em> annotated images do you have?  How big<br>
are the images</p>
</blockquote>
<p>About 2600 different images. They are about 1500x1500 to start. My latest attempt downsamples them to 224x224 (earlier versions used 64x64 and 128x128).</p>
</blockquote>
</aside>
<p>This is a reasonable number, but more data might well help.  (Again,<br>
the size of your images may help in the sense that different regions<br>
of a given image might be sufficiently independent so as to count,<br>
roughly speaking, as separate data samples.)</p>
<p>How much computing power do you have for this project?  Can you<br>
reasonably do a lot of long training runs on large models with large<br>
datasets?</p>
<p>Are you downsampling because of hardware limitations, or are you<br>
able to train with the full 1500x1500 images (or more modestly<br>
downsampled images)?  I imagine an image of gravel and could<br>
believe that downsampling to 244x244 would lose lots of relevant<br>
information.</p>
<aside class="quote no-group">
<blockquote>
<blockquote>
<p>Is it important that you use neural networks / machine learning for this<br>
task, or would you be open to using other computer-vision techniques?</p>
</blockquote>
<p>We’ve actually tried some analytical machine vision to identify individual particles. Results were quite poor. I am hoping ML can do better.</p>
<p>Any suggestions would be welcome! (Including your thoughts about non-ML methods.)</p>
</blockquote>
</aside>
<p>Well, if you’ve already tried non-neural-network “particle analysis” and<br>
it doesn’t work, I don’t have much to offer.</p>
<p>In this vein, however, if a person looks at one of your images, are<br>
the individual grains of gravel readily apparent?  Do most pixels in<br>
your images belong to a grain of gravel, or do you have a significant<br>
fraction of “background” pixels (e.g., maybe a piece of paper that<br>
your gravel was spread out on)?</p>
<p>How often in your images is a grain of gravel partially occluded by<br>
another grain?</p>
<p>One approach to “particle analysis” – to identify individual grains of<br>
of gravel – would be to train (or fine tune) a segmentation network.<br>
This could either be <em>semantic segmentation</em> where you label each<br>
pixel as gravel body, vs., for example, gravel boundary and / or<br>
background.  Or it could be <em>instance segmentation</em> where the network<br>
further infers that one set of “gravel pixels” belongs instance-A of a<br>
gravel grain, and some other set belongs to instance-B.</p>
<p>The significant <em>disadvantage</em> of segmentation is that your training<br>
images would need to be annotated with pixel-by-pixel labels, a<br>
tedious, and potentially expensive task.</p>
<p>Would it be possible for you to post a representative image from your<br>
training set?  If the image is too large, posting a cropped, rather than<br>
a downsampled image would be preferable.</p>
<p>Best.</p>
<p>K. Frank</p>

Sally_Goldin: <p>Hello, K.Frank,</p>
<p>Sorry about the delay in replying. I really appreciate your thoughtful responses.</p>
<ol>
<li>
<p>Loss function - I can understand why you’d recommend cross-entropy loss. I chose RMSE because the end users would like to be able to compare the output “error” from the model with the results from the classical machine vision approach. RMSE intuitively captures the overall deviation between an output distribution and the ideal distribution. Cross-entropy loss would be more difficult from them to interpret.</p>
</li>
<li>
<p>I was startled by your question about downsampling, since I thought this was a standard approach in building CNNs. Wouldn’t a network designed to process 1500x1500 images be enormous?  In any case, the particles are quite visible in a 224x224 image (see below). I’m also switching from color to gray scale. Do you think this is a bad idea?</p>
</li>
</ol>
<p>Here’s a downsampled example (downsampled to 224x224 using ImageMagick, not Pytorch transforms):</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/6/56c4a834157c7a612c1189b061a55d188963f7d2.jpeg" alt="DownSampledExample" data-base62-sha1="cnAoGOgheCRcbf8dFIN7w06huBc" width="224" height="224"></p>
<p>(The images are normalized before training. This has not been.)</p>
<p>The image area has been cropped to a circle because I am using rotations to do data augmentation.</p>
<ol start="3">
<li>
<p>I think you are right about needing a greater diversity of training data. All these images were taken in the same lab, with the same lighting. I’ll have to discuss this with our end users, to see what they can do.</p>
</li>
<li>
<p>I recently became aware of some work doing segmentation analysis of this general sort of image using ML, specifically the Mask R-CNN model. Results are excellent, but for training, they had to draw polygons around every single particle in each training image!</p>
</li>
<li>
<p>My hardware resources are fairly modest, but the ML machine is more or less dedicated to that task, so I can start it up and let it run.</p>
</li>
</ol>
<p>Thank you again for your suggestions. I will update this thread with any progress.</p>
<p>Best regards,<br>
Sally</p>

gphilip: <aside class="quote no-group" data-username="Sally_Goldin" data-post="5" data-topic="131110">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sally_goldin/48/18108_2.png" class="avatar"> Sally_Goldin:</div>
<blockquote>
<p>RMSE intuitively captures the overall deviation between an output distribution and the ideal distribution. Cross-entropy loss would be more difficult from them to interpret.</p>
</blockquote>
</aside>
<p>If you have the (hardware/time) bandwidth, it may be a good idea to try training with Cross-entropy loss as the loss function. You may find that this <em>also</em> makes RMSE decrease, possibly even faster than when training with RMSE as the measure of loss.</p>

KFrank: <p>Hi Sally!</p>
<aside class="quote no-group quote-modified" data-username="Sally_Goldin" data-post="5" data-topic="131110" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sally_goldin/48/18108_2.png" class="avatar"> Sally_Goldin:</div>
<blockquote>
<ol>
<li>Loss function -<br>
I chose RMSE because the end users would like to be able to compare the output “error” from the model with the results from the classical machine vision approach.<br>
…<br>
Cross-entropy loss would be more difficult from them to interpret.</li>
</ol>
</blockquote>
</aside>
<p>I agree with G. Philip’s comment.  It is perfectly reasonable to use one<br>
loss function for training and one or more distinct loss-like functions<br>
as evaluation metrics.  These evaluation metrics might be in some<br>
substantive sense better metrics, or they might be technically not as<br>
good but easier for non-specialists to interpret.  Either way, it’s common<br>
practice to use multiple evaluation metrics.</p>
<p>It’s also true, as G. Philip notes, that metric X may improve more<br>
rapidly and end up at a better level if you train with loss A than if<br>
you trained directly with metric X.</p>
<p>In any event, I would strongly recommend that you try training<br>
with cross entropy (or perhaps the <a href="https://pytorch.org/docs/1.9.0/generated/torch.nn.KLDivLoss.html" rel="noopener nofollow ugc">Kullback-Leibler divergence</a>,<br>
a related measure of the discrepancy between two probability<br>
distributions).  Doing so might not work better than training with<br>
root-mean-squared-error (RMSE), but it certainly could, even<br>
with RMSE as your evaluation metric.</p>
<aside class="quote no-group">
<blockquote>
<ol start="2">
<li>Wouldn’t a network designed to process 1500x1500 images be enormous?</li>
</ol>
</blockquote>
</aside>
<p>The network, in and of itself, would not be significantly larger, as it<br>
would likely consist of upstream convolutional and downsampling<br>
layers whose sizes are independent of the size of the input images.<br>
However the memory and computation time required to train on<br>
(and evaluate) the large images would be potentially enormous.</p>
<aside class="quote no-group">
<blockquote>
<p>In any case, the particles are quite visible in a 224x224 image (see below).</p>
</blockquote>
</aside>
<p>The larger particles look distinct to me.  In the upper right of your<br>
sample image, around two o’clock, the smaller particles – if that’s<br>
what they are – look to my eye to be harder to distinguish.  It’s a<br>
judgement call, but if it’s hard for your eye to see, you would be<br>
making it harder on your network,  Perhaps something like 500x500<br>
could provide a happy medium.</p>
<aside class="quote no-group">
<blockquote>
<p>I’m also switching from color to gray scale. Do you think this is a bad idea?</p>
</blockquote>
</aside>
<p>My intuition is that gray scale should be fine.  Are different particles<br>
of significantly different hues?  Are the particles significantly easier<br>
to distinguish by eye in the color image?  If not, it doesn’t seem like<br>
a gray-scale conversion would discard much relevant information.</p>
<aside class="quote no-group">
<blockquote>
<p>(The images are normalized before training. This has not been.)</p>
</blockquote>
</aside>
<p>This is perfectly reasonable and possibly helpful.  If you do train<br>
with normalization, be sure to evaluate and infer with similarly<br>
normalized inputs.</p>
<aside class="quote no-group">
<blockquote>
<ol start="4">
<li>for training, they had to draw polygons around every single particle in each training image!</li>
</ol>
</blockquote>
</aside>
<p>Indeed!</p>
<aside class="quote no-group">
<blockquote>
<ol start="5">
<li>My hardware resources are fairly modest, but the ML machine is more or less dedicated to that task, so I can start it up and let it run.</li>
</ol>
</blockquote>
</aside>
<p>If you can perform a forward pass and backpropagation on a single<br>
image with your hardware – typically an issue of gpu memory – you<br>
should be able to train, at the cost of hours or days (or even weeks)<br>
of clock / calendar time.  (If you can’t run a single sample through<br>
your model, things become much more painful, and it wouldn’t bode<br>
well for managing to carry out training in a reasonable amount of time.)</p>
<p>Good luck.</p>
<p>K. Frank</p>