Maybe we should generalize. If the max-min range for any channel is less than 50 (or something) maybe it doesn't make any sense to enhance that channel... It seems like it would just cause posterization type artifacts.

I'd guess that we should treat each channel independently, however.

So:

const MIN_RANGE_TO_ENHANCE = 50;
if (maxRed - minRed < MIN_RANGE_TO_ENHANCE) {
  minRed = 0;
  maxRed = 255;
}
// Repeat for green and blue

I don't really know whether 50 is a reasonable constant. I'd imagine that any photo would be > 50 for all channels.  But non-photographic images (including solid colors) might be under it.