Free handle if we can't spawn a thread.  Found by codesonar
Rework min/max/spare handling

So we don't let idle connections make us go below "min"