The affected flowers require days or weeks to get back to same point of growth as before they were stressed. Repeatedly stressing your impatiens this way causes significant decline in overall plant quality and flowering performance. Larger plants are not better than smaller plants. The reason is smaller impatiens grow into your landscape soil faster and more thoroughly. Smaller starter plants actually end up growing bigger with more flowers and lasting longer into the Spring than larger starter plants.
Planting impatiens with a high quality time-release fertilizer is the biggest secret of all.
Impatiens in pots can be tricky. Impatiens are very sensitive to wet feet and fungus from over watering. Usually the over watering is from nature. When we get into a heavy rain pattern, potted impatiens die easily. Consider moving pots under cover is a rain spell is predicted to persist.