The degree of leaf coloration or loss is a function of parentage and not the severity of the winter. In unusually cold winters, certain azaleas may lose more leaves than they would in mild winters. My azalea foliage looks white, and there are black bugs underneath the leaves. You undoubtedly have a problem with the azalea lace bug. This insect hatches early in spring as the new foliage begins to mature and its numbers may build to damaging levels with successive generations.
Lace bugs reach their peak in late summer and do their worst in sunny, exposed sites.
Spiders are important predators of lace bugs and since they shy away from sunny, hot places, plant your azaleas where there is some shade. Insecticidal soap, horticultural oil, or a systemic insecticide may spare your azaleas the damage if applied in spring when the first hatchlings are noticed. All the leaves on my azaleas are notched, and they are growing poorly. I don't see any insect or pest causing this damage. The damage you describe is that of the black vine weevil, a small nocturnal weevil that, in its larval stage, feeds on azalea and rhododendron roots.