Once I'm happy with the results on the residual layer (or it doesn't need any work), I'll look at the largest detail scale (usually Wavelet scale 5).
This has been the scale level that usually produces the greatest impact quickly.