That way the mulch will decompose over the winter and add all that good stuff to the soil. In our area, any time after May 15 for perennials, and after May 30 for annuals. Perennial gardens take time to mature. While adding plants to your gardens, allow enough space for your plants to grow and mature. Your garden has a front and a back.
Color counts, too.
Consider the color both of the flowers and the foliage as you choose your plants. Try grouping plants together at the nursery to see how they look, and ask for advice. Feel free to experiment. Just because it's in the ground someplace doesn't mean it has to stay there. Ulster and Duchess counties are in the coldest part of Zone 5.