Yes, although the rosemary flavor may not be as strong. Try using 1 teaspoon of dried rosemary for every 1 cup of water. Taste, and add more rosemary for a stronger flavor if needed.