You can absolutely skip the rosemary. You can use mint, thyme or sage instead, or simply skip the herb garnish altogether.