Definitely! Just allow the milk and the tea to cool down (you can also cold brew the tea) and serve chilled and/or over ice.