Yes, you can! Chamomile tea has a mild taste and pairs well with many other ingredients. It will not have a problem melding with the milk flavor.