In my experience it gets a little too thick without milk, but you may still give it a try. It sounds like it would taste also great with just cream and sugar.