Yes of course you can drink earl grey without milk. It's really up to you and what you prefer.