Yes you can. However, the texture of your drink is likely to be less thick and you may want to up the proportion of the milk for better balance.