Yes, water works great. Sometimes the milk sets before you have a chance to stir in the flour, so water is a good option. Water works well if you want a non-dairy substitute too.