You can certainly substitute water for milk in the batter, just be aware that the mixture will not be quite as creamy.