While water can be used in place of milk in some recipes, it will affect the texture and taste of your baked goods. Milk adds moisture, richness, and flavor to baked goods, so using water will result in a drier, less flavorful final product.