Eggnog is not a direct substitute for milk in baking because you also have to account for the eggs. However, if you adjust your eggs in your baked goods, you can substitute milk. Either experiment yourself or use a tested recipe to take the guesswork out of it!