No, you don't need milk! You can use water, non-dairy milk, or even just cream though it will not have the same texture.