If you want to make the pudding without whole milk and prefer using plant-based milk like oat milk, soy milk, or almond milk, you can do that too. Keep in mind that some non-dairy milk tends to set just fine, but others don’t. Plus, it also depends on the brand of non-dairy milk you are using, so you’ll have to experiment to see what works.