Replace whole milk with oat milk, unsweetened soy milk, almond milk and even coconut milk. Do note that the inherent flavours of some non-dairy milks like almond milk and coconut milk can come through in your baked muffins.