No, you don't have to. But I find that yogurt gives a wonderfully rich and creamy texture. If you're dairy-free, you can always use dairy-free milk and dairy-free yogurt.