No, you don't have to use tofu. You could try using plant-based milk instead. I like using tofu because it gives the drink a lovely creamy texture.