No, you don't. You can swap out the buttermilk with regular or alternative milk. Buttermilk definitely adds a tanginess and tenderness that you can't get with those other milks, but the ricotta and lemon do much of the same work.