Yes, you can. I love the zing and sweetness the glaze gives the cake but it is still delicious on its own.