No, without them, you’ll still have a wonderful Orange Pound Cake. If you just want to add one, I highly recommend trying the orange syrup - it really does take this cake to a whole new level. The glaze or drizzle is more for looks and can definitely be skipped.