Yes, it’s important to pit the olives before using them in the cake. The pits can affect the texture and taste of the final product.