Yes, you can omit the olives and capers if you prefer. These ingredients contribute to a specific flavor profile, but the dish will still be tasty without them.