Sure, if you aren't a fan of capers, feel free to omit them entirely. If you'd like a substitute, try chopped green olives, mushrooms, or artichoke hearts.