Considering olives are fruits, all olive oils would technically be "fruity." But technicalities aside, olive oils can range in flavor from grassy and buttery to peppery and the sought-after fruity. Finer-quality extra-virgin olive oils tend to be perceived as fruity with aromas and notes of tart apple, apricot, pineapple, or banana. There are even olive oils infused with orange. But you don't need to go crazy trying to find the perfect oil for this cake. Just make sure the one you choose isn't peppery.