When eating an olive oil cake you can taste the fruity character of the olive oil and the moistness it provides. That's why it pairs very well will lemon and orange flavors. If you're afraid that it will taste exactly like olive oil, then the answer is no.