Olive oil cake should be moist but not oily. If your cake came out oily it's most likely that the ingredients were not measured correctly and the ingredient ratios are off.