Yes, technically, you can, but keep in mind that olive oil has a strong and robust flavor and will make the mayonnaise strong in flavor too.