The mustard helps the oil and egg emulsify together. You will have a higher chance of the mixture breaking or separating, but it could be left out. You will not taste the mustard in the mayonnaise. {I highly recommend adding it if making this in a food processor or blender as the chance of it separating is higher if the oil is added too fast.}