You can back out at any point before things really hot up, if you've changed your mind. This might not be a bad idea if you've entered into a romance with Gale or Wyll, as they will break up with you immediately the next morning after finding out about your tryst with Mizora.