He tells me that it doesn't matter if I did or didn't he would still love me just the same, but I feel that if thats the case he wouldn't keep harassing me about it. I think he is just using this as an excuse to not get married because he knows I'm a person that if I feel that I am right I will stand my ground. Dear Harassed, Your question has been asked by countless lovers for centuries. Men are driven by the green-eyed monster of jealousy to know, and then when they're told, they're sorry they asked. Sometimes much worse.
Smart women avoid answering or tell a little white lie to keep from hurting their lover's feelings.
You know he doesn't want to know that you've been with someone else. He wants to hear that you've been faithful. Confessing might make you feel better, but it won't do your relationship any good and could destroy it. If you confess to having sex while your Fianc was in jail, you'll be reinforcing his worst fears. And if you tell him you had sex once or twice two years ago, he'll think there was more to it and want more details.