It could be because of multiple reasons, including your past experiences and the kind of person you are. Both combine and make the person you are today. For example, your past might have been painful and traumatic, or you could just not be a person who likes physical affection so much.