When it is operating fully, a person doesn't even need society to punish him for his misdeeds. Our guilty consciences make us feel terrible enough, so bad that we won't make the same mistake again. The process operates largely unnoticed, as it exists in part in our unconscious minds. Our sense of guilt, then, is a result of suppressing our instinctive natures for the sake of the larger good we call civilization. Freud thought this was inevitable and even necessary.
Guilt is the price for having a conscience.
Guilty feelings arise when we have violated a moral norm that we accept as valid. A person who feels guilty, notes philosopher Herbert Morris, is one who has internalized norms and, as such, is committed to avoiding wrong. The mere fact that the wrong is believed to have occurred, regardless of who bears responsibility for it, naturally causes distress. When we are attached to a person, injury to that person causes us pain regardless of who or what has occasioned the injury. Psychologists Nico Frijda and Batja Mesquita of the University of Amsterdam find that people feel guilty about having harmed someone even when it was accidental.