Mostly because one rather nasty version attacks the genital area and is a long lasting sexually transmitted disease (STD) that you can never get rid of.
The more common varieties of herpes (which most people suffer from) only affects the lips and mouth and does not contain the same stigma as genital herpes.