Only worry about a skin tag if you find them to be unsightly or irritating. If the skin tags are rubbing against the skin, clothing, or jewelry more frequently, then you can consider removal.