Check your technique - you don’t need to aggressively rub off this skin. And don’t scrub too often. If your lips are hurting, lay off exfoliating that area for a bit.