Try placing a damp cloth on them or dabbing on calamine lotion - this will provide some relief and can be repeated as often as necessary. Avoid scratching your armpits. While this can provide momentary relief, it further irritates your skin, which may cause an infection.