Wearing white means you can afford to get out of the city on vacations elesewhere just after the summer ended. Labor day represents the end of summer and the rule was estabilished that no one wear white after the Labor day if they don’t have enough money to take fall and winter vacations.