If we care at all about gender equality, we must ditch the assumption that all men enjoy similar privilege.