Make flags smaller when they have no user associated