libfreerdp-codec: detect planar overflow per set of planes, not individual planes