Yes, I understand. The same problem exist with vnsi but we can't wait forever. If we don't get the remaining bytes withing a given timeout, we lost sync and need to terminate connection. For vnsi I do this already.
Waiting here for longer does harm because it blocks player thread. You won't make player hang in case of a nasty behaving backend, do you?