Fix bug where packets weren't interleaving

Packets were not interleaving, thus new data was being sent out with
potentially non-monotonically increasing timestamps