fixes invalidStateError when trying to remove from buffer after signalEndOfStream was called