Also don't assert on spurious row/col replies.

This can happen in a similar situation as in the commit before:
Request for one Node, Node gets deleted and another added, also
create request for that node, get two Replies.

In such a case, the row/col count must equal what we got before.