I don't remember being told any such thing. Ser Perth looked at Nema, who nodded.

Does it follow that "Ser Perth did not remember being told that."?
yes