Follow-up I569098: Rebuild reference list when reopening as it can change

Filed bug 68890 for fixing the performance regression from this.

Bug: 68869
Change-Id: I0564c7a9c61d33cdc8b61b2e69d01733b06de32f
Merge "Follow-up I569098: Rebuild reference list when reopening as it can change"