We are adding elements to this cache, and despite we clean it each time we do a buildContacts (when we load the first time, or when we change the order of the contacts), we don't always add, but I would suggest to remove this cache when we remove an item from the list.

Or even better remove it when we finish the  lazyloadsearch, the more memory we save the better :)