Speed up memoization by storing unique ids in lineages