Memoizing neighbor patches in topologies