- Only construct Neighbors on active patches