Move neighbor finding into Graph