It's possible, but this is actually a pretty efficient way to do it. If `sklearn` or something else has some functions for automatically returning the neighbors in a 3D grid, then we could use that. This has the advantage of giving the exact same list, in the exact same order, as the C code, which is nice.