When we're rendering a sub-graph, we really don't want to check for all
neighbors.  Fixes #2799