Revert "Publish changes from TreeAgentSet to allow nw ext to update graph context"

This reverts commit 523b89018fa16101dcce924e5afdf7a93e03f09f.

Will turn into a pull request instead.