Hold a reference to the busy object while accessing its members.

This fixes a couple of races having become apparent when testing
streaming