Fix some refcounting and stream control parent setting issues