raop: Fix sink getting destroyed after one use

It is expected to get disconnected after switching back to a
different sink.