Refactor subscriber

Prevent some deadlocks and panics.