Unsubscribe from ensure only when needed

Fixes #305.