Fix the bug that Switch doesn't propagate 'unsubscribe'

Found in #1666