kick the workers even if same vbucketmap

Because the workers might want to rebuild streams and such.