Patch remove tags

Basically using the score isn't reliable apparently there is a way to end up with a negative score. (I am unsure of the exact conditions this occurs under, I believe it has something to do with the cancelling path)
Instead lets use the actual size of the tag set to determine if we should remove from the global tags set.

also fixed some comments.