Patch remove tags

Basically using the score isn't reliable apparently there is a way to end up with a negative score.
Instead lets use the actual size of the tag set to determine if we should remove from the global tags set.

also fixed some comments.