Fixed the logic bug! At the end of the button touched event I was always adding the card anyways, even if it was matched earlier in the block.