Modified Difference method to only iterate once.  Thanks g0dfried at http://www.reddit.com/r/golang/comments/1hlijg/a_simple_set_type_for_the_go_language/