Moved comments to more appropriate place. Also, added casesensitive parameter.
Added more tests
It looks like HashSet.Add automatically checks for duplicate values, no need to check manually. So, I simplified the code.